novox/mesh-lab
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Rename mesh-control -> mesh-controller, substrate -> foundation

Browse Source 
One name per thing, per the HQ glossary: the module/container/image/binary/repo
becomes mesh-controller, the seat the-controller, and the store+broker pair the
foundation (embedded base bundles, default template and example lock renamed with
their go:embed directives). No behaviour change — a pure vocabulary rename.

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
This commit is contained in:
jschoubben
2026-09-16 18:40:40 +02:00
parent 49b80d8516
commit 5d6e8fbe7a
89 changed files with 785 additions and 785 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/integration/whole-mesh-full.test.ts
+55 -55
View File
@@ -5,12 +5,12 @@
*
* hosting (public) home (private, behind a NAT access point)
* novox 192.0.2.20 — the ANCHOR: ace 10.99.1.10 the home server, media/IoT set
* substrate (store/broker/ shanks 10.99.1.20 workstation (light: portainer only)
* foundation (store/broker/ shanks 10.99.1.20 workstation (light: portainer only)
* control) + the whole novox g14 10.99.1.30 workstation (light: portainer only)
* set + overlay hub + ingress
*
* There is NO separate anchor: novox IS the anchor. The substrate runs on novox, and novox also
* enrols as a node and receives its own service set — the substrate host and a service node at once.
* There is NO separate anchor: novox IS the anchor. The foundation runs on novox, and novox also
* enrols as a node and receives its own service set — the foundation host and a service node at once.
*
* TWO ACTS, AND THE BED NOW DISTINGUISHES THEM (novox/hq ADR 0067).
*
@@ -22,9 +22,9 @@
* is a WORKING MESH OF ONE, and this bed asserts exactly that before going any further.
*
* JOINING — ace, shanks and g14 then join a mesh that already exists: host binary, token,
* `enrol`, run the agent. No bootstrap, no substrate, no registry. novox is NOT enrolled again.
* `enrol`, run the agent. No bootstrap, no foundation, no registry. novox is NOT enrolled again.
*
* The bed used to do neither. It applied the substrate bundle itself and looped enrolment over all
* The bed used to do neither. It applied the foundation bundle itself and looped enrolment over all
* four machines as one continuous operation — which got the order right by accident and modelled
* the wrong shape, and is why ADR 0067's own acceptance check ("the bed bootstraps through the
* installer rather than around it") went unmet. Genesis GATES joining: if it stops, the bed says
@@ -39,10 +39,10 @@
* BEFORE any heavy module lands, so the cross-segment-overlay verdict survives whatever the module
* convergence then does. Phase B converges the full node sets and reports per node.
*
* SUBSTRATE-ON-NOVOX PORT COLLISIONS (a real consequence of collapsing the anchor onto novox that the
* separate-anchor beds never hit): the substrate store binds 127.0.0.1:5432 and novox's postgres
* provider publishes 5432; the substrate broker binds 5671 + 127.0.0.1:5672 and novox's lavinmq
* provider publishes 5672. The two provider host publishes are REMAPPED off the substrate's ports
* FOUNDATION-ON-NOVOX PORT COLLISIONS (a real consequence of collapsing the anchor onto novox that the
* separate-anchor beds never hit): the foundation store binds 127.0.0.1:5432 and novox's postgres
* provider publishes 5432; the foundation broker binds 5671 + 127.0.0.1:5672 and novox's lavinmq
* provider publishes 5672. The two provider host publishes are REMAPPED off the foundation's ports
* (REMAP below); consumers reach the providers over the mesh network on the container port, so the
* host side is free to move. Reported as a topology finding.
*
@@ -50,7 +50,7 @@
* (whole-mesh-full-live) and NOT torn down — it is left standing and browsable. Without it the bed
* behaves like every other: raise in before(), destroy in after().
*
* MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
* MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/foundation-first-node.lock
* MESH_LAB_BOOTSTRAP_BINARY=.../mesh-bootstrap MESH_LAB_CATALOG=.../mesh-catalog/modules
*/
@@ -65,7 +65,7 @@ import { destroy, exec, instanceNameOf, push } from "../../src/lifecycle/operate
import {
bootstrapBinaryPath, hostBinaryPath, placeBootstrap, BOOTSTRAP_PATH, HOST_PATH,
} from "../../src/lifecycle/place.ts";
import { labIsUsable, destroyAll, substrateBundle, onTheMachine } from "./harness.ts";
import { labIsUsable, destroyAll, foundationBundle, onTheMachine } from "./harness.ts";
import { referenceFor, type HeldImage } from "../../src/pinning.ts";
const capability = await labIsUsable();
@@ -84,7 +84,7 @@ const skip = !capability.usable
: !binary || !existsSync(binary)
? "MESH_LAB_HOST_BINARY is not set to a built mesh-host"
: !bundle || !existsSync(bundle)
? "MESH_LAB_BUNDLE is not set to a substrate bundle (mesh-host examples/)"
? "MESH_LAB_BUNDLE is not set to a foundation bundle (mesh-host examples/)"
: !installer || !existsSync(installer)
? "MESH_LAB_BOOTSTRAP_BINARY is not set to a built mesh-bootstrap (mesh-host `make " +
"bootstrap IMAGE=mesh-builder:development`). The anchor is raised BY the installer now, " +
@@ -96,7 +96,7 @@ const skip = !capability.usable
: false;
const SCENARIO = "whole-mesh-full";
/** novox hosts the substrate and the control plane; it is where `mesh` commands run. */
/** novox hosts the foundation and the control plane; it is where `mesh` commands run. */
const CONTROL = "novox";
/** Every node in the mesh. novox is on hosting; the rest are behind the home gateway. */
const NODES = ["novox", "ace", "shanks", "g14"];
@@ -118,7 +118,7 @@ const CATALOGUE_ON_MACHINE = "/opt/mesh-catalog";
* `internal/bootstrap` RegistryModule and ControlPlaneModule. If it ever opens a third, this list
* is where the bed finds out, by the installer saying which manifest it could not read.
*/
const CATALOGUE_MODULES = ["registry", "mesh-control", "builder"];
const CATALOGUE_MODULES = ["registry", "mesh-controller", "builder"];
/**
* Where this mesh keeps its own images, as the anchor reaches it.
@@ -221,7 +221,7 @@ const CORE_NOVOX = new Set([
const GAPS_NOVOX = new Set([
"umami", "mailu", "firewall", "fail2ban", "only-office", "de-spiegel", "amqp-email-forwarder",
// step-ca is reported, not gated: the internal-CA ISSUANCE path is still being fixed in
// mesh-control, and this bed is not the place to discover that a fix has not landed yet. What is
// mesh-controller, and this bed is not the place to discover that a fix has not landed yet. What is
// gated is the half that is decided and cheap — see the ADR 0066 section at the end.
"step-ca",
]);
@@ -274,13 +274,13 @@ const PLAN: { node: string; mods: Mod[]; core: Set<string>; gaps: Set<string> }[
/**
* Host-port remaps (per module; host ports are per-VM so novox's and ace's never clash across nodes).
* The two SUBSTRATE collisions are the new ones: postgres 5432 and lavinmq 5672 are moved off the
* substrate store/broker's host ports, which only exist on novox because that is where the substrate
* The two FOUNDATION collisions are the new ones: postgres 5432 and lavinmq 5672 are moved off the
* foundation store/broker's host ports, which only exist on novox because that is where the foundation
* runs. The rest break the novox web/app host-port collisions (route-proxy fronts 80/443).
*/
const REMAP: Record<string, Record<string, string>> = {
// Moved, NOT bound to loopback. These two carried `127.0.0.1:` and it broke a consumer on a node
// with no substrate at all: a module is told to reach its provider at `<node>.internal`, that
// with no foundation at all: a module is told to reach its provider at `<node>.internal`, that
// name resolves to the node's overlay address, and a provider listening only on loopback refuses
// it. letta on ace died of exactly this — "is the server running on that host and accepting
// TCP/IP connections?" — while postgres sat healthy beside it.
@@ -344,7 +344,7 @@ async function must(machine: string, command: string, timeoutMs?: number): Promi
/** The control plane, a container on novox (the anchor). */
async function mesh(command: string, timeoutMs?: number): Promise<string> {
return must(CONTROL, `docker exec mesh-control /mesh-control ${command}`, timeoutMs);
return must(CONTROL, `docker exec mesh-controller /mesh-controller ${command}`, timeoutMs);
}
/** What a manifest's image reference becomes on the machine — ours by ID, everything else as written. */
@@ -353,7 +353,7 @@ function pinned(reference: string): string {
}
function bundleFor(images: HeldImage[]): string {
return substrateBundle(bundle, images);
return foundationBundle(bundle, images);
}
function loadManifest(name: string): { manifest: string; broker: boolean } {
@@ -405,7 +405,7 @@ interface NodeState {
}
async function nodeState(node: string): Promise<NodeState> {
const asked = await on(CONTROL, `docker exec mesh-control /mesh-control status --json`);
const asked = await on(CONTROL, `docker exec mesh-controller /mesh-controller status --json`);
if (!asked.ok) return { reached: false, applied: false, current: false, waiting: false, raw: asked.out };
let state: {
wrong: { node: string; outcome: string; refused?: string; failed?: { id: string; error: string }[] }[];
@@ -467,9 +467,9 @@ async function deliverCaRoot(): Promise<boolean> {
// Safe here and nowhere else: these three exist for the seconds between being written and
// being sealed to the machine, on a lab node, for a CA thrown away with the scenario.
"chmod 0644 /tmp/ca/root.crt /tmp/ca/root.key /tmp/ca/key-password",
"docker cp /tmp/ca/root.crt mesh-control:/ca-root-cert",
"docker cp /tmp/ca/root.key mesh-control:/ca-root-key",
"docker cp /tmp/ca/key-password mesh-control:/ca-root-key-password",
"docker cp /tmp/ca/root.crt mesh-controller:/ca-root-cert",
"docker cp /tmp/ca/root.key mesh-controller:/ca-root-key",
"docker cp /tmp/ca/key-password mesh-controller:/ca-root-key-password",
].join("\n"), 180_000);
if (!made.ok) {
console.log(`CA ROOT NOT MADE on ${CONTROL}:\n${made.out.split("\n").slice(-8).join("\n")}`);
@@ -527,7 +527,7 @@ function stepIn(said: string): string {
* Put on the anchor what the installer needs to read, and run it.
*
* **This is the whole of what changed, and it is not a refactor.** The bed used to apply the
* substrate bundle itself, by hand, and then enrol four machines in one loop. It got the order
* foundation bundle itself, by hand, and then enrol four machines in one loop. It got the order
* right by accident and it modelled the wrong shape: an install procedure that exists only as a
* test fixture is exercised by whoever writes tests and never by whoever installs, which is why
* every bootstrap fault this year was found late (novox/hq ADR 0067). The anchor is now raised by
@@ -566,31 +566,31 @@ async function genesis(images: HeldImage[]): Promise<GenesisResult> {
}
report.push(` catalogue ${CATALOGUE_MODULES.join(", ")} at ${CATALOGUE_ON_MACHINE}`);
// The substrate TEMPLATE — not the bundle. The installer produces the bundle from it: it replaces
// The foundation TEMPLATE — not the bundle. The installer produces the bundle from it: it replaces
// the control plane's image with the id of the image it carries, renames that container
// `temp-mesh-control`, and writes the result where a person can read it.
// `temp-mesh-controller`, and writes the result where a person can read it.
//
// Two substitutions still happen here, and both belong to the bed rather than to the installer.
// The example names three images at a registry the lab no longer raises: the store and the broker
// become the upstream references mesh-catalog pins (harness), and the machine pulls them over its
// uplink like any first node. The third, mesh-control, is deliberately LEFT naming that dead
// uplink like any first node. The third, mesh-controller, is deliberately LEFT naming that dead
// registry — the installer overwrites it, and leaving it proves that it does.
//
// And the broker's advertised address. The template hardcodes 192.0.2.10:5671, the old
// separate-anchor address; a token carries MESH_BROKER_ADDRESS verbatim as the endpoint an
// enrolling node dials, so with the substrate on novox it must be novox's own public address or
// enrolling node dials, so with the foundation on novox it must be novox's own public address or
// every node would enrol against a dead one. The installer refuses to guess this and says so
// loudly, which is right — it does not know what this machine is called from outside.
const template = bundleFor(images).replaceAll("192.0.2.10:5671", "192.0.2.20:5671");
const local = join(tmpdir(), `mesh-lab-substrate-${process.pid}.lock`);
const local = join(tmpdir(), `mesh-lab-foundation-${process.pid}.lock`);
writeFileSync(local, template);
await push(instanceId, CONTROL, local, "/tmp/substrate-template.lock");
await push(instanceId, CONTROL, local, "/tmp/foundation-template.lock");
const command = [
BOOTSTRAP_PATH,
`--source ${source}`,
`--source-ref ${sourceRef}`,
`--bundle /tmp/substrate-template.lock`,
`--bundle /tmp/foundation-template.lock`,
`--catalog ${CATALOGUE_ON_MACHINE}`,
`--node ${CONTROL}`,
`--registry ${MESH_REGISTRY}`,
@@ -628,10 +628,10 @@ async function genesis(images: HeldImage[]): Promise<GenesisResult> {
// ------------------------------------------------------------------------------------------
// 1. The control plane answers. Asked of the PERMANENT container by name — `status` opens all
// three stores, so a reply proves the connections it was given are the substrate's own.
const answered = await on(CONTROL, `docker exec mesh-control /mesh-control status`, 60_000);
// three stores, so a reply proves the connections it was given are the foundation's own.
const answered = await on(CONTROL, `docker exec mesh-controller /mesh-controller status`, 60_000);
report.push(` control plane ${answered.ok ? answered.out.split("\n")[0] : "NO ANSWER"}`);
if (!answered.ok) return stop("after step 10", `mesh-control does not answer:\n${answered.out}`);
if (!answered.ok) return stop("after step 10", `mesh-controller does not answer:\n${answered.out}`);
// 2. The registry replies on /v2/ — the registry API's own "yes, I am one and I am ready". A
// container that is up is not a registry that serves.
@@ -643,22 +643,22 @@ async function genesis(images: HeldImage[]): Promise<GenesisResult> {
// 3. THE PIVOT COMPLETED. ADR 0067 states this check in as many words: after installing, the
// running control plane's image is pinned by a digest THE MESH'S OWN REGISTRY ASSIGNED — not
// by an image id. If it is still an image id, the substrate's container is what is running,
// by an image id. If it is still an image id, the foundation's container is what is running,
// nothing was published, and this mesh can never roll out its own upgrades.
const pinnedTo = (await on(CONTROL, `docker inspect --format '{{.Config.Image}}' mesh-control`))
const pinnedTo = (await on(CONTROL, `docker inspect --format '{{.Config.Image}}' mesh-controller`))
.out.trim();
report.push(` pinned to ${pinnedTo || "(nothing)"}`);
if (/^sha256:[0-9a-f]{64}$/.test(pinnedTo)) {
return stop("after step 10",
`mesh-control is running from ${pinnedTo}, which is an IMAGE ID — the digest of the image's ` +
`mesh-controller is running from ${pinnedTo}, which is an IMAGE ID — the digest of the image's ` +
`own configuration, which no registry ever served. The pivot did not happen: what is ` +
`running is the image the installer carried, not one this mesh published, so this mesh ` +
`cannot upgrade itself (novox/hq ADR 0067, "the pivot completed").`);
}
if (!new RegExp(`^${MESH_REGISTRY.replaceAll(".", "\\.")}/mesh-control@sha256:[0-9a-f]{64}$`)
if (!new RegExp(`^${MESH_REGISTRY.replaceAll(".", "\\.")}/mesh-controller@sha256:[0-9a-f]{64}$`)
.test(pinnedTo)) {
return stop("after step 10",
`mesh-control is running from ${pinnedTo || "nothing this bed could read"}, which is not a ` +
`mesh-controller is running from ${pinnedTo || "nothing this bed could read"}, which is not a ` +
`digest assigned by ${MESH_REGISTRY}.`);
}
@@ -669,36 +669,36 @@ async function genesis(images: HeldImage[]): Promise<GenesisResult> {
// outside to one that can — same container, same digest, same registry. The difference is
// whether a build happened, and the only place that is visible is the installer saying so.
const wanted = sourceRef.slice(0, 8);
if (!new RegExp(`built mesh-control from ${wanted}`).test(said)) {
if (!new RegExp(`built mesh-controller from ${wanted}`).test(said)) {
return stop("after the last step",
`the installer never said it built mesh-control from ${wanted}. What runs may have been ` +
`the installer never said it built mesh-controller from ${wanted}. What runs may have been ` +
`carried rather than made here, which is a mesh that cannot rebuild its own control plane.`);
}
report.push(` built here mesh-control from ${wanted}, by the carried builder`);
report.push(` built here mesh-controller from ${wanted}, by the carried builder`);
// 3b. And the registry really serves it, asked of the registry rather than of the container. A
// reference is a claim; a tag list is the registry agreeing.
const tags = await on(CONTROL,
`curl -s --max-time 10 http://${MESH_REGISTRY}/v2/mesh-control/tags/list`);
`curl -s --max-time 10 http://${MESH_REGISTRY}/v2/mesh-controller/tags/list`);
report.push(` registry holds ${tags.out.trim() || "nothing"}`);
if (!tags.out.includes("genesis")) {
return stop("after step 10",
`${MESH_REGISTRY} does not serve mesh-control, so the digest the container is pinned to ` +
`${MESH_REGISTRY} does not serve mesh-controller, so the digest the container is pinned to ` +
`names an image nothing can pull: ${tags.out.trim()}`);
}
// 4. The temporary control plane is GONE. Two control planes is the half-finished state, and the
// name is the audit: a machine running mesh-control and not temp-mesh-control has pivoted.
const temp = await on(CONTROL, `docker inspect --format '{{.State.Status}}' temp-mesh-control`);
report.push(` temp-mesh-control ${temp.ok ? `STILL HERE (${temp.out.trim()})` : "gone"}`);
// name is the audit: a machine running mesh-controller and not temp-mesh-controller has pivoted.
const temp = await on(CONTROL, `docker inspect --format '{{.State.Status}}' temp-mesh-controller`);
report.push(` temp-mesh-controller ${temp.ok ? `STILL HERE (${temp.out.trim()})` : "gone"}`);
if (temp.ok) {
return stop("after step 10",
`temp-mesh-control is still ${temp.out.trim()}. Two control planes are consuming this ` +
`temp-mesh-controller is still ${temp.out.trim()}. Two control planes are consuming this ` +
`mesh's broker queues; neither is wrong and the pivot is not finished.`);
}
// 5. And the mesh has heard from its one node. Everything the join phase does next depends on it.
const nodes = await on(CONTROL, `docker exec mesh-control /mesh-control node list`);
const nodes = await on(CONTROL, `docker exec mesh-controller /mesh-controller node list`);
report.push(` node list ${nodes.out.trim().split("\n").join(" | ")}`);
const line = nodes.out.split("\n").map((l) => l.trim()).find((l) => l.startsWith(`${CONTROL} `));
if (!line || !/^\S+\s+here\b/.test(line)) {
@@ -715,7 +715,7 @@ async function genesis(images: HeldImage[]): Promise<GenesisResult> {
// ==================================================================================================
/**
* ace, shanks and g14 join. Host binary plus a token — no bootstrap, no substrate, no registry.
* ace, shanks and g14 join. Host binary plus a token — no bootstrap, no foundation, no registry.
*
* **novox is not in this loop.** It was enrolled by the installer, as part of becoming a mesh, and
* enrolling it again would present the mesh with a second identity for a node it already knows —
@@ -866,7 +866,7 @@ test("the full mesh forms across the access point and both server sets converge"
const known = added.get(name);
if (known !== undefined) return known;
const { manifest, broker } = loadManifest(name);
await must(CONTROL, `printf %s ${quote(manifest)} > /tmp/${name}.json && docker cp /tmp/${name}.json mesh-control:/${name}.json`);
await must(CONTROL, `printf %s ${quote(manifest)} > /tmp/${name}.json && docker cp /tmp/${name}.json mesh-controller:/${name}.json`);
await mesh(`module add /${name}.json`);
added.set(name, broker);
return broker;
@@ -892,7 +892,7 @@ test("the full mesh forms across the access point and both server sets converge"
// Operator-provided app credentials (own-secrets), delivered as fake values through `secret accept`.
const credentialDelivered = new Map<string, boolean>();
for (const name of new Set(CREDENTIALS.map((c) => c.name))) {
await must(CONTROL, `printf %s ${quote(`fake-${name}-value`)} > /tmp/fake-${name} && docker cp /tmp/fake-${name} mesh-control:/fake-${name}`);
await must(CONTROL, `printf %s ${quote(`fake-${name}-value`)} > /tmp/fake-${name} && docker cp /tmp/fake-${name} mesh-controller:/fake-${name}`);
}
for (const c of CREDENTIALS) {
if (!assigned[c.node]!.has(c.module)) {
@@ -912,7 +912,7 @@ test("the full mesh forms across the access point and both server sets converge"
if (!assigned[s.node]!.has(s.module)) continue;
try {
const inControl = `/secret-${s.module}-${s.name}`;
await must(CONTROL, `printf %s ${quote(s.value)} > /tmp${inControl} && docker cp /tmp${inControl} mesh-control:${inControl}`);
await must(CONTROL, `printf %s ${quote(s.value)} > /tmp${inControl} && docker cp /tmp${inControl} mesh-controller:${inControl}`);
await mesh(`secret accept ${s.node} ${s.module} ${s.name} --from ${inControl}`);
} catch (err) {
console.log(`OPERATOR SECRET FAILED ${s.node}/${s.module}/${s.name}: ${(err as Error).message.split("\n").slice(0, 2).join(" | ")}`);
@@ -1021,7 +1021,7 @@ test("the full mesh forms across the access point and both server sets converge"
// and before this bed set a public domain it composed to nothing on every node, silently.
//
// What is NOT checked here is issuance: whether route-proxy actually obtains a certificate from
// step-ca over ACME. That path is being fixed in mesh-control as this is written, and a bed that
// step-ca over ACME. That path is being fixed in mesh-controller as this is written, and a bed that
// gated on it would be reporting somebody else's in-flight work as this bed's failure.
// ================================================================================================
const adr: string[] = ["================ ADR 0066: LABELLED ROUTES ================"];