novox/mesh-lab
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

The lab had a registry that production does not, so it tested a fiction

Browse Source 
The lab raised a `registry` VM, pushed ~73 images into it from the workstation, and
rewrote every manifest reference — third-party ones included — to point at it. No
production mesh has such a thing. So every bed proved that a machine could fetch an
image from a registry that exists nowhere else, and the bootstrap problems that only
appear when a machine has to fetch for itself went unfound.

What replaces it is the two things that are true in the world:

**Public images come from the public internet.** mesh-lab already created a NAT'd
uplink for exactly this and attached it to any machine declaring `egress`; no scenario
ever declared it. They do now, and third-party references are left exactly as the
catalogue writes them.

**The mesh's own images have no registry and never will.** mesh-control, mesh-builder,
mesh-route-proxy and the per-module runtimes are built from source and exist in no
registry. A machine gets them the way an operator's machine does — they are built here
and loaded onto it — and is then named by the digest of its own image configuration,
which mesh-host now accepts as "an image this machine already holds".

`images:` therefore means only *ours*, and a third-party entry is refused rather than
quietly loaded: otherwise the fiction returns one convenient line at a time. It is
per-machine as well, because "everything, everywhere" was never a description of
anything real — handing whole-mesh-full's union to its two 30GiB workstations would
fill the disk with runtimes nothing on them will start.

**The uplink and the declared gateway would have fought, silently.** A gateway container
and the transit router reach the scenario and nothing else; a default route through
either is a black hole for anything outside, and it beats the uplink's DHCP route on
metric. So a machine with egress states the scenario's ranges explicitly — through the
same gateway or transit it would have defaulted to, so the overlay-across-NAT path is
unchanged — and leaves the default to the uplink. A range with no path inside the
scenario becomes `unreachable` rather than falling through: 192.168.1.0/24 is an
ordinary private range in fact, and letting it escape would put scenario traffic on
whatever network the workstation is sitting on. `scenarioRoutesFor` is pure and tested,
because a decision only a full raise could check is one nobody checks.

The registry-reachability check the raise gained earlier is kept, pointed at the real
thing: every machine with egress must resolve a name and reach the internet before the
raise says it finished. Same failure it was written for — a raise that returns, an apply
that dies on its first pull, an instance left a bare shell — now guarding the path that
actually carries.

The base image's trust of the documentation ranges as plain-HTTP registries STAYS. It
was never only for the lab's registry: the mesh has one of its own, the `registry`
module, serving artifacts to the whole mesh over plain HTTP from whatever node runs it.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
jschoubben
2026-09-10 23:16:05 +02:00
parent 0a0c57b610
commit 751948f0f9
13 changed files with 558 additions and 602 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lifecycle/raise.ts
+24 -40
View File
@@ -22,9 +22,10 @@ import { applyAddresses, applyDefaultRoutes } from "./address.ts";
import { assertSupported } from "./supported.ts";
import { planRouters, raiseRouters, raiseTransit } from "./router.ts";
import { applyHostFirewalls } from "./firewall.ts";
import { IMAGE_PREFIX, BASE_IMAGE_ALIAS, BASE_IMAGE_HOWTO, planPlacements, applyPlacements } from "./place.ts";
import { IMAGE_PREFIX, BASE_IMAGE_ALIAS, BASE_IMAGE_HOWTO, planPlacements, applyPlacements, loadHeldImages } from "./place.ts";
import { baseImageExists, UPSTREAM_IMAGE } from "./base.ts";
import { confirmRegistryServes, discardStock, raiseRegistry, stockRegistry } from "./registry.ts";
import { confirmEgress } from "./egress.ts";
import type { HeldImage } from "../pinning.ts";
import { log as record } from "../log.ts";
/** Drivers whose snapshots are copy-on-write. On `dir` a snapshot is a full copy. */
@@ -47,12 +48,15 @@ export interface RaisedScenario {
networks: string[];
pool: string;
/**
* Images the scenario's registry serves, as references a declaration can pin.
* The mesh's own images, as loaded onto the machines, and what a declaration should call them.
*
* Reported rather than declared, because the digest is the one this registry assigned and
* is not knowable before it was raised.
* Reported rather than declared: an image built from source has no digest until it has been
* built, and what names it here is the digest of its own configuration.
*
* **Only ours.** Everything third-party is pulled from the internet by the machine that needs
* it, so it is not in this list and nothing rewrites it.
*/
images: string[];
images: HeldImage[];
}
export class RaiseError extends Error {
@@ -314,24 +318,6 @@ export async function raise(
const routers = await raiseRouters(scenario, instanceId, planRouters(scenario, instanceId), log);
if (transit) routers.push(transit);
// Stocked on this workstation, where there is a network, and served from inside the
// scenario, where there is not (novox/hq 04-ISSUES/009).
enter("stocking the registry");
const stock = await stockRegistry(scenario.images ?? [], log);
let registry: Awaited<ReturnType<typeof raiseRegistry>> = null;
try {
enter("raising the registry");
registry = await raiseRegistry(scenario, instanceId, stock, log, pool);
} finally {
// Cleaning up scratch must not fail a raise that succeeded. The scenario is standing
// and usable; a directory left behind is untidy, and saying so is the honest report.
try {
await discardStock(stock);
} catch (err) {
log(` (could not remove the registry's scratch directory: ${(err as Error).message})`);
}
}
enter("routing machines through their gateways");
await applyDefaultRoutes(scenario, byMachine, log);
@@ -340,31 +326,29 @@ export async function raise(
enter("applying host firewalls");
await applyHostFirewalls(scenario, byMachine, log);
// **Only now can "the registry is serving" be said truthfully.** Raising it proved the
// registry answers on its own machine; a machine pulls across a segment, and the home nodes
// pull through a gateway whose route and firewall were applied in the two steps above. So the
// path is checked here, where it is finally the one a pull will take — and before `placing`,
// which is minutes of work that a machine unable to fetch an image cannot use.
//
// The alternative is what happened: `raise` returned, the caller applied a substrate whose
// every image is pinned to this registry, the first pull failed, no node enrolled, and the
// instance was left a bare shell. A raise that reports success owes the next step the fact it
// depends on.
if (registry) {
enter("confirming the registry serves the machines");
await confirmRegistryServes(registry, [...byMachine.values()], stock, log);
}
// **Only now is "this machine can reach the outside" a true statement.** The route, the
// gateway and the machine's own filtering are all in place, so this is the path a pull takes.
// A raise that returned without checking would hand the next step a fact it depends on and
// has no way to test — which is how a substrate apply used to die on its first pull.
enter("confirming egress reaches the internet");
await confirmEgress(scenario, byMachine, log);
// Last, and only once the underlay is real. Placing before the machines can reach each
// other would test the host against a network the scenario does not describe.
enter("placing");
await applyPlacements(scenario, byMachine, log);
// After `placing`, because loading an image needs the container runtime that `placing`
// confirmed. The mesh's own images only — everything third-party is pulled by the machine
// itself, over its uplink, exactly as it is on a real one.
enter("loading the mesh's own images onto the machines");
const images = await loadHeldImages(scenario, byMachine, log);
return {
instanceId,
scenario: scenario.scenario,
images: registry?.pinned ?? [],
machines: [...created, ...routers, ...(registry ? [registry.machine] : [])],
images,
machines: [...created, ...routers],
networks,
pool,
};