A registry inside the scenario: the mechanism, verified
Issue 009's resolution, proven manually end to end before any of it was written. A sealed machine pulled an image BY DIGEST from a registry on its own segment and ran it; then the host applied all four shapes -- package, service with boot, container from that digest, and an action inside it -- idempotently. That is the first time the container shape has worked anywhere but a workstation, and it was the shape blocking the whole substrate bootstrap. The registry's digests are its own, not Docker Hub's, and that is correct rather than a compromise: ADR 0046 requires a reference that is exact and cannot move, and a digest this registry assigned is both. It is also not a lab workaround -- 0048 names an OCI registry as substrate and 0046 says a first node fetches "upstream, wherever the image ordinarily lives". This IS that upstream, scenery in the same sense the transit router is the internet. The base image now trusts the RFC 5737 and RFC 3849 documentation ranges as plain-HTTP registries. Scoped to those rather than an address because they never route on the real internet, so it cannot make a real machine trust a real registry whatever it is copied onto. Three faults found while verifying, two of them mine: My probe script picked an interface with `ls /sys/class/net | head -1`, which returns docker0 once a runtime exists -- so it addressed the wrong interface and then, because that address overlapped the segment, broke routing on the machine entirely. The lab itself is immune: it matches by MAC, for a related reason it already recorded (bus-position naming on multi-homed machines). And a test that proved nothing: I asserted `sha256:tooshort` is rejected, but its letters fall outside a-f, so it failed the character class rather than the length check. Replaced with hex of the wrong length, after which removing the length check bites.
This commit is contained in:
@@ -54,6 +54,19 @@ export async function buildBaseImage(
|
||||
|
||||
log(" installing a container runtime");
|
||||
await incus(["exec", BUILDER, "--", "pacman", "-Sy", "--noconfirm", "docker"], 600_000);
|
||||
|
||||
// Trust the documentation ranges as plain-HTTP registries.
|
||||
//
|
||||
// A scenario's registry is scenery inside the scenario, serving over HTTP, and a runtime
|
||||
// will not pull from one without being told. Scoped to RFC 5737 and RFC 3849 ranges rather
|
||||
// than a specific address, because those never route on the real internet — so this cannot
|
||||
// make a real machine trust a real registry, whatever it is copied onto.
|
||||
await incus([
|
||||
"exec", BUILDER, "--", "sh", "-c",
|
||||
`mkdir -p /etc/docker && printf '%s' '${JSON.stringify({
|
||||
"insecure-registries": ["192.0.2.0/24", "198.51.100.0/24", "203.0.113.0/24"],
|
||||
})}' > /etc/docker/daemon.json`,
|
||||
], 60_000);
|
||||
await incus(["exec", BUILDER, "--", "systemctl", "enable", "docker"], 60_000);
|
||||
await incus(["exec", BUILDER, "--", "systemctl", "start", "docker"], 120_000);
|
||||
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* A registry inside the scenario.
|
||||
*
|
||||
* A sealed machine cannot reach a registry, and an image placed from an archive cannot keep its
|
||||
* digest — `docker save` of a digest reference produces an archive with no repo tag, because a
|
||||
* repo digest only exists for an image a registry served (novox/hq 04-ISSUES/009). So an image
|
||||
* pinned by digest, which is the only kind the host accepts
|
||||
* ([ADR 0046](../../02-DECISIONS/0046-the-installer-fetches-what-it-pins.md)), could not be
|
||||
* placed at all.
|
||||
*
|
||||
* The answer is a registry, and it is not a workaround for the lab: ADR 0048 names an OCI
|
||||
* registry as substrate, and ADR 0046 says a first node fetches "upstream, wherever the image
|
||||
* ordinarily lives". **This is that upstream** — scenery, like the transit router is the
|
||||
* internet ([ADR 0033](../../02-DECISIONS/0033-a-router-is-scenery-not-a-node.md)).
|
||||
*
|
||||
* The digests it serves are its own, not Docker Hub's, and that is correct rather than a
|
||||
* compromise. What ADR 0046 requires is a reference that is exact and cannot move. A digest
|
||||
* assigned by this registry is both.
|
||||
*/
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
/** The image the registry itself runs from. Placed by tag, which archives keep. */
|
||||
export const REGISTRY_IMAGE = "registry:2";
|
||||
|
||||
/** Where the registry serves, inside its machine. */
|
||||
export const REGISTRY_PORT = 5000;
|
||||
|
||||
export class RegistryError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = "RegistryError";
|
||||
}
|
||||
}
|
||||
|
||||
export interface StockedImage {
|
||||
/** What the scenario asked for, as written. */
|
||||
requested: string;
|
||||
/** The repository path the registry serves it under. */
|
||||
repository: string;
|
||||
/** The digest THIS registry assigned. What a declaration pins. */
|
||||
digest: string;
|
||||
}
|
||||
|
||||
export interface Stock {
|
||||
/** A directory holding the registry's data, ready to be placed in a machine. */
|
||||
dataDir: string;
|
||||
images: StockedImage[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a registry's data directory on this workstation, with the given images in it.
|
||||
*
|
||||
* Runs a throwaway registry here — where there IS a network — pushes into it, and keeps what
|
||||
* it wrote. Research 012's reframing again: fetch at build time on a machine that has a
|
||||
* network, apply on a target that needs nothing.
|
||||
*
|
||||
* The caller owns the returned directory and must remove it.
|
||||
*/
|
||||
export async function stockRegistry(
|
||||
references: string[],
|
||||
log: (message: string) => void = () => {},
|
||||
): Promise<Stock> {
|
||||
if (references.length === 0) return { dataDir: "", images: [] };
|
||||
|
||||
const dataDir = await mkdtemp(join(tmpdir(), "mesh-lab-registry-"));
|
||||
const container = `mesh-lab-stock-${process.pid}`;
|
||||
const port = 5000 + (process.pid % 1000);
|
||||
|
||||
await docker(["rm", "-f", container], 60_000);
|
||||
const started = await docker(
|
||||
["run", "-d", "--name", container, "-p", `${port}:5000`, "-v", `${dataDir}:/var/lib/registry`,
|
||||
REGISTRY_IMAGE],
|
||||
300_000,
|
||||
);
|
||||
if (!started.ok) {
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
throw new RegistryError(
|
||||
`cannot run ${REGISTRY_IMAGE} on this workstation to stock a registry: ${started.stderr.trim()}`,
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
await waitForRegistry(port);
|
||||
const images: StockedImage[] = [];
|
||||
|
||||
for (const reference of references) {
|
||||
// The repository path a machine will pull from. A tag is dropped: what a declaration
|
||||
// pins is the digest, and carrying the tag as well would invite pinning the wrong one.
|
||||
const repository = repositoryFor(reference);
|
||||
const target = `localhost:${port}/${repository}`;
|
||||
|
||||
const tagged = await docker(["tag", reference, target], 60_000);
|
||||
if (!tagged.ok) {
|
||||
throw new RegistryError(
|
||||
`${reference} is not on this workstation, and the lab does not fetch on a scenario's ` +
|
||||
`behalf. Pull it here first.\n ${tagged.stderr.trim()}`,
|
||||
);
|
||||
}
|
||||
const pushed = await docker(["push", target], 900_000);
|
||||
if (!pushed.ok) throw new RegistryError(`cannot push ${reference}: ${pushed.stderr.trim()}`);
|
||||
|
||||
const digest = digestFrom(pushed.stdout + pushed.stderr);
|
||||
if (!digest) {
|
||||
throw new RegistryError(
|
||||
`${reference} was pushed and the registry did not report a digest. Without one there ` +
|
||||
`is nothing for a declaration to pin.`,
|
||||
);
|
||||
}
|
||||
images.push({ requested: reference, repository, digest });
|
||||
log(` stocked ${repository}@${digest}`);
|
||||
}
|
||||
|
||||
return { dataDir, images };
|
||||
} catch (err) {
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
throw err;
|
||||
} finally {
|
||||
await docker(["rm", "-f", container], 60_000);
|
||||
}
|
||||
}
|
||||
|
||||
/** `alpine:3.20` and `alpine` both serve from `alpine`; `foo/bar:1` from `foo/bar`. */
|
||||
export function repositoryFor(reference: string): string {
|
||||
const withoutDigest = reference.split("@")[0] ?? reference;
|
||||
const lastColon = withoutDigest.lastIndexOf(":");
|
||||
const lastSlash = withoutDigest.lastIndexOf("/");
|
||||
return lastColon > lastSlash ? withoutDigest.slice(0, lastColon) : withoutDigest;
|
||||
}
|
||||
|
||||
/** `docker push` prints `<tag>: digest: sha256:… size: …` on its last useful line. */
|
||||
export function digestFrom(output: string): string | null {
|
||||
const match = output.match(/digest:\s*(sha256:[a-f0-9]{64})/);
|
||||
return match?.[1] ?? null;
|
||||
}
|
||||
|
||||
async function waitForRegistry(port: number): Promise<void> {
|
||||
for (let i = 0; i < 30; i++) {
|
||||
const probe = await docker(["run", "--rm", "--network", "host", REGISTRY_IMAGE,
|
||||
"sh", "-c", `wget -q -O- http://localhost:${port}/v2/ >/dev/null 2>&1`], 30_000);
|
||||
if (probe.ok) return;
|
||||
await new Promise((r) => setTimeout(r, 1_000));
|
||||
}
|
||||
throw new RegistryError("a registry was started on this workstation and never answered");
|
||||
}
|
||||
|
||||
function docker(
|
||||
args: string[],
|
||||
timeoutMs: number,
|
||||
): Promise<{ ok: boolean; stdout: string; stderr: string }> {
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn("docker", args, { stdio: ["ignore", "pipe", "pipe"] });
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const timer = setTimeout(() => child.kill("SIGKILL"), timeoutMs);
|
||||
child.stdout.on("data", (d) => (stdout += d));
|
||||
child.stderr.on("data", (d) => (stderr += d));
|
||||
child.on("error", (err) => {
|
||||
clearTimeout(timer);
|
||||
resolve({ ok: false, stdout, stderr: err.message });
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
clearTimeout(timer);
|
||||
resolve({ ok: code === 0, stdout, stderr });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// --- the registry inside a scenario ------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Where the registry sits on its segment.
|
||||
*
|
||||
* A convention rather than a declaration, like the router's. `.250` is chosen to sit well away
|
||||
* from the low addresses scenarios give their machines, so a scenario can be written without
|
||||
* thinking about it and a collision is obvious when it happens.
|
||||
*/
|
||||
export const REGISTRY_HOST_OCTET = 250;
|
||||
|
||||
/** The address the registry answers on, given the segment it is attached to. */
|
||||
export function registryAddress(cidr: string): string {
|
||||
const [network] = cidr.split("/");
|
||||
const parts = (network ?? "").split(".");
|
||||
if (parts.length !== 4) {
|
||||
throw new RegistryError(
|
||||
`cannot place a registry on '${cidr}': it is not an IPv4 network, and the registry needs ` +
|
||||
`an address a machine can be pointed at.`,
|
||||
);
|
||||
}
|
||||
return `${parts[0]}.${parts[1]}.${parts[2]}.${REGISTRY_HOST_OCTET}`;
|
||||
}
|
||||
|
||||
/** What a declaration should pin, once a scenario is raised. */
|
||||
export function pinnedReference(address: string, image: StockedImage): string {
|
||||
return `${address}:${REGISTRY_PORT}/${image.repository}@${image.digest}`;
|
||||
}
|
||||
Reference in New Issue
Block a user