A machine is as big as the scenario says, and may reach the world

Three changes, found by one failing test.

The forge failed three runs in a row as "status hangs", and it was
diagnosed twice as contention — real defects, fixed, and not the cause.
The heartbeats told the truth in the end: every exec on anchor crawled
from 15s to 105s, because eleven containers plus a database pull were
running in a 1GiB machine. Starvation presents as whatever you were
doing when the page-outs start, which is why it wore two other bugs'
clothes first.

So machine size is now the scenario's to declare — memory and cpus per
machine, default unchanged. The anchor that carries the whole substrate
is bigger than the laptop that joins it, and the comment on the
scenario says why in terms of what lands there.

`egress: true` gives a machine one extra interface on a lab-supplied
NAT network, addressed by DHCP because the one address a scenario has
no business choosing is on the host's side of the fence. Declared
per machine and off by default: a closed scenario stays the rule
(novox/hq ADR 0016), and the exception exists because a first node
fetches its images before any mesh can serve them — which is now the
tested path (04-ISSUES/029), and a lab that can never reach upstream
cannot prove the bootstrap it exists to prove. The uplink route is
metric-4096, so it never shadows a route the scenario declared. A
detached machine declaring egress is refused, not ignored.

And settled() treats a poll that threw as a poll that missed. An exec
timeout at minute four of a wait is "could not ask", not a verdict on
the machine.
This commit is contained in:
2026-09-01 21:42:18 +02:00
parent f85dbb0713
commit 4a343a2652
9 changed files with 152 additions and 6 deletions
+46 -3
View File
@@ -131,12 +131,42 @@ async function createNetwork(
return name;
}
/**
* The one network the lab supplies rather than the declaration.
*
* Every segment a scenario describes is an isolated bridge with no addresses, no DHCP and no NAT,
* because the declaration owns addressing. This is the opposite of that on purpose: it is not part
* of the scenario, it carries no scenario traffic, and what is routable on it is the host's fact.
*
* It exists so a machine can fetch what it starts from. A first node pulls three images before
* there is any mesh, and the module that gives a mesh its own store pulls one more
* (novox/hq 04-ISSUES/029) — none of which anything inside a scenario can serve.
*
* Tagged like everything else, so tearing the scenario down takes it too.
*/
async function createUplink(instanceId: string): Promise<string> {
const name = networkName(instanceId, "uplink");
if (await succeeds(["network", "show", name], 15_000)) return name;
await incus([
"network", "create", name,
"ipv4.address=auto",
"ipv4.nat=true",
"ipv6.address=none",
`user.mesh-lab.instance=${instanceId}`,
"user.mesh-lab.segment=uplink",
]);
return name;
}
async function createMachine(
instanceId: string,
machine: string,
attachments: { segment: string }[],
image: string,
pool: string,
egress = false,
memory = "1GiB",
cpus = 2,
): Promise<string> {
const name = machineName(instanceId, machine);
if (await succeeds(["config", "show", name], 15_000)) return name;
@@ -148,8 +178,8 @@ async function createMachine(
// Arch images refuse to boot under secureboot with the shipped keys. Discovered by
// the first launch failing with exactly that message.
"-c", "security.secureboot=false",
"-c", "limits.memory=1GiB",
"-c", "limits.cpu=2",
"-c", `limits.memory=${memory}`,
"-c", `limits.cpu=${cpus}`,
"-c", `user.mesh-lab.instance=${instanceId}`,
"-c", `user.mesh-lab.machine=${machine}`,
];
@@ -168,6 +198,17 @@ async function createMachine(
`hwaddr=${macFor(instanceId, machine, index)}`,
]);
}
// After every declared attachment, so eth0..ethN keep meaning what the scenario said and the
// uplink is whatever comes next. A machine that never asked for one has no such interface at
// all, which is the difference between a closed scenario and an open one.
if (egress) {
await incus([
"config", "device", "add", name, `eth${attachments.length}`, "nic",
"nictype=bridged",
`parent=${await createUplink(instanceId)}`,
`hwaddr=${macFor(instanceId, machine, attachments.length)}`,
]);
}
return name;
}
@@ -242,7 +283,9 @@ export async function raise(
const byMachine = new Map<string, string>();
for (const [machine, spec] of Object.entries(scenario.machines)) {
const attachments = spec.at === "detached" ? [] : spec.at;
const name = await createMachine(instanceId, machine, attachments, image, pool);
const name = await createMachine(
instanceId, machine, attachments, image, pool,
spec.at !== "detached" && spec.egress === true, spec.memory, spec.cpus);
created.push(name);
byMachine.set(machine, name);
log(` machine ${machine}${spec.at === "detached" ? " (detached)" : ""}`);