Scenario lifecycle: raise, exec, snapshot, restore, destroy

A declaration goes in and a disposable mesh comes out. Verified on a
workstation, not asserted: two machines raised and addressed in 14.6s,
snapshot 0.28s, restore-to-usable 11.6s, both families pinging with no
loss, and the workstation with no route into any of it.

The declaration layer implements the model in full — three positions a
machine can be in, keyed on forwardability; gateways carrying the address
the world sees them as; both address families; multi-homing; MTU;
inter-segment policy. It is validated hard because the failures it prevents
are silent: a private range on a public segment produces no error, the mesh
simply never forms. Public segments are refused unless they use RFC 5737 or
RFC 3849 space, and a range wider than the reserved block is refused too.
33 tests, all offline.

The runtime implements less than the model, and refuses the difference.
A scenario declaring gateways, published ports, policy, inbound deny or
place is rejected at raise with every gap named. Raising it would produce a
mesh that silently lacks what it declared, which is the fault this lab
exists to catch — 04-ISSUES/003, where a firewall key is declared in five
manifests and read by no code.

Three bugs found by review and by running it, all of one family:

The readiness check truthiness-tested incusOk's return. `exec … true`
succeeds with EMPTY output, so every machine reported unreachable while
incus exec on it worked perfectly. succeeds() now exists so the mistake is
not available, and network delete had the same bug — it counted zero
segments removed while removing them.

list() split instance from machine on the last dash, so a machine called
home-server absorbed half the instance id and destroy found nothing.
Resources are now found by the metadata they carry, never by name.

restore reported success in 0.79s while the machine's agent was still
starting, so the next command failed. Both raise and restore now wait for
usable and say how long that took — reporting the earlier number is
transport reported as effect, which is the fault the lab is being built to
find.

Two incus behaviours worth recording. Its CLI reads a YAML definition from
stdin when stdin is not a terminal, so a spawned command hangs until the
timeout kills it and arrives with empty stderr — a failure with no
explanation, on a command that works when typed. And it assigns a MAC at
runtime without recording it in device config, so MACs are derived and set
explicitly, which the guest needs anyway: it names interfaces by bus
position, and matching by name configures the wrong one on a multi-homed
machine.

No build step; Node strips the types. The lifecycle has no unit tests
because a fake hypervisor would assert that the fake behaves as expected,
which is the shape of test this project exists to stop shipping.

src/lifecycle/address.ts

@@ -0,0 +1,100 @@
+/**
+ * Put the declared addresses on the machines.
+ *
+ * The lab provides the underlay, and an address is underlay — it is what a hosting
+ * provider or a home router would have given the machine before any of our software ran.
+ * So the lab assigns it, and the mesh is left to build everything above it.
+ *
+ * Addresses are applied by matching on MAC rather than interface name. A guest names its
+ * interfaces by bus position — `enp5s0`, not `eth0` — so the name the hypervisor uses and
+ * the name the guest uses are different, and matching by name silently configures the
+ * wrong interface on a multi-homed machine.
+ *
+ * See novox/hq 02-DECISIONS/0031-the-lab-provides-the-underlay.md
+ */
+
+import type { Scenario } from "../declaration/types.ts";
+import { incus } from "../incus/client.ts";
+import { macFor } from "./names.ts";
+
+export interface Wire {
+  device: string;
+  mac: string;
+  addresses: string[];
+  mtu: number | undefined;
+}
+
+/**
+ * A systemd-networkd unit per interface. Static, because the declaration is the authority:
+ * a scenario that let the hypervisor hand out addresses would be the lab supplying facts
+ * the declaration is supposed to own.
+ */
+function networkUnit(wire: Wire): string {
+  const lines = [
+    "[Match]",
+    `MACAddress=${wire.mac}`,
+    "",
+    "[Network]",
+    ...wire.addresses.map((address) => `Address=${address}`),
+    // No gateway and no DNS on purpose. Routing off the segment is a gateway's job, and a
+    // scenario that pre-wired it would be arranging what it is supposed to observe.
+    "IPv6AcceptRA=no",
+  ];
+  if (wire.mtu !== undefined) {
+    lines.push("", "[Link]", `MTUBytes=${wire.mtu}`);
+  }
+  return lines.join("\n") + "\n";
+}
+
+export async function applyAddresses(
+  scenario: Scenario,
+  instanceId: string,
+  machineNames: Map<string, string>,
+  log: (message: string) => void = () => {},
+): Promise<void> {
+  for (const [machine, spec] of Object.entries(scenario.machines)) {
+    if (spec.at === "detached") continue;
+    const name = machineNames.get(machine);
+    if (!name) continue;
+
+    const wires: Wire[] = [];
+    for (const [index, attachment] of spec.at.entries()) {
+      wires.push({
+        device: `eth${index}`,
+        mac: macFor(instanceId, machine, index),
+        addresses: attachment.address.map((address) => withPrefix(scenario, attachment.segment, address)),
+        mtu: scenario.segments[attachment.segment]?.mtu,
+      });
+    }
+
+    for (const [index, wire] of wires.entries()) {
+      const unit = networkUnit(wire);
+      await incus(
+        ["exec", name, "--", "sh", "-c",
+         `mkdir -p /etc/systemd/network && cat > /etc/systemd/network/10-mlab-${index}.network <<'MLAB'\n${unit}MLAB`],
+        30_000,
+      );
+    }
+
+    await incus(["exec", name, "--", "systemctl", "enable", "--now", "systemd-networkd"], 60_000);
+    await incus(["exec", name, "--", "systemctl", "restart", "systemd-networkd"], 60_000);
+    log(`  addressed ${machine} (${wires.map((w) => w.addresses.join(",")).join(" | ")})`);
+  }
+}
+
+/**
+ * systemd-networkd wants a prefix length on the address. The declaration gives a bare
+ * address and the segment gives the range, so the two are combined here rather than making
+ * every scenario repeat the prefix on every machine.
+ */
+function withPrefix(scenario: Scenario, segment: string, address: string): string {
+  const ranges = scenario.segments[segment]?.cidr ?? [];
+  const wantV6 = address.includes(":");
+  for (const range of ranges) {
+    const slash = range.lastIndexOf("/");
+    if (slash === -1) continue;
+    const isV6 = range.slice(0, slash).includes(":");
+    if (isV6 === wantV6) return `${address}${range.slice(slash)}`;
+  }
+  return address;
+}