The danger is not that the suite breaks. It is that nobody notices it
stopped running (novox/hq 04-ISSUES/005). The harness this replaces had
not built for two and a half months and nothing said so — and this suite
needs a hypervisor, so it inherits exactly that: it runs when somebody
remembers, and remembering is not a mechanism.
So running, recording, and rebuilding are one act:
- the host binary, control-plane image and builder are rebuilt from
source first. The last two both parse manifests; building one and not
the other left a binary eleven hours old refusing a field the mesh had
just renamed, found by a full run.
- a receipt lands in XDG state — outside git, because the question is
whether *this machine* has run it, and a receipt in git would be a
claim about everybody's machine made by whoever committed last.
- `last-run` judges it and exits non-zero when it no longer counts.
Three faults found by running the thing rather than reading it, each now
held by a test confirmed to fail without it:
- counted() passed every test while parsing nothing. The runner colours
its summary even into a pipe; the fixtures were clean text that had
been imagined rather than captured. A fixture that agrees with the
mistake proves the mistake.
- a receipt for `suite test/lastrun.test.ts` was indistinguishable from
one for the real thing — 005's own symptom, rebuilt inside its remedy.
The receipt now records what ran.
- a tree with uncommitted work reported the bare commit, claiming
coverage of code nobody can check out. Nothing else could tell: the
hash is identical either way.
Proven on real machines: 22/22, against all three repositories.
The suite next door raises one scenario and asks deep questions of it. This one
asks shallow questions of every scenario — the half that was missing, since both
faults found by hand lived in scenarios nothing ever built.
Adds bootstrap-single, the cheapest, and the loop that lets the list grow. Also
adds the second universal invariant: every address a scenario declared is one
the machine actually holds. A machine that came up bare looks identical to one
that came up correctly until something asks it.
Verified to bite rather than assumed: against a live instance, the real
declaration passes and a declaration claiming an address nothing holds fails
with 'anchor declared 192.0.2.99 on hosting but holds 192.0.2.10'.
Integration now runs with --test-concurrency=1. Two files raise real instances,
node --test runs files in parallel by default, and two concurrent runs of this
suite already produced a whole-suite failure once — every test red, from
resource contention rather than from any fault in the code.
Gate: 45.7s -> 60.2s.
`mesh-lab diagram` renders a scenario as draw.io, from either source, through
one layout — so a difference between what was asked for and what exists is a
difference you can see.
The shape says what a resource is and is fixed per kind. The badges say what is
true about that particular one and come entirely from metadata: translation,
forwardability, mapping expiry, refuses-inbound, container-or-VM, running. The
interesting properties of a network are exactly the ones with no visual
consequence — a translated address looks identical to an untranslated one.
For the live picture to be a record rather than a restatement, raise now writes
down what it applied: a segment's kind, ranges and MTU on the link; a gateway's
translation, forwardability and expiry on the gateway; inbound: deny on the
machine. Every behavioural tag is written AFTER the thing works, never at
creation — a failed raise leaves wreckage standing on purpose, and a picture of
that wreckage must not badge translation the router never got.
The pairing earned itself immediately: drawn side by side, every virtual machine
held no addresses. A container's interface carries the device's name and a VM
names its own, so joining them by name silently dropped one whole class of
machine. Fixed by joining on MAC.
Also brings tests under the typecheck gate, which caught integration timeouts
being passed as a 4th argument and therefore ignored entirely.
Reviewed and the criticism was right: 1,072 of 2,128 lines untested, all of
it the half that touches the hypervisor, and no gate. The verification I had
done was real — pings across NAT, TTL counts, ruleset comparisons — and none
of it survived the terminal it ran in, which is 04-ISSUES/005 in miniature.
Ten integration tests against a real hypervisor, each named for what it
defends. ADR 0031: a raised machine carries no overlay, no wireguard, no
mesh config — a scenario that pre-built peering would certify its own work.
ADR 0032: exec is the only way in. ADR 0033: routers are containers while
machines are virtual machines. And the design's claims: raise waits for
usable, snapshots are whole-scenario, NAT hides a private address,
published reaches the machine at the gateway's address.
Mocking the hypervisor is forbidden, so they skip with a reason on a
machine that cannot raise scenarios rather than passing green having
checked nothing.
The suite earned itself on its first run. It found that a snapshot of a
running machine could miss a file written seconds earlier — not stale,
absent — because the write was still in the guest's page cache. That is
exactly the question the lifecycle design listed as open: does a scenario
snapshot need the machines stopped? It does not, but it does need them
flushed. snapshot now syncs every machine before capturing, and the design
records the answer.
The fix buys write-durability, not application-consistency: anything
mid-transaction is still captured mid-transaction, and that is now stated
rather than assumed.
npm run check is the gate — typecheck, 40 unit tests, 10 integration tests.
A declaration goes in and a disposable mesh comes out. Verified on a
workstation, not asserted: two machines raised and addressed in 14.6s,
snapshot 0.28s, restore-to-usable 11.6s, both families pinging with no
loss, and the workstation with no route into any of it.
The declaration layer implements the model in full — three positions a
machine can be in, keyed on forwardability; gateways carrying the address
the world sees them as; both address families; multi-homing; MTU;
inter-segment policy. It is validated hard because the failures it prevents
are silent: a private range on a public segment produces no error, the mesh
simply never forms. Public segments are refused unless they use RFC 5737 or
RFC 3849 space, and a range wider than the reserved block is refused too.
33 tests, all offline.
The runtime implements less than the model, and refuses the difference.
A scenario declaring gateways, published ports, policy, inbound deny or
place is rejected at raise with every gap named. Raising it would produce a
mesh that silently lacks what it declared, which is the fault this lab
exists to catch — 04-ISSUES/003, where a firewall key is declared in five
manifests and read by no code.
Three bugs found by review and by running it, all of one family:
The readiness check truthiness-tested incusOk's return. `exec … true`
succeeds with EMPTY output, so every machine reported unreachable while
incus exec on it worked perfectly. succeeds() now exists so the mistake is
not available, and network delete had the same bug — it counted zero
segments removed while removing them.
list() split instance from machine on the last dash, so a machine called
home-server absorbed half the instance id and destroy found nothing.
Resources are now found by the metadata they carry, never by name.
restore reported success in 0.79s while the machine's agent was still
starting, so the next command failed. Both raise and restore now wait for
usable and say how long that took — reporting the earlier number is
transport reported as effect, which is the fault the lab is being built to
find.
Two incus behaviours worth recording. Its CLI reads a YAML definition from
stdin when stdin is not a terminal, so a spawned command hangs until the
timeout kills it and arrives with empty stderr — a failure with no
explanation, on a command that works when typed. And it assigns a MAC at
runtime without recording it in device config, so MACs are derived and set
explicitly, which the guest needs anyway: it names interfaces by bus
position, and matching by name configures the wrong one on a multi-homed
machine.
No build step; Node strips the types. The lifecycle has no unit tests
because a fake hypervisor would assert that the fake behaves as expected,
which is the shape of test this project exists to stop shipping.
jschoubben