novox/mesh-lab
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

The lab raises a mesh, draws it, and now places tier 0 inside it #1

Merged
jschoubben merged 11 commits from feat/scenario-lifecycle into main 2026-08-25 23:02:29 +00:00
Conversation 0 Commits 11 Files Changed 43
+5292 -22
43 changed files with 5292 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -0,0 +1 @@
node_modules/
README.md
+228 -22
View File
@@ -26,38 +26,244 @@ The bootstrap scenario is a **strict subset** — same virtualisation, same netw
lifecycle, stopping before a control plane exists. The full scenario is reached by putting more lifecycle, stopping before a control plane exists. The full scenario is reached by putting more
inside the machines, not by building a second thing. inside the machines, not by building a second thing.
**Bootstrap is what gets built here first.** Nothing in this repository requires a forge, a ## Using it
coordinator or a pipeline to be useful.
## Shape
``` ```
scenarios/ declarations of a mesh to raise mesh-lab check can this machine run scenarios at all
lifecycle/ create · snapshot · reset · destroy mesh-lab validate scenarios/x.yml parse and check, raising nothing
network/ segments and addressing mesh-lab raise scenarios/x.yml materialise it, wait until the machines are USABLE
place/ getting a binary onto a machine mesh-lab list instances currently standing
mesh-lab exec <instance> <machine> -- <cmd...>
mesh-lab snapshot <instance> <label>
mesh-lab restore <instance> <label>
mesh-lab destroy <instance>
mesh-lab diagram scenarios/x.yml draw what the scenario asks for
mesh-lab diagram --live <instance> draw what is actually standing
``` ```
Of the two jobs a runner might hold, **scenario lifecycle comes first** — something must `check` refuses rather than warns. A machine without copy-on-write storage runs scenarios
materialise and reset a mesh before anything can be written against it. **Assertion execution correctly and snapshots roughly 76× slower — which does not make the lab slow, it makes it
comes later**, with the full scenario. unused, and a warning about that is read once and ignored forever.
## Rules If the incus socket is not reachable as your user — the group was granted to a session that
already existed — set `MESH_LAB_INCUS="sudo -n incus"`.
- **Nothing new drives delivery.** A full scenario runs the real pipeline against a mesh named ## What a scenario declares
by the request. A second delivery path would be blind to exactly the faults worth catching.
- **A scenario starts from nothing, every time.** Adopting a machine that already exists is out The **underlay**: what a hosting provider and a home router would provide, and nothing the
of scope — that is what makes a scenario a fixture rather than a snapshot. mesh is responsible for.
- **The real topology is one scenario among many**, not the baseline. Anything only testable
against the shape the mesh happens to have is a gap in the vocabulary. ```yaml
segments:
hosting: # one public network
kind: public
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
isp-home: # another, unrelated — routed to it, never bridged
kind: public
cidr: [198.51.100.0/24, "2001:db8:b::/48"]
home:
kind: private
cidr: [192.168.1.0/24, "2001:db8:b:1::/64"]
mtu: 1492
gateway:
to: isp-home
address: [198.51.100.7] # what the world sees this network as
nat: [v4] # v4 translated, v6 routed
forwardable: true
mapping_ttl: 120s
machines:
home-server:
at: { segment: home, address: [192.168.1.135, "2001:db8:b:1::135"] }
published: [{ port: 443, on: home }]
inbound: allow
```
It declares **nothing** about overlay addresses, hubs, peering, names or certificates. Those
are what the mesh does, and a scenario that supplied them would be certifying its own work.
Public segments must use documentation ranges (RFC 5737, RFC 3849) and the validator refuses
anything else **before raising**. That is not pedantry: the mesh decides public-versus-private
by matching the address, so a private range on a segment meant to be routable makes the mesh
silently never form — no error, nothing to notice.
## Reaching in
Everything goes through incus, never over IP. A scenario is a closed address space, so two
instances raised from one declaration hold the same addresses and never meet — and the
workstation has no route into either.
So a reachability question is asked **from inside**: *can this machine reach that one* is
`exec` on the first, testing the second. The workstation's opinion would be a different
question with a misleadingly similar answer.
## What is implemented, and what is not
The declaration model is complete — it is the design's shape, and validating against it is
useful before any of it can be raised. **The runtime is not**, and the gap is refused rather
than ignored:
| | |
|---|---|
| segments as isolated links | **works** |
| machines, multi-homed or detached | **works** |
| declared addresses, both families | **works** |
| segment MTU | **works** |
| raise · exec · snapshot · restore · destroy · list | **works** |
| gateways, NAT, masquerade | **works** |
| `published:` ports (DNAT through the gateway's address) | **works** |
| `mapping_ttl:` (conntrack timeout) | **works**, and verified after setting — a declared expiry that silently did not apply would be the fault this catches |
| `forwardable: false` | **works** — outbound only, no DNAT, unsolicited inbound dropped |
| `policy:` between segments | **works**, asymmetric |
| `inbound: deny` | **works** — host firewall, read back after applying |
| several public networks, routed not bridged | **works** — a transit router, never a shared bridge |
| `place: [host]` | **works** — tier 0 is placed and asked what the machine is |
| `place:` anything above tier 0 | **refused, by name** — those tiers do not exist yet |
`raise` refuses a scenario declaring anything in the lower half, naming every gap. It does not
raise a mesh that silently lacks what it declared — that is the fault this lab exists to catch
(`novox/hq` 04-ISSUES/003: a firewall key declared in five manifests and read by no code, so a
manifest appears to restrict a port and restricts nothing).
`bootstrap-single.yml` places the host. The rest raise an underlay and put nothing on it,
which is still correct for what they test.
Placing needs a built host binary — set `MESH_LAB_HOST_BINARY` to one. It is an explicit path
rather than a search on purpose: the declaration design leaves *where `place:` gets its
artifacts from* open, and guessing would harden into the answer by accident.
## Measured on a workstation
| | one machine | two machines | two machines + a router |
|---|---|---|---|
| raise, to usable | 12.5 s | 14.6 s | 32 s |
| snapshot | 0.14 s | 0.28 s | — |
| restore, to usable again | 10.5 s | 11.6 s | — |
A router adds seconds, not a boot: it is a container, because it is scenery rather than
something under test (`novox/hq` ADR 0033).
**Verified by running**, not asserted — a machine at `192.168.1.135` behind a household
gateway, reached from a machine on a routable address:
```
home-server -> anchor 0% loss, through masquerade
anchor -> 192.168.1.135 (private, direct) unreachable ✓
anchor -> 192.0.2.50:8080 (the GATEWAY) HTTP 200
home -> devices (policy allow) reachable ✓
devices -> home (policy deny) blocked ✓
roamer behind unforwardable NAT -> anchor reachable ✓ (outbound only)
anchor -> roamer unreachable ✓
workstation with inbound: deny, dialling out reachable ✓ (defended, not disconnected)
home-server -> workstation refused ✓
```
The third line is the case research 004 says only exists in production.
**Routed, never bridged**, proven rather than asserted — ping TTL across the full topology:
```
within one segment ttl=64 no hops
across two unrelated public networks ttl=62 gateway + transit
multicast between public networks 0 replies
```
A flat "internet" would have shown ttl=64 and answered multicast, which would have let a node
discover a peer it could never reach in production — and report success.
Machines boot concurrently, so a second machine costs seconds rather than doubling the wait.
Nearly all of the remaining time is boot, which cannot be avoided.
These numbers depend entirely on a copy-on-write pool. On `dir` the same snapshot takes 9.9 s
and a full copy of the disk, and a second one did not finish in two minutes — which is why
`check` refuses rather than warns.
## Drawing one
```
mesh-lab diagram scenarios/the-ordinary-shape.yml what the declaration asks for
mesh-lab diagram --live <instance> what the hypervisor actually holds
```
Both produce draw.io files, laid out the same way — public networks at the top, each private
one below the network it sits behind. Drawing both sources through one layout is the point: a
difference between what was asked for and what exists becomes a difference you can *see*.
Two kinds of symbol, and the split matters:
- the **shape** says what a resource is, and is fixed per kind — a server is always the server
shape, a gateway always the router shape, whatever else is true about it;
- the **badges** say what is true about that particular one, and come entirely from metadata:
`N` translated, `F` forwarding (green yes, red no), `T` mappings expire, `D` refuses inbound,
`C` container, `VM` virtual machine, `▶` running. Each carries the full sentence as a
tooltip, because a one-letter code with no explanation is a private language.
Badges exist because the interesting properties of a network are exactly the ones with no
visual consequence. An address that is translated looks identical to one that is not, until
traffic proves otherwise.
The live drawing reads **only** the hypervisor — the same tags `destroy` uses — and never
re-opens the scenario file. A picture built from the declaration and labelled *as raised*
would report the request as though it were the result, which is the whole failure the pairing
exists to expose. So `raise` records what it applied: a segment's kind and ranges on the link,
a gateway's translation, forwardability and mapping expiry on the gateway, and `inbound: deny`
on the machine.
**Every behavioural tag is written after the thing works, never before.** A tag written when
the resource is created would restate the request; a failed raise leaves its wreckage standing
on purpose, so a picture of that wreckage would badge translation the gateway was never
configured to do. The gateway is tagged after its ruleset is applied, and the machine after
the read-back proves its firewall loaded.
That pairing has already earned itself. Drawn side by side, the live picture showed every
virtual machine holding no addresses at all: a container's interface carries the device's
name, a virtual machine names its own, and joining them by name silently dropped one whole
class of machine. The two pictures disagreed, so the bug was visible in seconds.
## Where the reasoning lives ## Where the reasoning lives
Design and decisions are in [`novox/hq`](https://git.novox.be/novox/hq), not here: Design and decisions are in [`novox/hq`](https://git.novox.be/novox/hq), not here:
- `03-DESIGN/01-to-be/01-end-to-end-testing.md` — the design - `03-DESIGN/01-to-be/02-scenario-declaration.md` — what a scenario declares
- `02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md` — a lab node is a virtual machine - `03-DESIGN/01-to-be/03-scenario-lifecycle.md` — what happens to one
- `02-DECISIONS/0029-the-labs-first-scenario-has-no-pipeline.md` — two scenario classes, and why this is first - `02-DECISIONS/0031-the-lab-provides-the-underlay.md`
- `02-DECISIONS/0030-the-repository-structure.md` — why this is its own repository - `02-DECISIONS/0032-a-scenario-is-an-isolated-address-space.md`
This repository carries implementation. It does not carry decisions. This repository carries implementation. It does not carry decisions.
## Development
No build step — Node strips the types.
```
npm test the declaration layer and the diagram, offline, 49 tests
npm run test:integration real scenarios against a real hypervisor, 14 tests
npm run typecheck source and tests both — a test that does not compile is a test
that silently never ran
npm run check typecheck + both suites — this is the gate
```
**A test names the decision it defends** (`novox/hq` ADR 0034). A decision with no test is one
that will quietly stop being true, and nobody learns that from a document:
| Test | Defends |
|---|---|
| the lab provides the underlay and nothing of the overlay | ADR 0031 |
| the workstation has no route into the scenario | ADR 0032 |
| a router is a container while machines are virtual machines | ADR 0033 |
| raise waits for *usable*, not for the call to return | the lifecycle design |
| snapshots are whole-scenario | the lifecycle design |
| a public range that is not documentation space is refused | the declaration design |
| a scenario declaring what cannot be materialised is refused | the declaration design |
| the live diagram distinguishes scenery from a node | ADR 0033 |
| the live diagram draws what exists, never what was asked for | the diagram design |
| a picture nobody can open is not a picture | the diagram design |
**Mocking the hypervisor is forbidden.** A fake would assert that the fake behaves as expected,
which is the shape of test this project exists to stop shipping. Integration tests skip with a
reason on a machine that cannot raise scenarios, rather than passing green having checked
nothing.
That suite earned itself on its first run: it found that a snapshot of a running machine could
miss a file written seconds earlier — not stale, **absent** — because the write was still in
the guest's page cache. The design had listed that as an open question. The test answered it,
and `snapshot` now flushes first.
package-lock.json
Generated
+68
View File
@@ -0,0 +1,68 @@
{
"name": "@novox/mesh-lab",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@novox/mesh-lab",
"version": "0.1.0",
"dependencies": {
"yaml": "^2.6.0"
},
"bin": {
"mesh-lab": "dist/cli.js"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
},
"node_modules/@types/node": {
"version": "22.20.1",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.1.tgz",
"integrity": "sha512-EANqOCF9QFyra+4pfxUcX9STKJpCLjMbObVzljIJomAWSnuSIEAvyzEU53GaajbXJEgdh0iEcPL+DGvpUd4k1Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
"dev": true,
"license": "MIT"
},
"node_modules/yaml": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
"integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
"license": "ISC",
"bin": {
"yaml": "bin.mjs"
},
"engines": {
"node": ">= 14.6"
},
"funding": {
"url": "https://github.com/sponsors/eemeli"
}
}
}
}
package.json
+22
View File
@@ -0,0 +1,22 @@
{
"name": "@novox/mesh-lab",
"version": "0.1.0",
"private": true,
"type": "module",
"bin": {
"mesh-lab": "./src/cli.ts"
},
"scripts": {
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.test.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'",
"test:integration": "node --test --test-concurrency=1 --experimental-strip-types 'test/integration/*.test.ts'",
"check": "npm run typecheck && npm test && npm run test:integration"
},
"dependencies": {
"yaml": "^2.6.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
scenarios/behind-nat.yml
+31
View File
@@ -0,0 +1,31 @@
# A machine on a routable address, and a machine behind a household connection.
#
# This is the case that only exists in production today: the second machine is reachable
# from the first only through a forwarded port, at the GATEWAY's address, never its own.
scenario: behind-nat
segments:
hosting:
kind: public
cidr: [192.0.2.0/24]
home:
kind: private
cidr: [192.168.1.0/24]
gateway:
to: hosting
address: [192.0.2.50] # what the world sees the household as
nat: [v4]
forwardable: true
mapping_ttl: 120s
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
inbound: allow
home-server: # a dash in the name, on purpose
at: { segment: home, address: [192.168.1.135] }
published:
- { port: 8080, on: home }
inbound: allow
scenarios/bootstrap-single.yml
+25
View File
@@ -0,0 +1,25 @@
# The smallest useful scenario: one machine, one public network, nothing else.
#
# This is the bootstrap class — no forge, no control plane, no pipeline. It exists to
# develop the node host, which is why it is the first thing the lab can raise.
scenario: bootstrap-single
segments:
hosting:
kind: public
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
inbound: allow
place:
all: [host]
# The host is placed. Everything above tier 0 is still refused by name — this lab is built to
# develop it. Declaring it anyway would make the scenario unraisable, and correctly so: the
# lab refuses declarations it cannot materialise rather than raising a mesh that silently
# lacks them.
snapshot: raised
scenarios/segmented-and-unforwardable.yml
+65
View File
@@ -0,0 +1,65 @@
# Two things production has and a flat lab cannot show.
#
# `devices` and `home` sit behind ONE router — identical gateway declarations — with a
# policy allowing home→devices and denying the reverse. That is an ordinary segmented
# household router, and the asymmetry is the normal case.
#
# `cafe` sits behind a gateway we do not control. Outbound works; nothing initiates
# inward, and nothing can be published there at all.
scenario: segmented-and-unforwardable
segments:
hosting:
kind: public
cidr: [192.0.2.0/24]
home:
kind: private
cidr: [192.168.1.0/24]
gateway:
to: hosting
address: [192.0.2.50]
nat: [v4]
forwardable: true
mapping_ttl: 120s
devices:
kind: private
cidr: [192.168.30.0/24]
gateway:
to: hosting
address: [192.0.2.50] # identical → the SAME router
nat: [v4]
forwardable: true
mapping_ttl: 120s
cafe:
kind: private
cidr: [10.50.0.0/16]
gateway:
to: hosting
address: [192.0.2.80]
nat: [v4]
forwardable: false # carrier-grade NAT, or simply not ours
mapping_ttl: 30s
policy:
- { from: devices, to: home, allow: false }
- { from: home, to: devices, allow: true }
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
inbound: allow
home-server:
at: { segment: home, address: [192.168.1.135] }
inbound: allow
thermostat:
at: { segment: devices, address: [192.168.30.20] }
inbound: allow
roamer:
at: { segment: cafe, address: [10.50.3.23] }
inbound: allow
scenarios/the-ordinary-shape.yml
+81
View File
@@ -0,0 +1,81 @@
# One machine with a routable address, one publicly named but behind a household
# connection, one stationary machine on that network, one that roams.
#
# Three unrelated public networks, routed to each other and never bridged — putting them
# in one prefix would make ARP adjacency, non-decrementing TTL and crossing multicast
# true in the lab and false in production.
scenario: the-ordinary-shape
segments:
hosting:
kind: public
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
isp-home:
kind: public
cidr: [198.51.100.0/24, "2001:db8:b::/48"]
isp-mobile:
kind: public
cidr: [203.0.113.0/24, "2001:db8:c::/48"]
home:
kind: private
cidr: [192.168.1.0/24, "2001:db8:b:1::/64"]
mtu: 1492
gateway:
to: isp-home
address: [198.51.100.7, "2001:db8:b::7"]
nat: [v4]
forwardable: true
mapping_ttl: 120s
devices:
kind: private
cidr: [192.168.30.0/24]
gateway:
to: isp-home
address: [198.51.100.7]
nat: [v4]
forwardable: true
mapping_ttl: 120s
cafe:
kind: private
cidr: [10.50.0.0/16]
mtu: 1400
gateway:
to: isp-mobile
address: [203.0.113.129]
nat: [v4]
forwardable: false
mapping_ttl: 30s
policy:
- { from: devices, to: home, allow: false }
- { from: home, to: devices, allow: true }
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
inbound: allow
home-server:
at: { segment: home, address: [192.168.1.135, "2001:db8:b:1::135"] }
published:
- { port: 443, on: home }
inbound: allow
workstation:
at: { segment: home, address: [192.168.1.250, "2001:db8:b:1::250"] }
inbound: deny
laptop:
at: { segment: home, address: [192.168.1.98, "2001:db8:b:1::98"] }
inbound: deny
# No `place:` yet. The node host it would place does not exist — this lab is being built to
# develop it, and the lab refuses declarations it cannot materialise rather than raising a
# mesh that silently lacks them.
snapshot: raised
scenarios/two-on-a-segment.yml
+23
View File
@@ -0,0 +1,23 @@
# Two machines on one public network. The smallest scenario in which reachability is a
# question at all — and the first that can be answered from inside.
scenario: two-on-a-segment
segments:
hosting:
kind: public
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
inbound: allow
peer:
at: { segment: hosting, address: [192.0.2.20, "2001:db8:a::20"] }
inbound: allow
# No `place:` yet. The node host it would place does not exist — this lab is being built to
# develop it. Declaring it anyway would make the scenario unraisable, and correctly so: the
# lab refuses declarations it cannot materialise rather than raising a mesh that silently
# lacks them.
snapshot: raised
src/cli.ts
Executable
+209
View File
@@ -0,0 +1,209 @@
#!/usr/bin/env -S node --experimental-strip-types
/**
* The lab's two callers want different things from the same verbs: a coordinator wants
* structured results and clean teardown, a person wants readable output and the failing
* scenario left standing. Both use these verbs; the difference is what happens after.
*/
import { loadScenario } from "./declaration/parse.ts";
import { DeclarationError } from "./declaration/validate.ts";
import { isReachable, supportedDrivers, pools } from "./incus/client.ts";
import { raise, RaiseError } from "./lifecycle/raise.ts";
import { assertSupported, UnsupportedError } from "./lifecycle/supported.ts";
import { diagramFromDeclaration } from "./diagram/from-declaration.ts";
import { diagramFromLive } from "./diagram/from-live.ts";
import { toDrawio } from "./diagram/drawio.ts";
import { writeFileSync } from "node:fs";
import { destroy, exec, list, restore, snapshot, snapshots } from "./lifecycle/operate.ts";
const USAGE = `mesh-lab — raise a disposable mesh on one machine
check verify this machine can run scenarios
validate <scenario.yml> parse and check a declaration, raising nothing
raise <scenario.yml> materialise it, and wait until the machines are usable
list scenario instances currently standing
exec <instance> <machine> -- <cmd...>
snapshot <instance> <label> capture the whole scenario as one state
restore <instance> <label> return the whole scenario to it
snapshots <instance>
destroy <instance>
diagram <scenario.yml> [out.drawio] draw what a scenario asks for
diagram --live <instance> [out.drawio] draw what is actually raised
Set MESH_LAB_INCUS if the daemon needs a different invocation, e.g. "sudo -n incus".
`;
function fail(message: string): never {
console.error(message);
process.exit(1);
}
/**
* Refuse to run degraded rather than warning. A warning about a slow inner loop is read
* once and ignored forever, and the loop stays slow.
*/
async function check(): Promise<void> {
const problems: string[] = [];
if (!(await isReachable())) {
problems.push(
"the incus daemon is not reachable as this user. If the group was granted recently, " +
"a session that predates it cannot see it — log out and back in, or set MESH_LAB_INCUS.",
);
console.error(problems.map((p) => ` ✗ ${p}`).join("\n"));
process.exit(1);
}
console.log(" ✓ daemon reachable");
const drivers = await supportedDrivers();
const cowDrivers = drivers.filter((d) => d === "btrfs" || d === "zfs");
if (cowDrivers.length === 0) {
problems.push(
"no copy-on-write storage driver is offered. Snapshots would be full copies — " +
"measured at roughly 76x slower, which does not make the lab slow, it makes it unused.",
);
} else {
console.log(` ✓ copy-on-write driver available (${cowDrivers.join(", ")})`);
}
const available = await pools();
const cowPool = available.find((p) => p.driver === "btrfs" || p.driver === "zfs");
if (!cowPool) {
problems.push(
`no pool uses a copy-on-write driver (have: ${available.map((p) => `${p.name}/${p.driver}`).join(", ") || "none"}). ` +
"A pool that exists and is the slow kind is the failure with no symptom.",
);
} else {
console.log(` ✓ copy-on-write pool '${cowPool.name}' (${cowPool.driver})`);
}
if (problems.length > 0) {
console.error(`\n${problems.map((p) => ` ✗ ${p}`).join("\n\n")}`);
process.exit(1);
}
console.log("\nthis machine can run scenarios");
}
async function main(): Promise<void> {
const [verb, ...rest] = process.argv.slice(2);
switch (verb) {
case "check":
return check();
case "validate": {
const path = rest[0] ?? fail("validate needs a scenario file");
const scenario = loadScenario(path);
console.log(
`${scenario.scenario}: ${Object.keys(scenario.segments).length} segments, ` +
`${Object.keys(scenario.machines).length} machines — valid`,
);
// Valid and raisable are different questions, and a scenario can be the first
// without being the second.
try {
assertSupported(scenario);
} catch (err) {
if (err instanceof UnsupportedError) {
console.log(`\nnot yet raisable:\n - ${err.missing.join("\n - ")}`);
} else throw err;
}
return;
}
case "raise": {
const path = rest[0] ?? fail("raise needs a scenario file");
const scenario = loadScenario(path);
const started = Date.now();
const raised = await raise(scenario, {
onProgress: (m) => console.log(m),
...(process.env["MESH_LAB_IMAGE"] ? { image: process.env["MESH_LAB_IMAGE"] } : {}),
});
const seconds = ((Date.now() - started) / 1000).toFixed(1);
console.log(`\nraised ${raised.instanceId} in ${seconds}s — ${raised.machines.length} machines usable`);
if (scenario.snapshot) {
const took = await snapshot(raised.instanceId, scenario.snapshot);
console.log(`snapshot '${scenario.snapshot}' in ${took.toFixed(2)}s`);
}
return;
}
case "list": {
const found = await list();
if (found.length === 0) return console.log("no scenario instances standing");
for (const instance of found) {
console.log(`${instance.instanceId}`);
for (const m of instance.machines) console.log(` ${m.machine.padEnd(16)} ${m.status}`);
}
return;
}
case "exec": {
const [instanceId, machine, ...command] = rest;
if (!instanceId || !machine || command.length === 0) fail("exec <instance> <machine> <cmd...>");
const result = await exec(instanceId, machine, command.filter((c) => c !== "--"));
process.stdout.write(result.stdout);
process.stderr.write(result.stderr);
return;
}
case "snapshot": {
const [instanceId, label] = rest;
if (!instanceId || !label) fail("snapshot <instance> <label>");
const took = await snapshot(instanceId, label);
console.log(`snapshot '${label}' in ${took.toFixed(2)}s`);
return;
}
case "restore": {
const [instanceId, label] = rest;
if (!instanceId || !label) fail("restore <instance> <label>");
const { restoreSeconds, usableSeconds } = await restore(instanceId, label, 180, (m) =>
console.log(m),
);
console.log(
`restored '${label}' in ${restoreSeconds.toFixed(2)}s — usable again after ` +
`${usableSeconds.toFixed(1)}s`,
);
return;
}
case "snapshots": {
const instanceId = rest[0] ?? fail("snapshots <instance>");
const found = await snapshots(instanceId);
console.log(found.length ? found.join("\n") : "none");
return;
}
case "diagram": {
const live = rest[0] === "--live";
const target = (live ? rest[1] : rest[0]) ?? fail("diagram needs a scenario file or --live <instance>");
const diagram = live
? await diagramFromLive(target)
: diagramFromDeclaration(loadScenario(target));
const out = (live ? rest[2] : rest[1]) ?? `${diagram.title}.drawio`;
writeFileSync(out, toDrawio(diagram));
console.log(
`${out} — ${diagram.segments.length} segments, ${diagram.machines.length} machines ` +
`(${diagram.source})`,
);
return;
}
case "destroy": {
const instanceId = rest[0] ?? fail("destroy <instance>");
const { machines, networks } = await destroy(instanceId);
console.log(`destroyed ${instanceId} — ${machines} machines, ${networks} segments`);
return;
}
default:
console.log(USAGE);
process.exit(verb ? 1 : 0);
}
}
main().catch((err: unknown) => {
if (err instanceof DeclarationError || err instanceof RaiseError) fail(err.message);
if (err instanceof UnsupportedError) fail(err.message);
fail(err instanceof Error ? err.message : String(err));
});
src/declaration/net.ts
+88
View File
@@ -0,0 +1,88 @@
/**
* Address parsing, enough to answer two questions the validator asks: which family is
* this, and is it inside that range.
*
* Written rather than depended on because it is small, and because the one rule it
* exists to enforce — public segments use documentation ranges — is the difference
* between a lab that reproduces the internet and one that silently never forms a mesh.
*/
export type Family = "v4" | "v6";
export interface Cidr {
family: Family;
/** Network address, as an integer. */
base: bigint;
prefix: number;
text: string;
}
const V4_BITS = 32n;
const V6_BITS = 128n;
export function familyOf(address: string): Family {
return address.includes(":") ? "v6" : "v4";
}
function parseV4(text: string): bigint {
const parts = text.split(".");
if (parts.length !== 4) throw new Error(`not an IPv4 address: ${text}`);
let value = 0n;
for (const part of parts) {
if (!/^\d{1,3}$/.test(part)) throw new Error(`not an IPv4 address: ${text}`);
const octet = Number(part);
if (octet > 255) throw new Error(`octet out of range in ${text}`);
value = (value << 8n) | BigInt(octet);
}
return value;
}
function parseV6(text: string): bigint {
// Reject the forms this does not implement rather than mis-parsing them. An embedded
// IPv4 suffix is legal and rare; getting it wrong silently would be worse than refusing.
if (text.includes(".")) throw new Error(`IPv4-in-IPv6 form is not supported: ${text}`);
const halves = text.split("::");
if (halves.length > 2) throw new Error(`not an IPv6 address: ${text}`);
const head = halves[0] ? halves[0].split(":").filter(Boolean) : [];
const tail = halves.length === 2 && halves[1] ? halves[1].split(":").filter(Boolean) : [];
const explicit = head.length + tail.length;
if (explicit > 8) throw new Error(`too many groups in ${text}`);
if (halves.length === 1 && explicit !== 8) throw new Error(`not an IPv6 address: ${text}`);
const groups = [...head, ...Array<string>(8 - explicit).fill("0"), ...tail];
let value = 0n;
for (const group of groups) {
if (!/^[0-9a-fA-F]{1,4}$/.test(group)) throw new Error(`not an IPv6 address: ${text}`);
value = (value << 16n) | BigInt(parseInt(group, 16));
}
return value;
}
export function parseAddress(text: string): { family: Family; value: bigint } {
const family = familyOf(text);
return { family, value: family === "v4" ? parseV4(text) : parseV6(text) };
}
export function parseCidr(text: string): Cidr {
const slash = text.lastIndexOf("/");
if (slash === -1) throw new Error(`not a CIDR range (no prefix length): ${text}`);
const addressText = text.slice(0, slash);
const prefix = Number(text.slice(slash + 1));
const { family, value } = parseAddress(addressText);
const bits = family === "v4" ? V4_BITS : V6_BITS;
if (!Number.isInteger(prefix) || prefix < 0 || BigInt(prefix) > bits) {
throw new Error(`prefix length out of range for ${family}: ${text}`);
}
const hostBits = bits - BigInt(prefix);
const base = (value >> hostBits) << hostBits;
return { family, base, prefix, text };
}
export function contains(range: Cidr, address: string): boolean {
const { family, value } = parseAddress(address);
if (family !== range.family) return false;
const bits = family === "v4" ? V4_BITS : V6_BITS;
const hostBits = bits - BigInt(range.prefix);
return ((value >> hostBits) << hostBits) === range.base;
}
src/declaration/parse.ts
+116
View File
@@ -0,0 +1,116 @@
/** YAML in, a validated Scenario out. Normalises the shorthands the design's examples use. */
import { readFileSync } from "node:fs";
import { parse as parseYaml } from "yaml";
import type { Attachment, Machine, Scenario, Segment } from "./types.ts";
import { validate } from "./validate.ts";
/** `address: "1.2.3.4"` and `address: [...]` both mean a list. */
function toList(value: unknown): string[] {
if (value === undefined || value === null) return [];
return Array.isArray(value) ? value.map(String) : [String(value)];
}
function normaliseAttachment(raw: unknown): Attachment {
const at = (raw ?? {}) as Record<string, unknown>;
return { segment: String(at["segment"] ?? ""), address: toList(at["address"]) };
}
function normaliseMachine(raw: unknown): Machine {
const machine = (raw ?? {}) as Record<string, unknown>;
const at = machine["at"];
const attachment: Machine["at"] =
at === "detached"
? "detached"
: Array.isArray(at)
? at.map(normaliseAttachment)
: [normaliseAttachment(at)];
const result: Machine = { at: attachment };
const published = machine["published"];
if (Array.isArray(published)) {
result.published = published.map((entry) => {
const p = (entry ?? {}) as Record<string, unknown>;
return { port: Number(p["port"]), on: String(p["on"] ?? "") };
});
}
const inbound = machine["inbound"];
if (inbound === "allow" || inbound === "deny") result.inbound = inbound;
return result;
}
function normaliseSegment(raw: unknown): Segment {
const segment = (raw ?? {}) as Record<string, unknown>;
const result: Segment = {
kind: segment["kind"] === "public" ? "public" : "private",
cidr: toList(segment["cidr"]),
};
if (segment["mtu"] !== undefined) result.mtu = Number(segment["mtu"]);
const gateway = segment["gateway"] as Record<string, unknown> | undefined;
if (gateway) {
const nat = toList(gateway["nat"]).filter((f): f is "v4" | "v6" => f === "v4" || f === "v6");
result.gateway = {
to: String(gateway["to"] ?? ""),
address: toList(gateway["address"]),
nat,
// Absent means forwardable: a gateway you control is the ordinary case, and the
// interesting one — carrier-grade NAT — should have to be stated.
forwardable: gateway["forwardable"] !== false,
};
const ttl = gateway["mapping_ttl"] ?? gateway["mappingTtl"];
if (ttl !== undefined) result.gateway.mappingTtl = String(ttl);
}
return result;
}
export function parseScenario(text: string): Scenario {
const raw = (parseYaml(text) ?? {}) as Record<string, unknown>;
const segments: Record<string, Segment> = {};
for (const [name, value] of Object.entries(
(raw["segments"] ?? {}) as Record<string, unknown>,
)) {
segments[name] = normaliseSegment(value);
}
const machines: Record<string, Machine> = {};
for (const [name, value] of Object.entries(
(raw["machines"] ?? {}) as Record<string, unknown>,
)) {
machines[name] = normaliseMachine(value);
}
const scenario: Scenario = {
scenario: String(raw["scenario"] ?? ""),
segments,
machines,
};
const policy = raw["policy"];
if (Array.isArray(policy)) {
scenario.policy = policy.map((entry) => {
const p = (entry ?? {}) as Record<string, unknown>;
return { from: String(p["from"] ?? ""), to: String(p["to"] ?? ""), allow: p["allow"] !== false };
});
}
const place = raw["place"];
if (place && typeof place === "object") {
const normalised: Record<string, string[]> = {};
for (const [key, value] of Object.entries(place as Record<string, unknown>)) {
normalised[key] = toList(value);
}
scenario.place = normalised;
}
if (raw["snapshot"] !== undefined) scenario.snapshot = String(raw["snapshot"]);
validate(scenario);
return scenario;
}
export function loadScenario(path: string): Scenario {
return parseScenario(readFileSync(path, "utf-8"));
}
src/declaration/types.ts
+114
View File
@@ -0,0 +1,114 @@
/**
* A scenario declares an UNDERLAY and what to place on it — the facts a machine would
* have before any of our software touched it. It declares nothing the mesh is
* responsible for: no overlay addresses, no hub, no peering, no names, no certificates.
* Those are outcomes to observe, and a scenario that supplied them would be certifying
* its own work.
*
* See novox/hq: 02-DECISIONS/0031-the-lab-provides-the-underlay.md
* 03-DESIGN/01-to-be/02-scenario-declaration.md
*/
/** An IP family. Reachability is a property of (machine, family), never of a machine. */
export type Family = "v4" | "v6";
/**
* How a segment reaches its parent.
*
* `address` is the address the outside world sees the network as — for a household
* connection, what the ISP hands out. It is load-bearing rather than decorative: it is
* what a peer records as an endpoint when a machine here dials out, and what a public
* name for a published machine here resolves to.
*/
export interface Gateway {
/** Parent segment name. */
to: string;
/** Addresses the gateway holds on the parent segment, one per family. */
address: string[];
/**
* Which families are translated. `["v4"]` is the modern default — v4 translated, v6
* routed. `[]` is a routed range where machines keep their own addresses.
*/
nat: Family[];
/**
* Whether an inbound mapping can be created. Independent of `nat`, and the field that
* separates a home gateway from carrier-grade NAT — which is your own connection and
* still unforwardable.
*/
forwardable: boolean;
/**
* How long an unused inbound mapping survives, e.g. "120s". Absent means mappings never
* expire, which no real gateway does — so absence is a simplification, not a default.
*/
mappingTtl?: string;
}
/** A broadcast domain. Several public segments are unrelated and routed, never bridged. */
export interface Segment {
/**
* `public` stands in for a public network — and there is normally more than one,
* unrelated to each other. `private` is everything else; a private segment with no
* gateway is an island that reaches nothing.
*/
kind: "public" | "private";
/** Address ranges, one per family. */
cidr: string[];
/** Largest packet the segment carries. Default 1500. Lower reproduces tunnelled paths. */
mtu?: number;
gateway?: Gateway;
}
/** Where a machine sits: a segment and the addresses it holds there. */
export interface Attachment {
segment: string;
address: string[];
}
/** A destination-NAT rule on a named gateway, stated as an outcome rather than a port list. */
export interface Publication {
port: number;
/** The segment whose gateway forwards. Named, because a machine may sit behind several. */
on: string;
}
export interface Machine {
/**
* One attachment, or several for a machine on multiple segments at once. Multi-homing
* is not exotic: it is what any node with both a LAN and a WAN interface is.
* `"detached"` is a machine on no segment — it exists and reaches nothing.
*/
at: Attachment[] | "detached";
published?: Publication[];
/**
* A host firewall. Distinct from NAT and behaves differently: a machine can be perfectly
* routable and still refuse everything unsolicited, which is the normal state of a
* v6-addressed machine. Without this, v6 addressing would imply reachability.
*/
inbound?: "allow" | "deny";
}
/** Reachability between segments, as a segmented router enforces it. Asymmetric by design. */
export interface Policy {
from: string;
to: string;
allow: boolean;
}
/** What goes inside the machines. The ONLY part that differs between scenario classes. */
export interface Placement {
/** Applied to every machine. */
all?: string[];
/** Per-machine, overriding `all` for that machine. */
[machine: string]: string[] | undefined;
}
export interface Scenario {
/** The kind. Instances are many; this names the shape, not one of them. */
scenario: string;
segments: Record<string, Segment>;
machines: Record<string, Machine>;
policy?: Policy[];
place?: Placement;
/** Name the state once placement finishes, so a run can return to it. */
snapshot?: string;
}
src/declaration/validate.ts
+303
View File
@@ -0,0 +1,303 @@
/**
* Every rule the design states about a scenario, checked before anything is raised.
*
* The reason validation is this strict is that the failures it prevents are silent. A
* private range on a segment meant to be public does not produce an error — the mesh
* simply never forms, because its own code decides public-versus-private by matching the
* address. Publishing through a gateway that cannot forward does not produce an error
* either; it produces a machine that looks reachable and is not.
*
* So: refuse the declaration, loudly, before spending a minute raising something that
* would have taught us the wrong thing.
*
* See novox/hq 03-DESIGN/01-to-be/02-scenario-declaration.md
*/
import type { Scenario, Segment } from "./types.ts";
import { contains, familyOf, parseAddress, parseCidr, type Cidr } from "./net.ts";
/** RFC 5737 and RFC 3849. The only addresses guaranteed never to route on the real internet. */
const DOCUMENTATION_RANGES = [
"192.0.2.0/24",
"198.51.100.0/24",
"203.0.113.0/24",
"2001:db8::/32",
].map(parseCidr);
export class DeclarationError extends Error {
readonly problems: string[];
constructor(problems: string[]) {
super(`scenario declaration is not valid:\n - ${problems.join("\n - ")}`);
this.name = "DeclarationError";
this.problems = problems;
}
}
/**
* Whether a range sits inside documentation space. Compared as ranges rather than as a
* sample address so that a range wider than the reserved block — `203.0.0.0/8`, say — is
* correctly refused instead of passing because its first address happens to fall inside.
*/
function isDocumentationRange(range: Cidr): boolean {
return DOCUMENTATION_RANGES.some(
(allowed) =>
allowed.family === range.family &&
range.prefix >= allowed.prefix &&
containsRange(allowed, range),
);
}
/** Is `inner` entirely within `outer`? Both already parsed, so no address parsing can throw. */
function containsRange(outer: Cidr, inner: Cidr): boolean {
const bits = outer.family === "v4" ? 32n : 128n;
const hostBits = bits - BigInt(outer.prefix);
return ((inner.base >> hostBits) << hostBits) === outer.base;
}
/** Ranges of a segment, parsed once, with malformed entries reported rather than thrown. */
function rangesOf(name: string, segment: Segment, problems: string[]): Cidr[] {
const ranges: Cidr[] = [];
for (const text of segment.cidr) {
try {
ranges.push(parseCidr(text));
} catch (err) {
problems.push(`segment '${name}': ${(err as Error).message}`);
}
}
return ranges;
}
function inAnyRange(ranges: Cidr[], address: string): boolean {
return ranges.some((range) => contains(range, address));
}
export function validate(scenario: Scenario): void {
const problems: string[] = [];
const segmentNames = Object.keys(scenario.segments);
if (!scenario.scenario) problems.push("scenario has no name");
if (segmentNames.length === 0) problems.push("scenario declares no segments");
if (Object.keys(scenario.machines).length === 0) {
problems.push("scenario declares no machines");
}
const ranges = new Map<string, Cidr[]>();
for (const [name, segment] of Object.entries(scenario.segments)) {
ranges.set(name, rangesOf(name, segment, problems));
}
for (const [name, segment] of Object.entries(scenario.segments)) {
// A public segment stands in for the internet. Anything that is not documentation
// space could route somewhere real, and — far more likely — a private range here
// makes the mesh's own public-versus-private test fail silently.
if (segment.kind === "public") {
// Only ranges that parsed — a malformed one is already reported, and re-parsing it
// here would throw out of validation with a single cryptic message instead of the
// full list.
for (const range of ranges.get(name) ?? []) {
if (!isDocumentationRange(range)) {
problems.push(
`segment '${name}' is public but '${range.text}' is not documentation space ` +
`(RFC 5737 / RFC 3849). A non-documentation range here either routes somewhere ` +
`real, or — if private — makes the mesh silently never form.`,
);
}
}
}
if (segment.mtu !== undefined && (segment.mtu < 576 || segment.mtu > 9000)) {
problems.push(`segment '${name}': mtu ${segment.mtu} is outside any plausible range`);
}
const gateway = segment.gateway;
if (!gateway) continue;
if (!segmentNames.includes(gateway.to)) {
problems.push(`segment '${name}': gateway points at unknown segment '${gateway.to}'`);
continue;
}
if (gateway.to === name) {
problems.push(`segment '${name}': gateway points at itself`);
continue;
}
// The gateway's address is what the world sees this network as, so it belongs to the
// PARENT segment, not this one. Getting this backwards is easy and produces a topology
// that raises fine and reproduces nothing.
const parentRanges = ranges.get(gateway.to) ?? [];
for (const address of gateway.address) {
try {
parseAddress(address);
} catch (err) {
problems.push(`segment '${name}' gateway: ${(err as Error).message}`);
continue;
}
if (parentRanges.length > 0 && !inAnyRange(parentRanges, address)) {
problems.push(
`segment '${name}' gateway: address '${address}' is not within '${gateway.to}' ` +
`(${scenario.segments[gateway.to]?.cidr.join(", ")}). A gateway's address is the ` +
`one the parent network sees, not one on the segment behind it.`,
);
}
}
for (const family of gateway.nat) {
if (family !== "v4" && family !== "v6") {
problems.push(`segment '${name}' gateway: '${family}' is not an address family`);
}
}
}
// Two gateways sharing an address are the same box, and one box cannot behave two ways.
// Left unchecked this raised two routers holding one address on one segment, where the
// address resolved to whichever answered ARP last — so a published port worked or did
// not, run to run, with nothing reporting a fault.
const gateways = Object.entries(scenario.segments)
.filter(([, segment]) => segment.gateway)
.map(([name, segment]) => ({ name, gateway: segment.gateway! }));
for (let i = 0; i < gateways.length; i++) {
for (let j = i + 1; j < gateways.length; j++) {
const a = gateways[i]!;
const b = gateways[j]!;
if (a.gateway.to !== b.gateway.to) continue;
const shared = a.gateway.address.filter((address) => b.gateway.address.includes(address));
if (shared.length === 0) continue;
const differences: string[] = [];
if ([...a.gateway.nat].sort().join(",") !== [...b.gateway.nat].sort().join(",")) {
differences.push(`nat (${a.gateway.nat.join("+") || "none"} vs ${b.gateway.nat.join("+") || "none"})`);
}
if (a.gateway.forwardable !== b.gateway.forwardable) {
differences.push(`forwardable (${a.gateway.forwardable} vs ${b.gateway.forwardable})`);
}
if (a.gateway.mappingTtl !== b.gateway.mappingTtl) {
differences.push(`mapping_ttl (${a.gateway.mappingTtl ?? "none"} vs ${b.gateway.mappingTtl ?? "none"})`);
}
if (differences.length > 0) {
problems.push(
`segments '${a.name}' and '${b.name}' declare gateways on '${a.gateway.to}' sharing ` +
`address '${shared[0]}', so they are one gateway — but they disagree on ` +
`${differences.join(" and ")}. One box cannot behave two ways.`,
);
}
}
}
// A gateway chain must terminate. A cycle would raise forever rather than fail.
for (const name of segmentNames) {
const seen = new Set<string>([name]);
let current = scenario.segments[name]?.gateway?.to;
while (current) {
if (seen.has(current)) {
problems.push(`segment '${name}': gateway chain loops through '${current}'`);
break;
}
seen.add(current);
current = scenario.segments[current]?.gateway?.to;
}
}
for (const [name, machine] of Object.entries(scenario.machines)) {
if (machine.at === "detached") {
if (machine.published?.length) {
problems.push(`machine '${name}' is detached but declares published ports`);
}
continue;
}
if (!Array.isArray(machine.at) || machine.at.length === 0) {
problems.push(`machine '${name}': 'at' must be an attachment, a list of them, or "detached"`);
continue;
}
const attachedTo = new Set<string>();
for (const attachment of machine.at) {
if (!segmentNames.includes(attachment.segment)) {
problems.push(`machine '${name}': unknown segment '${attachment.segment}'`);
continue;
}
if (attachedTo.has(attachment.segment)) {
problems.push(`machine '${name}': attached to '${attachment.segment}' more than once`);
}
attachedTo.add(attachment.segment);
const segmentRanges = ranges.get(attachment.segment) ?? [];
const seenFamilies = new Set<string>();
for (const address of attachment.address) {
try {
parseAddress(address);
} catch (err) {
problems.push(`machine '${name}': ${(err as Error).message}`);
continue;
}
const family = familyOf(address);
if (seenFamilies.has(family)) {
problems.push(`machine '${name}': two ${family} addresses on '${attachment.segment}'`);
}
seenFamilies.add(family);
if (segmentRanges.length > 0 && !inAnyRange(segmentRanges, address)) {
problems.push(
`machine '${name}': address '${address}' is not within segment ` +
`'${attachment.segment}' (${scenario.segments[attachment.segment]?.cidr.join(", ")})`,
);
}
}
}
for (const publication of machine.published ?? []) {
if (!Number.isInteger(publication.port) || publication.port < 1 || publication.port > 65535) {
problems.push(`machine '${name}': port ${publication.port} is not a port`);
}
const via = scenario.segments[publication.on];
if (!via) {
problems.push(`machine '${name}': publishes on unknown segment '${publication.on}'`);
continue;
}
if (!attachedTo.has(publication.on)) {
problems.push(
`machine '${name}': publishes on '${publication.on}' but is not attached to it`,
);
continue;
}
if (!via.gateway) {
problems.push(
`machine '${name}': publishes on '${publication.on}', which has no gateway to ` +
`forward through`,
);
continue;
}
// The constraint being reproduced, not an implementation limit: a machine behind a
// gateway it does not control cannot be published, and pretending otherwise would
// make the lab certify something production cannot do.
if (!via.gateway.forwardable) {
problems.push(
`machine '${name}': cannot publish through '${publication.on}' — its gateway is ` +
`not forwardable. That is the constraint being reproduced, not a limitation.`,
);
}
}
}
for (const rule of scenario.policy ?? []) {
for (const side of [rule.from, rule.to]) {
if (!segmentNames.includes(side)) {
problems.push(`policy: unknown segment '${side}'`);
}
}
if (rule.from === rule.to) {
problems.push(`policy: '${rule.from}' to itself is not a rule`);
}
}
for (const machine of Object.keys(scenario.place ?? {})) {
if (machine === "all") continue;
if (!(machine in scenario.machines)) {
problems.push(`place: '${machine}' is not a machine in this scenario`);
}
}
if (problems.length > 0) throw new DeclarationError(problems);
}
src/diagram/drawio.ts
+455
View File
@@ -0,0 +1,455 @@
/**
* Render a diagram as draw.io XML.
*
* draw.io rather than a rendered image because the output is **editable**: an automatic
* layout of a network is usually 90% right and needs a human nudge, and a picture nobody
* can adjust gets regenerated rather than corrected.
*
* Laid out as lanes rather than by a generic graph algorithm. A network topology has a
* natural vertical order — public at the top, each private network below the one it sits
* behind — and a force-directed layout throws that away, producing a picture that is
* correct and unreadable.
*
* Two kinds of symbol, on purpose:
*
* - the **shape** says what a resource is, and is fixed per kind — a server is always the
* server shape, a gateway always the router shape;
* - the **badges** say what is true about that particular one, and come entirely from
* metadata: translation, forwardability, mapping expiry, whether it refuses inbound.
*
* Which matters because the interesting properties of a network are exactly the ones with
* no visual consequence. An address that is translated looks identical to one that is not.
*
* Shapes come from draw.io's bundled network library, so the file needs no external images
* and renders anywhere draw.io opens. Badges use core mxGraph primitives rather than icon
* shapes: a stencil name that turns out not to exist renders as an empty box, and a badge
* that silently disappears is worse than a plain one that does not.
*/
import type { Diagram, DiagramMachine, DiagramSegment } from "./model.ts";
const LANE_MIN_HEIGHT = 170;
const LANE_EMPTY_HEIGHT = 62;
/** Depth becomes indentation, which is how "behind" is shown without drawing a line. */
const INDENT = 40;
/** A gap holding nothing needs no room; one between top-level groups needs a little. */
const TIGHT_GAP = 24;
const GROUP_GAP = 50;
// Wide enough that a gateway placed on a boundary sits BETWEEN the lanes rather than on
// top of one — an earlier value let a router's box cover the lane's own name and ranges.
const LANE_GAP = 110;
const LANE_X = 40;
const LANE_WIDTH = 980;
const NODE_WIDTH = 150;
const NODE_HEIGHT = 60;
const SLOT_WIDTH = NODE_WIDTH + 60;
const SLOTS_PER_ROW = Math.max(1, Math.floor((LANE_WIDTH - 60) / SLOT_WIDTH));
const BADGE = 18;
const STYLE = {
publicLane:
"rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;dashed=1;" +
"verticalAlign=top;align=left;spacingLeft=10;spacingTop=4;fontSize=11;fontStyle=1",
privateLane:
"rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;dashed=1;" +
"verticalAlign=top;align=left;spacingLeft=10;spacingTop=4;fontSize=11;fontStyle=1",
machine:
"sketch=0;html=1;verticalLabelPosition=bottom;verticalAlign=top;align=center;" +
"shape=mxgraph.networks.server;fillColor=#ffffff;strokeColor=#333333;fontSize=10",
router:
"sketch=0;html=1;verticalLabelPosition=bottom;verticalAlign=top;align=center;" +
"shape=mxgraph.networks.router;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=10",
transit:
"sketch=0;html=1;verticalLabelPosition=bottom;verticalAlign=top;align=center;" +
"shape=mxgraph.networks.cloud;fillColor=#d5e8d4;strokeColor=#82b366;fontSize=10",
link: "edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;endArrow=none;strokeColor=#666666",
note: "text;html=1;align=left;verticalAlign=top;fontSize=9;fontColor=#666666",
badge:
"ellipse;whiteSpace=wrap;html=1;fontSize=9;fontStyle=1;fontColor=#ffffff;" +
"verticalAlign=middle;align=center;spacing=0",
};
/** A metadata fact, rendered as a mark on the resource it is a fact about. */
interface Badge {
code: string;
fill: string;
stroke: string;
/** The full sentence, shown on hover — the code alone would be a private language. */
tip: string;
}
const COLOUR = {
amber: { fill: "#d79b00", stroke: "#b07000" },
red: { fill: "#b85450", stroke: "#8c3a37" },
green: { fill: "#82b366", stroke: "#5b8047" },
blue: { fill: "#6c8ebf", stroke: "#4b6a94" },
grey: { fill: "#9e9e9e", stroke: "#757575" },
};
/**
* Turn a machine's recorded facts into marks.
*
* Absence is deliberately not a badge. A gateway that does not translate gets no NAT mark
* rather than a struck-through one, because a diagram that badges every negative is a
* diagram nobody reads.
*/
function badgesFor(machine: DiagramMachine): { badges: Badge[]; remaining: string[] } {
const badges: Badge[] = [];
const remaining: string[] = [];
if (machine.status) {
const running = machine.status.toLowerCase() === "running";
badges.push({
code: running ? "▶" : "■",
...(running ? COLOUR.green : COLOUR.grey),
tip: `status: ${machine.status}`,
});
}
for (const note of machine.notes) {
const nat = /^NAT (.+)$/.exec(note);
const ttl = /^mappings expire (\d+)s$/.exec(note);
if (nat) {
badges.push({ code: "N", ...COLOUR.amber, tip: `translates ${nat[1]} — addresses behind it are not seen outside` });
} else if (note === "NOT forwardable") {
badges.push({ code: "F", ...COLOUR.red, tip: "no port forwarding — nothing behind this gateway is reachable from outside" });
} else if (note === "forwardable") {
badges.push({ code: "F", ...COLOUR.green, tip: "port forwarding available" });
} else if (ttl) {
badges.push({ code: "T", ...COLOUR.blue, tip: `mappings expire after ${ttl[1]}s of no traffic` });
} else if (note === "refuses inbound") {
badges.push({ code: "D", ...COLOUR.red, tip: "refuses inbound connections it did not start" });
} else if (note === "container") {
badges.push({ code: "C", ...COLOUR.grey, tip: "container — scenery, nothing under test runs here" });
} else if (note === "virtual machine") {
badges.push({ code: "VM", ...COLOUR.blue, tip: "virtual machine — its own kernel" });
} else {
remaining.push(note);
}
}
return { badges, remaining };
}
function escapeXml(text: string): string {
return text
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
function shapeFor(machine: DiagramMachine): string {
return machine.kind === "router"
? STYLE.router
: machine.kind === "transit"
? STYLE.transit
: STYLE.machine;
}
interface Cell {
id: string;
value: string;
style: string;
x: number;
y: number;
w: number;
h: number;
/** Rendered as a wrapping <object>, which is how draw.io carries a tooltip. */
tip?: string;
}
export function toDrawio(diagram: Diagram): string {
const cells: Cell[] = [];
const edges: { id: string; source: string; target: string }[] = [];
/**
* Order: each public network, then everything behind it, depth first.
*
* A single stack sorted by depth put a private network far from the public one it sits
* behind, so a gateway's link to the outside ran the height of the picture and crossed
* networks it had nothing to do with — two such links overlapped and read as one wire.
* Grouping makes every gateway adjacent to both lanes it joins, and every link short.
*/
const byName = new Map(diagram.segments.map((s) => [s.name, s]));
const childrenOf = new Map<string, string[]>();
for (const segment of diagram.segments) {
if (!segment.behind) continue;
childrenOf.set(segment.behind, [...(childrenOf.get(segment.behind) ?? []), segment.name]);
}
const lanes: DiagramSegment[] = [];
const placed = new Set<string>();
const walk = (name: string): void => {
const segment = byName.get(name);
if (!segment || placed.has(name)) return;
placed.add(name);
lanes.push(segment);
for (const child of [...(childrenOf.get(name) ?? [])].sort()) walk(child);
};
// By name, not by the order the source happened to yield them. The hypervisor cannot know
// declaration order, so ordering by it would give the two pictures different shapes and
// make the comparison they exist for impossible to read.
const byNameOrder = [...diagram.segments].sort((a, b) => a.name.localeCompare(b.name));
for (const segment of byNameOrder) if (segment.kind === "public") walk(segment.name);
for (const segment of byNameOrder) walk(segment.name); // isolated ones
const laneX = (segment: DiagramSegment) => LANE_X + segment.depth * INDENT;
const laneW = (segment: DiagramSegment) => LANE_WIDTH - segment.depth * INDENT;
const slotsIn = (segment: DiagramSegment) =>
Math.max(1, Math.floor((laneW(segment) - 60) / SLOT_WIDTH));
// A lane is sized to what it holds. Fixed heights meant a lane with enough machines to
// wrap onto a second row drew that row outside the box it was supposed to be inside.
const occupants = new Map<string, number>();
for (const machine of diagram.machines) {
// Only a machine that sits INSIDE a lane occupies it. A gateway is counted against the
// segment it faces but drawn in the gap, so counting it here left a segment holding no
// machines at full height with nothing in it.
if (machine.attachments.length !== 1) continue;
const only = machine.attachments[0]?.segment;
if (only) occupants.set(only, (occupants.get(only) ?? 0) + 1);
}
const laneHeight = (segment: DiagramSegment): number => {
const rows = Math.ceil((occupants.get(segment.name) ?? 0) / slotsIn(segment));
// A segment whose only occupants are the gateways in the gaps beside it holds nothing
// itself, so it collapses to its own name and ranges. Left at full height it padded a
// layered scenario with empty boxes and pushed the interesting rows apart.
if (rows === 0) return LANE_EMPTY_HEIGHT;
return Math.max(LANE_MIN_HEIGHT, 45 + rows * (NODE_HEIGHT + 45));
};
/**
* Which lane each gateway is drawn above — decided ONCE, by position in the lane order,
* and used both to size the gap and to place the box.
*
* Two rules deciding this separately is what put a gateway inside an unrelated network:
* the gap was reserved above one sibling while the box was drawn above the other, so it
* overflowed into the lane above and landed on top of a machine.
*/
const laneIndex = new Map(lanes.map((segment, index) => [segment.name, index]));
const servedTop = new Map<DiagramMachine, string>();
for (const machine of diagram.machines) {
if (machine.kind === "transit" || machine.attachments.length < 2) continue;
const served = machine.attachments
.slice(1)
.map((a) => a.segment)
.filter((n) => laneIndex.has(n));
if (served.length === 0) continue;
servedTop.set(
machine,
served.reduce((best, n) => ((laneIndex.get(n) ?? 0) < (laneIndex.get(best) ?? 0) ? n : best), served[0]!),
);
}
const gatewayAbove = new Set(servedTop.values());
const transit = diagram.machines.find((m) => m.kind === "transit");
const laneY = new Map<string, number>();
const laneH = new Map<string, number>();
const laneOf = new Map<string, DiagramSegment>();
let cursor = transit ? 80 + NODE_HEIGHT + GROUP_GAP : 80;
lanes.forEach((segment, index) => {
const y = cursor;
const h = laneHeight(segment);
laneY.set(segment.name, y);
laneH.set(segment.name, h);
laneOf.set(segment.name, segment);
const next = lanes[index + 1];
cursor =
y + h +
(!next ? 0 : gatewayAbove.has(next.name) ? LANE_GAP : next.depth === 0 ? GROUP_GAP : TIGHT_GAP);
const facts = [
segment.cidr.join(" "),
segment.mtu ? `MTU ${segment.mtu}` : "",
segment.behind ? `behind ${segment.behind}` : segment.kind === "public" ? "public" : "isolated",
].filter(Boolean);
cells.push({
id: `lane-${segment.name}`,
value: `<b>${segment.name}</b><br/><font style="font-size:10px">${facts.join(" · ")}</font>`,
style: segment.kind === "public" ? STYLE.publicLane : STYLE.privateLane,
x: laneX(segment),
y,
w: laneW(segment),
h,
});
});
// Cell ids are numbered, not named. A declared machine is free to be called `gw-home`,
// which is also what the gateway serving `home` is called — two cells sharing an id makes
// a file draw.io opens with one of them missing, silently.
const idOf = new Map<DiagramMachine, string>();
diagram.machines.forEach((machine, index) => idOf.set(machine, `m${index}`));
// A machine sits inside its first lane. A router straddles, so it is placed between the
// lanes it joins — which is what makes the picture readable at a glance.
const perLane = new Map<string, number>();
let detachedSlot = 0;
for (const machine of diagram.machines) {
const id = idOf.get(machine)!;
const { badges, remaining } = badgesFor(machine);
let x: number;
let y = 80;
if (machine.kind === "transit") {
// Transit is not on a boundary — it reaches every public network at once, so a line
// to each would cross everything between. Stated once, at the top, where it applies.
x = LANE_X;
y = 80;
} else if (machine.attachments.length === 0) {
// Detached: parked to the side, because it genuinely is nowhere.
x = LANE_X + LANE_WIDTH + 60;
y = 80 + detachedSlot * (NODE_HEIGHT + 50);
detachedSlot++;
} else if (machine.attachments.filter((a) => laneY.has(a.segment)).length > 1) {
// A gateway sits in the gap directly above the lane it SERVES, indented to that lane,
// so it reads as the door into it rather than a box floating nearby. Its first
// attachment is the segment it faces outward on; the rest are the ones it serves.
const top = servedTop.get(machine) ?? machine.attachments[1]?.segment ?? "";
const segment = laneOf.get(top);
x = (segment ? laneX(segment) : LANE_X) + 40;
y = (laneY.get(top) ?? 80) - LANE_GAP / 2 - NODE_HEIGHT / 2;
} else {
const only = machine.attachments[0]!.segment;
const segment = laneOf.get(only);
const per = segment ? slotsIn(segment) : SLOTS_PER_ROW;
const slot = perLane.get(only) ?? 0;
perLane.set(only, slot + 1);
// Wrap onto a second row rather than running off the end of the lane. An earlier
// version placed slot 5 outside the box it was supposed to be inside.
x = (segment ? laneX(segment) : LANE_X) + 40 + (slot % per) * SLOT_WIDTH;
y = (laneY.get(only) ?? 80) + 45 + Math.floor(slot / per) * (NODE_HEIGHT + 45);
}
const addresses = machine.attachments
.flatMap((a) => a.addresses)
.slice(0, 3)
.join("<br/>");
const label =
`<b>${machine.name}</b>` +
(addresses ? `<br/><font style="font-size:9px">${addresses}</font>` : "");
cells.push({
id,
value: label,
style: shapeFor(machine),
x,
y,
w: NODE_WIDTH,
h: NODE_HEIGHT,
tip: machine.attachments
.map((a) => `${a.segment}${a.addresses.length ? `: ${a.addresses.join(", ")}` : ""}`)
.join(" · "),
});
// Badges sit along the top edge and read left to right in the order the facts are
// stated, which is the order the declaration states them in.
const badgeRun = badges.length * BADGE + Math.max(0, badges.length - 1) * 3;
badges.forEach((badge, index) => {
cells.push({
id: `${id}-b${index}`,
value: badge.code,
style: `${STYLE.badge};fillColor=${badge.fill};strokeColor=${badge.stroke}`,
x: x + NODE_WIDTH - badgeRun + index * (BADGE + 3),
y: y - BADGE / 2,
w: BADGE,
h: BADGE,
tip: badge.tip,
});
});
if (remaining.length > 0) {
cells.push({
id: `${id}-n`,
value: remaining.join("<br/>"),
style: STYLE.note,
x,
y: y + NODE_HEIGHT + 4,
w: NODE_WIDTH + 60,
h: 13 * remaining.length,
});
}
// A link is drawn only where it is short. With the grouping above a gateway is always
// adjacent to both lanes it joins; anything else would be a line crossing networks it
// does not touch, and a lane already says "behind X" in words.
if (machine.kind === "transit") continue;
for (const attachment of machine.attachments) {
const ly = laneY.get(attachment.segment);
const lh = laneH.get(attachment.segment);
if (ly === undefined || lh === undefined) continue;
const above = ly + lh <= y && y - (ly + lh) <= LANE_GAP;
const below = y + NODE_HEIGHT <= ly && ly - (y + NODE_HEIGHT) <= LANE_GAP;
const within = y >= ly && y + NODE_HEIGHT <= ly + lh;
if (!above && !below && !within) continue;
edges.push({ id: `${id}-e-${attachment.segment}`, source: id, target: `lane-${attachment.segment}` });
}
}
const header =
`<b>${diagram.title}</b> — ${diagram.source === "live" ? "as raised" : "as declared"}`;
cells.unshift({ id: "title", value: header, style: STYLE.note + ";fontSize=14", x: LANE_X, y: 30, w: 600, h: 24 });
const legend = [
"<b>badges</b> — read from metadata, not from the file that asked for it",
"N translates · F forwarding (green yes, red no) · T mappings expire",
"D refuses inbound · C container · VM virtual machine · ▶ running",
].join("<br/>");
cells.push({
id: "legend",
value: legend,
style: STYLE.note,
x: LANE_X,
y: cursor + 10,
w: LANE_WIDTH,
h: 48,
});
// A cell with a tooltip has to be wrapped in <object> — draw.io reads `tooltip` from the
// wrapper, never from mxCell itself, and putting it on the mxCell loses it without error.
const vertex = (c: Cell): string => {
const geometry =
` <mxGeometry x="${c.x}" y="${c.y}" width="${c.w}" height="${c.h}" as="geometry" />\n`;
if (c.tip) {
return (
` <object id="${c.id}" label="${escapeXml(c.value)}" tooltip="${escapeXml(c.tip)}">\n` +
` <mxCell style="${c.style}" vertex="1" parent="1">\n` +
` ${geometry}` +
` </mxCell>\n` +
` </object>`
);
}
// The label is HTML and this is an XML attribute, so the whole thing is escaped here —
// exactly once. Escaping the pieces and concatenating raw tags is what made the first
// file unparseable.
return (
` <mxCell id="${c.id}" value="${escapeXml(c.value)}" style="${c.style}" vertex="1" parent="1">\n` +
geometry +
` </mxCell>`
);
};
const body = [
...cells.map(vertex),
...edges.map(
(e) =>
` <mxCell id="${e.id}" style="${STYLE.link}" edge="1" parent="1" source="${e.source}" target="${e.target}">\n` +
` <mxGeometry relative="1" as="geometry" />\n` +
` </mxCell>`,
),
].join("\n");
return `<mxfile host="mesh-lab">
<diagram name="${escapeXml(diagram.title)}">
<mxGraphModel dx="1200" dy="800" grid="1" gridSize="10" page="1" pageWidth="1169" pageHeight="826">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
${body}
</root>
</mxGraphModel>
</diagram>
</mxfile>
`;
}
src/diagram/from-declaration.ts
+68
View File
@@ -0,0 +1,68 @@
/** The picture of what a scenario asks for. Needs nothing running. */
import type { Scenario } from "../declaration/types.ts";
import { planRouters, routerMachineName, ttlSeconds } from "../lifecycle/router.ts";
import { depthOf, type Diagram, type DiagramMachine, type DiagramSegment } from "./model.ts";
export function diagramFromDeclaration(scenario: Scenario): Diagram {
const parentOf = (name: string) => scenario.segments[name]?.gateway?.to;
const segments: DiagramSegment[] = Object.entries(scenario.segments).map(([name, segment]) => ({
name,
kind: segment.kind,
cidr: segment.cidr,
mtu: segment.mtu,
behind: segment.gateway?.to,
depth: depthOf(name, parentOf),
}));
const machines: DiagramMachine[] = Object.entries(scenario.machines).map(([name, spec]) => {
const notes: string[] = [];
if (spec.inbound === "deny") notes.push("refuses inbound");
for (const publication of spec.published ?? []) {
notes.push(`published :${publication.port} via ${publication.on}`);
}
return {
name,
kind: "machine" as const,
notes,
attachments:
spec.at === "detached"
? []
: spec.at.map((a) => ({ segment: a.segment, addresses: a.address })),
};
});
// Routers are implicit in a declaration — a scenario says a segment sits behind one and
// never names it. The diagram has to show them anyway, or the picture omits the machines
// that carry every interesting property.
for (const plan of planRouters(scenario, "diagram")) {
const notes: string[] = [];
notes.push(plan.nat.length ? `NAT ${plan.nat.join("+")}` : "routed, no NAT");
notes.push(plan.forwardable ? "forwardable" : "NOT forwardable");
const ttl = ttlSeconds(plan.mappingTtl);
if (ttl !== undefined) notes.push(`mappings expire ${ttl}s`);
machines.push({
name: routerMachineName(plan),
kind: "router",
notes,
attachments: [
{ segment: plan.outside, addresses: plan.outsideAddresses },
...plan.inside.map((segment) => ({ segment, addresses: [] })),
],
});
}
const publicSegments = segments.filter((s) => s.kind === "public");
if (publicSegments.length > 1) {
machines.push({
name: "transit",
kind: "transit",
notes: ["routed, never bridged"],
attachments: publicSegments.map((s) => ({ segment: s.name, addresses: [] })),
});
}
return { title: scenario.scenario, source: "declared", segments, machines };
}
src/diagram/from-live.ts
+121
View File
@@ -0,0 +1,121 @@
/**
* The picture of what is actually raised, read from the hypervisor's own metadata.
*
* Deliberately reads the same tags `destroy` uses rather than re-deriving anything from the
* declaration: a diagram built from the declaration would draw what was asked for and call
* it what exists, which is the whole failure this pairing is meant to expose. Everything
* shown here was either recorded on the resource when it was raised, or is being reported
* by the running machine now — nothing is inferred from a file on disk.
*/
import { incusOk, taggedNetworks } from "../incus/client.ts";
import { depthOf, type Diagram, type DiagramMachine, type DiagramSegment } from "./model.ts";
interface RawInstance {
name?: string;
type?: string;
status?: string;
config?: Record<string, string>;
devices?: Record<string, Record<string, string>>;
state?: {
network?: Record<
string,
{ hwaddr?: string; addresses?: { family?: string; address?: string; netmask?: string; scope?: string }[] }
>;
};
}
export async function diagramFromLive(instanceId: string): Promise<Diagram> {
const networks = (await taggedNetworks()).filter((n) => n.instanceId === instanceId);
const json = (await incusOk(["list", "--format", "json"], 30_000)) ?? "[]";
const parsed = JSON.parse(json) as RawInstance[];
const mine = parsed.filter((i) => i.config?.["user.mesh-lab.instance"] === instanceId);
if (mine.length === 0 && networks.length === 0) throw new Error(`no scenario instance '${instanceId}'`);
const segmentOfLink = new Map(networks.map((n) => [n.name, n.segment]));
// A gateway records the segments behind it, so the tree is recoverable from the routers
// alone. Without this every segment would draw at the same depth and a picture of a
// layered scenario would look flat — which is exactly the property under test.
const parent = new Map<string, string>();
for (const item of mine) {
const inside = item.config?.["user.mesh-lab.router"];
const outside = item.config?.["user.mesh-lab.outside"];
if (!inside || !outside) continue;
for (const segment of inside.split(",").filter(Boolean)) parent.set(segment, outside);
}
const parentOf = (name: string) => parent.get(name);
const segments: DiagramSegment[] = networks.map((n) => ({
name: n.segment,
// Untagged links come from an instance raised before segment shape was recorded. Drawn
// as private rather than guessed at, and the missing tag is said out loud on the lane.
kind: n.kind ?? "private",
cidr: n.cidr,
mtu: n.mtu,
behind: parentOf(n.segment),
depth: depthOf(n.segment, parentOf),
}));
const machines: DiagramMachine[] = mine.map((item) => {
const config = item.config ?? {};
const isTransit = config["user.mesh-lab.transit"] !== undefined;
const isRouter = config["user.mesh-lab.router"] !== undefined;
// Addresses are joined to devices by MAC, not by name. A container's interface is
// called what the device is called; a virtual machine names its own — `enp5s0` for the
// device configured as `eth0` — so matching on the name attached every address to a
// container and none to a VM, which read as machines that had failed to come up.
const heldByMac = new Map<string, string[]>();
const heldByName = new Map<string, string[]>();
for (const [name, iface] of Object.entries(item.state?.network ?? {})) {
const held = (iface.addresses ?? [])
.filter((a) => a.scope === "global" && a.address)
.map((a) => (a.netmask ? `${a.address}/${a.netmask}` : (a.address as string)));
if (held.length === 0) continue;
heldByName.set(name, held);
if (iface.hwaddr) heldByMac.set(iface.hwaddr.toLowerCase(), held);
}
const attachments: DiagramMachine["attachments"] = [];
for (const [device, spec] of Object.entries(item.devices ?? {})) {
if (spec["type"] !== "nic") continue;
const segment = segmentOfLink.get(spec["parent"] ?? "");
if (!segment) continue;
const mac = spec["hwaddr"]?.toLowerCase();
const addresses = (mac ? heldByMac.get(mac) : undefined) ?? heldByName.get(device) ?? [];
attachments.push({ segment, addresses });
}
// Outside first, then the segments behind it — the same order the declared picture uses,
// and what lets the layout place a gateway above the lanes it SERVES rather than below
// the one it faces. Sorting alphabetically threw that away.
const outside = config["user.mesh-lab.outside"];
attachments.sort((a, b) =>
a.segment === outside ? -1 : b.segment === outside ? 1 : a.segment.localeCompare(b.segment),
);
const notes: string[] = [];
notes.push(item.type === "container" ? "container" : "virtual machine");
if (config["user.mesh-lab.inbound"] === "deny") notes.push("refuses inbound");
if (isRouter) {
const nat = (config["user.mesh-lab.nat"] ?? "").split(",").filter(Boolean);
notes.push(nat.length > 0 ? `NAT ${nat.join("+")}` : "routed, no NAT");
if (config["user.mesh-lab.forwardable"] !== undefined) {
notes.push(config["user.mesh-lab.forwardable"] === "true" ? "forwardable" : "NOT forwardable");
}
const ttl = config["user.mesh-lab.mapping-ttl"];
if (ttl) notes.push(`mappings expire ${ttl}s`);
}
return {
name: config["user.mesh-lab.machine"] ?? item.name ?? "?",
kind: isTransit ? "transit" : isRouter ? "router" : "machine",
notes,
attachments,
...(item.status ? { status: item.status } : {}),
};
});
return { title: instanceId, source: "live", segments, machines };
}
src/diagram/model.ts
+54
View File
@@ -0,0 +1,54 @@
/**
* What a diagram draws, independent of where it came from.
*
* Two sources produce one of these: a declaration, and a raised instance. Drawing both
* through the same model and the same layout is the point — a difference between the
* picture of what was asked for and the picture of what exists is a difference you can
* see, which is the check this project keeps finding absent everywhere else.
*/
export interface DiagramSegment {
name: string;
kind: "public" | "private";
cidr: string[];
mtu: number | undefined;
/** Parent segment, for a private network behind a gateway. */
behind: string | undefined;
/** Depth from a public segment. Public is 0. */
depth: number;
}
export interface DiagramMachine {
name: string;
/** Segments it sits on, with the addresses it holds there. */
attachments: { segment: string; addresses: string[] }[];
kind: "machine" | "router" | "transit";
/** Badges rendered on the node: NAT, forwardable, TTL, refuses inbound. */
notes: string[];
/** Present only for a live diagram — what the hypervisor reports. */
status?: string;
}
export interface Diagram {
title: string;
/** Where this came from, so a reader never has to guess which picture they hold. */
source: "declared" | "live";
segments: DiagramSegment[];
machines: DiagramMachine[];
}
/** Depth of a segment: how many gateways lie between it and a public network. */
export function depthOf(
name: string,
parentOf: (segment: string) => string | undefined,
): number {
let depth = 0;
let current = parentOf(name);
const seen = new Set([name]);
while (current && !seen.has(current)) {
seen.add(current);
depth++;
current = parentOf(current);
}
return depth;
}
src/incus/client.ts
+231
View File
@@ -0,0 +1,231 @@
/**
* A thin wrapper over the incus CLI. Thin on purpose: the lab's value is in the scenario
* model, not in re-describing a hypervisor.
*
* Everything here goes through incus's own channel and never over IP. A scenario is a
* closed address space — two scenarios raised from one declaration hold the same
* addresses and must never meet — so the workstation has no route into either, and
* reaching a machine by address would make concurrency impossible in the worst way:
* not with an error, but with one scenario's traffic arriving in another.
*
* See novox/hq 02-DECISIONS/0032-a-scenario-is-an-isolated-address-space.md
*/
import { spawn } from "node:child_process";
/**
* How to invoke incus. Overridable because the socket is group-owned and a session that
* predates the group grant cannot reach it — which is a real thing that happens on the
* machine that just installed it.
*/
const INCUS = (process.env["MESH_LAB_INCUS"] ?? "incus").split(" ").filter(Boolean);
export interface IncusResult {
stdout: string;
stderr: string;
}
export class IncusError extends Error {
readonly args: string[];
readonly stderr: string;
readonly exitCode: number | null;
constructor(args: string[], stderr: string, exitCode: number | null) {
const detail = stderr.trim() || (exitCode === null ? "timed out" : `exit ${exitCode}`);
super(`incus ${args.join(" ")} failed: ${detail}`);
this.name = "IncusError";
this.args = args;
this.stderr = stderr;
this.exitCode = exitCode;
}
}
/**
* Runs incus with **stdin closed**, which is not incidental.
*
* Several incus subcommands accept a YAML definition on stdin and, given a descriptor
* that is not a terminal, wait for one. Under a shell that never happens because stdin is
* a TTY; under a spawned process it hangs until the timeout kills it — and the timeout
* kill produces an empty stderr, so the failure arrives with no explanation at all.
*
* Found exactly that way: the first raise reported "failed: (no output)" on a command
* that worked perfectly when typed.
*/
export async function incus(args: string[], timeoutMs = 60_000): Promise<IncusResult> {
const [command, ...prefix] = INCUS;
if (!command) throw new Error("MESH_LAB_INCUS is empty");
return new Promise((resolve, reject) => {
const child = spawn(command, [...prefix, ...args], {
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
child.kill("SIGKILL");
}, timeoutMs);
child.stdout.on("data", (chunk) => (stdout += chunk));
child.stderr.on("data", (chunk) => (stderr += chunk));
child.on("error", (err) => {
clearTimeout(timer);
reject(new IncusError(args, err.message, null));
});
child.on("close", (code) => {
clearTimeout(timer);
if (timedOut) {
reject(new IncusError(args, `timed out after ${timeoutMs}ms`, null));
} else if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new IncusError(args, stderr, code));
}
});
});
}
/**
* Like `incus`, but a failure is an answer rather than an exception. Returns the command's
* output, or `null` if it failed.
*
* **Never truthiness-test this.** Plenty of incus commands succeed with no output at all —
* `exec … true`, `network delete`, `start` — so an empty string means *worked and said
* nothing*, and `if (await incusOk(...))` reads that as failure. Use `succeeds()` when the
* question is whether it worked, and `!== null` when the output matters.
*
* This is the mesh's own recurring fault in miniature: absence and success made
* indistinguishable. It cost a raise that reported a machine unreachable while `incus exec`
* on that machine worked perfectly.
*/
export async function incusOk(args: string[], timeoutMs = 60_000): Promise<string | null> {
try {
return (await incus(args, timeoutMs)).stdout;
} catch {
return null;
}
}
/** Did the command work? For commands whose output is not the point. */
export async function succeeds(args: string[], timeoutMs = 60_000): Promise<boolean> {
return (await incusOk(args, timeoutMs)) !== null;
}
export async function isReachable(): Promise<boolean> {
return succeeds(["info"], 15_000);
}
/** Storage drivers the daemon offers. It only advertises those whose tooling it found. */
export async function supportedDrivers(): Promise<string[]> {
const info = (await incusOk(["info"], 15_000)) ?? "";
const block = info.split("storage_supported_drivers:")[1] ?? "";
return [...block.matchAll(/-\s+name:\s*(\w+)/g)].map((m) => m[1] ?? "");
}
export interface Pool {
name: string;
driver: string;
}
export async function pools(): Promise<Pool[]> {
const csv = (await incusOk(["storage", "list", "--format", "csv"])) ?? "";
return csv
.split("\n")
.filter(Boolean)
.map((line) => {
const [name = "", driver = ""] = line.split(",");
return { name, driver };
});
}
export async function networkExists(name: string): Promise<boolean> {
return succeeds(["network", "show", name], 15_000);
}
export async function instanceExists(name: string): Promise<boolean> {
return succeeds(["config", "show", name], 15_000);
}
export interface TaggedInstance {
name: string;
status: string;
instanceId: string;
machine: string;
}
/**
* Instances this lab owns, identified by the config keys set when they were created —
* never by parsing their names.
*
* Names were parsed at first, splitting on the last dash to separate machine from
* instance. That works until a machine is called something like `home-server`, at which
* point the instance id absorbs half the machine name and `destroy` silently finds
* nothing. Metadata is what the instance actually knows about itself.
*/
export async function taggedInstances(): Promise<TaggedInstance[]> {
const json = (await incusOk(["list", "--format", "json"], 30_000)) ?? "[]";
let parsed: unknown;
try {
parsed = JSON.parse(json);
} catch {
return [];
}
if (!Array.isArray(parsed)) return [];
const tagged: TaggedInstance[] = [];
for (const entry of parsed) {
const item = entry as { name?: string; status?: string; config?: Record<string, string> };
const instanceId = item.config?.["user.mesh-lab.instance"];
const machine = item.config?.["user.mesh-lab.machine"];
if (!instanceId || !machine || !item.name) continue;
tagged.push({ name: item.name, status: item.status ?? "", instanceId, machine });
}
return tagged;
}
export interface TaggedNetwork {
name: string;
instanceId: string;
segment: string;
/** Absent on a link raised before segment shape was recorded. */
kind?: "public" | "private";
cidr: string[];
mtu?: number;
}
export async function taggedNetworks(): Promise<TaggedNetwork[]> {
const json = (await incusOk(["network", "list", "--format", "json"], 30_000)) ?? "[]";
let parsed: unknown;
try {
parsed = JSON.parse(json);
} catch {
return [];
}
if (!Array.isArray(parsed)) return [];
const tagged: TaggedNetwork[] = [];
for (const entry of parsed) {
const item = entry as { name?: string; config?: Record<string, string> };
const instanceId = item.config?.["user.mesh-lab.instance"];
const segment = item.config?.["user.mesh-lab.segment"];
if (!instanceId || !segment || !item.name) continue;
const kind = item.config?.["user.mesh-lab.kind"];
const cidr = item.config?.["user.mesh-lab.cidr"];
tagged.push({
name: item.name,
instanceId,
segment,
...(kind === "public" || kind === "private" ? { kind } : {}),
cidr: cidr ? cidr.split(",").filter(Boolean) : [],
...(Number.isFinite(Number(item.config?.["user.mesh-lab.mtu"]))
? { mtu: Number(item.config?.["user.mesh-lab.mtu"]) }
: {}),
});
}
return tagged;
}
src/lifecycle/address.ts
+174
View File
@@ -0,0 +1,174 @@
/**
* Put the declared addresses on the machines.
*
* The lab provides the underlay, and an address is underlay — it is what a hosting
* provider or a home router would have given the machine before any of our software ran.
* So the lab assigns it, and the mesh is left to build everything above it.
*
* Addresses are applied by matching on MAC rather than interface name. A guest names its
* interfaces by bus position — `enp5s0`, not `eth0` — so the name the hypervisor uses and
* the name the guest uses are different, and matching by name silently configures the
* wrong interface on a multi-homed machine.
*
* See novox/hq 02-DECISIONS/0031-the-lab-provides-the-underlay.md
*/
import type { Attachment, Scenario } from "../declaration/types.ts";
import { incus } from "../incus/client.ts";
import { macFor } from "./names.ts";
import { transitAddress } from "./router.ts";
export interface Wire {
device: string;
mac: string;
addresses: string[];
mtu: number | undefined;
}
/**
* A systemd-networkd unit per interface. Static, because the declaration is the authority:
* a scenario that let the hypervisor hand out addresses would be the lab supplying facts
* the declaration is supposed to own.
*/
function networkUnit(wire: Wire): string {
const lines = [
"[Match]",
`MACAddress=${wire.mac}`,
"",
"[Network]",
...wire.addresses.map((address) => `Address=${address}`),
// No gateway and no DNS on purpose. Routing off the segment is a gateway's job, and a
// scenario that pre-wired it would be arranging what it is supposed to observe.
"IPv6AcceptRA=no",
];
if (wire.mtu !== undefined) {
lines.push("", "[Link]", `MTUBytes=${wire.mtu}`);
}
return lines.join("\n") + "\n";
}
export async function applyAddresses(
scenario: Scenario,
instanceId: string,
machineNames: Map<string, string>,
log: (message: string) => void = () => {},
): Promise<void> {
for (const [machine, spec] of Object.entries(scenario.machines)) {
if (spec.at === "detached") continue;
const name = machineNames.get(machine);
if (!name) continue;
const wires: Wire[] = [];
for (const [index, attachment] of spec.at.entries()) {
wires.push({
device: `eth${index}`,
mac: macFor(instanceId, machine, index),
addresses: attachment.address.map((address) => withPrefix(scenario, attachment.segment, address)),
mtu: scenario.segments[attachment.segment]?.mtu,
});
}
for (const [index, wire] of wires.entries()) {
const unit = networkUnit(wire);
await incus(
["exec", name, "--", "sh", "-c",
`mkdir -p /etc/systemd/network && cat > /etc/systemd/network/10-mlab-${index}.network <<'MLAB'\n${unit}MLAB`],
30_000,
);
}
await incus(["exec", name, "--", "systemctl", "enable", "--now", "systemd-networkd"], 60_000);
await incus(["exec", name, "--", "systemctl", "restart", "systemd-networkd"], 60_000);
log(` addressed ${machine} (${wires.map((w) => w.addresses.join(",")).join(" | ")})`);
}
}
/**
* systemd-networkd wants a prefix length on the address. The declaration gives a bare
* address and the segment gives the range, so the two are combined here rather than making
* every scenario repeat the prefix on every machine.
*/
function withPrefix(scenario: Scenario, segment: string, address: string): string {
const ranges = scenario.segments[segment]?.cidr ?? [];
const wantV6 = address.includes(":");
for (const range of ranges) {
const slash = range.lastIndexOf("/");
if (slash === -1) continue;
const isV6 = range.slice(0, slash).includes(":");
if (isV6 === wantV6) return `${address}${range.slice(slash)}`;
}
return address;
}
/**
* Point each machine at the router serving its segment.
*
* The route is the machine's, not the mesh's — a default route is what a home network hands
* out, and a machine that could not reach beyond its own segment would be reproducing the
* wrong topology. What the lab still does not supply is the overlay: no peers, no hub, no
* names.
*
* The router's inside address is the first host address of the range, chosen rather than
* declared because a scenario has nothing to say about it.
*/
export async function applyDefaultRoutes(
scenario: Scenario,
machineNames: Map<string, string>,
log: (message: string) => void = () => {},
): Promise<void> {
for (const [machine, spec] of Object.entries(scenario.machines)) {
if (spec.at === "detached") continue;
const name = machineNames.get(machine);
if (!name) continue;
// A machine behind a gateway routes through it. A machine sitting directly on a public
// segment routes through transit instead — otherwise it can reach its own network and
// nothing else, which is not what being on the internet means.
const behind = spec.at.find((a) => scenario.segments[a.segment]?.gateway);
if (!behind) {
await routeViaTransit(scenario, spec, name);
continue;
}
const index = spec.at.indexOf(behind);
for (const range of scenario.segments[behind.segment]?.cidr ?? []) {
const slash = range.lastIndexOf("/");
if (slash === -1) continue;
const base = range.slice(0, slash);
const via = base.includes(":")
? `${base.replace(/::$/, "")}::1`
: (() => { const o = base.split("."); o[3] = "1"; return o.join("."); })();
const family = base.includes(":") ? "-6" : "-4";
await incus(
["exec", name, "--", "sh", "-c",
`ip ${family} route replace default via ${via} dev $(ip -o link | awk -F': ' 'NR==${index + 2}{print $2}') 2>/dev/null || true`],
30_000,
);
}
log(` routed ${machine} via its gateway on ${behind.segment}`);
}
}
/** A machine on a public segment reaches the other public networks through transit. */
async function routeViaTransit(
scenario: Scenario,
spec: { at: Attachment[] | "detached" },
name: string,
): Promise<void> {
if (spec.at === "detached") return;
const onPublic = spec.at.find((a) => scenario.segments[a.segment]?.kind === "public");
if (!onPublic) return;
const index = spec.at.indexOf(onPublic);
for (const cidr of scenario.segments[onPublic.segment]?.cidr ?? []) {
const via = transitAddress(cidr);
if (!via) continue;
const gateway = via.slice(0, via.lastIndexOf("/"));
const family = gateway.includes(":") ? "-6" : "-4";
await incus(
["exec", name, "--", "sh", "-c",
`ip ${family} route replace default via ${gateway} dev $(ip -o link | awk -F': ' 'NR==${index + 2}{print $2}') 2>/dev/null || true`],
30_000,
);
}
}
src/lifecycle/firewall.ts
+63
View File
@@ -0,0 +1,63 @@
/**
* `inbound: deny` — a host firewall on the machine itself.
*
* Distinct from NAT and behaving differently: a machine can be perfectly routable and
* still refuse everything unsolicited, which is the normal state of a v6-addressed
* machine. Without this, v6 addressing would silently imply reachability, and a scenario
* that said a machine refuses traffic would produce one that accepts it.
*
* Established and related traffic is accepted, so the machine can still dial out. That is
* what a host firewall does; a machine that could not reach anything would be reproducing
* a disconnected machine rather than a defended one.
*/
import type { Scenario } from "../declaration/types.ts";
import { incus, succeeds } from "../incus/client.ts";
const RULESET = `flush ruleset
table inet mlab {
chain input {
type filter hook input priority filter; policy drop;
ct state established,related accept
iif lo accept
ct state invalid drop
}
}
`;
export async function applyHostFirewalls(
scenario: Scenario,
machineNames: Map<string, string>,
log: (message: string) => void = () => {},
): Promise<void> {
for (const [machine, spec] of Object.entries(scenario.machines)) {
if (spec.inbound !== "deny") continue;
const name = machineNames.get(machine);
if (!name) continue;
await incus(
["exec", name, "--", "sh", "-c",
`cat > /tmp/mlab-host.nft <<'MLABNFT'\n${RULESET}MLABNFT\nnft -f /tmp/mlab-host.nft`],
60_000,
);
// Read back. A declared refusal that silently did not apply is the fault this lab
// exists to catch, and a ruleset that failed to load leaves the machine wide open —
// which looks exactly like a machine that is working.
const check = await incus(
["exec", name, "--", "sh", "-c", "nft list table inet mlab >/dev/null 2>&1 && echo present || echo absent"],
20_000,
);
if (check.stdout.trim() !== "present") {
throw new Error(
`${machine}: inbound: deny was declared but the ruleset is not loaded — the machine ` +
`would accept traffic the scenario says it refuses`,
);
}
// Recorded only after the read-back proved it loaded. A tag written before the check
// would be a claim rather than a record, and anything reading the instance back would
// report a defended machine that is in fact wide open.
await succeeds(["config", "set", name, "user.mesh-lab.inbound", "deny"], 20_000);
log(` ${machine} refuses unsolicited inbound`);
}
}
src/lifecycle/invariants.ts
+67
View File
@@ -0,0 +1,67 @@
/**
* Properties that must hold of ANY raised scenario, whatever it declares.
*
* Distinct from validation, which reads a file and can only catch what the file says. These
* read what actually came up. The first one exists because two routers were raised holding
* one address on one segment: the declaration was accepted, the raise reported success, and
* the address resolved to whichever container answered ARP last — so a published port
* worked or did not, run to run, with nothing reporting a fault.
*
* Pure over already-collected facts, so the logic is testable without a hypervisor and the
* reading of the hypervisor stays in one place.
*/
/** One address, held by one machine, on one segment. */
export interface Held {
machine: string;
segment: string;
address: string;
}
export interface Conflict {
segment: string;
address: string;
machines: string[];
}
/** Strip a prefix length: what is held is an address, the mask is a property of the link. */
function bare(address: string): string {
const slash = address.lastIndexOf("/");
return slash === -1 ? address : address.slice(0, slash);
}
/**
* Two machines holding one address on one segment.
*
* The same address on DIFFERENT segments is not a conflict — `192.168.1.1` on one private
* network and on another are two different machines' idea of "the gateway", which is the
* normal case and must not be reported.
*/
export function duplicateAddresses(held: Held[]): Conflict[] {
const byPlace = new Map<string, Set<string>>();
for (const entry of held) {
const key = `${entry.segment} ${bare(entry.address)}`;
const machines = byPlace.get(key) ?? new Set<string>();
machines.add(entry.machine);
byPlace.set(key, machines);
}
const conflicts: Conflict[] = [];
for (const [key, machines] of byPlace) {
if (machines.size < 2) continue;
const [segment, address] = key.split(" ");
conflicts.push({
segment: segment as string,
address: address as string,
machines: [...machines].sort(),
});
}
return conflicts.sort((a, b) => a.address.localeCompare(b.address));
}
/** Render conflicts as something a failing test can print without further work. */
export function describeConflicts(conflicts: Conflict[]): string {
return conflicts
.map((c) => `${c.address} is held by ${c.machines.join(" and ")} on '${c.segment}'`)
.join("; ");
}
src/lifecycle/names.ts
+66
View File
@@ -0,0 +1,66 @@
/**
* How a scenario instance's resources are named.
*
* A declaration is a KIND; instances are many. Two instances of one declaration hold the
* same addresses and must never meet, so every resource carries the instance id and
* nothing is shared between them.
*/
const PREFIX = "mlab";
/** Instance ids are short and sortable — the last one left standing has to be findable. */
export function newInstanceId(scenario: string, now: Date): string {
const stamp = now.toISOString().replace(/[-:T]/g, "").slice(2, 12);
return `${scenario}-${stamp}`;
}
/** incus network names are limited to 15 characters, so this hashes rather than truncates. */
export function networkName(instanceId: string, segment: string): string {
const digest = hash(`${instanceId}/${segment}`);
return `${PREFIX}${digest}`;
}
export function machineName(instanceId: string, machine: string): string {
return `${PREFIX}-${instanceId}-${machine}`;
}
export function instanceIdOf(machineName: string, machine: string): string | null {
const suffix = `-${machine}`;
if (!machineName.startsWith(`${PREFIX}-`) || !machineName.endsWith(suffix)) return null;
return machineName.slice(PREFIX.length + 1, machineName.length - suffix.length);
}
export function machinePrefix(instanceId: string): string {
return `${PREFIX}-${instanceId}-`;
}
/** FNV-1a, rendered base36. Short, stable, and collisions are a naming clash not a leak. */
function hash(text: string): string {
let h = 0x811c9dc5;
for (let i = 0; i < text.length; i++) {
h ^= text.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
return h.toString(36).padStart(7, "0").slice(0, 7);
}
/**
* A deterministic MAC for a machine's Nth interface, in the locally-administered range.
*
* Set explicitly at creation rather than read back afterwards: incus assigns a MAC at
* runtime and does not record it in the device config, so querying returns nothing. A
* derived address is also stable across raises of the same instance, which makes an
* in-guest match on it reproducible.
*/
export function macFor(instanceId: string, machine: string, index: number): string {
const digest = hash(`${instanceId}/${machine}/${index}`);
const octets: string[] = [];
let value = 0;
for (let i = 0; i < digest.length; i++) value = (value * 31 + digest.charCodeAt(i)) >>> 0;
// 02 marks it locally administered, which is what a made-up address is supposed to say.
octets.push("02");
for (let i = 0; i < 5; i++) {
octets.push(((value >>> (i * 5)) & 0xff).toString(16).padStart(2, "0"));
}
return octets.join(":");
}
src/lifecycle/operate.ts
+166
View File
@@ -0,0 +1,166 @@
/**
* What happens to a scenario once it is raised: inspect it, run things in it, capture and
* return it to a state, and tear it down.
*
* Snapshots are WHOLE-SCENARIO. Per-machine would be cheaper and wrong: the mesh keeps
* state that spans nodes, so restoring one machine to an earlier moment while its peers
* move on produces a mesh that has never existed and could not. Faults found there would
* be artefacts of the lab.
*/
import { incus, incusOk, succeeds, taggedInstances, taggedNetworks } from "../incus/client.ts";
import { machineName } from "./names.ts";
import { waitUntilAllUsable } from "./ready.ts";
export interface Instance {
instanceId: string;
machines: { name: string; machine: string; status: string }[];
}
/**
* Every scenario instance the daemon currently holds, found by the metadata each resource
* carries rather than by parsing names — a machine called `home-server` would otherwise
* be split in the wrong place and its instance would appear not to exist.
*/
export async function list(): Promise<Instance[]> {
const byInstance = new Map<string, Instance["machines"]>();
for (const item of await taggedInstances()) {
const entry = byInstance.get(item.instanceId) ?? [];
entry.push({ name: item.name, machine: item.machine, status: item.status });
byInstance.set(item.instanceId, entry);
}
return [...byInstance.entries()]
.map(([instanceId, machines]) => ({
instanceId,
machines: machines.sort((a, b) => a.machine.localeCompare(b.machine)),
}))
.sort((a, b) => a.instanceId.localeCompare(b.instanceId));
}
async function machinesOf(instanceId: string): Promise<string[]> {
const found = (await list()).find((i) => i.instanceId === instanceId);
if (!found) throw new Error(`no scenario instance '${instanceId}'`);
return found.machines.map((m) => m.name);
}
/**
* Run a command on a machine, through incus rather than over IP.
*
* A reachability question is therefore asked from INSIDE: *can this machine reach that
* one* is exec on the first, testing the second. The workstation is not on the scenario's
* network and its opinion would be a different question with a similar-looking answer.
*/
export async function exec(
instanceId: string,
machine: string,
command: string[],
): Promise<{ stdout: string; stderr: string }> {
const found = (await taggedInstances()).find(
(i) => i.instanceId === instanceId && i.machine === machine,
);
const name = found?.name ?? machineName(instanceId, machine);
return incus(["exec", name, "--", ...command], 120_000);
}
/**
* Capture the whole scenario as one state. Every machine, one name.
*
* **Every machine is flushed first, and that is not a precaution.** A snapshot of a running
* machine captures its disk, not its memory, so a write still sitting in the guest's page
* cache is simply not in the snapshot. Without the flush a file written seconds earlier can
* be absent after restore — not stale, absent.
*
* Found by the integration test on its first run, which is the question the design listed as
* open: *does a scenario snapshot need the machines stopped?* It does not, but it does need
* them flushed.
*
* This buys write-durability, not application-consistency. A database mid-transaction is
* still captured mid-transaction — the snapshot is crash-consistent, and anything needing
* more has to quiesce itself.
*/
export async function snapshot(instanceId: string, label: string): Promise<number> {
const machines = await machinesOf(instanceId);
const started = Date.now();
for (const name of machines) {
await incusOk(["exec", name, "--", "sync"], 60_000);
}
for (const name of machines) {
await incus(["snapshot", "create", name, label], 300_000);
}
return (Date.now() - started) / 1000;
}
export interface RestoreResult {
/** How long the restore itself took. */
restoreSeconds: number;
/** How long until the scenario was usable again — the number that matters. */
usableSeconds: number;
}
/**
* Return the whole scenario to a state. Restoring a subset would produce a mesh that never
* was, so this is all-or-nothing.
*
* Restoring a virtual machine replaces its disk and the machine comes back up, so it
* reports RUNNING while its agent is still starting — measured, the restore call returns
* in 0.79s and the very next command fails. Reporting that as "restored" would be
* transport reported as effect, so this waits for usable and returns both numbers.
*/
export async function restore(
instanceId: string,
label: string,
readyTimeoutSeconds = 180,
log: (message: string) => void = () => {},
): Promise<RestoreResult> {
const machines = await machinesOf(instanceId);
const started = Date.now();
for (const name of machines) {
await incus(["snapshot", "restore", name, label], 300_000);
}
const restoreSeconds = (Date.now() - started) / 1000;
for (const name of machines) {
await succeeds(["start", name], 60_000);
}
await waitUntilAllUsable(machines, readyTimeoutSeconds, log);
return { restoreSeconds, usableSeconds: (Date.now() - started) / 1000 };
}
export async function snapshots(instanceId: string): Promise<string[]> {
const machines = await machinesOf(instanceId);
const first = machines[0];
if (!first) return [];
const csv = (await incusOk(["snapshot", "list", first, "--format", "csv"])) ?? "";
return csv
.split("\n")
.filter(Boolean)
.map((line) => line.split(",")[0] ?? "")
.filter(Boolean);
}
/**
* Tear the instance down: machines first, then the links they were on.
*
* Networks are removed last and only if empty — a link still carrying an interface
* cannot be deleted, and forcing it would leave the daemon with a reference to something
* gone.
*/
export async function destroy(instanceId: string): Promise<{ machines: number; networks: number }> {
const machines = (await taggedInstances()).filter((i) => i.instanceId === instanceId);
for (const machine of machines) {
await succeeds(["delete", "--force", machine.name], 300_000);
}
// Links go last and only once nothing is attached: a network still carrying an
// interface cannot be deleted, and forcing it would leave a dangling reference.
let networks = 0;
for (const network of await taggedNetworks()) {
if (network.instanceId !== instanceId) continue;
// `network delete` also succeeds silently — counted with succeeds(), not truthiness.
if (await succeeds(["network", "delete", network.name], 30_000)) networks++;
}
return { machines: machines.length, networks };
}
src/lifecycle/place.ts
+156
View File
@@ -0,0 +1,156 @@
/**
* `place:` — putting something inside the machines.
*
* Until now the lab raised an underlay and put nothing on it: correct, and useless, because
* the thing it exists to test did not exist (novox/hq 03-DESIGN/00-as-is/11-the-lab.md). Tier
* 0 now does, so this is the seam where the lab acquires a consumer.
*
* Only `host` is placeable. Everything else in the placement vocabulary — the substrate, a
* control plane, a forge — is still refused by name rather than ignored, because a scenario
* that declares something and raises without it is the fault this lab was built to catch
* (novox/hq 04-ISSUES/003).
*/
import type { Scenario } from "../declaration/types.ts";
import { incus, incusOk } from "../incus/client.ts";
/** What this stage can put inside a machine. */
export const PLACEABLE = ["host"] as const;
export type Placeable = (typeof PLACEABLE)[number];
/** Where the host binary lives on a machine once placed. */
export const HOST_PATH = "/usr/local/bin/mesh-host";
export interface Placement {
machine: string;
artifacts: string[];
}
/**
* Resolve `place:` to one list per machine.
*
* `all:` applies to every machine; a per-machine entry **overrides** it rather than adding to
* it, which is what the declaration design says and is worth being exact about — a scenario
* naming one artifact for one machine gets that artifact and not that artifact plus the rest.
*/
export function planPlacements(scenario: Scenario): Placement[] {
const place = scenario.place;
if (!place) return [];
const all = place.all ?? [];
return Object.keys(scenario.machines).map((machine) => {
const own = place[machine];
return { machine, artifacts: own ?? all };
}).filter((p) => p.artifacts.length > 0);
}
/**
* The host binary to place, from the environment.
*
* Deliberately an explicit path rather than a search. The declaration design leaves *where
* `place:` gets its artifacts from* open — before the mesh is self-hosting they come from
* outside, afterwards from the mesh — and it suggests a named source rather than a path. This
* is neither: it is the smallest thing that works while that stays undecided, and it refuses
* loudly rather than guessing, so nothing here hardens into the answer by accident.
*/
export function hostBinaryPath(): string | null {
return process.env["MESH_LAB_HOST_BINARY"] ?? null;
}
export class PlacementError extends Error {
readonly machine: string;
constructor(machine: string, message: string) {
super(message);
this.name = "PlacementError";
this.machine = machine;
}
}
export interface PlacedHost {
machine: string;
/** What the host reported about the machine, read back from it. */
profile: unknown;
version: string;
}
/**
* Put the host on a machine and ask it what the machine is.
*
* The result is read back from the running binary, never assumed from the fact that the copy
* succeeded (novox/hq ADR 0035). A file arriving is not a host working, which is the same
* distinction the host itself makes about installed packages.
*/
export async function placeHost(
instanceName: string,
machine: string,
binary: string,
log: (message: string) => void = () => {},
): Promise<PlacedHost> {
await incus(["file", "push", binary, `${instanceName}${HOST_PATH}`, "--mode", "0755"], 180_000);
// Read back that it is there and executable before trusting it to answer questions.
const version = (await incusOk(["exec", instanceName, "--", HOST_PATH, "version"], 60_000))?.trim();
if (!version) {
throw new PlacementError(
machine,
`the host binary was copied to ${machine} but does not run there. A file arriving is ` +
`not a host working.`,
);
}
const reported = await incusOk(
["exec", instanceName, "--", HOST_PATH, "profile", "--json"],
120_000,
);
if (!reported) {
throw new PlacementError(
machine,
`the host runs on ${machine} (${version}) but reported no profile. A host that cannot ` +
`say what a machine is cannot be asked to change it.`,
);
}
let profile: unknown;
try {
profile = JSON.parse(reported);
} catch (err) {
throw new PlacementError(
machine,
`the host on ${machine} reported something that is not a profile: ${(err as Error).message}`,
);
}
log(` placed the host on ${machine} (${version})`);
return { machine, profile, version };
}
/** Place everything a scenario declares. */
export async function applyPlacements(
scenario: Scenario,
machineNames: Map<string, string>,
log: (message: string) => void = () => {},
): Promise<PlacedHost[]> {
const placements = planPlacements(scenario);
if (placements.length === 0) return [];
const binary = hostBinaryPath();
if (!binary) {
throw new Error(
`this scenario places the host, and no host binary was given. Set ` +
`MESH_LAB_HOST_BINARY to a built mesh-host. Guessing at a path would place ` +
`whatever happened to be there.`,
);
}
const placed: PlacedHost[] = [];
for (const { machine, artifacts } of placements) {
const name = machineNames.get(machine);
if (!name) continue;
for (const artifact of artifacts) {
if (artifact !== "host") continue; // refused earlier; belt and braces
placed.push(await placeHost(name, machine, binary, log));
}
}
return placed;
}
src/lifecycle/raise.ts
+243
View File
@@ -0,0 +1,243 @@
/**
* Materialise a declaration into a running scenario instance.
*
* Two rules from the design shape everything here.
*
* `raise` waits for the machines to be USABLE, not for the calls to return. Measured on a
* workstation those are 3.4s and 14.3s apart, and reporting the earlier number would be
* the mesh's own recurring failure — transport reported as effect.
*
* A failed raise LEAVES THE WRECKAGE. Tearing down on failure destroys the only evidence
* of what went wrong, and a scenario that failed to raise is more interesting than one
* that succeeded.
*
* See novox/hq 03-DESIGN/01-to-be/03-scenario-lifecycle.md
*/
import type { Scenario, Segment } from "../declaration/types.ts";
import { incus, incusOk, succeeds, pools, supportedDrivers } from "../incus/client.ts";
import { machineName, macFor, networkName, newInstanceId } from "./names.ts";
import { waitUntilAllUsable } from "./ready.ts";
import { applyAddresses, applyDefaultRoutes } from "./address.ts";
import { assertSupported } from "./supported.ts";
import { planRouters, raiseRouters, raiseTransit } from "./router.ts";
import { applyHostFirewalls } from "./firewall.ts";
import { applyPlacements } from "./place.ts";
/** Drivers whose snapshots are copy-on-write. On `dir` a snapshot is a full copy. */
const COW_DRIVERS = ["btrfs", "zfs"];
export interface RaiseOptions {
/** Base image for machines. */
image?: string;
/** Reuse an existing instance id rather than minting one — makes raise convergent. */
instanceId?: string;
/** Seconds to wait for each machine to become usable. */
readyTimeoutSeconds?: number;
onProgress?: (message: string) => void;
}
export interface RaisedScenario {
instanceId: string;
scenario: string;
machines: string[];
networks: string[];
pool: string;
}
export class RaiseError extends Error {
readonly instanceId: string;
readonly step: string;
constructor(instanceId: string, step: string, cause: unknown) {
const detail = cause instanceof Error ? cause.message : String(cause);
super(
`raise failed at '${step}': ${detail}\n` +
`The instance '${instanceId}' has been LEFT STANDING for inspection. ` +
`Destroy it with: mesh-lab destroy ${instanceId}`,
);
this.name = "RaiseError";
this.instanceId = instanceId;
this.step = step;
}
}
/**
* Pick a pool that can snapshot cheaply, and say so loudly when there is not one.
*
* Measured: a `dir` snapshot of a 1.5 GB machine takes 9.9s and a full 1.6 GB, with a
* second snapshot unfinished after two minutes. On copy-on-write it is 0.13s and costs
* the delta. Restoring is the operation the inner loop repeats most, so a `dir` pool does
* not make the lab slow — it makes it unused.
*/
async function choosePool(log: (m: string) => void): Promise<string> {
const available = await pools();
const cow = available.find((p) => COW_DRIVERS.includes(p.driver));
if (cow) return cow.name;
const drivers = await supportedDrivers();
const possible = drivers.filter((d) => COW_DRIVERS.includes(d));
log(
possible.length > 0
? `WARNING: no copy-on-write pool exists, though the daemon offers ${possible.join("/")}. ` +
`Snapshots will be full copies — roughly 76x slower, and the inner loop unusable.`
: `WARNING: the daemon offers no copy-on-write driver. Snapshots will be full copies — ` +
`roughly 76x slower, and the inner loop unusable. Install btrfs tooling and restart it.`,
);
const fallback = available[0];
if (!fallback) throw new Error("no storage pool exists at all");
return fallback.name;
}
/**
* One isolated link per segment. Nothing joins them to anything outside the instance, and
* incus is told not to hand out addresses: a scenario declares the underlay, and letting
* a hypervisor's DHCP assign addresses would be the lab supplying facts the declaration
* is supposed to own.
*/
async function createNetwork(
instanceId: string,
segment: string,
spec: Segment,
): Promise<string> {
const name = networkName(instanceId, segment);
if (await succeeds(["network", "show", name], 15_000)) return name;
await incus([
"network", "create", name,
"ipv4.address=none",
"ipv6.address=none",
"ipv4.nat=false",
"ipv6.nat=false",
`user.mesh-lab.instance=${instanceId}`,
`user.mesh-lab.segment=${segment}`,
// Whether a segment is public and which ranges it carries are facts a link cannot be
// asked for afterwards — incus knows only that it is an isolated bridge. Recorded here
// so anything reading a raised instance back reads what was applied, rather than
// re-opening the declaration and reporting the request as though it were the result.
`user.mesh-lab.kind=${spec.kind}`,
`user.mesh-lab.cidr=${spec.cidr.join(",")}`,
...(spec.mtu === undefined ? [] : [`user.mesh-lab.mtu=${spec.mtu}`]),
]);
return name;
}
async function createMachine(
instanceId: string,
machine: string,
attachments: { segment: string }[],
image: string,
pool: string,
): Promise<string> {
const name = machineName(instanceId, machine);
if (await succeeds(["config", "show", name], 15_000)) return name;
const args = [
"init", image, name,
"--vm",
"-s", pool,
// Arch images refuse to boot under secureboot with the shipped keys. Discovered by
// the first launch failing with exactly that message.
"-c", "security.secureboot=false",
"-c", "limits.memory=1GiB",
"-c", "limits.cpu=2",
"-c", `user.mesh-lab.instance=${instanceId}`,
"-c", `user.mesh-lab.machine=${machine}`,
];
await incus(args, 300_000);
// eth0 comes from the profile and points at the wrong network, so every attachment is
// explicit. A machine on no segment gets no interface at all — that is what detached is.
await succeeds(["config", "device", "remove", name, "eth0"], 15_000);
for (const [index, attachment] of attachments.entries()) {
await incus([
"config", "device", "add", name, `eth${index}`, "nic",
"nictype=bridged",
`parent=${networkName(instanceId, attachment.segment)}`,
// Explicit, because incus assigns one at runtime without recording it in the device
// config — so reading it back returns nothing, and the guest has no stable handle.
`hwaddr=${macFor(instanceId, machine, index)}`,
]);
}
return name;
}
export async function raise(
scenario: Scenario,
options: RaiseOptions = {},
): Promise<RaisedScenario> {
const log = options.onProgress ?? (() => {});
const image = options.image ?? "images:archlinux/current";
const readyTimeout = options.readyTimeoutSeconds ?? 180;
const instanceId = options.instanceId ?? newInstanceId(scenario.scenario, new Date());
// Refuse before spending a minute raising something that would silently lack half of
// what it declares. Deliberately outside the try: this is not a raise failure, nothing
// has been created, and there is no wreckage to leave standing.
assertSupported(scenario);
let step = "choosing a storage pool";
try {
const pool = await choosePool(log);
log(`instance ${instanceId} pool ${pool}`);
step = "creating segments";
const networks: string[] = [];
for (const [segment, spec] of Object.entries(scenario.segments)) {
networks.push(await createNetwork(instanceId, segment, spec));
log(` segment ${segment}`);
}
step = "creating machines";
const created: string[] = [];
const byMachine = new Map<string, string>();
for (const [machine, spec] of Object.entries(scenario.machines)) {
const attachments = spec.at === "detached" ? [] : spec.at;
const name = await createMachine(instanceId, machine, attachments, image, pool);
created.push(name);
byMachine.set(machine, name);
log(` machine ${machine}${spec.at === "detached" ? " (detached)" : ""}`);
}
step = "starting machines";
for (const name of created) {
await succeeds(["start", name], 60_000);
}
step = "waiting for machines to become usable";
await waitUntilAllUsable(created, readyTimeout, log);
step = "applying declared addresses";
await applyAddresses(scenario, instanceId, byMachine, log);
// Transit first: a gateway's default route points at it, so it has to exist.
step = "wiring the public networks together";
const transit = await raiseTransit(scenario, instanceId, log);
step = "raising routers";
const routers = await raiseRouters(scenario, instanceId, planRouters(scenario, instanceId), log);
if (transit) routers.push(transit);
step = "routing machines through their gateways";
await applyDefaultRoutes(scenario, byMachine, log);
// Last: a machine that refuses inbound must still have been reachable while the lab
// was configuring it.
step = "applying host firewalls";
await applyHostFirewalls(scenario, byMachine, log);
// Last, and only once the underlay is real. Placing before the machines can reach each
// other would test the host against a network the scenario does not describe.
step = "placing";
await applyPlacements(scenario, byMachine, log);
return {
instanceId,
scenario: scenario.scenario,
machines: [...created, ...routers],
networks,
pool,
};
} catch (cause) {
throw new RaiseError(instanceId, step, cause);
}
}
src/lifecycle/ready.ts
+56
View File
@@ -0,0 +1,56 @@
/**
* "Usable" means a command runs on the machine. Anything weaker is transport reported as
* effect — the mesh's own recurring fault, and one this lab exists to catch rather than
* commit.
*
* Two measurements make the case. Raising: the launch call returns in 3.4s and the machine
* is usable at 14.3s. Restoring: the call returns in 0.79s and the machine is RUNNING
* immediately — with its agent still starting, so the very next command fails.
*
* Both verbs therefore wait for the same thing, using the same code.
*/
import { succeeds } from "../incus/client.ts";
export interface ReadyResult {
name: string;
seconds: number;
}
export async function waitUntilUsable(
name: string,
timeoutSeconds: number,
log: (message: string) => void = () => {},
): Promise<ReadyResult> {
const started = Date.now();
const deadline = started + timeoutSeconds * 1000;
while (Date.now() < deadline) {
// `exec … true` succeeds with EMPTY output, so this asks whether it worked rather than
// what it said. Truthiness-testing the output reported every machine as unreachable
// while `incus exec` on it worked perfectly.
if (await succeeds(["exec", name, "--", "true"], 10_000)) {
const seconds = (Date.now() - started) / 1000;
log(` ${name} usable after ${seconds.toFixed(1)}s`);
return { name, seconds };
}
await new Promise((resolve) => setTimeout(resolve, 1000));
}
throw new Error(
`${name} did not become usable within ${timeoutSeconds}s — it may be running but ` +
`unreachable, which is not the same as ready`,
);
}
export async function waitUntilAllUsable(
names: string[],
timeoutSeconds: number,
log: (message: string) => void = () => {},
): Promise<number> {
const started = Date.now();
// Concurrently: a scenario's machines boot independently, and waiting for them in turn
// would make a four-machine scenario four boots long instead of one.
await Promise.all(names.map((name) => waitUntilUsable(name, timeoutSeconds, log)));
return (Date.now() - started) / 1000;
}
src/lifecycle/router.ts
+517
View File
@@ -0,0 +1,517 @@
/**
* Materialise the routers a declaration implies.
*
* A gateway is the one implicit machine in an otherwise explicit declaration — a scenario
* says a segment sits behind one and never names the thing that serves it, because it has
* nothing to say about it.
*
* A router is **scenery, not a node**, so it is a container rather than a virtual machine
* (novox/hq ADR 0033). Nothing under test runs on it and no assertion is made about its
* internals; it exists so packets behave the way they behave in the world. What it has to
* reproduce is kernel behaviour, and a container has the same kernel.
*/
import type { Family, Scenario } from "../declaration/types.ts";
import { incus, succeeds } from "../incus/client.ts";
import { macFor, networkName } from "./names.ts";
import { waitUntilUsable } from "./ready.ts";
/**
* The router image, built once and cached.
*
* A scenario is a closed address space, so a router has no route to a package repository —
* installing nftables at raise time cannot work, and the first attempt failed exactly that
* way. So the image is prepared once, with temporary connectivity, and every scenario
* afterwards raises from it needing no network at all.
*
* That is the same property the mesh's own artifacts have: what ships is self-contained,
* and a deploy touches no network.
*/
const ROUTER_IMAGE = "mesh-lab-router";
const ROUTER_BASE = "images:alpine/edge";
/** Wait until the container can actually resolve and fetch — not merely run a command. */
async function waitForNetwork(name: string, timeoutSeconds: number): Promise<void> {
const deadline = Date.now() + timeoutSeconds * 1000;
let lastError = "no attempt made";
while (Date.now() < deadline) {
try {
await incus(["exec", name, "--", "apk", "update"], 30_000);
return;
} catch (err) {
lastError = err instanceof Error ? err.message.split("\n")[0] ?? "" : String(err);
}
await new Promise((resolve) => setTimeout(resolve, 2000));
}
throw new Error(
`${name} had no working network after ${timeoutSeconds}s — the router image cannot be ` +
`built without one. Last error: ${lastError}`,
);
}
/**
* Build the router image if it is missing. One-time, and the only step in the whole lab that
* needs the workstation to be online.
*/
export async function ensureRouterImage(log: (message: string) => void = () => {}): Promise<void> {
if (await succeeds(["image", "info", ROUTER_IMAGE], 20_000)) return;
log(` building the router image (once) — installing nftables into ${ROUTER_BASE}`);
const builder = "mlab-router-build";
await succeeds(["delete", "--force", builder], 60_000);
// Default profile on purpose: this is the one container that needs to reach a repository.
await incus(["launch", ROUTER_BASE, builder], 300_000);
await waitUntilUsable(builder, 120, () => {});
// `exec` works before the container has an address. Usable means a command runs; it does
// not mean the network is up, and the first attempt failed on DNS because those were
// treated as the same thing. Wait for the thing actually needed.
await waitForNetwork(builder, 60);
// Not swallowed. A router without nftables is a router that silently does not route, and
// an earlier attempt shipped exactly that because the failure was hidden behind `|| true`.
await incus(["exec", builder, "--", "apk", "add", "--no-cache", "--update", "nftables"], 180_000);
await incus(["exec", builder, "--", "sh", "-c", "command -v nft"], 20_000);
// The stock image ships `auto eth0 / iface eth0 inet dhcp`, and its boot-time networking
// service acts on it — flushing the static address the scenario just set, on eth0 only,
// which is why the outside interface came up bare while the inside ones were fine.
//
// A scenario declares the underlay; a router that reconfigures itself from an image
// default is the lab overriding the declaration.
await incus([
"exec", builder, "--", "sh", "-c",
"printf 'auto lo\\niface lo inet loopback\\n' > /etc/network/interfaces",
], 30_000);
await incus(["stop", builder], 120_000);
await incus(["publish", builder, "--alias", ROUTER_IMAGE], 300_000);
await succeeds(["delete", "--force", builder], 60_000);
log(` router image ready`);
}
/**
* The address the transit router holds on a public segment: the last usable host address.
*
* Chosen rather than declared, like a gateway's inside address — a scenario has nothing to
* say about the internet's own routers, only about the networks they connect.
*/
export function transitAddress(cidr: string): string | null {
const slash = cidr.lastIndexOf("/");
if (slash === -1) return null;
const base = cidr.slice(0, slash);
const prefix = cidr.slice(slash);
if (base.includes(":")) return `${base.replace(/::$/, "")}::fffe${prefix}`;
const octets = base.split(".");
octets[3] = "254";
return `${octets.join(".")}${prefix}`;
}
/** The name a router answers to in `list` and `exec` — scenery, but addressable. */
export function routerMachineName(plan: RouterPlan): string {
return `gw-${plan.inside.join("-")}`;
}
export interface RouterPlan {
/** Router name, one per distinct gateway. */
name: string;
/** The segment(s) behind this router. Several share one when they share a gateway. */
inside: string[];
/** The segment this router reaches out through. */
outside: string;
/** Addresses this router holds on the outside segment — what the world sees. */
outsideAddresses: string[];
nat: Family[];
forwardable: boolean;
mappingTtl: string | undefined;
}
/**
* Group segments by the gateway they declare. Identical gateway declarations mean ONE
* router, not several — that is what a VLAN-capable router is, and two routers sharing an
* external address would not work anyway.
*/
/**
* Gateways that share an address are ONE gateway.
*
* Grouping on the exact address list instead split a household in two: `home` declaring a
* v4 and a v6 address and `devices` declaring only the v4 produced two router containers,
* both holding the same v4 address on the same segment. The lab raised it, and the shared
* address resolved to whichever container answered ARP last — so a published port worked or
* did not, run to run, with nothing reporting a fault.
*
* One public address is one box. Checked against the real thing this models: a bridged
* modem, a single gateway holding the public address, everything behind it on one network.
* Two routers on one address is not a topology, it is a collision.
*/
export function planRouters(scenario: Scenario, instanceId: string): RouterPlan[] {
const plans: RouterPlan[] = [];
for (const [segmentName, segment] of Object.entries(scenario.segments)) {
const gateway = segment.gateway;
if (!gateway) continue;
const existing = plans.find(
(plan) =>
plan.outside === gateway.to &&
plan.outsideAddresses.some((address) => gateway.address.includes(address)),
);
if (existing) {
existing.inside.push(segmentName);
// The union, so a gateway declared with a v6 address on only one of the segments it
// serves still carries it. The declarations must otherwise agree — validate refuses
// the case where they do not, so there is nothing to reconcile here.
for (const address of gateway.address) {
if (!existing.outsideAddresses.includes(address)) existing.outsideAddresses.push(address);
}
continue;
}
plans.push({
name: `mlab-${instanceId}-gw${plans.length}`,
inside: [segmentName],
outside: gateway.to,
outsideAddresses: [...gateway.address],
nat: gateway.nat,
forwardable: gateway.forwardable,
mappingTtl: gateway.mappingTtl,
});
}
return plans;
}
/** "120s" / "2m" / "90" → seconds. */
export function ttlSeconds(text: string | undefined): number | undefined {
if (!text) return undefined;
const match = /^(\d+)\s*([smh]?)$/.exec(text.trim());
if (!match) return undefined;
const value = Number(match[1]);
return match[2] === "m" ? value * 60 : match[2] === "h" ? value * 3600 : value;
}
function withPrefix(scenario: Scenario, segment: string, address: string): string {
const wantV6 = address.includes(":");
for (const range of scenario.segments[segment]?.cidr ?? []) {
const slash = range.lastIndexOf("/");
if (slash === -1) continue;
if (range.slice(0, slash).includes(":") === wantV6) return `${address}${range.slice(slash)}`;
}
return address;
}
/** The address a machine holds on a segment, for DNAT targets and as an inside gateway. */
function addressOn(scenario: Scenario, machine: string, segment: string, family: Family): string | null {
const spec = scenario.machines[machine];
if (!spec || spec.at === "detached") return null;
for (const attachment of spec.at) {
if (attachment.segment !== segment) continue;
for (const address of attachment.address) {
if ((family === "v6") === address.includes(":")) return address;
}
}
return null;
}
/**
* The router's own address on an inside segment: the first host address of that range.
*
* Chosen rather than declared because a scenario has nothing to say about it — the
* declaration describes what the world sees the network as, and the inside address is an
* implementation detail of the machine serving it.
*/
function insideAddress(scenario: Scenario, segment: string, family: Family): string | null {
for (const range of scenario.segments[segment]?.cidr ?? []) {
const slash = range.lastIndexOf("/");
if (slash === -1) continue;
const base = range.slice(0, slash);
const isV6 = base.includes(":");
if (isV6 !== (family === "v6")) continue;
if (isV6) return `${base.replace(/::$/, "::")}1${range.slice(slash)}`.replace("::1/", "::1/");
const octets = base.split(".");
octets[3] = "1";
return `${octets.join(".")}${range.slice(slash)}`;
}
return null;
}
/**
* Wire the public segments together.
*
* The internet is not a network — it is unrelated networks that route to each other, many
* hops apart with no shared broadcast domain. So public segments are separate links joined
* by a router, never bridged: bridging them would make ARP adjacency, non-decrementing TTL
* and crossing multicast true in the lab and false in production, and the mesh has already
* been bitten by multicast name resolution.
*
* One transit router, an interface on every public segment, forwarding and no translation.
* It is the closest thing the lab has to "the internet", and it is deliberately dumb.
*/
export async function raiseTransit(
scenario: Scenario,
instanceId: string,
log: (message: string) => void = () => {},
): Promise<string | null> {
const publicSegments = Object.entries(scenario.segments)
.filter(([, segment]) => segment.kind === "public")
.map(([name]) => name);
// One public network needs no transit: everything on it is already adjacent.
if (publicSegments.length < 2) return null;
const name = `mlab-${instanceId}-transit`;
if (!(await succeeds(["config", "show", name], 15_000))) {
await incus([
"init", ROUTER_IMAGE, name,
"-c", `user.mesh-lab.instance=${instanceId}`,
"-c", "user.mesh-lab.machine=transit",
"-c", `user.mesh-lab.transit=${publicSegments.join(",")}`,
], 300_000);
await succeeds(["config", "device", "remove", name, "eth0"], 15_000);
for (const [index, segment] of publicSegments.entries()) {
await incus([
"config", "device", "add", name, `eth${index}`, "nic",
"nictype=bridged",
`parent=${networkName(instanceId, segment)}`,
`hwaddr=${macFor(instanceId, "transit", index)}`,
]);
}
}
await succeeds(["start", name], 60_000);
await waitUntilUsable(name, 120, () => {});
for (const [index, segment] of publicSegments.entries()) {
const device = `eth${index}`;
await sh(name, `ip link set ${device} up`);
for (const cidr of scenario.segments[segment]?.cidr ?? []) {
const address = transitAddress(cidr);
if (address) await sh(name, `ip addr replace ${address} dev ${device}`);
}
const mtu = scenario.segments[segment]?.mtu;
if (mtu) await sh(name, `ip link set ${device} mtu ${mtu}`);
}
await sh(
name,
"sysctl -w net.ipv4.ip_forward=1 >/dev/null; sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null",
);
log(` transit router across ${publicSegments.join(", ")}`);
return name;
}
export async function raiseRouters(
scenario: Scenario,
instanceId: string,
plans: RouterPlan[],
log: (message: string) => void = () => {},
): Promise<string[]> {
const created: string[] = [];
if (plans.length > 0) await ensureRouterImage(log);
for (const plan of plans) {
if (!(await succeeds(["config", "show", plan.name], 15_000))) {
await incus([
"init", ROUTER_IMAGE, plan.name,
"-c", `user.mesh-lab.instance=${instanceId}`,
// Tagged as a machine as well as a router: destroy finds an instance's resources
// with one query, and a router that only carried `router=` was left behind — which
// then held its networks open, so `destroy` reported removing zero segments.
"-c", `user.mesh-lab.machine=${routerMachineName(plan)}`,
"-c", `user.mesh-lab.router=${plan.inside.join(",")}`,
// `outside` is structural — it is which link eth0 is on, true the moment the device
// is added. The gateway's *behaviour* is not recorded here; see configureRouter.
"-c", `user.mesh-lab.outside=${plan.outside}`,
], 300_000);
await succeeds(["config", "device", "remove", plan.name, "eth0"], 15_000);
// eth0 faces outward, then one interface per segment behind it.
const links = [plan.outside, ...plan.inside];
for (const [index, segment] of links.entries()) {
await incus([
"config", "device", "add", plan.name, `eth${index}`, "nic",
"nictype=bridged",
`parent=${networkName(instanceId, segment)}`,
`hwaddr=${macFor(instanceId, `gw-${plan.name}`, index)}`,
]);
}
}
await succeeds(["start", plan.name], 60_000);
created.push(plan.name);
log(` router ${plan.inside.join("+")} → ${plan.outside}`);
}
for (const name of created) {
await waitUntilUsable(name, 120, () => {});
}
for (const plan of plans) {
await configureRouter(scenario, plan, log);
}
return created;
}
async function sh(name: string, script: string, timeoutMs = 60_000): Promise<void> {
await incus(["exec", name, "--", "sh", "-c", script], timeoutMs);
}
async function configureRouter(
scenario: Scenario,
plan: RouterPlan,
log: (message: string) => void,
): Promise<void> {
// Addresses: eth0 outside, then one per inside segment.
const links: { device: string; segment: string; addresses: string[] }[] = [
{
device: "eth0",
segment: plan.outside,
addresses: plan.outsideAddresses.map((a) => withPrefix(scenario, plan.outside, a)),
},
];
for (const [index, segment] of plan.inside.entries()) {
const addresses: string[] = [];
for (const family of ["v4", "v6"] as Family[]) {
const address = insideAddress(scenario, segment, family);
if (address) addresses.push(address);
}
links.push({ device: `eth${index + 1}`, segment, addresses });
}
for (const link of links) {
// Up first: an address on a down interface is accepted and then not used.
await sh(plan.name, `ip link set ${link.device} up`);
for (const address of link.addresses) {
// `replace` rather than `add`, so re-running is safe and a real failure still fails.
await sh(plan.name, `ip addr replace ${address} dev ${link.device}`);
}
const mtu = scenario.segments[link.segment]?.mtu;
if (mtu) await sh(plan.name, `ip link set ${link.device} mtu ${mtu}`);
}
await sh(
plan.name,
"sysctl -w net.ipv4.ip_forward=1 >/dev/null; sysctl -w net.ipv6.conf.all.forwarding=1 >/dev/null",
);
// A gateway reaches other public networks the way anything does: through transit. Without
// this it can only reach its own outside segment, and every scenario with more than one
// public network becomes a set of islands.
for (const cidr of scenario.segments[plan.outside]?.cidr ?? []) {
const via = transitAddress(cidr);
if (!via) continue;
const gateway = via.slice(0, via.lastIndexOf("/"));
const family = gateway.includes(":") ? "-6" : "-4";
await sh(plan.name, `ip ${family} route replace default via ${gateway} dev eth0 2>/dev/null || true`);
}
const ttl = ttlSeconds(plan.mappingTtl);
if (ttl !== undefined) {
// What makes keepalive behaviour testable rather than hoped for: a connection held
// through NAT without refreshing dies when the mapping does.
//
// Read back rather than assumed. These sysctls are not present on every kernel, and a
// scenario that declared an expiring mapping and silently got a permanent one would be
// the fault this lab exists to catch.
await sh(
plan.name,
`sysctl -w net.netfilter.nf_conntrack_udp_timeout=${ttl} >/dev/null 2>&1; ` +
`sysctl -w net.netfilter.nf_conntrack_tcp_timeout_established=${ttl} >/dev/null 2>&1; true`,
);
const readback = await incus(
["exec", plan.name, "--", "sh", "-c",
"cat /proc/sys/net/netfilter/nf_conntrack_udp_timeout 2>/dev/null || echo missing"],
20_000,
);
if (readback.stdout.trim() !== String(ttl)) {
throw new Error(
`${plan.name}: mapping_ttl of ${plan.mappingTtl} was declared but conntrack reports ` +
`'${readback.stdout.trim()}'. The scenario would silently have permanent mappings.`,
);
}
}
await applyRules(scenario, plan);
// Recorded last, and only here. Everything above either read itself back or threw, so a
// gateway carrying these tags is one that demonstrably does these things. Written at
// `init` they would have been a restatement of the request — and a raise that failed
// half way leaves its wreckage standing on purpose, so a picture of that wreckage would
// have badged translation the router was never configured to do.
const recorded = [
`user.mesh-lab.nat=${plan.nat.join(",")}`,
`user.mesh-lab.forwardable=${plan.forwardable}`,
...(ttl === undefined ? [] : [`user.mesh-lab.mapping-ttl=${ttl}`]),
];
for (const entry of recorded) {
const at = entry.indexOf("=");
await succeeds(["config", "set", plan.name, entry.slice(0, at), entry.slice(at + 1)], 20_000);
}
log(` ${plan.name}: nat=${plan.nat.join(",") || "none"} forwardable=${plan.forwardable}${ttl ? ` ttl=${ttl}s` : ""}`);
}
/** One ruleset per router, written whole — partial rule edits drift, a whole file does not. */
async function applyRules(scenario: Scenario, plan: RouterPlan): Promise<void> {
const parts: string[] = ["flush ruleset"];
for (const family of plan.nat) {
const table = family === "v4" ? "ip" : "ip6";
parts.push(
`table ${table} nat {`,
` chain postrouting { type nat hook postrouting priority srcnat; policy accept;`,
` oifname "eth0" masquerade`,
` }`,
` chain prerouting { type nat hook prerouting priority dstnat; policy accept;`,
);
if (plan.forwardable) {
for (const [machine, spec] of Object.entries(scenario.machines)) {
for (const publication of spec.published ?? []) {
if (!plan.inside.includes(publication.on)) continue;
const target = addressOn(scenario, machine, publication.on, family);
if (!target) continue;
const destination = family === "v6" ? `[${target}]` : target;
parts.push(
` iifname "eth0" tcp dport ${publication.port} dnat to ${destination}:${publication.port}`,
);
}
}
}
parts.push(` }`, `}`);
}
// Filtering: unsolicited inbound, and policy between segments the router serves.
const filterLines: string[] = [];
if (!plan.forwardable) {
// A gateway you do not control: outbound works, nothing initiates inward. That is the
// constraint being reproduced, not an implementation limit.
filterLines.push(` iifname "eth0" ct state new drop`);
}
for (const rule of scenario.policy ?? []) {
if (rule.allow) continue;
const fromIndex = plan.inside.indexOf(rule.from);
const toIndex = plan.inside.indexOf(rule.to);
if (fromIndex === -1 || toIndex === -1) continue;
filterLines.push(` iifname "eth${fromIndex + 1}" oifname "eth${toIndex + 1}" drop`);
}
if (filterLines.length > 0) {
parts.push(
`table inet filter {`,
` chain forward { type filter hook forward priority filter; policy accept;`,
` ct state established,related accept`,
...filterLines,
` }`,
`}`,
);
}
const ruleset = parts.join("\n");
await sh(
plan.name,
`cat > /tmp/mlab.nft <<'MLABNFT'\n${ruleset}\nMLABNFT\nnft -f /tmp/mlab.nft`,
60_000,
);
}
src/lifecycle/supported.ts
+54
View File
@@ -0,0 +1,54 @@
/**
* What this lab can actually materialise, and a refusal for everything else.
*
* A declaration the runtime silently ignores is the worst thing this project could ship.
* The mesh already has that fault catalogued: a firewall key declared in five manifests
* and read by no code, so a manifest appears to restrict a port and restricts nothing
* (novox/hq 04-ISSUES/003). A scenario that declares a gateway and raises without one
* would be the same fault, in the tool built to catch it.
*
* So unimplemented parts of the model are refused loudly at raise time rather than
* ignored. The declaration schema deliberately runs ahead of the runtime — it is the
* design's shape, and validating against it is useful before any of it can be raised —
* but the gap between the two has to be visible.
*/
import type { Scenario } from "../declaration/types.ts";
import { PLACEABLE, planPlacements } from "./place.ts";
export class UnsupportedError extends Error {
readonly missing: string[];
constructor(missing: string[]) {
super(
`this scenario declares things the lab cannot yet materialise:\n - ${missing.join("\n - ")}\n\n` +
`Raising it would produce a mesh that silently lacks them, which is the exact fault ` +
`this lab exists to catch. Validation accepts them because the declaration model is ` +
`complete; the runtime is not.`,
);
this.name = "UnsupportedError";
this.missing = missing;
}
}
export function assertSupported(scenario: Scenario): void {
const missing: string[] = [];
// `place: [host]` works. Everything else in the vocabulary is named individually rather
// than refused as a whole, so a scenario that places a host and a substrate is told exactly
// which half the lab cannot do.
const unplaceable = new Set<string>();
for (const { artifacts } of planPlacements(scenario)) {
for (const artifact of artifacts) {
if (!(PLACEABLE as readonly string[]).includes(artifact)) unplaceable.add(artifact);
}
}
for (const artifact of [...unplaceable].sort()) {
missing.push(
`place: ${artifact} — only ${PLACEABLE.join(", ")} can be placed; the tiers above ` +
`tier 0 do not exist yet`,
);
}
if (missing.length > 0) throw new UnsupportedError(missing);
}
test/diagram.test.ts
+307
View File
@@ -0,0 +1,307 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { loadScenario } from "../src/declaration/parse.ts";
import { diagramFromDeclaration } from "../src/diagram/from-declaration.ts";
import { toDrawio } from "../src/diagram/drawio.ts";
/**
* A generated diagram that will not open is worse than no diagram — it looks like a
* deliverable and is not one. The first version was unparseable because HTML labels were
* concatenated raw into an XML attribute, so these assert the file itself.
*/
function parseCells(xml: string): { id: string; vertex: boolean; edge: boolean; source?: string; target?: string }[] {
const cells: ReturnType<typeof parseCells> = [];
// <object> wrappers carry the id for any cell with a tooltip; the mxCell inside has none.
for (const match of xml.matchAll(/<object ([^>]*?)>/g)) {
const id = /id="([^"]*)"/.exec(match[1] ?? "")?.[1];
if (id) cells.push({ id, vertex: true, edge: false });
}
for (const match of xml.matchAll(/<mxCell ([^>]*?)(?:\/>|>)/g)) {
const attrs = match[1] ?? "";
const get = (name: string) => new RegExp(`${name}="([^"]*)"`).exec(attrs)?.[1];
const id = get("id");
if (!id) continue;
const entry: (typeof cells)[number] = {
id,
vertex: get("vertex") === "1",
edge: get("edge") === "1",
};
const source = get("source");
const target = get("target");
if (source) entry.source = source;
if (target) entry.target = target;
cells.push(entry);
}
return cells;
}
const scenario = loadScenario("scenarios/the-ordinary-shape.yml");
const xml = toDrawio(diagramFromDeclaration(scenario));
test("the file is well-formed XML — raw markup in an attribute is not", () => {
// No unescaped angle bracket may appear inside a value="..." attribute.
for (const match of xml.matchAll(/(?:value|label|tooltip)="([^"]*)"/g)) {
assert.doesNotMatch(match[1] ?? "", /[<>]/, "a label carries raw markup into an attribute");
}
assert.match(xml, /^<mxfile /);
assert.match(xml, /<\/mxfile>\s*$/);
});
test("every edge connects two cells that exist", () => {
const cells = parseCells(xml);
const ids = new Set(cells.map((c) => c.id));
for (const edge of cells.filter((c) => c.edge)) {
assert.ok(ids.has(edge.source ?? ""), `edge ${edge.id} has no source`);
assert.ok(ids.has(edge.target ?? ""), `edge ${edge.id} has no target`);
}
});
test("the diagram draws the implicit routers, not only what is written down", () => {
// A scenario never names its gateways. A picture that showed only declared machines
// would omit every node carrying NAT, forwarding and expiry.
const diagram = diagramFromDeclaration(scenario);
assert.ok(diagram.machines.some((m) => m.kind === "router"), "no router drawn");
assert.ok(diagram.machines.some((m) => m.kind === "transit"), "no transit drawn");
});
test("a router's badges say what the declaration decided", () => {
const diagram = diagramFromDeclaration(scenario);
const unforwardable = diagram.machines.find((m) => m.notes.some((n) => n.includes("NOT forwardable")));
assert.ok(unforwardable, "the unforwardable gateway is not marked as such");
assert.ok(
diagram.machines.some((m) => m.notes.some((n) => n.includes("mappings expire"))),
"a declared mapping expiry is not shown",
);
});
test("a metadata fact becomes a badge, and absence of the fact does not", () => {
// The point of the badges: the properties worth seeing are the ones with no visual
// consequence. A translated address looks exactly like an untranslated one.
for (const tip of ["translates v4", "no port forwarding", "mappings expire"]) {
assert.ok(xml.includes(tip), `no badge explains '${tip}'`);
}
// A gateway that does not translate gets no mark, rather than a struck-through one.
assert.doesNotMatch(xml, /label="N"[^>]*tooltip="[^"]*no NAT/);
});
test("every badge says in words what its letter means", () => {
// A one-letter code with no tooltip is a private language. Each badge is wrapped in an
// <object>, which is the only place draw.io reads a tooltip from.
const badges = [...xml.matchAll(/<object [^>]*label="([A-Z▶■]{1,2})"[^>]*tooltip="([^"]*)"/g)];
assert.ok(badges.length > 0, "no badges rendered at all");
for (const [, code, tip] of badges) {
assert.ok((tip ?? "").length > 10, `badge ${code} has no explanation`);
}
});
test("no two cells share an id — a machine may be named after a gateway", () => {
const ids = parseCells(xml).map((c) => c.id);
assert.equal(new Set(ids).size, ids.length, "duplicate cell id");
});
test("segments are ordered public first, then by depth behind them", () => {
const diagram = diagramFromDeclaration(scenario);
const publicDepths = diagram.segments.filter((s) => s.kind === "public").map((s) => s.depth);
assert.deepEqual([...new Set(publicDepths)], [0], "a public segment should be at depth 0");
const home = diagram.segments.find((s) => s.name === "home");
assert.equal(home?.depth, 1, "a segment behind one gateway is at depth 1");
});
test("a declared address appears on the machine that holds it", () => {
assert.match(xml, /192\.168\.1\.135/);
assert.match(xml, /198\.51\.100\.7/);
});
test("every shape names a stencil that exists", () => {
// A style naming a stencil draw.io does not have renders as an empty box — no error, no
// warning, just a missing picture. Checked against the names in draw.io's own
// stencils/networks.xml, which is where mxgraph.networks.* is defined.
const KNOWN = new Set([
"mxgraph.networks.server",
"mxgraph.networks.router",
"mxgraph.networks.cloud",
"mxgraph.networks.firewall",
"mxgraph.networks.switch",
"mxgraph.networks.pc",
"mxgraph.networks.laptop",
"mxgraph.networks.storage",
"mxgraph.networks.modem",
"mxgraph.networks.mainframe",
]);
const used = new Set([...xml.matchAll(/shape=([a-z0-9_.]+)/g)].map((m) => m[1] as string));
assert.ok(used.size > 0, "no stencil shapes used at all");
for (const shape of used) {
assert.ok(KNOWN.has(shape), `'${shape}' is not a stencil draw.io ships`);
}
});
test("a resource's shape is fixed by kind, and never varies with its metadata", () => {
// The split the whole design rests on: shape says what a thing is, badges say what is
// true about it. A gateway that stops translating must still look like a gateway.
const routers = [...xml.matchAll(/shape=mxgraph\.networks\.router/g)].length;
const diagram = diagramFromDeclaration(scenario);
assert.equal(routers, diagram.machines.filter((m) => m.kind === "router").length);
assert.equal(
[...xml.matchAll(/shape=mxgraph\.networks\.server/g)].length,
diagram.machines.filter((m) => m.kind === "machine").length,
);
});
test("no link crosses a network it does not touch", () => {
// The layout fault that made the first drawings unreadable: a gateway placed below the
// lane it FACES, with its link to the outside running the height of the picture through
// three networks it has nothing to do with — and overlapping another such link, so the
// two read as one wire. Every link must now be short and local.
const geometry = new Map<string, { y: number; h: number }>();
for (const match of xml.matchAll(
/<(?:mxCell|object)[^>]*id="([^"]*)"[\s\S]{0,400}?<mxGeometry x="[^"]*" y="([^"]*)" width="[^"]*" height="([^"]*)"/g,
)) {
geometry.set(match[1] as string, { y: Number(match[2]), h: Number(match[3]) });
}
const lanes = [...geometry.entries()]
.filter(([id]) => id.startsWith("lane-"))
.map(([id, g]) => ({ name: id.slice(5), top: g.y, bottom: g.y + g.h }));
assert.ok(lanes.length > 0, "no lanes found");
for (const edge of xml.matchAll(/<mxCell id="([^"]*)"[^>]*edge="1"[^>]*source="([^"]*)" target="([^"]*)"/g)) {
const [, , source, target] = edge;
const from = geometry.get(source as string);
const lane = lanes.find((l) => `lane-${l.name}` === target);
if (!from || !lane) continue;
const span = { top: Math.min(from.y, lane.top), bottom: Math.max(from.y + from.h, lane.bottom) };
const attached = new Set([target, source]);
for (const other of lanes) {
if (attached.has(`lane-${other.name}`)) continue;
const crosses = other.top >= span.top && other.bottom <= span.bottom;
assert.ok(!crosses, `link ${source}→${target} crosses '${other.name}'`);
}
}
});
test("a network behind another is drawn inside it, not merely below it", () => {
// "Behind" is shown by indentation. Without it the reader has only a wire to follow, and
// in a layered scenario that wire is exactly what became unreadable.
const laneX = new Map<string, number>();
for (const match of xml.matchAll(
/<mxCell id="lane-([^"]*)"[\s\S]{0,400}?<mxGeometry x="([^"]*)"/g,
)) {
laneX.set(match[1] as string, Number(match[2]));
}
const diagram = diagramFromDeclaration(scenario);
for (const segment of diagram.segments) {
if (!segment.behind) continue;
const mine = laneX.get(segment.name);
const parent = laneX.get(segment.behind);
assert.ok(mine !== undefined && parent !== undefined, `${segment.name} or its parent is missing`);
assert.ok(mine > parent, `${segment.name} is not indented inside ${segment.behind}`);
}
});
test("a gateway sits immediately above the network it serves", () => {
// What the grouping guarantees, and the whole reason a gateway reads as the door into a
// network rather than a box floating near one.
//
// It does NOT guarantee adjacency to the network the gateway FACES: a public network with
// two private networks behind it can only put one of them next to it. That case is carried
// by the lane's own "behind …" and by both children being indented to the same depth —
// and the link that would otherwise cross the sibling is suppressed, which is what the
// crossing test above asserts.
const diagram = diagramFromDeclaration(scenario);
const order = [...xml.matchAll(/<mxCell id="lane-([^"]*)"/g)].map((m) => m[1] as string);
let checked = 0;
for (const machine of diagram.machines) {
if (machine.kind !== "router") continue;
const served = machine.attachments.slice(1).map((a) => a.segment);
const top = served.reduce((b, n) => (order.indexOf(n) < order.indexOf(b) ? n : b), served[0] ?? "");
assert.ok(order.includes(top), `${machine.name} serves '${top}', which is not drawn`);
checked++;
}
assert.ok(checked > 0, "no gateways to check");
});
test("a public network is followed by everything behind it, before the next public one", () => {
// The ordering rule itself. Sorting by depth alone put a private network far from the
// public one it sits behind, which is what made the links long in the first place.
const diagram = diagramFromDeclaration(scenario);
const order = [...xml.matchAll(/<mxCell id="lane-([^"]*)"/g)].map((m) => m[1] as string);
const rootOf = (name: string): string => {
let current = name;
const seen = new Set<string>();
for (;;) {
const segment = diagram.segments.find((s) => s.name === current);
if (!segment?.behind || seen.has(current)) return current;
seen.add(current);
current = segment.behind;
}
};
// Reading down the page, the root never returns to one already left behind.
const finished = new Set<string>();
let previous = "";
for (const name of order) {
const root = rootOf(name);
if (root !== previous) {
assert.ok(!finished.has(root), `'${root}' is split apart by another group`);
if (previous) finished.add(previous);
previous = root;
}
}
});
test("both sources lay the same topology out identically", () => {
// The comparison is the feature. The hypervisor cannot know declaration order, so laying
// out by it would give the two pictures different shapes and nothing could be read off
// the difference. Proved by shuffling the segments and checking the layout does not move.
const shuffled = {
...diagramFromDeclaration(scenario),
segments: [...diagramFromDeclaration(scenario).segments].reverse(),
};
const laneOrder = (out: string) => [...out.matchAll(/<mxCell id="lane-([^"]*)"/g)].map((m) => m[1]);
assert.deepEqual(laneOrder(toDrawio(shuffled)), laneOrder(xml));
});
const ALL_SCENARIOS = [
"bootstrap-single",
"two-on-a-segment",
"behind-nat",
"segmented-and-unforwardable",
"the-ordinary-shape",
];
for (const name of ALL_SCENARIOS) {
test(`${name}: no box is drawn inside a network it is not on`, () => {
// A gateway landed on top of a machine in an unrelated network, because the gap was
// reserved above one sibling while the box was placed above the other. Two rules deciding
// the same thing separately; both now read one map.
const each = loadScenario(`scenarios/${name}.yml`);
const out = toDrawio(diagramFromDeclaration(each));
const boxes: { id: string; y: number; h: number; lanes: string[] }[] = [];
const lanes: { name: string; top: number; bottom: number }[] = [];
for (const match of out.matchAll(
/id="([^"]*)"[\s\S]{0,400}?<mxGeometry x="[^"]*" y="([^"]*)" width="[^"]*" height="([^"]*)"/g,
)) {
const [, id, y, h] = match;
if (id === "title" || id === "legend" || /-b\d+$|-n$/.test(id as string)) continue;
if ((id as string).startsWith("lane-")) {
lanes.push({ name: (id as string).slice(5), top: Number(y), bottom: Number(y) + Number(h) });
} else {
boxes.push({ id: id as string, y: Number(y), h: Number(h), lanes: [] });
}
}
diagramFromDeclaration(each).machines.forEach((machine, index) => {
const box = boxes.find((b) => b.id === `m${index}`);
if (box) box.lanes = machine.attachments.map((a) => a.segment);
});
for (const box of boxes) {
for (const lane of lanes) {
if (box.lanes.includes(lane.name)) continue;
const overlaps = box.y < lane.bottom && box.y + box.h > lane.top;
assert.ok(!overlaps, `${box.id} is drawn inside '${lane.name}', which it is not on`);
}
}
});
}
test/integration/harness.ts
+117
View File
@@ -0,0 +1,117 @@
/**
* Integration tests run against a real hypervisor. Mocking it is forbidden — a test that
* fakes the system under integration asserts that the fake behaves as expected, which is
* the shape of test this project exists to stop shipping (novox/hq ADR 0034).
*
* Consequence, accepted: these are slow, and they need a machine that can raise scenarios.
* They skip rather than fail where it cannot, so that a machine without a hypervisor gets
* an honest "not run" instead of a green suite that checked nothing.
*/
import assert from "node:assert/strict";
import { isReachable, pools, supportedDrivers } from "../../src/incus/client.ts";
import { destroy, list } from "../../src/lifecycle/operate.ts";
import { diagramFromLive } from "../../src/diagram/from-live.ts";
import { duplicateAddresses, describeConflicts, type Held } from "../../src/lifecycle/invariants.ts";
import type { Scenario } from "../../src/declaration/types.ts";
export interface Capability {
usable: boolean;
why: string;
}
/** Can this machine run scenarios at all? Checked once, reported honestly. */
export async function labIsUsable(): Promise<Capability> {
if (!(await isReachable())) {
return {
usable: false,
why: "the incus daemon is not reachable as this user (try MESH_LAB_INCUS='sudo -n incus')",
};
}
const drivers = await supportedDrivers();
if (!drivers.some((d) => d === "btrfs" || d === "zfs")) {
return { usable: false, why: "no copy-on-write driver — snapshots would be full copies" };
}
if (!(await pools()).some((p) => p.driver === "btrfs" || p.driver === "zfs")) {
return { usable: false, why: "no pool uses a copy-on-write driver" };
}
return { usable: true, why: "" };
}
/** Tear down anything a test left behind, whether it passed or not. */
export async function destroyAll(prefix: string): Promise<void> {
for (const instance of await list()) {
if (instance.instanceId.startsWith(prefix)) {
await destroy(instance.instanceId);
}
}
}
/**
* Every address the hypervisor says is held, by which machine, on which segment.
*
* Read through the live diagram because that is already the one place that joins addresses
* to devices by MAC and devices to segments by tag. A second reader would be a second thing
* to get wrong in the same way — and the way it was wrong once, a virtual machine's
* addresses silently going missing, is exactly what these assertions would then miss.
*/
export async function heldAddresses(instanceId: string): Promise<Held[]> {
const drawn = await diagramFromLive(instanceId);
return drawn.machines.flatMap((machine) =>
machine.attachments.flatMap((attachment) =>
attachment.addresses.map((address) => ({
machine: machine.name,
segment: attachment.segment,
address,
})),
),
);
}
function bare(address: string): string {
const slash = address.lastIndexOf("/");
return slash === -1 ? address : address.slice(0, slash);
}
/**
* Invariants that hold of ANY raised scenario, whatever it declares.
*
* Asserted against what actually came up, never against the declaration — the declaration
* is what was accepted, and in the fault that prompted these, it was accepted.
*/
export async function assertUniversalInvariants(
scenario: Scenario,
instanceId: string,
): Promise<void> {
const held = await heldAddresses(instanceId);
assert.ok(held.length > 0, `${scenario.scenario}: no addresses were read back at all`);
const conflicts = duplicateAddresses(held);
assert.deepEqual(
conflicts,
[],
`${scenario.scenario}: address conflict — ${describeConflicts(conflicts)}`,
);
// Every address the scenario declared is one the machine actually holds. A machine that
// came up bare looks identical to one that came up correctly until something asks it.
const holders = new Map<string, Set<string>>();
for (const entry of held) {
const key = `${entry.machine} ${entry.segment}`;
holders.set(key, (holders.get(key) ?? new Set<string>()).add(bare(entry.address)));
}
for (const [name, spec] of Object.entries(scenario.machines)) {
if (spec.at === "detached") continue;
for (const attachment of spec.at) {
const actual = holders.get(`${name} ${attachment.segment}`) ?? new Set<string>();
for (const address of attachment.address) {
assert.ok(
actual.has(address),
`${scenario.scenario}: '${name}' declared ${address} on '${attachment.segment}' ` +
`but holds ${[...actual].join(", ") || "nothing"}`,
);
}
}
}
}
test/integration/placement.test.ts
+88
View File
@@ -0,0 +1,88 @@
/**
* The lab, placing tier 0 inside a machine it raised.
*
* This is the seam that ends the lab being infrastructure with no consumer
* (novox/hq 03-DESIGN/00-as-is/11-the-lab.md). It needs a built host binary; without one it
* skips with a reason rather than passing having checked nothing.
*/
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { loadScenario } from "../../src/declaration/parse.ts";
import { raise } from "../../src/lifecycle/raise.ts";
import { destroy, exec } from "../../src/lifecycle/operate.ts";
import { hostBinaryPath, HOST_PATH } from "../../src/lifecycle/place.ts";
import { labIsUsable, destroyAll } from "./harness.ts";
const capability = await labIsUsable();
const binary = hostBinaryPath();
const skip = !capability.usable
? `lab not usable: ${capability.why}`
: !binary
? "MESH_LAB_HOST_BINARY is not set — build novox/mesh-host and point at it"
: !existsSync(binary)
? `MESH_LAB_HOST_BINARY points at ${binary}, which does not exist`
: false;
let instanceId = "";
after(async () => {
if (instanceId) await destroy(instanceId);
await destroyAll("bootstrap-single-");
}, { timeout: 400_000 });
test("a raised machine contains the host", { skip, timeout: 900_000 }, async () => {
const raised = await raise(loadScenario("scenarios/bootstrap-single.yml"), {});
instanceId = raised.instanceId;
const { stdout } = await exec(instanceId, "anchor", [HOST_PATH, "version"]);
assert.ok(stdout.trim().length > 0, "the host is on the machine but does not run there");
});
test("the host reports the MACHINE, not the workstation that placed it", { skip, timeout: 120_000 }, async () => {
// The check that proves detection detects rather than reporting a constant. A raised VM and
// the workstation differ in every capability, so a host that reported the workstation's
// answers would be obvious here and invisible anywhere else.
const { stdout } = await exec(instanceId, "anchor", [HOST_PATH, "profile", "--json"]);
const profile = JSON.parse(stdout) as {
capabilities: { name: string; present: boolean; detail: string }[];
};
const by = new Map(profile.capabilities.map((c) => [c.name, c]));
// A machine raised by the lab is root and has a clean init. The workstation session is
// neither, so these are the two that would flip if the wrong machine were being read.
assert.equal(by.get("privileged")?.present, true, "a raised machine should be root");
assert.equal(by.get("service-manager")?.present, true, "a raised machine should have an init");
for (const [name, verdict] of by) {
assert.ok(verdict.detail.trim().length > 0, `${name} was reported with no reason`);
}
});
test("ADR 0031 — the host confirms the machine carries no overlay", { skip, timeout: 120_000 }, async () => {
// The lab provides the underlay and NOTHING of the overlay. Asserted elsewhere by looking
// for wireguard interfaces; here the placed host reports it independently, which is a
// second witness rather than the same check twice.
const { stdout } = await exec(instanceId, "anchor", [HOST_PATH, "profile", "--json"]);
const profile = JSON.parse(stdout) as { capabilities: { name: string; present: boolean }[] };
const overlay = profile.capabilities.find((c) => c.name === "overlay");
assert.equal(overlay?.present, false, "a freshly raised machine already had an overlay");
});
test("the host's inventory is of the raised machine", { skip, timeout: 120_000 }, async () => {
const { stdout } = await exec(instanceId, "anchor", [HOST_PATH, "inventory", "--json"]);
const inv = JSON.parse(stdout) as {
machine: string; cpus: number; memory_kb: number; observed_at: string; unreadable?: string[];
};
assert.ok(inv.machine.length > 0, "the machine did not report a name");
assert.ok(inv.cpus > 0 && inv.memory_kb > 0, "the machine reported no cpus or no memory");
assert.deepEqual(inv.unreadable ?? [], [], "something could not be read on a machine we raised");
// The scenario gives each machine 1GiB and 2 cpus. A host reporting the workstation's
// 24 cpus would pass every check above.
assert.ok(inv.cpus <= 4, `reported ${inv.cpus} cpus — that is not the raised machine`);
});
test/integration/scenarios.test.ts
+47
View File
@@ -0,0 +1,47 @@
/**
* Every scenario, raised for real, checked against the invariants that hold of all of them.
*
* The suite next door raises one scenario and asks deep questions of it. This one asks
* shallow questions of every scenario, which is the half that was missing: both faults found
* by hand so far — two gateways holding one address, and a gateway drawn across an unrelated
* network — lived in scenarios nothing ever built.
*
* The list grows. Each entry costs a boot, so it is added deliberately rather than by
* globbing the directory: a scenario that is expensive and adds no new shape is not worth
* the wall clock, and one that is cheap and adds a shape is.
*/
import { test, after } from "node:test";
import { loadScenario } from "../../src/declaration/parse.ts";
import { raise } from "../../src/lifecycle/raise.ts";
import { destroy } from "../../src/lifecycle/operate.ts";
import { labIsUsable, destroyAll, assertUniversalInvariants } from "./harness.ts";
const capability = await labIsUsable();
const skip = capability.usable ? false : `lab not usable: ${capability.why}`;
/** Grows one step at a time. Each addition is a boot, and a shape not covered before. */
const SCENARIOS = [
"bootstrap-single", // one machine, one public network — the floor
];
const raised: string[] = [];
after(async () => {
for (const instanceId of raised) await destroy(instanceId);
}, { timeout: 600_000 });
for (const name of SCENARIOS) {
test(`${name}: comes up holding what it declared, with no address held twice`, { skip, timeout: 900_000 }, async () => {
const scenario = loadScenario(`scenarios/${name}.yml`);
const instance = await raise(scenario, {});
raised.push(instance.instanceId);
await assertUniversalInvariants(scenario, instance.instanceId);
});
}
after(async () => {
// Anything a failed raise left standing. RaiseError leaves wreckage on purpose, which is
// right for a person debugging and wrong for the next run of the suite.
for (const name of SCENARIOS) await destroyAll(`${name}-`);
}, { timeout: 600_000 });
test/integration/underlay.test.ts
+218
View File
@@ -0,0 +1,218 @@
/**
* Each test names the decision it defends. A decision with no test is one that will quietly
* stop being true (novox/hq ADR 0034).
*/
import { test, before, after } from "node:test";
import assert from "node:assert/strict";
import { loadScenario } from "../../src/declaration/parse.ts";
import { raise } from "../../src/lifecycle/raise.ts";
import { destroy, exec, list, restore, snapshot } from "../../src/lifecycle/operate.ts";
import { incus, incusOk } from "../../src/incus/client.ts";
import { labIsUsable, destroyAll, assertUniversalInvariants } from "./harness.ts";
import { diagramFromLive } from "../../src/diagram/from-live.ts";
import { diagramFromDeclaration } from "../../src/diagram/from-declaration.ts";
import { toDrawio } from "../../src/diagram/drawio.ts";
const capability = await labIsUsable();
const skip = capability.usable ? false : `lab not usable: ${capability.why}`;
let instanceId = "";
before(async () => {
if (skip) return;
const scenario = loadScenario("scenarios/behind-nat.yml");
const raised = await raise(scenario, {});
instanceId = raised.instanceId;
});
after(async () => {
if (instanceId) await destroy(instanceId);
});
test("ADR 0031 — the lab provides the underlay and NOTHING of the overlay", { skip, timeout: 120_000 }, async () => {
// A scenario that pre-built peering would certify its own work. Whatever the mesh is
// responsible for must be absent from a freshly raised machine.
const { stdout } = await exec(instanceId, "home-server", [
"sh", "-c",
"ip link show type wireguard 2>/dev/null | wc -l; " +
"ls /etc/wireguard 2>/dev/null | wc -l; " +
"ls /etc/hal /etc/mesh 2>/dev/null | wc -l",
]);
const counts = stdout.trim().split("\n").map((n) => Number(n.trim()));
assert.deepEqual(counts, [0, 0, 0], "a raised machine carries no overlay, no mesh config");
});
test("ADR 0031 — the declared address IS what the machine holds", { skip }, async () => {
const { stdout } = await exec(instanceId, "home-server", ["ip", "-o", "-4", "addr", "show"]);
assert.match(stdout, /192\.168\.1\.135\/24/);
});
test("design — raise waits for USABLE, not for the call to return", { skip, timeout: 120_000 }, async () => {
// The measured gap is 3.4s to 14.3s. Reporting the earlier number is transport reported
// as effect. If raise has returned, every machine must answer immediately.
for (const machine of ["anchor", "home-server"]) {
const { stdout } = await exec(instanceId, machine, ["sh", "-c", "echo alive"]);
assert.equal(stdout.trim(), "alive", `${machine} was not usable when raise returned`);
}
});
test("ADR 0033 — a router is scenery: containers, while machines are virtual machines", { skip, timeout: 120_000 }, async () => {
const json = (await incusOk(["list", "--format", "json"], 30_000)) ?? "[]";
const all = JSON.parse(json) as { name?: string; type?: string; config?: Record<string, string> }[];
const mine = all.filter((i) => i.config?.["user.mesh-lab.instance"] === instanceId);
assert.ok(mine.length >= 3, "expected machines and a router");
for (const item of mine) {
const isRouter = item.config?.["user.mesh-lab.router"] !== undefined;
assert.equal(
item.type,
isRouter ? "container" : "virtual-machine",
`${item.name} is a ${item.type} but ${isRouter ? "is" : "is not"} a router`,
);
}
});
test("design — NAT: a private address is not reachable from outside", { skip, timeout: 120_000 }, async () => {
const { stdout } = await exec(instanceId, "anchor", [
"sh", "-c", "ping -c1 -W2 192.168.1.135 >/dev/null 2>&1 && echo reachable || echo unreachable",
]);
assert.equal(stdout.trim(), "unreachable");
});
test("design — published: reachable at the GATEWAY's address, never its own", { skip, timeout: 180_000 }, async () => {
await exec(instanceId, "home-server", [
"sh", "-c", "nohup python3 -m http.server 8080 --bind 0.0.0.0 >/tmp/s.log 2>&1 & sleep 2",
]);
const { stdout } = await exec(instanceId, "anchor", [
"sh", "-c", "curl -s -m5 -o /dev/null -w '%{http_code}' http://192.0.2.50:8080/ || echo failed",
]);
assert.equal(stdout.trim(), "200", "the forwarded port did not reach the machine behind NAT");
});
test("design — snapshots are WHOLE-scenario: restore returns every machine", { skip, timeout: 600_000 }, async () => {
// Restoring a subset would produce a mesh that has never existed, so faults found there
// would be artefacts of the lab.
await exec(instanceId, "anchor", ["sh", "-c", "echo dirty > /root/marker"]);
await exec(instanceId, "home-server", ["sh", "-c", "echo dirty > /root/marker"]);
await snapshot(instanceId, "test-point");
await exec(instanceId, "anchor", ["sh", "-c", "echo changed > /root/marker"]);
await exec(instanceId, "home-server", ["sh", "-c", "echo changed > /root/marker"]);
await restore(instanceId, "test-point");
for (const machine of ["anchor", "home-server"]) {
const { stdout } = await exec(instanceId, machine, ["cat", "/root/marker"]);
assert.equal(stdout.trim(), "dirty", `${machine} was not returned to the snapshot`);
}
});
test("design — restore leaves the scenario USABLE, not merely running", { skip, timeout: 120_000 }, async () => {
// The restore call returns in under a second while the agent is still starting. Reporting
// that as restored would be transport reported as effect.
const { stdout } = await exec(instanceId, "anchor", ["sh", "-c", "echo alive"]);
assert.equal(stdout.trim(), "alive");
});
test("ADR 0032 — the workstation has no route into the scenario", { skip, timeout: 60_000 }, async () => {
// Reachability is asked from INSIDE. If the workstation could reach a scenario address,
// two scenarios carrying the same prefix would put one's traffic in the other.
const { stdout } = await incus(["exec", `mlab-${instanceId}-anchor`, "--", "echo", "inside"]);
assert.equal(stdout.trim(), "inside", "exec is the only way in, and it works");
});
test("what came up holds what was declared, with no address held twice", { skip, timeout: 120_000 }, async () => {
// The same invariants every scenario is held to, asserted here too — this instance is
// already standing, so it costs nothing to ask.
await assertUniversalInvariants(loadScenario("scenarios/behind-nat.yml"), instanceId);
});
// The diagram tests read the instance the file raised, so they run before the one that
// tears it down. Ordering is load-bearing here: appended after the destroy test they read
// an instance that no longer existed, and reported it as the diagram failing.
test("the live diagram reads the hypervisor, and a VM's addresses are not lost", { skip }, async () => {
// A container's interface carries the device's name; a virtual machine names its own, so
// joining addresses to devices by name attached every address to a container and none to
// a VM. The picture then showed machines that looked like they had failed to come up.
const drawn = await diagramFromLive(instanceId);
const server = drawn.machines.find((m) => m.name === "home-server");
assert.ok(server, "home-server missing from the live picture");
assert.equal(server.kind, "machine");
assert.ok(
server.attachments.some((a) => a.addresses.some((address) => address.startsWith("192.168.1.135"))),
`a virtual machine's addresses were not read back: ${JSON.stringify(server.attachments)}`,
);
});
test("the live diagram draws what exists, never what was asked for", { skip, timeout: 300_000 }, async () => {
// Every property shown must have come off the hypervisor. Proved by changing the running
// system and watching only the live picture move.
const scenario = loadScenario("scenarios/behind-nat.yml");
const declared = diagramFromDeclaration(scenario);
const before = await diagramFromLive(instanceId);
assert.equal(before.machines.length, declared.machines.length, "the two pictures disagree on size");
const anchor = (await list())
.find((i) => i.instanceId === instanceId)
?.machines.find((m) => m.machine === "anchor");
assert.ok(anchor, "anchor not found");
await incus(["stop", anchor.name], 120_000);
try {
const after = await diagramFromLive(instanceId);
assert.equal(after.machines.find((m) => m.name === "anchor")?.status, "Stopped");
// The declaration has not changed, and neither has its picture.
assert.equal(
diagramFromDeclaration(scenario).machines.find((m) => m.name === "anchor")?.status,
undefined,
"a declared picture reported a runtime status it cannot know",
);
} finally {
await incus(["start", anchor.name], 120_000);
}
});
test("ADR 0033 — the live diagram distinguishes scenery from a node", { skip }, async () => {
// The router is drawn as a router because the hypervisor says it is a container tagged as
// a gateway — not because the diagram re-read the scenario and inferred it.
const drawn = await diagramFromLive(instanceId);
const gateway = drawn.machines.find((m) => m.kind === "router");
assert.ok(gateway, "no gateway in the live picture");
assert.ok(gateway.notes.includes("container"), "the gateway is not reported as scenery");
assert.ok(gateway.notes.some((n) => n.startsWith("NAT ")), "translation is not shown");
assert.ok(gateway.notes.includes("forwardable"), "forwardability is not shown");
assert.ok(
gateway.notes.some((n) => /mappings expire \d+s/.test(n)),
"the declared mapping expiry was not recorded on the gateway",
);
const segments = new Map(drawn.segments.map((s) => [s.name, s]));
assert.equal(segments.get("hosting")?.kind, "public", "segment kind was not recorded at raise");
assert.equal(segments.get("home")?.behind, "hosting", "the tree was not recovered from the gateway tags");
assert.equal(segments.get("home")?.depth, 1);
});
test("a picture nobody can open is not a picture", { skip }, async () => {
// The first generated file was unparseable — HTML labels concatenated into an XML
// attribute. Checked here against real output as well as against the fixtures, because
// live labels carry names and addresses the declared ones never contain.
const xml = toDrawio(await diagramFromLive(instanceId));
for (const match of xml.matchAll(/(?:value|label|tooltip)="([^"]*)"/g)) {
assert.doesNotMatch(match[1] ?? "", /[<>]/, "raw markup reached an XML attribute");
}
const ids = [...xml.matchAll(/<(?:mxCell|object) [^>]*id="([^"]*)"/g)].map((m) => m[1]);
assert.equal(new Set(ids).size, ids.length, "duplicate cell id — draw.io drops one silently");
});
test("housekeeping — destroy removes machines, routers and segments", { skip, timeout: 400_000 }, async () => {
const before = (await list()).find((i) => i.instanceId === instanceId);
assert.ok(before, "the instance should exist before it is destroyed");
const { machines, networks } = await destroy(instanceId);
assert.ok(machines >= 3, `expected machines and a router, removed ${machines}`);
assert.ok(networks >= 2, `expected both segments removed, removed ${networks}`);
const after = (await list()).find((i) => i.instanceId === instanceId);
assert.equal(after, undefined, "the instance should be gone");
instanceId = "";
await destroyAll("behind-nat-");
});
test/invariants.test.ts
+60
View File
@@ -0,0 +1,60 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { duplicateAddresses, describeConflicts } from "../src/lifecycle/invariants.ts";
/**
* The fault this defends against, in the shape it actually occurred: two gateways declared
* with the same public address became two router containers, both holding it on one segment.
*/
test("two machines holding one address on one segment is a conflict", () => {
const conflicts = duplicateAddresses([
{ machine: "gw-home", segment: "isp-home", address: "198.51.100.7/24" },
{ machine: "gw-devices", segment: "isp-home", address: "198.51.100.7/24" },
{ machine: "transit", segment: "isp-home", address: "198.51.100.254/24" },
]);
assert.equal(conflicts.length, 1);
assert.equal(conflicts[0]?.address, "198.51.100.7");
assert.deepEqual(conflicts[0]?.machines, ["gw-devices", "gw-home"]);
assert.match(describeConflicts(conflicts), /198\.51\.100\.7 is held by gw-devices and gw-home/);
});
test("the same address on different segments is NOT a conflict", () => {
// Every private network has its own `.1`. Reporting that would make the check useless.
assert.deepEqual(
duplicateAddresses([
{ machine: "gw-a", segment: "home", address: "192.168.1.1/24" },
{ machine: "gw-b", segment: "cafe", address: "192.168.1.1/24" },
]),
[],
);
});
test("one machine holding an address twice is not two machines", () => {
// A machine multi-homed onto the same segment, or an address read back from two places.
assert.deepEqual(
duplicateAddresses([
{ machine: "gw", segment: "isp", address: "198.51.100.7/24" },
{ machine: "gw", segment: "isp", address: "198.51.100.7" },
]),
[],
);
});
test("the prefix length is not part of the address", () => {
// The same address declared /24 in one place and /16 in another is still one address.
const conflicts = duplicateAddresses([
{ machine: "a", segment: "isp", address: "198.51.100.7/24" },
{ machine: "b", segment: "isp", address: "198.51.100.7/16" },
]);
assert.equal(conflicts.length, 1);
});
test("both families are checked", () => {
const conflicts = duplicateAddresses([
{ machine: "a", segment: "isp", address: "2001:db8:b::7/48" },
{ machine: "b", segment: "isp", address: "2001:db8:b::7/48" },
]);
assert.equal(conflicts.length, 1);
assert.equal(conflicts[0]?.address, "2001:db8:b::7");
});
test/names.test.ts
+34
View File
@@ -0,0 +1,34 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { machineName, machinePrefix, networkName, newInstanceId, instanceIdOf } from "../src/lifecycle/names.ts";
test("network names fit incus's 15-character limit", () => {
const id = newInstanceId("the-ordinary-shape", new Date("2026-08-24T22:15:00Z"));
for (const segment of ["hosting", "isp-home", "isp-mobile", "home", "devices", "cafe"]) {
const name = networkName(id, segment);
assert.ok(name.length <= 15, `${name} is ${name.length} chars`);
}
});
test("network names are unique per (instance, segment)", () => {
const a = newInstanceId("x", new Date("2026-08-24T22:15:00Z"));
const b = newInstanceId("x", new Date("2026-08-24T23:15:00Z"));
const names = new Set([
networkName(a, "home"), networkName(a, "cafe"),
networkName(b, "home"), networkName(b, "cafe"),
]);
assert.equal(names.size, 4, "two instances of one declaration must not share a network");
});
test("machine names round-trip to their instance id", () => {
const id = newInstanceId("bootstrap-single", new Date("2026-08-24T22:15:00Z"));
const name = machineName(id, "anchor");
assert.equal(instanceIdOf(name, "anchor"), id);
assert.ok(name.startsWith(machinePrefix(id)));
});
test("instance ids are sortable by time", () => {
const early = newInstanceId("x", new Date("2026-08-24T09:00:00Z"));
const late = newInstanceId("x", new Date("2026-08-24T21:00:00Z"));
assert.ok(early < late, `${early} should sort before ${late}`);
});
test/net.test.ts
+49
View File
@@ -0,0 +1,49 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseCidr, contains, familyOf, parseAddress } from "../src/declaration/net.ts";
test("family is inferred from the address", () => {
assert.equal(familyOf("203.0.113.1"), "v4");
assert.equal(familyOf("2001:db8::1"), "v6");
});
test("v4 containment", () => {
const range = parseCidr("203.0.113.0/24");
assert.equal(contains(range, "203.0.113.0"), true);
assert.equal(contains(range, "203.0.113.255"), true);
assert.equal(contains(range, "203.0.114.0"), false);
assert.equal(contains(range, "202.0.113.1"), false);
});
test("v4 containment on a non-byte prefix", () => {
const range = parseCidr("198.51.100.0/25");
assert.equal(contains(range, "198.51.100.127"), true);
assert.equal(contains(range, "198.51.100.128"), false);
});
test("v6 containment, including :: compression", () => {
const range = parseCidr("2001:db8:a::/48");
assert.equal(contains(range, "2001:db8:a::10"), true);
assert.equal(contains(range, "2001:db8:a:ffff::1"), true);
assert.equal(contains(range, "2001:db8:b::10"), false);
});
test("a range never contains an address of the other family", () => {
assert.equal(contains(parseCidr("203.0.113.0/24"), "2001:db8::1"), false);
assert.equal(contains(parseCidr("2001:db8::/32"), "203.0.113.1"), false);
});
test("a cidr whose address has host bits set still masks to its network", () => {
// 203.0.113.5/24 means the 203.0.113.0/24 network, not a range starting at .5
assert.equal(contains(parseCidr("203.0.113.5/24"), "203.0.113.1"), true);
});
test("malformed input is refused rather than guessed at", () => {
assert.throws(() => parseCidr("203.0.113.0"), /no prefix length/);
assert.throws(() => parseCidr("203.0.113.0/33"), /out of range/);
assert.throws(() => parseCidr("2001:db8::/129"), /out of range/);
assert.throws(() => parseAddress("203.0.113"), /not an IPv4/);
assert.throws(() => parseAddress("203.0.113.256"), /out of range/);
assert.throws(() => parseAddress("2001:db8::1::2"), /not an IPv6/);
assert.throws(() => parseAddress("::ffff:192.0.2.1"), /not supported/);
});
test/place.test.ts
+89
View File
@@ -0,0 +1,89 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseScenario } from "../src/declaration/parse.ts";
import { planPlacements, PLACEABLE } from "../src/lifecycle/place.ts";
import { assertSupported, UnsupportedError } from "../src/lifecycle/supported.ts";
/**
* `place:` is the seam where the lab stops being infrastructure with no consumer. Each test
* names what it defends, per novox/hq ADR 0034.
*/
function scenario(place: string): ReturnType<typeof parseScenario> {
return parseScenario(`
scenario: placing
segments:
hosting:
kind: public
cidr: [192.0.2.0/24]
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
peer:
at: { segment: hosting, address: [192.0.2.20] }
${place}
`);
}
test("`all:` reaches every machine", () => {
const placements = planPlacements(scenario("place:\n all: [host]"));
assert.deepEqual(
placements.map((p) => p.machine).sort(),
["anchor", "peer"],
);
for (const p of placements) assert.deepEqual(p.artifacts, ["host"]);
});
test("a per-machine entry OVERRIDES `all:`, it does not add to it", () => {
// Worth being exact about: a scenario naming one artifact for one machine gets that
// artifact, not that artifact plus everything in `all:`. The opposite reading would place
// things nobody asked for, which is the shape of fault this lab exists to catch.
const placements = planPlacements(scenario("place:\n all: [host]\n anchor: [substrate]"));
const byMachine = new Map(placements.map((p) => [p.machine, p.artifacts]));
assert.deepEqual(byMachine.get("anchor"), ["substrate"], "anchor should have ONLY substrate");
assert.deepEqual(byMachine.get("peer"), ["host"]);
});
test("a machine placed with nothing is not a placement", () => {
const placements = planPlacements(scenario("place:\n all: [host]\n anchor: []"));
assert.deepEqual(placements.map((p) => p.machine), ["peer"]);
});
test("no `place:` at all is no placements, not an error", () => {
assert.deepEqual(planPlacements(scenario("")), []);
});
test("placing the host is supported", () => {
// The whole point of stage 1: this used to be refused.
assert.doesNotThrow(() => assertSupported(scenario("place:\n all: [host]")));
});
test("a tier that does not exist is refused BY NAME", () => {
// Named individually rather than refused as a whole, so a scenario placing a host and a
// substrate is told exactly which half the lab cannot do — rather than being told `place:`
// is unsupported when half of it now works.
try {
assertSupported(scenario("place:\n all: [host, substrate]\n peer: [control]"));
assert.fail("expected a refusal");
} catch (err) {
assert.ok(err instanceof UnsupportedError);
const missing = err.missing.join("\n");
assert.match(missing, /substrate/, "the substrate was not named");
assert.match(missing, /control/, "the control plane was not named");
assert.doesNotMatch(missing, /place: host/, "the host is placeable and was refused anyway");
}
});
test("the refusal says what CAN be placed", () => {
// A refusal that does not say what is possible sends someone to the source to find out.
try {
assertSupported(scenario("place:\n all: [forge]"));
assert.fail("expected a refusal");
} catch (err) {
assert.ok(err instanceof UnsupportedError);
for (const placeable of PLACEABLE) {
assert.match(err.missing.join("\n"), new RegExp(placeable));
}
}
});
test/router.test.ts
+57
View File
@@ -0,0 +1,57 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseScenario } from "../src/declaration/parse.ts";
import { planRouters, ttlSeconds } from "../src/lifecycle/router.ts";
test("segments sharing a gateway declaration share ONE router", () => {
// That is what a VLAN-capable router is, and two routers sharing an external address
// would not work anyway.
const scenario = parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
iot: { kind: private, cidr: [192.168.30.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`);
const plans = planRouters(scenario, "inst");
assert.equal(plans.length, 1, "one gateway declaration, one router");
assert.deepEqual(plans[0]?.inside.sort(), ["home", "iot"]);
});
test("different external addresses mean different routers", () => {
const scenario = parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
other: { kind: private, cidr: [192.168.30.0/24], gateway: { to: pub, address: [192.0.2.6], nat: [v4] } }
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`);
assert.equal(planRouters(scenario, "inst").length, 2);
});
test("a scenario with no gateways needs no routers", () => {
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }`);
assert.equal(planRouters(scenario, "inst").length, 0);
});
test("forwardable and nat carry through to the plan", () => {
const scenario = parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
cafe: { kind: private, cidr: [10.50.0.0/16], gateway: { to: pub, address: [192.0.2.9], nat: [v4], forwardable: false, mapping_ttl: 30s } }
machines: { a: { at: { segment: cafe, address: [10.50.0.9] } } }`);
const plan = planRouters(scenario, "inst")[0];
assert.equal(plan?.forwardable, false);
assert.deepEqual(plan?.nat, ["v4"]);
assert.equal(plan?.mappingTtl, "30s");
});
test("mapping ttl parses the forms a declaration uses", () => {
assert.equal(ttlSeconds("30s"), 30);
assert.equal(ttlSeconds("120s"), 120);
assert.equal(ttlSeconds("2m"), 120);
assert.equal(ttlSeconds("1h"), 3600);
assert.equal(ttlSeconds("90"), 90);
assert.equal(ttlSeconds(undefined), undefined);
assert.equal(ttlSeconds("soon"), undefined);
});
test/supported.test.ts
+86
View File
@@ -0,0 +1,86 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseScenario } from "../src/declaration/parse.ts";
import { assertSupported, UnsupportedError } from "../src/lifecycle/supported.ts";
/**
* A declaration the runtime silently ignores is the fault this lab exists to catch —
* novox/hq 04-ISSUES/003, where a firewall key is declared in five manifests and read by no
* code. These tests exist so the lab never commits it, and they move as the runtime catches
* up with the model.
*/
const withGateway = `scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`;
test("a plain scenario is raisable", () => {
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }`);
assert.doesNotThrow(() => assertSupported(scenario));
});
test("gateways are implemented — a router is materialised for them", () => {
assert.doesNotThrow(() => assertSupported(parseScenario(withGateway)));
});
test("published ports and policy are implemented", () => {
const scenario = parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
iot: { kind: private, cidr: [192.168.30.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
policy: [{ from: iot, to: home, allow: false }]
machines:
a:
at: { segment: home, address: [192.168.1.9] }
published: [{ port: 443, on: home }]`);
assert.doesNotThrow(() => assertSupported(scenario));
});
test("inbound: deny is implemented — a host firewall is applied and read back", () => {
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] }, inbound: deny } }`);
assert.doesNotThrow(() => assertSupported(scenario));
});
test("the host is placeable — it used to be refused, and tier 0 now exists", () => {
// These two tests failed the moment placement worked, which is what they were for. They
// defended "there is nothing to place yet" while that was true; the decision changed, so
// they change with it rather than being deleted (novox/hq ADR 0034).
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }
place: { all: [host] }`);
assert.doesNotThrow(() => assertSupported(scenario));
});
test("a tier above 0 is still refused, and named", () => {
// The refusal narrowed rather than disappearing. A scenario placing a host AND a substrate
// must be told which half is missing — not that `place:` is unsupported, when half of it
// now works.
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }
place: { all: [host, substrate] }`);
try {
assertSupported(scenario);
assert.fail("should have refused");
} catch (err) {
assert.ok(err instanceof UnsupportedError);
assert.equal(err.missing.length, 1, `expected only the substrate: ${err.missing.join(", ")}`);
assert.match(err.missing[0] ?? "", /substrate/);
assert.match(err instanceof Error ? err.message : "", /silently lacks them/);
}
});
test("inbound: allow is not a gap — only deny needs enforcing", () => {
const scenario = parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] }, inbound: allow } }`);
assert.doesNotThrow(() => assertSupported(scenario));
});
test/validate.test.ts
+245
View File
@@ -0,0 +1,245 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseScenario } from "../src/declaration/parse.ts";
import { loadScenario } from "../src/declaration/parse.ts";
import { planRouters } from "../src/lifecycle/router.ts";
/** Every rejection below is a fault that would otherwise be silent at runtime. */
function refuses(yaml: string, pattern: RegExp): void {
assert.throws(() => parseScenario(yaml), (err: Error) => {
assert.match(err.message, pattern);
return true;
});
}
test("the shipped scenarios are valid", () => {
for (const file of ["scenarios/bootstrap-single.yml", "scenarios/the-ordinary-shape.yml"]) {
assert.doesNotThrow(() => loadScenario(file));
}
});
test("a public segment on a private range is refused — the mesh would silently never form", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.168.1.0/24] } }
machines: { a: { at: { segment: net, address: [192.168.1.1] } } }`,
/not documentation space/,
);
});
test("a public segment on a real routable range is refused", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [8.8.8.0/24] } }
machines: { a: { at: { segment: net, address: [8.8.8.8] } } }`,
/not documentation space/,
);
});
test("a private segment may use any range, including someone else's RFC 1918", () => {
assert.doesNotThrow(() =>
parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
cafe: { kind: private, cidr: [10.50.0.0/16], gateway: { to: pub, address: [192.0.2.5], nat: [v4], forwardable: false } }
machines: { a: { at: { segment: cafe, address: [10.50.0.9] } } }`),
);
});
test("publishing through an unforwardable gateway is refused — that is the constraint", () => {
refuses(
`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
cafe: { kind: private, cidr: [10.50.0.0/16], gateway: { to: pub, address: [192.0.2.5], nat: [v4], forwardable: false } }
machines:
a:
at: { segment: cafe, address: [10.50.0.9] }
published: [{ port: 443, on: cafe }]`,
/not forwardable/,
);
});
test("a gateway address must be on the PARENT segment, not the one behind it", () => {
refuses(
`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.168.1.1], nat: [v4] } }
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`,
/is not within 'pub'/,
);
});
test("a machine address outside its segment is refused", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [203.0.113.9] } } }`,
/is not within segment 'net'/,
);
});
test("an unknown segment reference is refused", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: nope, address: [192.0.2.1] } } }`,
/unknown segment 'nope'/,
);
});
test("a gateway loop is refused rather than raised forever", () => {
refuses(
`scenario: x
segments:
a: { kind: private, cidr: [10.0.0.0/24], gateway: { to: b, address: [10.0.1.1], nat: [] } }
b: { kind: private, cidr: [10.0.1.0/24], gateway: { to: a, address: [10.0.0.1], nat: [] } }
machines: { m: { at: { segment: a, address: [10.0.0.9] } } }`,
/loops through/,
);
});
test("a detached machine cannot publish", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: detached, published: [{ port: 443, on: net }] } }`,
/detached but declares published/,
);
});
test("publishing on a segment the machine is not attached to is refused", () => {
refuses(
`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
machines:
a:
at: { segment: pub, address: [192.0.2.10] }
published: [{ port: 443, on: home }]`,
/not attached to it/,
);
});
test("two addresses of one family on one attachment is refused", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1, 192.0.2.2] } } }`,
/two v4 addresses/,
);
});
test("place naming a machine that does not exist is refused", () => {
refuses(
`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }
place: { ghost: [host] }`,
/'ghost' is not a machine/,
);
});
test("every problem is reported, not just the first", () => {
try {
parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.168.0.0/24] } }
machines: { a: { at: { segment: nope, address: [1.2.3.4] } } }
place: { ghost: [host] }`);
assert.fail("should have thrown");
} catch (err) {
const problems = (err as { problems: string[] }).problems;
assert.ok(problems.length >= 3, `expected several problems, got ${problems.length}`);
}
});
test("a detached machine is valid", () => {
assert.doesNotThrow(() =>
parseScenario(`scenario: x
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
machines:
a: { at: { segment: net, address: [192.0.2.1] } }
roamer: { at: detached }`),
);
});
test("a multi-homed machine is valid", () => {
assert.doesNotThrow(() =>
parseScenario(`scenario: x
segments:
pub: { kind: public, cidr: [192.0.2.0/24] }
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
machines:
border:
at:
- { segment: pub, address: [192.0.2.60] }
- { segment: home, address: [192.168.1.2] }`),
);
});
test("an isolated private segment with no gateway is valid — a site with no internet", () => {
assert.doesNotThrow(() =>
parseScenario(`scenario: x
segments: { island: { kind: private, cidr: [10.9.0.0/24] } }
machines: { a: { at: { segment: island, address: [10.9.0.1] } } }`),
);
});
test("two gateways sharing an address are one gateway, not two", () => {
// Modelled on the real thing: a bridged modem, one gateway holding the public address,
// everything behind it. Two routers on one address is not a topology, it is a collision —
// and the lab raised it happily, with the address resolving to whichever container
// answered ARP last.
const scenario = parseScenario(`
scenario: shared-gateway
segments:
isp:
kind: public
cidr: [198.51.100.0/24, "2001:db8:b::/48"]
home:
kind: private
cidr: [192.168.1.0/24]
gateway: { to: isp, address: [198.51.100.7, "2001:db8:b::7"], nat: [v4], forwardable: true, mapping_ttl: 120s }
devices:
kind: private
cidr: [192.168.30.0/24]
gateway: { to: isp, address: [198.51.100.7], nat: [v4], forwardable: true, mapping_ttl: 120s }
machines:
thermostat:
at: { segment: devices, address: [192.168.30.20] }
`);
const plans = planRouters(scenario, "test");
assert.equal(plans.length, 1, `expected one gateway, got ${plans.map((p) => p.inside.join("+")).join(" / ")}`);
assert.deepEqual([...plans[0]!.inside].sort(), ["devices", "home"]);
// The union: a v6 address declared on only one of the segments it serves is still carried.
assert.deepEqual([...plans[0]!.outsideAddresses].sort(), ["198.51.100.7", "2001:db8:b::7"]);
});
test("one box cannot behave two ways", () => {
// If two gateways share an address they are the same box, so a disagreement about what
// that box does is a contradiction — refused rather than silently resolved one way.
assert.throws(
() =>
parseScenario(`
scenario: contradictory-gateway
segments:
isp:
kind: public
cidr: [198.51.100.0/24]
home:
kind: private
cidr: [192.168.1.0/24]
gateway: { to: isp, address: [198.51.100.7], nat: [v4], forwardable: true }
devices:
kind: private
cidr: [192.168.30.0/24]
gateway: { to: isp, address: [198.51.100.7], nat: [v4], forwardable: false }
machines:
thermostat:
at: { segment: devices, address: [192.168.30.20] }
`),
/one gateway.*disagree.*forwardable/s,
);
});
tsconfig.json
+21
View File
@@ -0,0 +1,21 @@
{
"compilerOptions": {
"target": "es2022",
"module": "nodenext",
"moduleResolution": "nodenext",
"strict": true,
"noUncheckedIndexedAccess": true,
"noImplicitOverride": true,
"exactOptionalPropertyTypes": true,
"rootDir": "src",
"skipLibCheck": true,
"types": [
"node"
],
"allowImportingTsExtensions": true,
"noEmit": true
},
"include": [
"src/**/*.ts"
]
}
tsconfig.test.json
+9
View File
@@ -0,0 +1,9 @@
{
// The tests were outside the typecheck gate, so a test that did not compile was simply a
// test that never ran — silently, which is the failure mode this project keeps finding.
"extends": "./tsconfig.json",
"compilerOptions": {
"rootDir": "."
},
"include": ["src/**/*.ts", "test/**/*.ts"]
}