The lab raises a mesh, draws it, and now places tier 0 inside it #1
@@ -0,0 +1 @@
|
||||
node_modules/
|
||||
@@ -26,38 +26,136 @@ The bootstrap scenario is a **strict subset** — same virtualisation, same netw
|
||||
lifecycle, stopping before a control plane exists. The full scenario is reached by putting more
|
||||
inside the machines, not by building a second thing.
|
||||
|
||||
**Bootstrap is what gets built here first.** Nothing in this repository requires a forge, a
|
||||
coordinator or a pipeline to be useful.
|
||||
|
||||
## Shape
|
||||
## Using it
|
||||
|
||||
```
|
||||
scenarios/ declarations of a mesh to raise
|
||||
lifecycle/ create · snapshot · reset · destroy
|
||||
network/ segments and addressing
|
||||
place/ getting a binary onto a machine
|
||||
mesh-lab check can this machine run scenarios at all
|
||||
mesh-lab validate scenarios/x.yml parse and check, raising nothing
|
||||
mesh-lab raise scenarios/x.yml materialise it, wait until the machines are USABLE
|
||||
mesh-lab list instances currently standing
|
||||
mesh-lab exec <instance> <machine> -- <cmd...>
|
||||
mesh-lab snapshot <instance> <label>
|
||||
mesh-lab restore <instance> <label>
|
||||
mesh-lab destroy <instance>
|
||||
```
|
||||
|
||||
Of the two jobs a runner might hold, **scenario lifecycle comes first** — something must
|
||||
materialise and reset a mesh before anything can be written against it. **Assertion execution
|
||||
comes later**, with the full scenario.
|
||||
`check` refuses rather than warns. A machine without copy-on-write storage runs scenarios
|
||||
correctly and snapshots roughly 76× slower — which does not make the lab slow, it makes it
|
||||
unused, and a warning about that is read once and ignored forever.
|
||||
|
||||
## Rules
|
||||
If the incus socket is not reachable as your user — the group was granted to a session that
|
||||
already existed — set `MESH_LAB_INCUS="sudo -n incus"`.
|
||||
|
||||
- **Nothing new drives delivery.** A full scenario runs the real pipeline against a mesh named
|
||||
by the request. A second delivery path would be blind to exactly the faults worth catching.
|
||||
- **A scenario starts from nothing, every time.** Adopting a machine that already exists is out
|
||||
of scope — that is what makes a scenario a fixture rather than a snapshot.
|
||||
- **The real topology is one scenario among many**, not the baseline. Anything only testable
|
||||
against the shape the mesh happens to have is a gap in the vocabulary.
|
||||
## What a scenario declares
|
||||
|
||||
The **underlay**: what a hosting provider and a home router would provide, and nothing the
|
||||
mesh is responsible for.
|
||||
|
||||
```yaml
|
||||
segments:
|
||||
hosting: # one public network
|
||||
kind: public
|
||||
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
|
||||
isp-home: # another, unrelated — routed to it, never bridged
|
||||
kind: public
|
||||
cidr: [198.51.100.0/24, "2001:db8:b::/48"]
|
||||
home:
|
||||
kind: private
|
||||
cidr: [192.168.1.0/24, "2001:db8:b:1::/64"]
|
||||
mtu: 1492
|
||||
gateway:
|
||||
to: isp-home
|
||||
address: [198.51.100.7] # what the world sees this network as
|
||||
nat: [v4] # v4 translated, v6 routed
|
||||
forwardable: true
|
||||
mapping_ttl: 120s
|
||||
machines:
|
||||
home-server:
|
||||
at: { segment: home, address: [192.168.1.135, "2001:db8:b:1::135"] }
|
||||
published: [{ port: 443, on: home }]
|
||||
inbound: allow
|
||||
```
|
||||
|
||||
It declares **nothing** about overlay addresses, hubs, peering, names or certificates. Those
|
||||
are what the mesh does, and a scenario that supplied them would be certifying its own work.
|
||||
|
||||
Public segments must use documentation ranges (RFC 5737, RFC 3849) and the validator refuses
|
||||
anything else **before raising**. That is not pedantry: the mesh decides public-versus-private
|
||||
by matching the address, so a private range on a segment meant to be routable makes the mesh
|
||||
silently never form — no error, nothing to notice.
|
||||
|
||||
## Reaching in
|
||||
|
||||
Everything goes through incus, never over IP. A scenario is a closed address space, so two
|
||||
instances raised from one declaration hold the same addresses and never meet — and the
|
||||
workstation has no route into either.
|
||||
|
||||
So a reachability question is asked **from inside**: *can this machine reach that one* is
|
||||
`exec` on the first, testing the second. The workstation's opinion would be a different
|
||||
question with a misleadingly similar answer.
|
||||
|
||||
## What is implemented, and what is not
|
||||
|
||||
The declaration model is complete — it is the design's shape, and validating against it is
|
||||
useful before any of it can be raised. **The runtime is not**, and the gap is refused rather
|
||||
than ignored:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| segments as isolated links | **works** |
|
||||
| machines, multi-homed or detached | **works** |
|
||||
| declared addresses, both families | **works** |
|
||||
| segment MTU | **works** |
|
||||
| raise · exec · snapshot · restore · destroy · list | **works** |
|
||||
| gateways, NAT, forwarding | **refused at raise** |
|
||||
| `published:` ports | **refused at raise** |
|
||||
| `policy:` between segments | **refused at raise** |
|
||||
| `inbound: deny` | **refused at raise** |
|
||||
| `place:` | **refused at raise** |
|
||||
|
||||
`raise` refuses a scenario declaring anything in the lower half, naming every gap. It does not
|
||||
raise a mesh that silently lacks what it declared — that is the fault this lab exists to catch
|
||||
(`novox/hq` 04-ISSUES/003: a firewall key declared in five manifests and read by no code, so a
|
||||
manifest appears to restrict a port and restricts nothing).
|
||||
|
||||
`the-ordinary-shape.yml` therefore validates and does not raise. That is the intended state:
|
||||
it is the topology being built toward, and the tool says exactly what is missing.
|
||||
|
||||
## Measured on a workstation
|
||||
|
||||
| | one machine | two machines |
|
||||
|---|---|---|
|
||||
| raise, to usable | 12.5 s | 14.6 s |
|
||||
| snapshot | 0.14 s | 0.28 s |
|
||||
| restore, to usable again | 10.5 s | 11.6 s |
|
||||
|
||||
Machines boot concurrently, so a second machine costs seconds rather than doubling the wait.
|
||||
Nearly all of the remaining time is boot, which cannot be avoided.
|
||||
|
||||
These numbers depend entirely on a copy-on-write pool. On `dir` the same snapshot takes 9.9 s
|
||||
and a full copy of the disk, and a second one did not finish in two minutes — which is why
|
||||
`check` refuses rather than warns.
|
||||
|
||||
## Where the reasoning lives
|
||||
|
||||
Design and decisions are in [`novox/hq`](https://git.novox.be/novox/hq), not here:
|
||||
|
||||
- `03-DESIGN/01-to-be/01-end-to-end-testing.md` — the design
|
||||
- `02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md` — a lab node is a virtual machine
|
||||
- `02-DECISIONS/0029-the-labs-first-scenario-has-no-pipeline.md` — two scenario classes, and why this is first
|
||||
- `02-DECISIONS/0030-the-repository-structure.md` — why this is its own repository
|
||||
- `03-DESIGN/01-to-be/02-scenario-declaration.md` — what a scenario declares
|
||||
- `03-DESIGN/01-to-be/03-scenario-lifecycle.md` — what happens to one
|
||||
- `02-DECISIONS/0031-the-lab-provides-the-underlay.md`
|
||||
- `02-DECISIONS/0032-a-scenario-is-an-isolated-address-space.md`
|
||||
|
||||
This repository carries implementation. It does not carry decisions.
|
||||
|
||||
## Development
|
||||
|
||||
No build step — Node strips the types.
|
||||
|
||||
```
|
||||
npm test the declaration layer, offline
|
||||
npm run typecheck
|
||||
```
|
||||
|
||||
The lifecycle is not unit-tested. It talks to a hypervisor, and a fake one would assert that
|
||||
the fake behaves as expected — which is the shape of test this project exists to stop
|
||||
shipping. It is exercised by raising real scenarios.
|
||||
|
||||
Generated
+68
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"name": "@novox/mesh-lab",
|
||||
"version": "0.1.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "@novox/mesh-lab",
|
||||
"version": "0.1.0",
|
||||
"dependencies": {
|
||||
"yaml": "^2.6.0"
|
||||
},
|
||||
"bin": {
|
||||
"mesh-lab": "dist/cli.js"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"typescript": "^5.6.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/node": {
|
||||
"version": "22.20.1",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.1.tgz",
|
||||
"integrity": "sha512-EANqOCF9QFyra+4pfxUcX9STKJpCLjMbObVzljIJomAWSnuSIEAvyzEU53GaajbXJEgdh0iEcPL+DGvpUd4k1Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"undici-types": "~6.21.0"
|
||||
}
|
||||
},
|
||||
"node_modules/typescript": {
|
||||
"version": "5.9.3",
|
||||
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
|
||||
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"tsc": "bin/tsc",
|
||||
"tsserver": "bin/tsserver"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/undici-types": {
|
||||
"version": "6.21.0",
|
||||
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
|
||||
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/yaml": {
|
||||
"version": "2.9.0",
|
||||
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
|
||||
"integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"yaml": "bin.mjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 14.6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/eemeli"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"name": "@novox/mesh-lab",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"mesh-lab": "./src/cli.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
|
||||
},
|
||||
"dependencies": {
|
||||
"yaml": "^2.6.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"typescript": "^5.6.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
# The smallest useful scenario: one machine, one public network, nothing else.
|
||||
#
|
||||
# This is the bootstrap class — no forge, no control plane, no pipeline. It exists to
|
||||
# develop the node host, which is why it is the first thing the lab can raise.
|
||||
scenario: bootstrap-single
|
||||
|
||||
segments:
|
||||
hosting:
|
||||
kind: public
|
||||
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
|
||||
|
||||
machines:
|
||||
anchor:
|
||||
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
|
||||
inbound: allow
|
||||
|
||||
# No `place:` yet. The node host it would place does not exist — this lab is being built to
|
||||
# develop it. Declaring it anyway would make the scenario unraisable, and correctly so: the
|
||||
# lab refuses declarations it cannot materialise rather than raising a mesh that silently
|
||||
# lacks them.
|
||||
|
||||
snapshot: raised
|
||||
@@ -0,0 +1,81 @@
|
||||
# One machine with a routable address, one publicly named but behind a household
|
||||
# connection, one stationary machine on that network, one that roams.
|
||||
#
|
||||
# Three unrelated public networks, routed to each other and never bridged — putting them
|
||||
# in one prefix would make ARP adjacency, non-decrementing TTL and crossing multicast
|
||||
# true in the lab and false in production.
|
||||
scenario: the-ordinary-shape
|
||||
|
||||
segments:
|
||||
hosting:
|
||||
kind: public
|
||||
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
|
||||
|
||||
isp-home:
|
||||
kind: public
|
||||
cidr: [198.51.100.0/24, "2001:db8:b::/48"]
|
||||
|
||||
isp-mobile:
|
||||
kind: public
|
||||
cidr: [203.0.113.0/24, "2001:db8:c::/48"]
|
||||
|
||||
home:
|
||||
kind: private
|
||||
cidr: [192.168.1.0/24, "2001:db8:b:1::/64"]
|
||||
mtu: 1492
|
||||
gateway:
|
||||
to: isp-home
|
||||
address: [198.51.100.7, "2001:db8:b::7"]
|
||||
nat: [v4]
|
||||
forwardable: true
|
||||
mapping_ttl: 120s
|
||||
|
||||
devices:
|
||||
kind: private
|
||||
cidr: [192.168.30.0/24]
|
||||
gateway:
|
||||
to: isp-home
|
||||
address: [198.51.100.7]
|
||||
nat: [v4]
|
||||
forwardable: true
|
||||
mapping_ttl: 120s
|
||||
|
||||
cafe:
|
||||
kind: private
|
||||
cidr: [10.50.0.0/16]
|
||||
mtu: 1400
|
||||
gateway:
|
||||
to: isp-mobile
|
||||
address: [203.0.113.129]
|
||||
nat: [v4]
|
||||
forwardable: false
|
||||
mapping_ttl: 30s
|
||||
|
||||
policy:
|
||||
- { from: devices, to: home, allow: false }
|
||||
- { from: home, to: devices, allow: true }
|
||||
|
||||
machines:
|
||||
anchor:
|
||||
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
|
||||
inbound: allow
|
||||
|
||||
home-server:
|
||||
at: { segment: home, address: [192.168.1.135, "2001:db8:b:1::135"] }
|
||||
published:
|
||||
- { port: 443, on: home }
|
||||
inbound: allow
|
||||
|
||||
workstation:
|
||||
at: { segment: home, address: [192.168.1.250, "2001:db8:b:1::250"] }
|
||||
inbound: deny
|
||||
|
||||
laptop:
|
||||
at: { segment: home, address: [192.168.1.98, "2001:db8:b:1::98"] }
|
||||
inbound: deny
|
||||
|
||||
place:
|
||||
all: [host]
|
||||
anchor: [substrate]
|
||||
|
||||
snapshot: raised
|
||||
@@ -0,0 +1,23 @@
|
||||
# Two machines on one public network. The smallest scenario in which reachability is a
|
||||
# question at all — and the first that can be answered from inside.
|
||||
scenario: two-on-a-segment
|
||||
|
||||
segments:
|
||||
hosting:
|
||||
kind: public
|
||||
cidr: [192.0.2.0/24, "2001:db8:a::/48"]
|
||||
|
||||
machines:
|
||||
anchor:
|
||||
at: { segment: hosting, address: [192.0.2.10, "2001:db8:a::10"] }
|
||||
inbound: allow
|
||||
peer:
|
||||
at: { segment: hosting, address: [192.0.2.20, "2001:db8:a::20"] }
|
||||
inbound: allow
|
||||
|
||||
# No `place:` yet. The node host it would place does not exist — this lab is being built to
|
||||
# develop it. Declaring it anyway would make the scenario unraisable, and correctly so: the
|
||||
# lab refuses declarations it cannot materialise rather than raising a mesh that silently
|
||||
# lacks them.
|
||||
|
||||
snapshot: raised
|
||||
Executable
+188
@@ -0,0 +1,188 @@
|
||||
#!/usr/bin/env -S node --experimental-strip-types
|
||||
/**
|
||||
* The lab's two callers want different things from the same verbs: a coordinator wants
|
||||
* structured results and clean teardown, a person wants readable output and the failing
|
||||
* scenario left standing. Both use these verbs; the difference is what happens after.
|
||||
*/
|
||||
|
||||
import { loadScenario } from "./declaration/parse.ts";
|
||||
import { DeclarationError } from "./declaration/validate.ts";
|
||||
import { isReachable, supportedDrivers, pools } from "./incus/client.ts";
|
||||
import { raise, RaiseError } from "./lifecycle/raise.ts";
|
||||
import { assertSupported, UnsupportedError } from "./lifecycle/supported.ts";
|
||||
import { destroy, exec, list, restore, snapshot, snapshots } from "./lifecycle/operate.ts";
|
||||
|
||||
const USAGE = `mesh-lab — raise a disposable mesh on one machine
|
||||
|
||||
check verify this machine can run scenarios
|
||||
validate <scenario.yml> parse and check a declaration, raising nothing
|
||||
raise <scenario.yml> materialise it, and wait until the machines are usable
|
||||
list scenario instances currently standing
|
||||
exec <instance> <machine> -- <cmd...>
|
||||
snapshot <instance> <label> capture the whole scenario as one state
|
||||
restore <instance> <label> return the whole scenario to it
|
||||
snapshots <instance>
|
||||
destroy <instance>
|
||||
|
||||
Set MESH_LAB_INCUS if the daemon needs a different invocation, e.g. "sudo -n incus".
|
||||
`;
|
||||
|
||||
function fail(message: string): never {
|
||||
console.error(message);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuse to run degraded rather than warning. A warning about a slow inner loop is read
|
||||
* once and ignored forever, and the loop stays slow.
|
||||
*/
|
||||
async function check(): Promise<void> {
|
||||
const problems: string[] = [];
|
||||
|
||||
if (!(await isReachable())) {
|
||||
problems.push(
|
||||
"the incus daemon is not reachable as this user. If the group was granted recently, " +
|
||||
"a session that predates it cannot see it — log out and back in, or set MESH_LAB_INCUS.",
|
||||
);
|
||||
console.error(problems.map((p) => ` ✗ ${p}`).join("\n"));
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(" ✓ daemon reachable");
|
||||
|
||||
const drivers = await supportedDrivers();
|
||||
const cowDrivers = drivers.filter((d) => d === "btrfs" || d === "zfs");
|
||||
if (cowDrivers.length === 0) {
|
||||
problems.push(
|
||||
"no copy-on-write storage driver is offered. Snapshots would be full copies — " +
|
||||
"measured at roughly 76x slower, which does not make the lab slow, it makes it unused.",
|
||||
);
|
||||
} else {
|
||||
console.log(` ✓ copy-on-write driver available (${cowDrivers.join(", ")})`);
|
||||
}
|
||||
|
||||
const available = await pools();
|
||||
const cowPool = available.find((p) => p.driver === "btrfs" || p.driver === "zfs");
|
||||
if (!cowPool) {
|
||||
problems.push(
|
||||
`no pool uses a copy-on-write driver (have: ${available.map((p) => `${p.name}/${p.driver}`).join(", ") || "none"}). ` +
|
||||
"A pool that exists and is the slow kind is the failure with no symptom.",
|
||||
);
|
||||
} else {
|
||||
console.log(` ✓ copy-on-write pool '${cowPool.name}' (${cowPool.driver})`);
|
||||
}
|
||||
|
||||
if (problems.length > 0) {
|
||||
console.error(`\n${problems.map((p) => ` ✗ ${p}`).join("\n\n")}`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log("\nthis machine can run scenarios");
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const [verb, ...rest] = process.argv.slice(2);
|
||||
|
||||
switch (verb) {
|
||||
case "check":
|
||||
return check();
|
||||
|
||||
case "validate": {
|
||||
const path = rest[0] ?? fail("validate needs a scenario file");
|
||||
const scenario = loadScenario(path);
|
||||
console.log(
|
||||
`${scenario.scenario}: ${Object.keys(scenario.segments).length} segments, ` +
|
||||
`${Object.keys(scenario.machines).length} machines — valid`,
|
||||
);
|
||||
// Valid and raisable are different questions, and a scenario can be the first
|
||||
// without being the second.
|
||||
try {
|
||||
assertSupported(scenario);
|
||||
} catch (err) {
|
||||
if (err instanceof UnsupportedError) {
|
||||
console.log(`\nnot yet raisable:\n - ${err.missing.join("\n - ")}`);
|
||||
} else throw err;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
case "raise": {
|
||||
const path = rest[0] ?? fail("raise needs a scenario file");
|
||||
const scenario = loadScenario(path);
|
||||
const started = Date.now();
|
||||
const raised = await raise(scenario, {
|
||||
onProgress: (m) => console.log(m),
|
||||
...(process.env["MESH_LAB_IMAGE"] ? { image: process.env["MESH_LAB_IMAGE"] } : {}),
|
||||
});
|
||||
const seconds = ((Date.now() - started) / 1000).toFixed(1);
|
||||
console.log(`\nraised ${raised.instanceId} in ${seconds}s — ${raised.machines.length} machines usable`);
|
||||
if (scenario.snapshot) {
|
||||
const took = await snapshot(raised.instanceId, scenario.snapshot);
|
||||
console.log(`snapshot '${scenario.snapshot}' in ${took.toFixed(2)}s`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
case "list": {
|
||||
const found = await list();
|
||||
if (found.length === 0) return console.log("no scenario instances standing");
|
||||
for (const instance of found) {
|
||||
console.log(`${instance.instanceId}`);
|
||||
for (const m of instance.machines) console.log(` ${m.machine.padEnd(16)} ${m.status}`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
case "exec": {
|
||||
const [instanceId, machine, ...command] = rest;
|
||||
if (!instanceId || !machine || command.length === 0) fail("exec <instance> <machine> <cmd...>");
|
||||
const result = await exec(instanceId, machine, command.filter((c) => c !== "--"));
|
||||
process.stdout.write(result.stdout);
|
||||
process.stderr.write(result.stderr);
|
||||
return;
|
||||
}
|
||||
|
||||
case "snapshot": {
|
||||
const [instanceId, label] = rest;
|
||||
if (!instanceId || !label) fail("snapshot <instance> <label>");
|
||||
const took = await snapshot(instanceId, label);
|
||||
console.log(`snapshot '${label}' in ${took.toFixed(2)}s`);
|
||||
return;
|
||||
}
|
||||
|
||||
case "restore": {
|
||||
const [instanceId, label] = rest;
|
||||
if (!instanceId || !label) fail("restore <instance> <label>");
|
||||
const { restoreSeconds, usableSeconds } = await restore(instanceId, label, 180, (m) =>
|
||||
console.log(m),
|
||||
);
|
||||
console.log(
|
||||
`restored '${label}' in ${restoreSeconds.toFixed(2)}s — usable again after ` +
|
||||
`${usableSeconds.toFixed(1)}s`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
case "snapshots": {
|
||||
const instanceId = rest[0] ?? fail("snapshots <instance>");
|
||||
const found = await snapshots(instanceId);
|
||||
console.log(found.length ? found.join("\n") : "none");
|
||||
return;
|
||||
}
|
||||
|
||||
case "destroy": {
|
||||
const instanceId = rest[0] ?? fail("destroy <instance>");
|
||||
const { machines, networks } = await destroy(instanceId);
|
||||
console.log(`destroyed ${instanceId} — ${machines} machines, ${networks} segments`);
|
||||
return;
|
||||
}
|
||||
|
||||
default:
|
||||
console.log(USAGE);
|
||||
process.exit(verb ? 1 : 0);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err: unknown) => {
|
||||
if (err instanceof DeclarationError || err instanceof RaiseError) fail(err.message);
|
||||
if (err instanceof UnsupportedError) fail(err.message);
|
||||
fail(err instanceof Error ? err.message : String(err));
|
||||
});
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* Address parsing, enough to answer two questions the validator asks: which family is
|
||||
* this, and is it inside that range.
|
||||
*
|
||||
* Written rather than depended on because it is small, and because the one rule it
|
||||
* exists to enforce — public segments use documentation ranges — is the difference
|
||||
* between a lab that reproduces the internet and one that silently never forms a mesh.
|
||||
*/
|
||||
|
||||
export type Family = "v4" | "v6";
|
||||
|
||||
export interface Cidr {
|
||||
family: Family;
|
||||
/** Network address, as an integer. */
|
||||
base: bigint;
|
||||
prefix: number;
|
||||
text: string;
|
||||
}
|
||||
|
||||
const V4_BITS = 32n;
|
||||
const V6_BITS = 128n;
|
||||
|
||||
export function familyOf(address: string): Family {
|
||||
return address.includes(":") ? "v6" : "v4";
|
||||
}
|
||||
|
||||
function parseV4(text: string): bigint {
|
||||
const parts = text.split(".");
|
||||
if (parts.length !== 4) throw new Error(`not an IPv4 address: ${text}`);
|
||||
let value = 0n;
|
||||
for (const part of parts) {
|
||||
if (!/^\d{1,3}$/.test(part)) throw new Error(`not an IPv4 address: ${text}`);
|
||||
const octet = Number(part);
|
||||
if (octet > 255) throw new Error(`octet out of range in ${text}`);
|
||||
value = (value << 8n) | BigInt(octet);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function parseV6(text: string): bigint {
|
||||
// Reject the forms this does not implement rather than mis-parsing them. An embedded
|
||||
// IPv4 suffix is legal and rare; getting it wrong silently would be worse than refusing.
|
||||
if (text.includes(".")) throw new Error(`IPv4-in-IPv6 form is not supported: ${text}`);
|
||||
const halves = text.split("::");
|
||||
if (halves.length > 2) throw new Error(`not an IPv6 address: ${text}`);
|
||||
|
||||
const head = halves[0] ? halves[0].split(":").filter(Boolean) : [];
|
||||
const tail = halves.length === 2 && halves[1] ? halves[1].split(":").filter(Boolean) : [];
|
||||
const explicit = head.length + tail.length;
|
||||
if (explicit > 8) throw new Error(`too many groups in ${text}`);
|
||||
if (halves.length === 1 && explicit !== 8) throw new Error(`not an IPv6 address: ${text}`);
|
||||
|
||||
const groups = [...head, ...Array<string>(8 - explicit).fill("0"), ...tail];
|
||||
let value = 0n;
|
||||
for (const group of groups) {
|
||||
if (!/^[0-9a-fA-F]{1,4}$/.test(group)) throw new Error(`not an IPv6 address: ${text}`);
|
||||
value = (value << 16n) | BigInt(parseInt(group, 16));
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function parseAddress(text: string): { family: Family; value: bigint } {
|
||||
const family = familyOf(text);
|
||||
return { family, value: family === "v4" ? parseV4(text) : parseV6(text) };
|
||||
}
|
||||
|
||||
export function parseCidr(text: string): Cidr {
|
||||
const slash = text.lastIndexOf("/");
|
||||
if (slash === -1) throw new Error(`not a CIDR range (no prefix length): ${text}`);
|
||||
const addressText = text.slice(0, slash);
|
||||
const prefix = Number(text.slice(slash + 1));
|
||||
const { family, value } = parseAddress(addressText);
|
||||
const bits = family === "v4" ? V4_BITS : V6_BITS;
|
||||
if (!Number.isInteger(prefix) || prefix < 0 || BigInt(prefix) > bits) {
|
||||
throw new Error(`prefix length out of range for ${family}: ${text}`);
|
||||
}
|
||||
const hostBits = bits - BigInt(prefix);
|
||||
const base = (value >> hostBits) << hostBits;
|
||||
return { family, base, prefix, text };
|
||||
}
|
||||
|
||||
export function contains(range: Cidr, address: string): boolean {
|
||||
const { family, value } = parseAddress(address);
|
||||
if (family !== range.family) return false;
|
||||
const bits = family === "v4" ? V4_BITS : V6_BITS;
|
||||
const hostBits = bits - BigInt(range.prefix);
|
||||
return ((value >> hostBits) << hostBits) === range.base;
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
/** YAML in, a validated Scenario out. Normalises the shorthands the design's examples use. */
|
||||
|
||||
import { readFileSync } from "node:fs";
|
||||
import { parse as parseYaml } from "yaml";
|
||||
import type { Attachment, Machine, Scenario, Segment } from "./types.ts";
|
||||
import { validate } from "./validate.ts";
|
||||
|
||||
/** `address: "1.2.3.4"` and `address: [...]` both mean a list. */
|
||||
function toList(value: unknown): string[] {
|
||||
if (value === undefined || value === null) return [];
|
||||
return Array.isArray(value) ? value.map(String) : [String(value)];
|
||||
}
|
||||
|
||||
function normaliseAttachment(raw: unknown): Attachment {
|
||||
const at = (raw ?? {}) as Record<string, unknown>;
|
||||
return { segment: String(at["segment"] ?? ""), address: toList(at["address"]) };
|
||||
}
|
||||
|
||||
function normaliseMachine(raw: unknown): Machine {
|
||||
const machine = (raw ?? {}) as Record<string, unknown>;
|
||||
const at = machine["at"];
|
||||
|
||||
const attachment: Machine["at"] =
|
||||
at === "detached"
|
||||
? "detached"
|
||||
: Array.isArray(at)
|
||||
? at.map(normaliseAttachment)
|
||||
: [normaliseAttachment(at)];
|
||||
|
||||
const result: Machine = { at: attachment };
|
||||
const published = machine["published"];
|
||||
if (Array.isArray(published)) {
|
||||
result.published = published.map((entry) => {
|
||||
const p = (entry ?? {}) as Record<string, unknown>;
|
||||
return { port: Number(p["port"]), on: String(p["on"] ?? "") };
|
||||
});
|
||||
}
|
||||
const inbound = machine["inbound"];
|
||||
if (inbound === "allow" || inbound === "deny") result.inbound = inbound;
|
||||
return result;
|
||||
}
|
||||
|
||||
function normaliseSegment(raw: unknown): Segment {
|
||||
const segment = (raw ?? {}) as Record<string, unknown>;
|
||||
const result: Segment = {
|
||||
kind: segment["kind"] === "public" ? "public" : "private",
|
||||
cidr: toList(segment["cidr"]),
|
||||
};
|
||||
if (segment["mtu"] !== undefined) result.mtu = Number(segment["mtu"]);
|
||||
|
||||
const gateway = segment["gateway"] as Record<string, unknown> | undefined;
|
||||
if (gateway) {
|
||||
const nat = toList(gateway["nat"]).filter((f): f is "v4" | "v6" => f === "v4" || f === "v6");
|
||||
result.gateway = {
|
||||
to: String(gateway["to"] ?? ""),
|
||||
address: toList(gateway["address"]),
|
||||
nat,
|
||||
// Absent means forwardable: a gateway you control is the ordinary case, and the
|
||||
// interesting one — carrier-grade NAT — should have to be stated.
|
||||
forwardable: gateway["forwardable"] !== false,
|
||||
};
|
||||
const ttl = gateway["mapping_ttl"] ?? gateway["mappingTtl"];
|
||||
if (ttl !== undefined) result.gateway.mappingTtl = String(ttl);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export function parseScenario(text: string): Scenario {
|
||||
const raw = (parseYaml(text) ?? {}) as Record<string, unknown>;
|
||||
|
||||
const segments: Record<string, Segment> = {};
|
||||
for (const [name, value] of Object.entries(
|
||||
(raw["segments"] ?? {}) as Record<string, unknown>,
|
||||
)) {
|
||||
segments[name] = normaliseSegment(value);
|
||||
}
|
||||
|
||||
const machines: Record<string, Machine> = {};
|
||||
for (const [name, value] of Object.entries(
|
||||
(raw["machines"] ?? {}) as Record<string, unknown>,
|
||||
)) {
|
||||
machines[name] = normaliseMachine(value);
|
||||
}
|
||||
|
||||
const scenario: Scenario = {
|
||||
scenario: String(raw["scenario"] ?? ""),
|
||||
segments,
|
||||
machines,
|
||||
};
|
||||
|
||||
const policy = raw["policy"];
|
||||
if (Array.isArray(policy)) {
|
||||
scenario.policy = policy.map((entry) => {
|
||||
const p = (entry ?? {}) as Record<string, unknown>;
|
||||
return { from: String(p["from"] ?? ""), to: String(p["to"] ?? ""), allow: p["allow"] !== false };
|
||||
});
|
||||
}
|
||||
|
||||
const place = raw["place"];
|
||||
if (place && typeof place === "object") {
|
||||
const normalised: Record<string, string[]> = {};
|
||||
for (const [key, value] of Object.entries(place as Record<string, unknown>)) {
|
||||
normalised[key] = toList(value);
|
||||
}
|
||||
scenario.place = normalised;
|
||||
}
|
||||
|
||||
if (raw["snapshot"] !== undefined) scenario.snapshot = String(raw["snapshot"]);
|
||||
|
||||
validate(scenario);
|
||||
return scenario;
|
||||
}
|
||||
|
||||
export function loadScenario(path: string): Scenario {
|
||||
return parseScenario(readFileSync(path, "utf-8"));
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* A scenario declares an UNDERLAY and what to place on it — the facts a machine would
|
||||
* have before any of our software touched it. It declares nothing the mesh is
|
||||
* responsible for: no overlay addresses, no hub, no peering, no names, no certificates.
|
||||
* Those are outcomes to observe, and a scenario that supplied them would be certifying
|
||||
* its own work.
|
||||
*
|
||||
* See novox/hq: 02-DECISIONS/0031-the-lab-provides-the-underlay.md
|
||||
* 03-DESIGN/01-to-be/02-scenario-declaration.md
|
||||
*/
|
||||
|
||||
/** An IP family. Reachability is a property of (machine, family), never of a machine. */
|
||||
export type Family = "v4" | "v6";
|
||||
|
||||
/**
|
||||
* How a segment reaches its parent.
|
||||
*
|
||||
* `address` is the address the outside world sees the network as — for a household
|
||||
* connection, what the ISP hands out. It is load-bearing rather than decorative: it is
|
||||
* what a peer records as an endpoint when a machine here dials out, and what a public
|
||||
* name for a published machine here resolves to.
|
||||
*/
|
||||
export interface Gateway {
|
||||
/** Parent segment name. */
|
||||
to: string;
|
||||
/** Addresses the gateway holds on the parent segment, one per family. */
|
||||
address: string[];
|
||||
/**
|
||||
* Which families are translated. `["v4"]` is the modern default — v4 translated, v6
|
||||
* routed. `[]` is a routed range where machines keep their own addresses.
|
||||
*/
|
||||
nat: Family[];
|
||||
/**
|
||||
* Whether an inbound mapping can be created. Independent of `nat`, and the field that
|
||||
* separates a home gateway from carrier-grade NAT — which is your own connection and
|
||||
* still unforwardable.
|
||||
*/
|
||||
forwardable: boolean;
|
||||
/**
|
||||
* How long an unused inbound mapping survives, e.g. "120s". Absent means mappings never
|
||||
* expire, which no real gateway does — so absence is a simplification, not a default.
|
||||
*/
|
||||
mappingTtl?: string;
|
||||
}
|
||||
|
||||
/** A broadcast domain. Several public segments are unrelated and routed, never bridged. */
|
||||
export interface Segment {
|
||||
/**
|
||||
* `public` stands in for a public network — and there is normally more than one,
|
||||
* unrelated to each other. `private` is everything else; a private segment with no
|
||||
* gateway is an island that reaches nothing.
|
||||
*/
|
||||
kind: "public" | "private";
|
||||
/** Address ranges, one per family. */
|
||||
cidr: string[];
|
||||
/** Largest packet the segment carries. Default 1500. Lower reproduces tunnelled paths. */
|
||||
mtu?: number;
|
||||
gateway?: Gateway;
|
||||
}
|
||||
|
||||
/** Where a machine sits: a segment and the addresses it holds there. */
|
||||
export interface Attachment {
|
||||
segment: string;
|
||||
address: string[];
|
||||
}
|
||||
|
||||
/** A destination-NAT rule on a named gateway, stated as an outcome rather than a port list. */
|
||||
export interface Publication {
|
||||
port: number;
|
||||
/** The segment whose gateway forwards. Named, because a machine may sit behind several. */
|
||||
on: string;
|
||||
}
|
||||
|
||||
export interface Machine {
|
||||
/**
|
||||
* One attachment, or several for a machine on multiple segments at once. Multi-homing
|
||||
* is not exotic: it is what any node with both a LAN and a WAN interface is.
|
||||
* `"detached"` is a machine on no segment — it exists and reaches nothing.
|
||||
*/
|
||||
at: Attachment[] | "detached";
|
||||
published?: Publication[];
|
||||
/**
|
||||
* A host firewall. Distinct from NAT and behaves differently: a machine can be perfectly
|
||||
* routable and still refuse everything unsolicited, which is the normal state of a
|
||||
* v6-addressed machine. Without this, v6 addressing would imply reachability.
|
||||
*/
|
||||
inbound?: "allow" | "deny";
|
||||
}
|
||||
|
||||
/** Reachability between segments, as a segmented router enforces it. Asymmetric by design. */
|
||||
export interface Policy {
|
||||
from: string;
|
||||
to: string;
|
||||
allow: boolean;
|
||||
}
|
||||
|
||||
/** What goes inside the machines. The ONLY part that differs between scenario classes. */
|
||||
export interface Placement {
|
||||
/** Applied to every machine. */
|
||||
all?: string[];
|
||||
/** Per-machine, overriding `all` for that machine. */
|
||||
[machine: string]: string[] | undefined;
|
||||
}
|
||||
|
||||
export interface Scenario {
|
||||
/** The kind. Instances are many; this names the shape, not one of them. */
|
||||
scenario: string;
|
||||
segments: Record<string, Segment>;
|
||||
machines: Record<string, Machine>;
|
||||
policy?: Policy[];
|
||||
place?: Placement;
|
||||
/** Name the state once placement finishes, so a run can return to it. */
|
||||
snapshot?: string;
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
/**
|
||||
* Every rule the design states about a scenario, checked before anything is raised.
|
||||
*
|
||||
* The reason validation is this strict is that the failures it prevents are silent. A
|
||||
* private range on a segment meant to be public does not produce an error — the mesh
|
||||
* simply never forms, because its own code decides public-versus-private by matching the
|
||||
* address. Publishing through a gateway that cannot forward does not produce an error
|
||||
* either; it produces a machine that looks reachable and is not.
|
||||
*
|
||||
* So: refuse the declaration, loudly, before spending a minute raising something that
|
||||
* would have taught us the wrong thing.
|
||||
*
|
||||
* See novox/hq 03-DESIGN/01-to-be/02-scenario-declaration.md
|
||||
*/
|
||||
|
||||
import type { Scenario, Segment } from "./types.ts";
|
||||
import { contains, familyOf, parseAddress, parseCidr, type Cidr } from "./net.ts";
|
||||
|
||||
/** RFC 5737 and RFC 3849. The only addresses guaranteed never to route on the real internet. */
|
||||
const DOCUMENTATION_RANGES = [
|
||||
"192.0.2.0/24",
|
||||
"198.51.100.0/24",
|
||||
"203.0.113.0/24",
|
||||
"2001:db8::/32",
|
||||
].map(parseCidr);
|
||||
|
||||
export class DeclarationError extends Error {
|
||||
readonly problems: string[];
|
||||
|
||||
constructor(problems: string[]) {
|
||||
super(`scenario declaration is not valid:\n - ${problems.join("\n - ")}`);
|
||||
this.name = "DeclarationError";
|
||||
this.problems = problems;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a range sits inside documentation space. Compared as ranges rather than as a
|
||||
* sample address so that a range wider than the reserved block — `203.0.0.0/8`, say — is
|
||||
* correctly refused instead of passing because its first address happens to fall inside.
|
||||
*/
|
||||
function isDocumentationRange(range: Cidr): boolean {
|
||||
return DOCUMENTATION_RANGES.some(
|
||||
(allowed) =>
|
||||
allowed.family === range.family &&
|
||||
range.prefix >= allowed.prefix &&
|
||||
containsRange(allowed, range),
|
||||
);
|
||||
}
|
||||
|
||||
/** Is `inner` entirely within `outer`? Both already parsed, so no address parsing can throw. */
|
||||
function containsRange(outer: Cidr, inner: Cidr): boolean {
|
||||
const bits = outer.family === "v4" ? 32n : 128n;
|
||||
const hostBits = bits - BigInt(outer.prefix);
|
||||
return ((inner.base >> hostBits) << hostBits) === outer.base;
|
||||
}
|
||||
|
||||
/** Ranges of a segment, parsed once, with malformed entries reported rather than thrown. */
|
||||
function rangesOf(name: string, segment: Segment, problems: string[]): Cidr[] {
|
||||
const ranges: Cidr[] = [];
|
||||
for (const text of segment.cidr) {
|
||||
try {
|
||||
ranges.push(parseCidr(text));
|
||||
} catch (err) {
|
||||
problems.push(`segment '${name}': ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
return ranges;
|
||||
}
|
||||
|
||||
function inAnyRange(ranges: Cidr[], address: string): boolean {
|
||||
return ranges.some((range) => contains(range, address));
|
||||
}
|
||||
|
||||
export function validate(scenario: Scenario): void {
|
||||
const problems: string[] = [];
|
||||
const segmentNames = Object.keys(scenario.segments);
|
||||
|
||||
if (!scenario.scenario) problems.push("scenario has no name");
|
||||
if (segmentNames.length === 0) problems.push("scenario declares no segments");
|
||||
if (Object.keys(scenario.machines).length === 0) {
|
||||
problems.push("scenario declares no machines");
|
||||
}
|
||||
|
||||
const ranges = new Map<string, Cidr[]>();
|
||||
for (const [name, segment] of Object.entries(scenario.segments)) {
|
||||
ranges.set(name, rangesOf(name, segment, problems));
|
||||
}
|
||||
|
||||
for (const [name, segment] of Object.entries(scenario.segments)) {
|
||||
// A public segment stands in for the internet. Anything that is not documentation
|
||||
// space could route somewhere real, and — far more likely — a private range here
|
||||
// makes the mesh's own public-versus-private test fail silently.
|
||||
if (segment.kind === "public") {
|
||||
// Only ranges that parsed — a malformed one is already reported, and re-parsing it
|
||||
// here would throw out of validation with a single cryptic message instead of the
|
||||
// full list.
|
||||
for (const range of ranges.get(name) ?? []) {
|
||||
if (!isDocumentationRange(range)) {
|
||||
problems.push(
|
||||
`segment '${name}' is public but '${range.text}' is not documentation space ` +
|
||||
`(RFC 5737 / RFC 3849). A non-documentation range here either routes somewhere ` +
|
||||
`real, or — if private — makes the mesh silently never form.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (segment.mtu !== undefined && (segment.mtu < 576 || segment.mtu > 9000)) {
|
||||
problems.push(`segment '${name}': mtu ${segment.mtu} is outside any plausible range`);
|
||||
}
|
||||
|
||||
const gateway = segment.gateway;
|
||||
if (!gateway) continue;
|
||||
|
||||
if (!segmentNames.includes(gateway.to)) {
|
||||
problems.push(`segment '${name}': gateway points at unknown segment '${gateway.to}'`);
|
||||
continue;
|
||||
}
|
||||
if (gateway.to === name) {
|
||||
problems.push(`segment '${name}': gateway points at itself`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// The gateway's address is what the world sees this network as, so it belongs to the
|
||||
// PARENT segment, not this one. Getting this backwards is easy and produces a topology
|
||||
// that raises fine and reproduces nothing.
|
||||
const parentRanges = ranges.get(gateway.to) ?? [];
|
||||
for (const address of gateway.address) {
|
||||
try {
|
||||
parseAddress(address);
|
||||
} catch (err) {
|
||||
problems.push(`segment '${name}' gateway: ${(err as Error).message}`);
|
||||
continue;
|
||||
}
|
||||
if (parentRanges.length > 0 && !inAnyRange(parentRanges, address)) {
|
||||
problems.push(
|
||||
`segment '${name}' gateway: address '${address}' is not within '${gateway.to}' ` +
|
||||
`(${scenario.segments[gateway.to]?.cidr.join(", ")}). A gateway's address is the ` +
|
||||
`one the parent network sees, not one on the segment behind it.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const family of gateway.nat) {
|
||||
if (family !== "v4" && family !== "v6") {
|
||||
problems.push(`segment '${name}' gateway: '${family}' is not an address family`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A gateway chain must terminate. A cycle would raise forever rather than fail.
|
||||
for (const name of segmentNames) {
|
||||
const seen = new Set<string>([name]);
|
||||
let current = scenario.segments[name]?.gateway?.to;
|
||||
while (current) {
|
||||
if (seen.has(current)) {
|
||||
problems.push(`segment '${name}': gateway chain loops through '${current}'`);
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
current = scenario.segments[current]?.gateway?.to;
|
||||
}
|
||||
}
|
||||
|
||||
for (const [name, machine] of Object.entries(scenario.machines)) {
|
||||
if (machine.at === "detached") {
|
||||
if (machine.published?.length) {
|
||||
problems.push(`machine '${name}' is detached but declares published ports`);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!Array.isArray(machine.at) || machine.at.length === 0) {
|
||||
problems.push(`machine '${name}': 'at' must be an attachment, a list of them, or "detached"`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const attachedTo = new Set<string>();
|
||||
for (const attachment of machine.at) {
|
||||
if (!segmentNames.includes(attachment.segment)) {
|
||||
problems.push(`machine '${name}': unknown segment '${attachment.segment}'`);
|
||||
continue;
|
||||
}
|
||||
if (attachedTo.has(attachment.segment)) {
|
||||
problems.push(`machine '${name}': attached to '${attachment.segment}' more than once`);
|
||||
}
|
||||
attachedTo.add(attachment.segment);
|
||||
|
||||
const segmentRanges = ranges.get(attachment.segment) ?? [];
|
||||
const seenFamilies = new Set<string>();
|
||||
for (const address of attachment.address) {
|
||||
try {
|
||||
parseAddress(address);
|
||||
} catch (err) {
|
||||
problems.push(`machine '${name}': ${(err as Error).message}`);
|
||||
continue;
|
||||
}
|
||||
const family = familyOf(address);
|
||||
if (seenFamilies.has(family)) {
|
||||
problems.push(`machine '${name}': two ${family} addresses on '${attachment.segment}'`);
|
||||
}
|
||||
seenFamilies.add(family);
|
||||
|
||||
if (segmentRanges.length > 0 && !inAnyRange(segmentRanges, address)) {
|
||||
problems.push(
|
||||
`machine '${name}': address '${address}' is not within segment ` +
|
||||
`'${attachment.segment}' (${scenario.segments[attachment.segment]?.cidr.join(", ")})`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const publication of machine.published ?? []) {
|
||||
if (!Number.isInteger(publication.port) || publication.port < 1 || publication.port > 65535) {
|
||||
problems.push(`machine '${name}': port ${publication.port} is not a port`);
|
||||
}
|
||||
const via = scenario.segments[publication.on];
|
||||
if (!via) {
|
||||
problems.push(`machine '${name}': publishes on unknown segment '${publication.on}'`);
|
||||
continue;
|
||||
}
|
||||
if (!attachedTo.has(publication.on)) {
|
||||
problems.push(
|
||||
`machine '${name}': publishes on '${publication.on}' but is not attached to it`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (!via.gateway) {
|
||||
problems.push(
|
||||
`machine '${name}': publishes on '${publication.on}', which has no gateway to ` +
|
||||
`forward through`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
// The constraint being reproduced, not an implementation limit: a machine behind a
|
||||
// gateway it does not control cannot be published, and pretending otherwise would
|
||||
// make the lab certify something production cannot do.
|
||||
if (!via.gateway.forwardable) {
|
||||
problems.push(
|
||||
`machine '${name}': cannot publish through '${publication.on}' — its gateway is ` +
|
||||
`not forwardable. That is the constraint being reproduced, not a limitation.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const rule of scenario.policy ?? []) {
|
||||
for (const side of [rule.from, rule.to]) {
|
||||
if (!segmentNames.includes(side)) {
|
||||
problems.push(`policy: unknown segment '${side}'`);
|
||||
}
|
||||
}
|
||||
if (rule.from === rule.to) {
|
||||
problems.push(`policy: '${rule.from}' to itself is not a rule`);
|
||||
}
|
||||
}
|
||||
|
||||
for (const machine of Object.keys(scenario.place ?? {})) {
|
||||
if (machine === "all") continue;
|
||||
if (!(machine in scenario.machines)) {
|
||||
problems.push(`place: '${machine}' is not a machine in this scenario`);
|
||||
}
|
||||
}
|
||||
|
||||
if (problems.length > 0) throw new DeclarationError(problems);
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* A thin wrapper over the incus CLI. Thin on purpose: the lab's value is in the scenario
|
||||
* model, not in re-describing a hypervisor.
|
||||
*
|
||||
* Everything here goes through incus's own channel and never over IP. A scenario is a
|
||||
* closed address space — two scenarios raised from one declaration hold the same
|
||||
* addresses and must never meet — so the workstation has no route into either, and
|
||||
* reaching a machine by address would make concurrency impossible in the worst way:
|
||||
* not with an error, but with one scenario's traffic arriving in another.
|
||||
*
|
||||
* See novox/hq 02-DECISIONS/0032-a-scenario-is-an-isolated-address-space.md
|
||||
*/
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
/**
|
||||
* How to invoke incus. Overridable because the socket is group-owned and a session that
|
||||
* predates the group grant cannot reach it — which is a real thing that happens on the
|
||||
* machine that just installed it.
|
||||
*/
|
||||
const INCUS = (process.env["MESH_LAB_INCUS"] ?? "incus").split(" ").filter(Boolean);
|
||||
|
||||
export interface IncusResult {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
}
|
||||
|
||||
export class IncusError extends Error {
|
||||
readonly args: string[];
|
||||
readonly stderr: string;
|
||||
readonly exitCode: number | null;
|
||||
|
||||
constructor(args: string[], stderr: string, exitCode: number | null) {
|
||||
const detail = stderr.trim() || (exitCode === null ? "timed out" : `exit ${exitCode}`);
|
||||
super(`incus ${args.join(" ")} failed: ${detail}`);
|
||||
this.name = "IncusError";
|
||||
this.args = args;
|
||||
this.stderr = stderr;
|
||||
this.exitCode = exitCode;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs incus with **stdin closed**, which is not incidental.
|
||||
*
|
||||
* Several incus subcommands accept a YAML definition on stdin and, given a descriptor
|
||||
* that is not a terminal, wait for one. Under a shell that never happens because stdin is
|
||||
* a TTY; under a spawned process it hangs until the timeout kills it — and the timeout
|
||||
* kill produces an empty stderr, so the failure arrives with no explanation at all.
|
||||
*
|
||||
* Found exactly that way: the first raise reported "failed: (no output)" on a command
|
||||
* that worked perfectly when typed.
|
||||
*/
|
||||
export async function incus(args: string[], timeoutMs = 60_000): Promise<IncusResult> {
|
||||
const [command, ...prefix] = INCUS;
|
||||
if (!command) throw new Error("MESH_LAB_INCUS is empty");
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(command, [...prefix, ...args], {
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let timedOut = false;
|
||||
|
||||
const timer = setTimeout(() => {
|
||||
timedOut = true;
|
||||
child.kill("SIGKILL");
|
||||
}, timeoutMs);
|
||||
|
||||
child.stdout.on("data", (chunk) => (stdout += chunk));
|
||||
child.stderr.on("data", (chunk) => (stderr += chunk));
|
||||
|
||||
child.on("error", (err) => {
|
||||
clearTimeout(timer);
|
||||
reject(new IncusError(args, err.message, null));
|
||||
});
|
||||
|
||||
child.on("close", (code) => {
|
||||
clearTimeout(timer);
|
||||
if (timedOut) {
|
||||
reject(new IncusError(args, `timed out after ${timeoutMs}ms`, null));
|
||||
} else if (code === 0) {
|
||||
resolve({ stdout, stderr });
|
||||
} else {
|
||||
reject(new IncusError(args, stderr, code));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Like `incus`, but a failure is an answer rather than an exception. Returns the command's
|
||||
* output, or `null` if it failed.
|
||||
*
|
||||
* **Never truthiness-test this.** Plenty of incus commands succeed with no output at all —
|
||||
* `exec … true`, `network delete`, `start` — so an empty string means *worked and said
|
||||
* nothing*, and `if (await incusOk(...))` reads that as failure. Use `succeeds()` when the
|
||||
* question is whether it worked, and `!== null` when the output matters.
|
||||
*
|
||||
* This is the mesh's own recurring fault in miniature: absence and success made
|
||||
* indistinguishable. It cost a raise that reported a machine unreachable while `incus exec`
|
||||
* on that machine worked perfectly.
|
||||
*/
|
||||
export async function incusOk(args: string[], timeoutMs = 60_000): Promise<string | null> {
|
||||
try {
|
||||
return (await incus(args, timeoutMs)).stdout;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Did the command work? For commands whose output is not the point. */
|
||||
export async function succeeds(args: string[], timeoutMs = 60_000): Promise<boolean> {
|
||||
return (await incusOk(args, timeoutMs)) !== null;
|
||||
}
|
||||
|
||||
export async function isReachable(): Promise<boolean> {
|
||||
return succeeds(["info"], 15_000);
|
||||
}
|
||||
|
||||
/** Storage drivers the daemon offers. It only advertises those whose tooling it found. */
|
||||
export async function supportedDrivers(): Promise<string[]> {
|
||||
const info = (await incusOk(["info"], 15_000)) ?? "";
|
||||
const block = info.split("storage_supported_drivers:")[1] ?? "";
|
||||
return [...block.matchAll(/-\s+name:\s*(\w+)/g)].map((m) => m[1] ?? "");
|
||||
}
|
||||
|
||||
export interface Pool {
|
||||
name: string;
|
||||
driver: string;
|
||||
}
|
||||
|
||||
export async function pools(): Promise<Pool[]> {
|
||||
const csv = (await incusOk(["storage", "list", "--format", "csv"])) ?? "";
|
||||
return csv
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.map((line) => {
|
||||
const [name = "", driver = ""] = line.split(",");
|
||||
return { name, driver };
|
||||
});
|
||||
}
|
||||
|
||||
export async function networkExists(name: string): Promise<boolean> {
|
||||
return succeeds(["network", "show", name], 15_000);
|
||||
}
|
||||
|
||||
export async function instanceExists(name: string): Promise<boolean> {
|
||||
return succeeds(["config", "show", name], 15_000);
|
||||
}
|
||||
|
||||
export interface TaggedInstance {
|
||||
name: string;
|
||||
status: string;
|
||||
instanceId: string;
|
||||
machine: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Instances this lab owns, identified by the config keys set when they were created —
|
||||
* never by parsing their names.
|
||||
*
|
||||
* Names were parsed at first, splitting on the last dash to separate machine from
|
||||
* instance. That works until a machine is called something like `home-server`, at which
|
||||
* point the instance id absorbs half the machine name and `destroy` silently finds
|
||||
* nothing. Metadata is what the instance actually knows about itself.
|
||||
*/
|
||||
export async function taggedInstances(): Promise<TaggedInstance[]> {
|
||||
const json = (await incusOk(["list", "--format", "json"], 30_000)) ?? "[]";
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(json);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
if (!Array.isArray(parsed)) return [];
|
||||
|
||||
const tagged: TaggedInstance[] = [];
|
||||
for (const entry of parsed) {
|
||||
const item = entry as { name?: string; status?: string; config?: Record<string, string> };
|
||||
const instanceId = item.config?.["user.mesh-lab.instance"];
|
||||
const machine = item.config?.["user.mesh-lab.machine"];
|
||||
if (!instanceId || !machine || !item.name) continue;
|
||||
tagged.push({ name: item.name, status: item.status ?? "", instanceId, machine });
|
||||
}
|
||||
return tagged;
|
||||
}
|
||||
|
||||
export interface TaggedNetwork {
|
||||
name: string;
|
||||
instanceId: string;
|
||||
segment: string;
|
||||
}
|
||||
|
||||
export async function taggedNetworks(): Promise<TaggedNetwork[]> {
|
||||
const json = (await incusOk(["network", "list", "--format", "json"], 30_000)) ?? "[]";
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(json);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
if (!Array.isArray(parsed)) return [];
|
||||
|
||||
const tagged: TaggedNetwork[] = [];
|
||||
for (const entry of parsed) {
|
||||
const item = entry as { name?: string; config?: Record<string, string> };
|
||||
const instanceId = item.config?.["user.mesh-lab.instance"];
|
||||
const segment = item.config?.["user.mesh-lab.segment"];
|
||||
if (!instanceId || !segment || !item.name) continue;
|
||||
tagged.push({ name: item.name, instanceId, segment });
|
||||
}
|
||||
return tagged;
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* Put the declared addresses on the machines.
|
||||
*
|
||||
* The lab provides the underlay, and an address is underlay — it is what a hosting
|
||||
* provider or a home router would have given the machine before any of our software ran.
|
||||
* So the lab assigns it, and the mesh is left to build everything above it.
|
||||
*
|
||||
* Addresses are applied by matching on MAC rather than interface name. A guest names its
|
||||
* interfaces by bus position — `enp5s0`, not `eth0` — so the name the hypervisor uses and
|
||||
* the name the guest uses are different, and matching by name silently configures the
|
||||
* wrong interface on a multi-homed machine.
|
||||
*
|
||||
* See novox/hq 02-DECISIONS/0031-the-lab-provides-the-underlay.md
|
||||
*/
|
||||
|
||||
import type { Scenario } from "../declaration/types.ts";
|
||||
import { incus } from "../incus/client.ts";
|
||||
import { macFor } from "./names.ts";
|
||||
|
||||
export interface Wire {
|
||||
device: string;
|
||||
mac: string;
|
||||
addresses: string[];
|
||||
mtu: number | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* A systemd-networkd unit per interface. Static, because the declaration is the authority:
|
||||
* a scenario that let the hypervisor hand out addresses would be the lab supplying facts
|
||||
* the declaration is supposed to own.
|
||||
*/
|
||||
function networkUnit(wire: Wire): string {
|
||||
const lines = [
|
||||
"[Match]",
|
||||
`MACAddress=${wire.mac}`,
|
||||
"",
|
||||
"[Network]",
|
||||
...wire.addresses.map((address) => `Address=${address}`),
|
||||
// No gateway and no DNS on purpose. Routing off the segment is a gateway's job, and a
|
||||
// scenario that pre-wired it would be arranging what it is supposed to observe.
|
||||
"IPv6AcceptRA=no",
|
||||
];
|
||||
if (wire.mtu !== undefined) {
|
||||
lines.push("", "[Link]", `MTUBytes=${wire.mtu}`);
|
||||
}
|
||||
return lines.join("\n") + "\n";
|
||||
}
|
||||
|
||||
export async function applyAddresses(
|
||||
scenario: Scenario,
|
||||
instanceId: string,
|
||||
machineNames: Map<string, string>,
|
||||
log: (message: string) => void = () => {},
|
||||
): Promise<void> {
|
||||
for (const [machine, spec] of Object.entries(scenario.machines)) {
|
||||
if (spec.at === "detached") continue;
|
||||
const name = machineNames.get(machine);
|
||||
if (!name) continue;
|
||||
|
||||
const wires: Wire[] = [];
|
||||
for (const [index, attachment] of spec.at.entries()) {
|
||||
wires.push({
|
||||
device: `eth${index}`,
|
||||
mac: macFor(instanceId, machine, index),
|
||||
addresses: attachment.address.map((address) => withPrefix(scenario, attachment.segment, address)),
|
||||
mtu: scenario.segments[attachment.segment]?.mtu,
|
||||
});
|
||||
}
|
||||
|
||||
for (const [index, wire] of wires.entries()) {
|
||||
const unit = networkUnit(wire);
|
||||
await incus(
|
||||
["exec", name, "--", "sh", "-c",
|
||||
`mkdir -p /etc/systemd/network && cat > /etc/systemd/network/10-mlab-${index}.network <<'MLAB'\n${unit}MLAB`],
|
||||
30_000,
|
||||
);
|
||||
}
|
||||
|
||||
await incus(["exec", name, "--", "systemctl", "enable", "--now", "systemd-networkd"], 60_000);
|
||||
await incus(["exec", name, "--", "systemctl", "restart", "systemd-networkd"], 60_000);
|
||||
log(` addressed ${machine} (${wires.map((w) => w.addresses.join(",")).join(" | ")})`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* systemd-networkd wants a prefix length on the address. The declaration gives a bare
|
||||
* address and the segment gives the range, so the two are combined here rather than making
|
||||
* every scenario repeat the prefix on every machine.
|
||||
*/
|
||||
function withPrefix(scenario: Scenario, segment: string, address: string): string {
|
||||
const ranges = scenario.segments[segment]?.cidr ?? [];
|
||||
const wantV6 = address.includes(":");
|
||||
for (const range of ranges) {
|
||||
const slash = range.lastIndexOf("/");
|
||||
if (slash === -1) continue;
|
||||
const isV6 = range.slice(0, slash).includes(":");
|
||||
if (isV6 === wantV6) return `${address}${range.slice(slash)}`;
|
||||
}
|
||||
return address;
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* How a scenario instance's resources are named.
|
||||
*
|
||||
* A declaration is a KIND; instances are many. Two instances of one declaration hold the
|
||||
* same addresses and must never meet, so every resource carries the instance id and
|
||||
* nothing is shared between them.
|
||||
*/
|
||||
|
||||
const PREFIX = "mlab";
|
||||
|
||||
/** Instance ids are short and sortable — the last one left standing has to be findable. */
|
||||
export function newInstanceId(scenario: string, now: Date): string {
|
||||
const stamp = now.toISOString().replace(/[-:T]/g, "").slice(2, 12);
|
||||
return `${scenario}-${stamp}`;
|
||||
}
|
||||
|
||||
/** incus network names are limited to 15 characters, so this hashes rather than truncates. */
|
||||
export function networkName(instanceId: string, segment: string): string {
|
||||
const digest = hash(`${instanceId}/${segment}`);
|
||||
return `${PREFIX}${digest}`;
|
||||
}
|
||||
|
||||
export function machineName(instanceId: string, machine: string): string {
|
||||
return `${PREFIX}-${instanceId}-${machine}`;
|
||||
}
|
||||
|
||||
export function instanceIdOf(machineName: string, machine: string): string | null {
|
||||
const suffix = `-${machine}`;
|
||||
if (!machineName.startsWith(`${PREFIX}-`) || !machineName.endsWith(suffix)) return null;
|
||||
return machineName.slice(PREFIX.length + 1, machineName.length - suffix.length);
|
||||
}
|
||||
|
||||
export function machinePrefix(instanceId: string): string {
|
||||
return `${PREFIX}-${instanceId}-`;
|
||||
}
|
||||
|
||||
/** FNV-1a, rendered base36. Short, stable, and collisions are a naming clash not a leak. */
|
||||
function hash(text: string): string {
|
||||
let h = 0x811c9dc5;
|
||||
for (let i = 0; i < text.length; i++) {
|
||||
h ^= text.charCodeAt(i);
|
||||
h = Math.imul(h, 0x01000193) >>> 0;
|
||||
}
|
||||
return h.toString(36).padStart(7, "0").slice(0, 7);
|
||||
}
|
||||
|
||||
/**
|
||||
* A deterministic MAC for a machine's Nth interface, in the locally-administered range.
|
||||
*
|
||||
* Set explicitly at creation rather than read back afterwards: incus assigns a MAC at
|
||||
* runtime and does not record it in the device config, so querying returns nothing. A
|
||||
* derived address is also stable across raises of the same instance, which makes an
|
||||
* in-guest match on it reproducible.
|
||||
*/
|
||||
export function macFor(instanceId: string, machine: string, index: number): string {
|
||||
const digest = hash(`${instanceId}/${machine}/${index}`);
|
||||
const octets: string[] = [];
|
||||
let value = 0;
|
||||
for (let i = 0; i < digest.length; i++) value = (value * 31 + digest.charCodeAt(i)) >>> 0;
|
||||
// 02 marks it locally administered, which is what a made-up address is supposed to say.
|
||||
octets.push("02");
|
||||
for (let i = 0; i < 5; i++) {
|
||||
octets.push(((value >>> (i * 5)) & 0xff).toString(16).padStart(2, "0"));
|
||||
}
|
||||
return octets.join(":");
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* What happens to a scenario once it is raised: inspect it, run things in it, capture and
|
||||
* return it to a state, and tear it down.
|
||||
*
|
||||
* Snapshots are WHOLE-SCENARIO. Per-machine would be cheaper and wrong: the mesh keeps
|
||||
* state that spans nodes, so restoring one machine to an earlier moment while its peers
|
||||
* move on produces a mesh that has never existed and could not. Faults found there would
|
||||
* be artefacts of the lab.
|
||||
*/
|
||||
|
||||
import { incus, incusOk, succeeds, taggedInstances, taggedNetworks } from "../incus/client.ts";
|
||||
import { machineName } from "./names.ts";
|
||||
import { waitUntilAllUsable } from "./ready.ts";
|
||||
|
||||
export interface Instance {
|
||||
instanceId: string;
|
||||
machines: { name: string; machine: string; status: string }[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Every scenario instance the daemon currently holds, found by the metadata each resource
|
||||
* carries rather than by parsing names — a machine called `home-server` would otherwise
|
||||
* be split in the wrong place and its instance would appear not to exist.
|
||||
*/
|
||||
export async function list(): Promise<Instance[]> {
|
||||
const byInstance = new Map<string, Instance["machines"]>();
|
||||
for (const item of await taggedInstances()) {
|
||||
const entry = byInstance.get(item.instanceId) ?? [];
|
||||
entry.push({ name: item.name, machine: item.machine, status: item.status });
|
||||
byInstance.set(item.instanceId, entry);
|
||||
}
|
||||
return [...byInstance.entries()]
|
||||
.map(([instanceId, machines]) => ({
|
||||
instanceId,
|
||||
machines: machines.sort((a, b) => a.machine.localeCompare(b.machine)),
|
||||
}))
|
||||
.sort((a, b) => a.instanceId.localeCompare(b.instanceId));
|
||||
}
|
||||
|
||||
async function machinesOf(instanceId: string): Promise<string[]> {
|
||||
const found = (await list()).find((i) => i.instanceId === instanceId);
|
||||
if (!found) throw new Error(`no scenario instance '${instanceId}'`);
|
||||
return found.machines.map((m) => m.name);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a command on a machine, through incus rather than over IP.
|
||||
*
|
||||
* A reachability question is therefore asked from INSIDE: *can this machine reach that
|
||||
* one* is exec on the first, testing the second. The workstation is not on the scenario's
|
||||
* network and its opinion would be a different question with a similar-looking answer.
|
||||
*/
|
||||
export async function exec(
|
||||
instanceId: string,
|
||||
machine: string,
|
||||
command: string[],
|
||||
): Promise<{ stdout: string; stderr: string }> {
|
||||
const found = (await taggedInstances()).find(
|
||||
(i) => i.instanceId === instanceId && i.machine === machine,
|
||||
);
|
||||
const name = found?.name ?? machineName(instanceId, machine);
|
||||
return incus(["exec", name, "--", ...command], 120_000);
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture the whole scenario as one state. Every machine, one name.
|
||||
*
|
||||
* Machines are snapshotted while running, so what is captured is the disk and not memory —
|
||||
* crash-consistent rather than a paused mesh. Whether a mesh restored that way is coherent
|
||||
* is an open question in the design, not something this silently assumes away.
|
||||
*/
|
||||
export async function snapshot(instanceId: string, label: string): Promise<number> {
|
||||
const machines = await machinesOf(instanceId);
|
||||
const started = Date.now();
|
||||
for (const name of machines) {
|
||||
await incus(["snapshot", "create", name, label], 300_000);
|
||||
}
|
||||
return (Date.now() - started) / 1000;
|
||||
}
|
||||
|
||||
export interface RestoreResult {
|
||||
/** How long the restore itself took. */
|
||||
restoreSeconds: number;
|
||||
/** How long until the scenario was usable again — the number that matters. */
|
||||
usableSeconds: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the whole scenario to a state. Restoring a subset would produce a mesh that never
|
||||
* was, so this is all-or-nothing.
|
||||
*
|
||||
* Restoring a virtual machine replaces its disk and the machine comes back up, so it
|
||||
* reports RUNNING while its agent is still starting — measured, the restore call returns
|
||||
* in 0.79s and the very next command fails. Reporting that as "restored" would be
|
||||
* transport reported as effect, so this waits for usable and returns both numbers.
|
||||
*/
|
||||
export async function restore(
|
||||
instanceId: string,
|
||||
label: string,
|
||||
readyTimeoutSeconds = 180,
|
||||
log: (message: string) => void = () => {},
|
||||
): Promise<RestoreResult> {
|
||||
const machines = await machinesOf(instanceId);
|
||||
const started = Date.now();
|
||||
for (const name of machines) {
|
||||
await incus(["snapshot", "restore", name, label], 300_000);
|
||||
}
|
||||
const restoreSeconds = (Date.now() - started) / 1000;
|
||||
|
||||
for (const name of machines) {
|
||||
await succeeds(["start", name], 60_000);
|
||||
}
|
||||
await waitUntilAllUsable(machines, readyTimeoutSeconds, log);
|
||||
|
||||
return { restoreSeconds, usableSeconds: (Date.now() - started) / 1000 };
|
||||
}
|
||||
|
||||
export async function snapshots(instanceId: string): Promise<string[]> {
|
||||
const machines = await machinesOf(instanceId);
|
||||
const first = machines[0];
|
||||
if (!first) return [];
|
||||
const csv = (await incusOk(["snapshot", "list", first, "--format", "csv"])) ?? "";
|
||||
return csv
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.map((line) => line.split(",")[0] ?? "")
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
/**
|
||||
* Tear the instance down: machines first, then the links they were on.
|
||||
*
|
||||
* Networks are removed last and only if empty — a link still carrying an interface
|
||||
* cannot be deleted, and forcing it would leave the daemon with a reference to something
|
||||
* gone.
|
||||
*/
|
||||
export async function destroy(instanceId: string): Promise<{ machines: number; networks: number }> {
|
||||
const machines = (await taggedInstances()).filter((i) => i.instanceId === instanceId);
|
||||
for (const machine of machines) {
|
||||
await succeeds(["delete", "--force", machine.name], 300_000);
|
||||
}
|
||||
|
||||
// Links go last and only once nothing is attached: a network still carrying an
|
||||
// interface cannot be deleted, and forcing it would leave a dangling reference.
|
||||
let networks = 0;
|
||||
for (const network of await taggedNetworks()) {
|
||||
if (network.instanceId !== instanceId) continue;
|
||||
// `network delete` also succeeds silently — counted with succeeds(), not truthiness.
|
||||
if (await succeeds(["network", "delete", network.name], 30_000)) networks++;
|
||||
}
|
||||
|
||||
return { machines: machines.length, networks };
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
/**
|
||||
* Materialise a declaration into a running scenario instance.
|
||||
*
|
||||
* Two rules from the design shape everything here.
|
||||
*
|
||||
* `raise` waits for the machines to be USABLE, not for the calls to return. Measured on a
|
||||
* workstation those are 3.4s and 14.3s apart, and reporting the earlier number would be
|
||||
* the mesh's own recurring failure — transport reported as effect.
|
||||
*
|
||||
* A failed raise LEAVES THE WRECKAGE. Tearing down on failure destroys the only evidence
|
||||
* of what went wrong, and a scenario that failed to raise is more interesting than one
|
||||
* that succeeded.
|
||||
*
|
||||
* See novox/hq 03-DESIGN/01-to-be/03-scenario-lifecycle.md
|
||||
*/
|
||||
|
||||
import type { Scenario } from "../declaration/types.ts";
|
||||
import { incus, incusOk, succeeds, pools, supportedDrivers } from "../incus/client.ts";
|
||||
import { machineName, macFor, networkName, newInstanceId } from "./names.ts";
|
||||
import { waitUntilAllUsable } from "./ready.ts";
|
||||
import { applyAddresses } from "./address.ts";
|
||||
import { assertSupported } from "./supported.ts";
|
||||
|
||||
/** Drivers whose snapshots are copy-on-write. On `dir` a snapshot is a full copy. */
|
||||
const COW_DRIVERS = ["btrfs", "zfs"];
|
||||
|
||||
export interface RaiseOptions {
|
||||
/** Base image for machines. */
|
||||
image?: string;
|
||||
/** Reuse an existing instance id rather than minting one — makes raise convergent. */
|
||||
instanceId?: string;
|
||||
/** Seconds to wait for each machine to become usable. */
|
||||
readyTimeoutSeconds?: number;
|
||||
onProgress?: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface RaisedScenario {
|
||||
instanceId: string;
|
||||
scenario: string;
|
||||
machines: string[];
|
||||
networks: string[];
|
||||
pool: string;
|
||||
}
|
||||
|
||||
export class RaiseError extends Error {
|
||||
readonly instanceId: string;
|
||||
readonly step: string;
|
||||
|
||||
constructor(instanceId: string, step: string, cause: unknown) {
|
||||
const detail = cause instanceof Error ? cause.message : String(cause);
|
||||
super(
|
||||
`raise failed at '${step}': ${detail}\n` +
|
||||
`The instance '${instanceId}' has been LEFT STANDING for inspection. ` +
|
||||
`Destroy it with: mesh-lab destroy ${instanceId}`,
|
||||
);
|
||||
this.name = "RaiseError";
|
||||
this.instanceId = instanceId;
|
||||
this.step = step;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Pick a pool that can snapshot cheaply, and say so loudly when there is not one.
|
||||
*
|
||||
* Measured: a `dir` snapshot of a 1.5 GB machine takes 9.9s and a full 1.6 GB, with a
|
||||
* second snapshot unfinished after two minutes. On copy-on-write it is 0.13s and costs
|
||||
* the delta. Restoring is the operation the inner loop repeats most, so a `dir` pool does
|
||||
* not make the lab slow — it makes it unused.
|
||||
*/
|
||||
async function choosePool(log: (m: string) => void): Promise<string> {
|
||||
const available = await pools();
|
||||
const cow = available.find((p) => COW_DRIVERS.includes(p.driver));
|
||||
if (cow) return cow.name;
|
||||
|
||||
const drivers = await supportedDrivers();
|
||||
const possible = drivers.filter((d) => COW_DRIVERS.includes(d));
|
||||
log(
|
||||
possible.length > 0
|
||||
? `WARNING: no copy-on-write pool exists, though the daemon offers ${possible.join("/")}. ` +
|
||||
`Snapshots will be full copies — roughly 76x slower, and the inner loop unusable.`
|
||||
: `WARNING: the daemon offers no copy-on-write driver. Snapshots will be full copies — ` +
|
||||
`roughly 76x slower, and the inner loop unusable. Install btrfs tooling and restart it.`,
|
||||
);
|
||||
const fallback = available[0];
|
||||
if (!fallback) throw new Error("no storage pool exists at all");
|
||||
return fallback.name;
|
||||
}
|
||||
|
||||
/**
|
||||
* One isolated link per segment. Nothing joins them to anything outside the instance, and
|
||||
* incus is told not to hand out addresses: a scenario declares the underlay, and letting
|
||||
* a hypervisor's DHCP assign addresses would be the lab supplying facts the declaration
|
||||
* is supposed to own.
|
||||
*/
|
||||
async function createNetwork(instanceId: string, segment: string): Promise<string> {
|
||||
const name = networkName(instanceId, segment);
|
||||
if (await succeeds(["network", "show", name], 15_000)) return name;
|
||||
await incus([
|
||||
"network", "create", name,
|
||||
"ipv4.address=none",
|
||||
"ipv6.address=none",
|
||||
"ipv4.nat=false",
|
||||
"ipv6.nat=false",
|
||||
`user.mesh-lab.instance=${instanceId}`,
|
||||
`user.mesh-lab.segment=${segment}`,
|
||||
]);
|
||||
return name;
|
||||
}
|
||||
|
||||
async function createMachine(
|
||||
instanceId: string,
|
||||
machine: string,
|
||||
attachments: { segment: string }[],
|
||||
image: string,
|
||||
pool: string,
|
||||
): Promise<string> {
|
||||
const name = machineName(instanceId, machine);
|
||||
if (await succeeds(["config", "show", name], 15_000)) return name;
|
||||
|
||||
const args = [
|
||||
"init", image, name,
|
||||
"--vm",
|
||||
"-s", pool,
|
||||
// Arch images refuse to boot under secureboot with the shipped keys. Discovered by
|
||||
// the first launch failing with exactly that message.
|
||||
"-c", "security.secureboot=false",
|
||||
"-c", "limits.memory=1GiB",
|
||||
"-c", "limits.cpu=2",
|
||||
"-c", `user.mesh-lab.instance=${instanceId}`,
|
||||
"-c", `user.mesh-lab.machine=${machine}`,
|
||||
];
|
||||
await incus(args, 300_000);
|
||||
|
||||
// eth0 comes from the profile and points at the wrong network, so every attachment is
|
||||
// explicit. A machine on no segment gets no interface at all — that is what detached is.
|
||||
await succeeds(["config", "device", "remove", name, "eth0"], 15_000);
|
||||
for (const [index, attachment] of attachments.entries()) {
|
||||
await incus([
|
||||
"config", "device", "add", name, `eth${index}`, "nic",
|
||||
"nictype=bridged",
|
||||
`parent=${networkName(instanceId, attachment.segment)}`,
|
||||
// Explicit, because incus assigns one at runtime without recording it in the device
|
||||
// config — so reading it back returns nothing, and the guest has no stable handle.
|
||||
`hwaddr=${macFor(instanceId, machine, index)}`,
|
||||
]);
|
||||
}
|
||||
return name;
|
||||
}
|
||||
|
||||
export async function raise(
|
||||
scenario: Scenario,
|
||||
options: RaiseOptions = {},
|
||||
): Promise<RaisedScenario> {
|
||||
const log = options.onProgress ?? (() => {});
|
||||
const image = options.image ?? "images:archlinux/current";
|
||||
const readyTimeout = options.readyTimeoutSeconds ?? 180;
|
||||
const instanceId = options.instanceId ?? newInstanceId(scenario.scenario, new Date());
|
||||
|
||||
// Refuse before spending a minute raising something that would silently lack half of
|
||||
// what it declares. Deliberately outside the try: this is not a raise failure, nothing
|
||||
// has been created, and there is no wreckage to leave standing.
|
||||
assertSupported(scenario);
|
||||
|
||||
let step = "choosing a storage pool";
|
||||
try {
|
||||
const pool = await choosePool(log);
|
||||
log(`instance ${instanceId} pool ${pool}`);
|
||||
|
||||
step = "creating segments";
|
||||
const networks: string[] = [];
|
||||
for (const segment of Object.keys(scenario.segments)) {
|
||||
networks.push(await createNetwork(instanceId, segment));
|
||||
log(` segment ${segment}`);
|
||||
}
|
||||
|
||||
step = "creating machines";
|
||||
const created: string[] = [];
|
||||
const byMachine = new Map<string, string>();
|
||||
for (const [machine, spec] of Object.entries(scenario.machines)) {
|
||||
const attachments = spec.at === "detached" ? [] : spec.at;
|
||||
const name = await createMachine(instanceId, machine, attachments, image, pool);
|
||||
created.push(name);
|
||||
byMachine.set(machine, name);
|
||||
log(` machine ${machine}${spec.at === "detached" ? " (detached)" : ""}`);
|
||||
}
|
||||
|
||||
step = "starting machines";
|
||||
for (const name of created) {
|
||||
await succeeds(["start", name], 60_000);
|
||||
}
|
||||
|
||||
step = "waiting for machines to become usable";
|
||||
await waitUntilAllUsable(created, readyTimeout, log);
|
||||
|
||||
step = "applying declared addresses";
|
||||
await applyAddresses(scenario, instanceId, byMachine, log);
|
||||
|
||||
return { instanceId, scenario: scenario.scenario, machines: created, networks, pool };
|
||||
} catch (cause) {
|
||||
throw new RaiseError(instanceId, step, cause);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* "Usable" means a command runs on the machine. Anything weaker is transport reported as
|
||||
* effect — the mesh's own recurring fault, and one this lab exists to catch rather than
|
||||
* commit.
|
||||
*
|
||||
* Two measurements make the case. Raising: the launch call returns in 3.4s and the machine
|
||||
* is usable at 14.3s. Restoring: the call returns in 0.79s and the machine is RUNNING
|
||||
* immediately — with its agent still starting, so the very next command fails.
|
||||
*
|
||||
* Both verbs therefore wait for the same thing, using the same code.
|
||||
*/
|
||||
|
||||
import { succeeds } from "../incus/client.ts";
|
||||
|
||||
export interface ReadyResult {
|
||||
name: string;
|
||||
seconds: number;
|
||||
}
|
||||
|
||||
export async function waitUntilUsable(
|
||||
name: string,
|
||||
timeoutSeconds: number,
|
||||
log: (message: string) => void = () => {},
|
||||
): Promise<ReadyResult> {
|
||||
const started = Date.now();
|
||||
const deadline = started + timeoutSeconds * 1000;
|
||||
|
||||
while (Date.now() < deadline) {
|
||||
// `exec … true` succeeds with EMPTY output, so this asks whether it worked rather than
|
||||
// what it said. Truthiness-testing the output reported every machine as unreachable
|
||||
// while `incus exec` on it worked perfectly.
|
||||
if (await succeeds(["exec", name, "--", "true"], 10_000)) {
|
||||
const seconds = (Date.now() - started) / 1000;
|
||||
log(` ${name} usable after ${seconds.toFixed(1)}s`);
|
||||
return { name, seconds };
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`${name} did not become usable within ${timeoutSeconds}s — it may be running but ` +
|
||||
`unreachable, which is not the same as ready`,
|
||||
);
|
||||
}
|
||||
|
||||
export async function waitUntilAllUsable(
|
||||
names: string[],
|
||||
timeoutSeconds: number,
|
||||
log: (message: string) => void = () => {},
|
||||
): Promise<number> {
|
||||
const started = Date.now();
|
||||
// Concurrently: a scenario's machines boot independently, and waiting for them in turn
|
||||
// would make a four-machine scenario four boots long instead of one.
|
||||
await Promise.all(names.map((name) => waitUntilUsable(name, timeoutSeconds, log)));
|
||||
return (Date.now() - started) / 1000;
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
/**
|
||||
* What this lab can actually materialise, and a refusal for everything else.
|
||||
*
|
||||
* A declaration the runtime silently ignores is the worst thing this project could ship.
|
||||
* The mesh already has that fault catalogued: a firewall key declared in five manifests
|
||||
* and read by no code, so a manifest appears to restrict a port and restricts nothing
|
||||
* (novox/hq 04-ISSUES/003). A scenario that declares a gateway and raises without one
|
||||
* would be the same fault, in the tool built to catch it.
|
||||
*
|
||||
* So unimplemented parts of the model are refused loudly at raise time rather than
|
||||
* ignored. The declaration schema deliberately runs ahead of the runtime — it is the
|
||||
* design's shape, and validating against it is useful before any of it can be raised —
|
||||
* but the gap between the two has to be visible.
|
||||
*/
|
||||
|
||||
import type { Scenario } from "../declaration/types.ts";
|
||||
|
||||
export class UnsupportedError extends Error {
|
||||
readonly missing: string[];
|
||||
|
||||
constructor(missing: string[]) {
|
||||
super(
|
||||
`this scenario declares things the lab cannot yet materialise:\n - ${missing.join("\n - ")}\n\n` +
|
||||
`Raising it would produce a mesh that silently lacks them, which is the exact fault ` +
|
||||
`this lab exists to catch. Validation accepts them because the declaration model is ` +
|
||||
`complete; the runtime is not.`,
|
||||
);
|
||||
this.name = "UnsupportedError";
|
||||
this.missing = missing;
|
||||
}
|
||||
}
|
||||
|
||||
export function assertSupported(scenario: Scenario): void {
|
||||
const missing: string[] = [];
|
||||
|
||||
const withGateways = Object.entries(scenario.segments)
|
||||
.filter(([, segment]) => segment.gateway)
|
||||
.map(([name]) => name);
|
||||
if (withGateways.length > 0) {
|
||||
missing.push(
|
||||
`gateways (segments: ${withGateways.join(", ")}) — no router is materialised, so ` +
|
||||
`nothing routes between segments and NAT does not exist`,
|
||||
);
|
||||
}
|
||||
|
||||
const published = Object.entries(scenario.machines)
|
||||
.filter(([, machine]) => machine.published?.length)
|
||||
.map(([name]) => name);
|
||||
if (published.length > 0) {
|
||||
missing.push(
|
||||
`published ports (machines: ${published.join(", ")}) — requires a gateway to forward through`,
|
||||
);
|
||||
}
|
||||
|
||||
if (scenario.policy?.length) {
|
||||
missing.push("policy between segments — requires a gateway to enforce it");
|
||||
}
|
||||
|
||||
const inbound = Object.entries(scenario.machines)
|
||||
.filter(([, machine]) => machine.inbound === "deny")
|
||||
.map(([name]) => name);
|
||||
if (inbound.length > 0) {
|
||||
missing.push(
|
||||
`inbound: deny (machines: ${inbound.join(", ")}) — no host firewall is configured, so ` +
|
||||
`these machines would accept traffic the scenario says they refuse`,
|
||||
);
|
||||
}
|
||||
|
||||
if (scenario.place && Object.keys(scenario.place).length > 0) {
|
||||
missing.push(
|
||||
"place — nothing is placed inside the machines yet; they are raised bare",
|
||||
);
|
||||
}
|
||||
|
||||
if (missing.length > 0) throw new UnsupportedError(missing);
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { machineName, machinePrefix, networkName, newInstanceId, instanceIdOf } from "../src/lifecycle/names.ts";
|
||||
|
||||
test("network names fit incus's 15-character limit", () => {
|
||||
const id = newInstanceId("the-ordinary-shape", new Date("2026-08-24T22:15:00Z"));
|
||||
for (const segment of ["hosting", "isp-home", "isp-mobile", "home", "devices", "cafe"]) {
|
||||
const name = networkName(id, segment);
|
||||
assert.ok(name.length <= 15, `${name} is ${name.length} chars`);
|
||||
}
|
||||
});
|
||||
|
||||
test("network names are unique per (instance, segment)", () => {
|
||||
const a = newInstanceId("x", new Date("2026-08-24T22:15:00Z"));
|
||||
const b = newInstanceId("x", new Date("2026-08-24T23:15:00Z"));
|
||||
const names = new Set([
|
||||
networkName(a, "home"), networkName(a, "cafe"),
|
||||
networkName(b, "home"), networkName(b, "cafe"),
|
||||
]);
|
||||
assert.equal(names.size, 4, "two instances of one declaration must not share a network");
|
||||
});
|
||||
|
||||
test("machine names round-trip to their instance id", () => {
|
||||
const id = newInstanceId("bootstrap-single", new Date("2026-08-24T22:15:00Z"));
|
||||
const name = machineName(id, "anchor");
|
||||
assert.equal(instanceIdOf(name, "anchor"), id);
|
||||
assert.ok(name.startsWith(machinePrefix(id)));
|
||||
});
|
||||
|
||||
test("instance ids are sortable by time", () => {
|
||||
const early = newInstanceId("x", new Date("2026-08-24T09:00:00Z"));
|
||||
const late = newInstanceId("x", new Date("2026-08-24T21:00:00Z"));
|
||||
assert.ok(early < late, `${early} should sort before ${late}`);
|
||||
});
|
||||
@@ -0,0 +1,49 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { parseCidr, contains, familyOf, parseAddress } from "../src/declaration/net.ts";
|
||||
|
||||
test("family is inferred from the address", () => {
|
||||
assert.equal(familyOf("203.0.113.1"), "v4");
|
||||
assert.equal(familyOf("2001:db8::1"), "v6");
|
||||
});
|
||||
|
||||
test("v4 containment", () => {
|
||||
const range = parseCidr("203.0.113.0/24");
|
||||
assert.equal(contains(range, "203.0.113.0"), true);
|
||||
assert.equal(contains(range, "203.0.113.255"), true);
|
||||
assert.equal(contains(range, "203.0.114.0"), false);
|
||||
assert.equal(contains(range, "202.0.113.1"), false);
|
||||
});
|
||||
|
||||
test("v4 containment on a non-byte prefix", () => {
|
||||
const range = parseCidr("198.51.100.0/25");
|
||||
assert.equal(contains(range, "198.51.100.127"), true);
|
||||
assert.equal(contains(range, "198.51.100.128"), false);
|
||||
});
|
||||
|
||||
test("v6 containment, including :: compression", () => {
|
||||
const range = parseCidr("2001:db8:a::/48");
|
||||
assert.equal(contains(range, "2001:db8:a::10"), true);
|
||||
assert.equal(contains(range, "2001:db8:a:ffff::1"), true);
|
||||
assert.equal(contains(range, "2001:db8:b::10"), false);
|
||||
});
|
||||
|
||||
test("a range never contains an address of the other family", () => {
|
||||
assert.equal(contains(parseCidr("203.0.113.0/24"), "2001:db8::1"), false);
|
||||
assert.equal(contains(parseCidr("2001:db8::/32"), "203.0.113.1"), false);
|
||||
});
|
||||
|
||||
test("a cidr whose address has host bits set still masks to its network", () => {
|
||||
// 203.0.113.5/24 means the 203.0.113.0/24 network, not a range starting at .5
|
||||
assert.equal(contains(parseCidr("203.0.113.5/24"), "203.0.113.1"), true);
|
||||
});
|
||||
|
||||
test("malformed input is refused rather than guessed at", () => {
|
||||
assert.throws(() => parseCidr("203.0.113.0"), /no prefix length/);
|
||||
assert.throws(() => parseCidr("203.0.113.0/33"), /out of range/);
|
||||
assert.throws(() => parseCidr("2001:db8::/129"), /out of range/);
|
||||
assert.throws(() => parseAddress("203.0.113"), /not an IPv4/);
|
||||
assert.throws(() => parseAddress("203.0.113.256"), /out of range/);
|
||||
assert.throws(() => parseAddress("2001:db8::1::2"), /not an IPv6/);
|
||||
assert.throws(() => parseAddress("::ffff:192.0.2.1"), /not supported/);
|
||||
});
|
||||
@@ -0,0 +1,63 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { parseScenario } from "../src/declaration/parse.ts";
|
||||
import { assertSupported, UnsupportedError } from "../src/lifecycle/supported.ts";
|
||||
|
||||
/**
|
||||
* A declaration the runtime silently ignores is the fault this lab exists to catch —
|
||||
* novox/hq 04-ISSUES/003, where a firewall key is declared in five manifests and read by
|
||||
* no code. These tests exist so the lab never commits it.
|
||||
*/
|
||||
|
||||
const withGateway = `scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
|
||||
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`;
|
||||
|
||||
test("a scenario with no unimplemented features is raisable", () => {
|
||||
const scenario = parseScenario(`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }`);
|
||||
assert.doesNotThrow(() => assertSupported(scenario));
|
||||
});
|
||||
|
||||
test("a declared gateway is refused rather than silently absent", () => {
|
||||
assert.throws(() => assertSupported(parseScenario(withGateway)), UnsupportedError);
|
||||
});
|
||||
|
||||
test("the refusal names every missing capability, not just the first", () => {
|
||||
const scenario = parseScenario(`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
|
||||
policy: [{ from: home, to: pub, allow: false }]
|
||||
machines:
|
||||
a:
|
||||
at: { segment: home, address: [192.168.1.9] }
|
||||
published: [{ port: 443, on: home }]
|
||||
inbound: deny
|
||||
place: { all: [host] }`);
|
||||
try {
|
||||
assertSupported(scenario);
|
||||
assert.fail("should have refused");
|
||||
} catch (err) {
|
||||
const missing = (err as UnsupportedError).missing;
|
||||
assert.ok(missing.length >= 5, `expected every gap named, got ${missing.length}`);
|
||||
assert.match(err instanceof Error ? err.message : "", /silently lacks them/);
|
||||
}
|
||||
});
|
||||
|
||||
test("inbound: allow is not a missing capability — only deny needs enforcing", () => {
|
||||
const scenario = parseScenario(`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [192.0.2.1] }, inbound: allow } }`);
|
||||
assert.doesNotThrow(() => assertSupported(scenario));
|
||||
});
|
||||
|
||||
test("validation and raisability are different questions", () => {
|
||||
// The declaration model is complete; the runtime is not. A scenario may be valid and
|
||||
// still not raisable, and conflating the two would hide the gap.
|
||||
assert.doesNotThrow(() => parseScenario(withGateway), "should validate");
|
||||
assert.throws(() => assertSupported(parseScenario(withGateway)), "should not raise");
|
||||
});
|
||||
@@ -0,0 +1,187 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { parseScenario } from "../src/declaration/parse.ts";
|
||||
import { loadScenario } from "../src/declaration/parse.ts";
|
||||
|
||||
/** Every rejection below is a fault that would otherwise be silent at runtime. */
|
||||
function refuses(yaml: string, pattern: RegExp): void {
|
||||
assert.throws(() => parseScenario(yaml), (err: Error) => {
|
||||
assert.match(err.message, pattern);
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
test("the shipped scenarios are valid", () => {
|
||||
for (const file of ["scenarios/bootstrap-single.yml", "scenarios/the-ordinary-shape.yml"]) {
|
||||
assert.doesNotThrow(() => loadScenario(file));
|
||||
}
|
||||
});
|
||||
|
||||
test("a public segment on a private range is refused — the mesh would silently never form", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.168.1.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [192.168.1.1] } } }`,
|
||||
/not documentation space/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a public segment on a real routable range is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [8.8.8.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [8.8.8.8] } } }`,
|
||||
/not documentation space/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a private segment may use any range, including someone else's RFC 1918", () => {
|
||||
assert.doesNotThrow(() =>
|
||||
parseScenario(`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
cafe: { kind: private, cidr: [10.50.0.0/16], gateway: { to: pub, address: [192.0.2.5], nat: [v4], forwardable: false } }
|
||||
machines: { a: { at: { segment: cafe, address: [10.50.0.9] } } }`),
|
||||
);
|
||||
});
|
||||
|
||||
test("publishing through an unforwardable gateway is refused — that is the constraint", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
cafe: { kind: private, cidr: [10.50.0.0/16], gateway: { to: pub, address: [192.0.2.5], nat: [v4], forwardable: false } }
|
||||
machines:
|
||||
a:
|
||||
at: { segment: cafe, address: [10.50.0.9] }
|
||||
published: [{ port: 443, on: cafe }]`,
|
||||
/not forwardable/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a gateway address must be on the PARENT segment, not the one behind it", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.168.1.1], nat: [v4] } }
|
||||
machines: { a: { at: { segment: home, address: [192.168.1.9] } } }`,
|
||||
/is not within 'pub'/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a machine address outside its segment is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [203.0.113.9] } } }`,
|
||||
/is not within segment 'net'/,
|
||||
);
|
||||
});
|
||||
|
||||
test("an unknown segment reference is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: nope, address: [192.0.2.1] } } }`,
|
||||
/unknown segment 'nope'/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a gateway loop is refused rather than raised forever", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments:
|
||||
a: { kind: private, cidr: [10.0.0.0/24], gateway: { to: b, address: [10.0.1.1], nat: [] } }
|
||||
b: { kind: private, cidr: [10.0.1.0/24], gateway: { to: a, address: [10.0.0.1], nat: [] } }
|
||||
machines: { m: { at: { segment: a, address: [10.0.0.9] } } }`,
|
||||
/loops through/,
|
||||
);
|
||||
});
|
||||
|
||||
test("a detached machine cannot publish", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: detached, published: [{ port: 443, on: net }] } }`,
|
||||
/detached but declares published/,
|
||||
);
|
||||
});
|
||||
|
||||
test("publishing on a segment the machine is not attached to is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
|
||||
machines:
|
||||
a:
|
||||
at: { segment: pub, address: [192.0.2.10] }
|
||||
published: [{ port: 443, on: home }]`,
|
||||
/not attached to it/,
|
||||
);
|
||||
});
|
||||
|
||||
test("two addresses of one family on one attachment is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [192.0.2.1, 192.0.2.2] } } }`,
|
||||
/two v4 addresses/,
|
||||
);
|
||||
});
|
||||
|
||||
test("place naming a machine that does not exist is refused", () => {
|
||||
refuses(
|
||||
`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines: { a: { at: { segment: net, address: [192.0.2.1] } } }
|
||||
place: { ghost: [host] }`,
|
||||
/'ghost' is not a machine/,
|
||||
);
|
||||
});
|
||||
|
||||
test("every problem is reported, not just the first", () => {
|
||||
try {
|
||||
parseScenario(`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.168.0.0/24] } }
|
||||
machines: { a: { at: { segment: nope, address: [1.2.3.4] } } }
|
||||
place: { ghost: [host] }`);
|
||||
assert.fail("should have thrown");
|
||||
} catch (err) {
|
||||
const problems = (err as { problems: string[] }).problems;
|
||||
assert.ok(problems.length >= 3, `expected several problems, got ${problems.length}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("a detached machine is valid", () => {
|
||||
assert.doesNotThrow(() =>
|
||||
parseScenario(`scenario: x
|
||||
segments: { net: { kind: public, cidr: [192.0.2.0/24] } }
|
||||
machines:
|
||||
a: { at: { segment: net, address: [192.0.2.1] } }
|
||||
roamer: { at: detached }`),
|
||||
);
|
||||
});
|
||||
|
||||
test("a multi-homed machine is valid", () => {
|
||||
assert.doesNotThrow(() =>
|
||||
parseScenario(`scenario: x
|
||||
segments:
|
||||
pub: { kind: public, cidr: [192.0.2.0/24] }
|
||||
home: { kind: private, cidr: [192.168.1.0/24], gateway: { to: pub, address: [192.0.2.5], nat: [v4] } }
|
||||
machines:
|
||||
border:
|
||||
at:
|
||||
- { segment: pub, address: [192.0.2.60] }
|
||||
- { segment: home, address: [192.168.1.2] }`),
|
||||
);
|
||||
});
|
||||
|
||||
test("an isolated private segment with no gateway is valid — a site with no internet", () => {
|
||||
assert.doesNotThrow(() =>
|
||||
parseScenario(`scenario: x
|
||||
segments: { island: { kind: private, cidr: [10.9.0.0/24] } }
|
||||
machines: { a: { at: { segment: island, address: [10.9.0.1] } } }`),
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2022",
|
||||
"module": "nodenext",
|
||||
"moduleResolution": "nodenext",
|
||||
"strict": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"noImplicitOverride": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"rootDir": "src",
|
||||
"skipLibCheck": true,
|
||||
"types": [
|
||||
"node"
|
||||
],
|
||||
"allowImportingTsExtensions": true,
|
||||
"noEmit": true
|
||||
},
|
||||
"include": [
|
||||
"src/**/*.ts"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user