jschoubben
eab4598494
ADR 0016 makes a lab node a virtual machine, and its reasoning is fidelity: a node boots a stock image and runs the real install, so it has to be a real machine or the thing under test is not the thing that ships. That reasoning does not reach a router. Nothing under test runs on one, it holds no identity, the mesh never installs anything on it, and no assertion is ever made about its internals. It exists so packets behave the way they behave in the world, which is the definition of scenery. So a router is a system container. What it must reproduce is kernel behaviour — translation, connection tracking, filtering, forwarding — and a container has the same kernel. Verified before deciding rather than assumed. In a plain unprivileged container: ip_forward and ipv6 forwarding both settable, nftables masquerade accepted and listed back, and the conntrack timeouts that mapping_ttl depends on both writable. No privileged mode, no nesting, no capability grants. Rejected letting the hypervisor provide NAT, on a stronger ground than speed: it makes the lab provide what the declaration is supposed to own, and it cannot express a mapping that expires, a gateway that refuses to forward, or policy between siblings. The model would shrink to fit the tool. The distinction is now load-bearing and has to stay legible: node means something under test, scenery means something that makes the test real. If the mesh ever installs anything on a router, it has become a node and this record no longer covers it.
4.0 KiB
status, date, deciders, reconstructed, extends
| status | date | deciders | reconstructed | extends |
|---|---|---|---|---|
| accepted | 2026-08-24 | jochen | false | 0016-a-lab-node-is-a-virtual-machine.md |
33. A router is scenery, not a node — so it is a container
Context
ADR 0016 settles that a lab node is a virtual machine, and its reasoning is fidelity: a node boots a stock image and runs the real install, so it has to be a real machine or the thing under test is not the thing that ships.
A scenario also needs routers. NAT, port forwarding, policy between segments and mapping expiry are all things a router does, and until one is materialised a multi-segment scenario raises isolated islands (03-DESIGN/01-to-be/02-scenario-declaration.md). The declaration already implies them: a gateway is the one implicit machine in an otherwise explicit declaration.
The question is whether ADR 0016 binds those too.
Considered options
- A router is a node, so it is a virtual machine. Consistent, and pays for a consistency nobody needs. A router boots in roughly ten seconds against a container's one; a six-segment scenario wanting three routers spends thirty seconds per raise on scenery.
- The hypervisor provides NAT — bridges with translation switched on, and its own forwarding primitives. Rejected on a stronger ground than speed: it makes the lab provide what the declaration is supposed to own, and it cannot express a mapping that expires, a gateway that refuses to forward, or policy between siblings. The model would shrink to fit the tool.
- A router is scenery, and scenery is a container. Chosen.
Decision
ADR 0016 binds nodes. A router is not a node.
Nothing under test runs on a router. It is not a participant, it holds no identity, the mesh never installs anything on it, and no assertion is ever made about its internals. It exists so that packets between machines behave the way they behave in the world — which is the definition of scenery.
So a router is a system container, and the fidelity argument does not reach it: what a router must reproduce is kernel behaviour — translation, connection tracking, filtering, forwarding — and a container has the same kernel.
Verified before deciding, not assumed. In a plain unprivileged container:
| Needed for | Works |
|---|---|
| routing at all | net.ipv4.ip_forward, net.ipv6.conf.all.forwarding |
nat: |
nftables masquerade, rules accepted and listed back |
mapping_ttl: |
nf_conntrack_udp_timeout, nf_conntrack_tcp_timeout_established |
No privileged mode, no nesting, no capability grants.
Consequences
- A raise stops paying a boot per router. Scenery costs about a second where a node costs ten, and a scenario's cost tracks the machines actually under test.
- The distinction is now load-bearing and has to stay legible. Node means something under test; scenery means something that makes the test real. If anything is ever installed on a router by the mesh, it has become a node and this decision no longer covers it.
- Routers and nodes are different kinds of thing in the lab's own model, which is a small extra concept — justified by it being true, rather than by the saving.
- A container shares the host kernel, so a scenario cannot reproduce a router running a different kernel from the workstation. Nothing currently wants that; if something does, that router becomes a virtual machine and this record needs revisiting rather than bending.
- The gateway stays implicit in the declaration. A scenario declares
gateway:on a segment and never names the machine that serves it — which is right, because it is not a machine the scenario has anything to say about.
References
- ADR 0016 — what a lab node is, unchanged.
- Research 004 — the topology needing a router, and why published but behind NAT only exists in production today.