novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

ADR 0033: a router is scenery, not a node

Browse Source 
ADR 0016 makes a lab node a virtual machine, and its reasoning is
fidelity: a node boots a stock image and runs the real install, so it has
to be a real machine or the thing under test is not the thing that ships.

That reasoning does not reach a router. Nothing under test runs on one, it
holds no identity, the mesh never installs anything on it, and no assertion
is ever made about its internals. It exists so packets behave the way they
behave in the world, which is the definition of scenery.

So a router is a system container. What it must reproduce is kernel
behaviour — translation, connection tracking, filtering, forwarding — and a
container has the same kernel.

Verified before deciding rather than assumed. In a plain unprivileged
container: ip_forward and ipv6 forwarding both settable, nftables
masquerade accepted and listed back, and the conntrack timeouts that
mapping_ttl depends on both writable. No privileged mode, no nesting, no
capability grants.

Rejected letting the hypervisor provide NAT, on a stronger ground than
speed: it makes the lab provide what the declaration is supposed to own,
and it cannot express a mapping that expires, a gateway that refuses to
forward, or policy between siblings. The model would shrink to fit the
tool.

The distinction is now load-bearing and has to stay legible: node means
something under test, scenery means something that makes the test real. If
the mesh ever installs anything on a router, it has become a node and this
record no longer covers it.
This commit is contained in:
jschoubben
2026-08-24 01:28:08 +02:00
parent 132f6a0626
commit eab4598494
2 changed files with 86 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0033-a-router-is-scenery-not-a-node.md
+81
View File
@@ -0,0 +1,81 @@
---
status: accepted
date: 2026-08-24
deciders: jochen
reconstructed: false
extends: 0016-a-lab-node-is-a-virtual-machine.md
---
# 33. A router is scenery, not a node — so it is a container
## Context
[ADR 0016](0016-a-lab-node-is-a-virtual-machine.md) settles that **a lab node is a virtual
machine**, and its reasoning is fidelity: a node boots a stock image and runs the real install,
so it has to be a real machine or the thing under test is not the thing that ships.
A scenario also needs routers. NAT, port forwarding, policy between segments and mapping
expiry are all things a router does, and until one is materialised a multi-segment scenario
raises isolated islands
([03-DESIGN/01-to-be/02-scenario-declaration.md](../03-DESIGN/01-to-be/02-scenario-declaration.md)).
The declaration already implies them: a gateway is *the one implicit machine in an otherwise
explicit declaration*.
The question is whether ADR 0016 binds those too.
## Considered options
1. **A router is a node, so it is a virtual machine.** Consistent, and pays for a consistency
nobody needs. A router boots in roughly ten seconds against a container's one; a
six-segment scenario wanting three routers spends thirty seconds per raise on scenery.
2. **The hypervisor provides NAT** — bridges with translation switched on, and its own
forwarding primitives. Rejected on a stronger ground than speed: it makes the *lab* provide
what the declaration is supposed to own, and it cannot express a mapping that expires, a
gateway that refuses to forward, or policy between siblings. The model would shrink to fit
the tool.
3. **A router is scenery, and scenery is a container.** Chosen.
## Decision
**ADR 0016 binds nodes. A router is not a node.**
Nothing under test runs on a router. It is not a participant, it holds no identity, the mesh
never installs anything on it, and no assertion is ever made about its internals. It exists so
that packets between machines behave the way they behave in the world — which is the definition
of scenery.
So a router is a **system container**, and the fidelity argument does not reach it: what a
router must reproduce is kernel behaviour — translation, connection tracking, filtering,
forwarding — and a container has the same kernel.
**Verified before deciding, not assumed.** In a plain unprivileged container:
| Needed for | Works |
|---|---|
| routing at all | `net.ipv4.ip_forward`, `net.ipv6.conf.all.forwarding` |
| `nat:` | nftables masquerade, rules accepted and listed back |
| `mapping_ttl:` | `nf_conntrack_udp_timeout`, `nf_conntrack_tcp_timeout_established` |
No privileged mode, no nesting, no capability grants.
## Consequences
- A raise stops paying a boot per router. Scenery costs about a second where a node costs ten,
and a scenario's cost tracks the machines actually under test.
- **The distinction is now load-bearing and has to stay legible.** *Node* means something under
test; *scenery* means something that makes the test real. If anything is ever installed on a
router by the mesh, it has become a node and this decision no longer covers it.
- Routers and nodes are different kinds of thing in the lab's own model, which is a small extra
concept — justified by it being true, rather than by the saving.
- A container shares the host kernel, so a scenario cannot reproduce a router running a
*different* kernel from the workstation. Nothing currently wants that; if something does, that
router becomes a virtual machine and this record needs revisiting rather than bending.
- The gateway stays implicit in the declaration. A scenario declares `gateway:` on a segment and
never names the machine that serves it — which is right, because it is not a machine the
scenario has anything to say about.
## References
- [ADR 0016](0016-a-lab-node-is-a-virtual-machine.md) — what a lab *node* is, unchanged.
- [Research 004](../01-RESEARCH/004-lab-network/analysis.md) — the topology needing a router,
and why *published but behind NAT* only exists in production today.
03-DESIGN/01-to-be/02-scenario-declaration.md
+5
View File
@@ -256,6 +256,11 @@ reachability, and it does not.
The lab materialises a machine to be the gateway. That is the one implicit machine in an
otherwise explicit declaration, and it exists because NAT has to run somewhere.
It is a **container, not a virtual machine** — a router is scenery rather than something under
test, so the fidelity argument that makes a node a virtual machine does not reach it
([ADR 0033](../../02-DECISIONS/0033-a-router-is-scenery-not-a-node.md)). What a router must
reproduce is kernel behaviour, and a container has the same kernel.
**`machines[].at`** — segment and addresses, or a **list** of them for a machine on several
segments at once. Multi-homing is not exotic: it is what a border machine is, and what any node
with both a LAN and a WAN interface is. Each entry carries the addresses that machine holds on