novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

The scenario model, the lifecycle, and what the lab actually costs #6

Merged
jschoubben merged 9 commits from design/scenario-underlay-detail into main 2026-08-23 22:40:55 +00:00
Conversation 0 Commits 9 Files Changed 7
+122 -45
1 changed files with 122 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit e65e5809dc - Show all commits
03-DESIGN/01-to-be/02-scenario-declaration.md
+122 -45
View File
@@ -17,31 +17,60 @@ everything in the lab hangs off, so it is worth getting small.
It states what a hosting provider and a home router would provide, and nothing the mesh is
responsible for ([ADR 0031](../../02-DECISIONS/0031-the-lab-provides-the-underlay.md)).
## Three positions a machine can be in
The underlay's whole job is to reproduce **where a machine sits relative to the internet**,
because that is what the mesh has to cope with and what only production currently exercises.
There are three positions, and they are genuinely different:
| Position | Reachable from outside | Address | Example |
|---|---|---|---|
| **Directly attached** | yes, at its own address | fixed, its own | a hosted server |
| **Behind a gateway you control** | only through a forwarded port, at the *gateway's* address | private, plus the gateway's public one | a machine at home |
| **Behind a gateway you don't control** | **no** | private, and it changes | a laptop on someone else's network |
The third is the hard one and the reason this matters. A machine there can dial out and nothing
more: it cannot be published, its apparent address belongs to somebody else's router, and that
address changes when it moves. Every assumption a mesh makes about reachability breaks there
first.
A declaration has to be able to say all three, and to move a machine between them.
## The shape
```yaml
scenario: published-behind-nat
scenario: roaming-and-published
segments:
wan:
cidr: 203.0.113.0/24 # RFC 5737 — never routes on the real internet
lan:
internet:
cidr: 203.0.113.0/24 # the simulated public internet, RFC 5737
home:
cidr: 192.168.1.0/24
behind: wan # NAT; the lab materialises a router
gateway:
to: internet
address: 203.0.113.50 # what the world sees this network as
nat: true
elsewhere: # a network we do not control
cidr: 198.51.100.0/24
gateway:
to: internet
address: 203.0.113.80
nat: true
machines:
anchor:
segment: wan
address: 203.0.113.10
at: { segment: internet, address: 203.0.113.10 }
home-server:
segment: lan
address: 192.168.1.135
forwarded: [443] # reachable from wan through the router
at: { segment: home, address: 192.168.1.135 }
published:
- { port: 443, on: home } # DNAT: 203.0.113.50:443 → 192.168.1.135:443
workstation:
segment: lan
address: 192.168.1.250
at: { segment: home, address: 192.168.1.250 }
laptop:
segment: detached # reachable by nothing until attached
at: { segment: home, address: 192.168.1.98 }
place:
all: [host]
@@ -50,41 +79,87 @@ place:
snapshot: raised
```
That is a complete bootstrap scenario. Nothing in it mentions the overlay, a hub, peering,
names or certificates — all of which are outcomes to be observed.
## What each part means, precisely
## The four parts
**`segments`** — a broadcast domain with an address range. A segment with no `gateway:` *is*
the internet as far as the scenario is concerned. A segment with one sits behind it.
**`segments`** — the networks that exist. `behind:` declares NAT, and is the only place a
router comes from: the lab materialises one without being asked, because NAT has to run
somewhere. This is the one implicit machine in an otherwise explicit declaration.
**`gateway:`** — how a segment reaches its parent, and this is where the previous version was
too thin. It carries three facts, and all three are load-bearing:
**`machines`** — what sits where. A machine has a segment and an address, and that is nearly
all. `forwarded:` opens a port through the router, which is what makes *published but behind
NAT* reproducible — the case that exists only in production today. `segment: detached` is a
machine on no network, which is how a roaming node is expressed at rest.
- `to:` — the parent segment.
- `address:` — **the address the outside world sees this network as.** For a household
connection this is the public address the ISP hands out. It is not decoration: it is what a
peer records as the endpoint when a machine here dials out, and what a public name for a
published machine here resolves to.
- `nat:` — whether addresses are translated. `true` gives the ordinary household case: many
private machines behind one public address. `false` describes a routed range, where machines
keep their own addresses and the gateway only forwards.
**`place`** — what goes inside. `all:` applies to every machine; a machine name overrides for
that machine. This is the only part that differs between the two scenario classes.
The lab materialises a machine to be the gateway. That is the one implicit machine in an
otherwise explicit declaration, and it exists because NAT has to run somewhere.
**`snapshot`** — names the state once placement finishes, so a run can return to it without
raising everything again. Snapshots are what make repetition cheap, and cheap repetition is
what makes the bootstrap path the inner development loop rather than a ceremony.
**`machines[].at`** — segment and address. That pair alone determines which of the three
positions a machine is in: on a gateway-less segment it is directly attached; on a segment with
a gateway it is behind one.
**`machines[].published`** — a destination-NAT rule on a named gateway, stated as an outcome
rather than a port list. `{ port: 443, on: home }` means the `home` gateway forwards its own
`203.0.113.50:443` to this machine's `443`. The resulting public endpoint is derivable, which is
the point: a scenario never writes an endpoint down, and the mesh has to discover it.
A machine may be published on **any gateway between it and the internet** — which is how *"our
LAN also has a public IP"* is expressed, and why `on:` names the gateway rather than being
implied. It cannot be published at all on a gateway the scenario models as foreign; attempting
it is a declaration error, because that is precisely the constraint being reproduced.
**`at: detached`** — on no segment. A machine that exists and can reach nothing.
## Moving a machine is a lifecycle operation
`at:` states where a machine *starts*. Moving it is something a run does:
```
move laptop → { segment: elsewhere, address: 198.51.100.23 }
move laptop → detached
move laptop → { segment: home, address: 192.168.1.98 }
```
This is the roaming case made testable, and it is the one that finds the interesting faults.
The same machine, the same identity, three positions in one run: at home where its peers can
reach it directly, on a foreign network where it can only dial out and its apparent address
belongs to a router it does not control, and asleep.
Whether the overlay survives that, re-forms, and is noticed to have changed endpoint is
**observed**, never arranged
([ADR 0031](../../02-DECISIONS/0031-the-lab-provides-the-underlay.md)).
## Why the addresses are load-bearing
The routable segment uses RFC 5737 documentation space, and this is not a stylistic choice.
The internet segment uses RFC 5737 documentation space, and this is not a stylistic choice.
The mesh decides *public versus private* by matching the address. A private range on the
segment meant to be routable makes the hub test as unreachable, and **the mesh silently never
forms** — no error, no failed step, just a mesh that does not exist. Research 004 calls this
the single most important fact in its analysis.
segment meant to be routable makes a would-be hub test as unreachable, and **the mesh silently
never forms** — no error, no failed step, just a mesh that does not exist. Research 004 calls
this the single most important fact in its analysis.
So the format should make this hard to get wrong rather than merely documented: a segment
without `behind:` is a routable segment, and an address in it that is not documentation space
is a declaration error, refused before anything is raised. That is
RFC 5737 reserves three ranges, which is exactly enough for the topology above:
| Range | Used for |
|---|---|
| `203.0.113.0/24` | the internet segment itself — directly attached machines, and gateway addresses |
| `198.51.100.0/24` | a foreign network, so a roaming machine's apparent address is plainly not ours |
| `192.0.2.0/24` | spare — a second foreign network, or a second site |
Private segments use RFC 1918 and can be **byte-identical to production**, because those
addresses mean the same thing everywhere. Only the public side is substituted, and only because
it must be.
The format should make getting this wrong hard rather than merely documented: a segment without
a `gateway:` is a public segment, and an address in it — including a gateway's `address:` — that
is not documentation space is a declaration error, refused before anything is raised. That is
[ADR 0008](../../02-DECISIONS/0008-a-failed-step-fails-the-job.md) applied to a configuration
file — the failure it prevents is silent, so the check has to be loud.
file: the failure it prevents is silent, so the check has to be loud.
## The same declaration serves both classes
@@ -127,15 +202,17 @@ not first.
## Open
- **`user` and `edge` profiles have no scenario.** A lab machine is always privileged, so the
two profiles that exist for unprivileged and phone-like participation cannot currently be
exercised. Either the lab grows a way to run the host unprivileged, or those profiles are
developed against something that is not a virtual machine.
- **Attaching and detaching during a run.** `segment: detached` covers a machine at rest;
moving one between segments while a scenario is live is what makes a roaming node
interesting, and that is lifecycle rather than declaration.
two profiles that exist for unprivileged and phone-like participation cannot be exercised.
Either the lab grows a way to run the host unprivileged, or those profiles are developed
against something that is not a virtual machine. This is the largest gap.
- **Where `place:` gets its artifacts from.** Before the mesh is self-hosting these come from
outside; afterwards from the mesh itself. The declaration should not have to care, which
suggests a named source rather than a path.
- **Multiple scenarios at once.** Each needs its own segments and addresses. Whether the
declaration carries absolute addresses, as above, or a template the lab allocates from,
decides whether two scenarios can run side by side.
- **Multiple scenarios at once.** Each needs its own segments and addresses, and the shape above
writes addresses absolutely. Whether a scenario carries literal addresses or a template the
lab allocates from decides whether two can run side by side — and there are only three
documentation ranges to go round.
- **Gateway behaviour beyond forwarding.** A real household gateway also has a NAT table with
timeouts, and connection tracking that drops idle flows. Whether a scenario can express *"the
gateway forgets a mapping after N seconds"* decides whether keepalive behaviour is testable
or merely hoped for.