--- layer: to-be status: designed code: [mesh-lab] updated: 2026-08-23 decisions: - 02-DECISIONS/0029-the-labs-first-scenario-has-no-pipeline.md - 02-DECISIONS/0031-the-lab-provides-the-underlay.md - 02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md --- # The scenario declaration A scenario is a **declaration of an underlay**, plus what to put on it. It is the interface everything in the lab hangs off, so it is worth getting small. It states what a hosting provider and a home router would provide, and nothing the mesh is responsible for ([ADR 0031](../../02-DECISIONS/0031-the-lab-provides-the-underlay.md)). ## What NAT does, and why the design turns on it A household or office has **one** address the outside world can see, and **many** machines behind it. Network address translation is what reconciles those. When a machine inside dials out, the gateway rewrites the packet's source from the private address to the public one, **remembers the mapping**, and rewrites the replies on the way back. Four consequences follow, and every one of them shapes this design: 1. **Outbound works; inbound does not.** A mapping exists only because something inside started a conversation. Nothing outside can start one — there is no mapping to look up, and no way to know which internal machine was meant. 2. **A forwarded port is a permanent mapping made by hand**, in the inbound direction: *anything arriving at the public address on 443 goes to this machine.* That is the only way a machine behind NAT becomes reachable, and it requires control of the gateway. 3. **Mappings expire.** A gateway forgets one that goes unused. This is why anything holding a connection through NAT sends keepalives, and why a mesh that does not is fine until it is idle. 4. **From outside, every machine behind the gateway looks like one address.** Identity and address stop corresponding. This is why the mesh dials outward and never inward ([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)), why a hub exists at all, and why a node's endpoint is something a peer **learns** from arriving packets rather than something anyone configures. **Carrier-grade NAT** is the same mechanism applied by an ISP: your own gateway gets a private address too, and the public one is shared with strangers. Nothing can be forwarded, because the rule would have to live on equipment you do not own. Common on mobile connections and increasingly on fixed ones. ## Three positions a machine can be in The underlay's whole job is to reproduce **where a machine sits relative to the internet**, because that is what the mesh has to cope with and what only production currently exercises. There are three positions, and they are genuinely different: | Position | Reachable from outside | Apparent address | Example | |---|---|---|---| | **Attached** | yes, at its own address | its own | a hosted server | | **Behind a forwardable gateway** | only through a forwarded port, at the *gateway's* address | the gateway's | a machine at home | | **Behind an unforwardable gateway** | **no** | someone else's, and it changes | a laptop on a café network; anything behind carrier-grade NAT | The axis is **forwardability, not ownership** — which is worth stating because the obvious framing gets it wrong. Carrier-grade NAT is *your* connection and is still unforwardable, so it belongs in the third row alongside the café. What the mesh has to cope with is whether an inbound mapping can be made, not who owns the equipment. The third position is the hard one. A machine there can dial out and nothing more: it cannot be published, its apparent address belongs to a router it does not control, and that address changes when it moves. Every assumption a mesh makes about reachability breaks there first. A declaration has to be able to say all three, and to move a machine between them. ## Reachability is per address family, not per machine Adding IPv6 is not a field. It changes the position model, and the reason is worth stating before the syntax. **IPv6 usually has no NAT.** A machine behind a household gateway can hold a *globally routable* v6 address while its v4 address is private and unforwardable. The same machine, at the same moment, is in **two different positions at once**: | | IPv4 | IPv6 | |---|---|---| | a typical machine at home | behind an unforwardable-or-forwardable gateway | **attached**, directly reachable | | a machine on mobile data | behind carrier-grade NAT | often attached, sometimes absent entirely | | a machine on an older network | attached or behind NAT | **no address at all** | So the three positions apply **per family**, and a machine's reachability is a property of *(machine, family)* rather than of the machine. A mesh that treats reachability as one fact per node will reach a peer over one family, fail over the other, and report whichever it tried. That has a direct consequence for what the lab is for: *"can these two nodes reach each other"* stops being a yes/no question. It is asked once per family, and the interesting answers are the asymmetric ones. The v6 documentation prefix is `2001:db8::/32` (RFC 3849) — the exact counterpart of the RFC 5737 rule, and load-bearing for the same reason. ## The shape ```yaml scenario: roaming-and-published segments: internet: kind: public # RFC 5737 for v4, RFC 3849 for v6 cidr: [203.0.113.0/24, 2001:db8::/32] home: kind: private cidr: [192.168.1.0/24, 2001:db8:1::/64] mtu: 1500 gateway: to: internet address: 203.0.113.50 # what the world sees this network as, on v4 nat: [v4] # v4 is translated; v6 is routed, not translated forwardable: true mapping_ttl: 120s # an unused inbound mapping is forgotten after this elsewhere: # a network we do not control kind: private cidr: [198.51.100.0/24] # v4 only — no v6 offered here at all mtu: 1400 # a tunnelled path, smaller than standard gateway: to: internet address: 203.0.113.80 nat: [v4] forwardable: false # café wifi, or carrier-grade NAT mapping_ttl: 30s # aggressive, as carrier NAT tends to be machines: anchor: at: { segment: internet, address: [203.0.113.10, 2001:db8::10] } home-server: at: { segment: home, address: [192.168.1.135, 2001:db8:1::135] } published: - { port: 443, on: home } # v4 only: 203.0.113.50:443 → 192.168.1.135:443 inbound: allow # v6 is routable here, so this decides whether it is reachable workstation: at: { segment: home, address: [192.168.1.250, 2001:db8:1::250] } inbound: deny # a host firewall: dials out, accepts nothing laptop: at: { segment: home, address: [192.168.1.98, 2001:db8:1::98] } border: # a machine on two segments at once at: - { segment: home, address: [192.168.1.2] } - { segment: internet, address: [203.0.113.60] } place: all: [host] anchor: [substrate] snapshot: raised ``` ## What each part means, precisely **`segments`** — a broadcast domain with an address range, and a `kind:`. `kind: public` marks the segment that stands in for the internet. `kind: private` is everything else. This is stated rather than inferred, and the earlier version inferred it — *a segment with no gateway is the internet* — which made an **isolated network inexpressible**: a LAN with no route out is a private segment with no gateway, and would have been read as the internet and forced to use documentation addresses. A mesh spanning a site with no internet access is a real topology, and the model has to be able to say it. **`gateway:`** — how a segment reaches its parent, and this is where the previous version was too thin. It carries three facts, and all three are load-bearing: - `to:` — the parent segment. - `address:` — **the address the outside world sees this network as.** For a household connection this is the public address the ISP hands out. It is not decoration: it is what a peer records as the endpoint when a machine here dials out, and what a public name for a published machine here resolves to. - `nat:` — **which families are translated**, as a list. `[v4]` is the ordinary modern case: v4 translated, v6 routed. `[v4, v6]` describes a gateway that translates both, which exists and is worth being able to reproduce. `[]` is a routed range, where machines keep their own addresses and the gateway only forwards. - `forwardable:` — whether an inbound mapping can be created. Independent of `nat:`, and the field that separates a home gateway from carrier-grade NAT. Publishing through a gateway with `forwardable: false` is a declaration error, because that is exactly the constraint being reproduced. - `mapping_ttl:` — how long an unused inbound mapping survives. This is what makes keepalive behaviour testable: a mesh that holds a connection through NAT without refreshing it works perfectly until the far side goes quiet for longer than this. Aggressive values reproduce carrier NAT; omitting it means mappings never expire, which no real gateway does. **`segments[].mtu`** — the largest packet the segment carries, defaulting to 1500. Lower values reproduce tunnelled and PPPoE paths. This matters because an overlay adds its own header: a tunnel over a 1400-byte path establishes a connection and then silently drops large packets, which is the shape of fault this whole effort exists to stop shipping. **`machines[].inbound`** — `allow` or `deny`, a host firewall. Distinct from NAT and behaves differently: a machine can be perfectly routable and still refuse everything unsolicited, which is the normal state of a v6-addressed machine. Without this, v6 addressing would imply reachability, and it does not. The lab materialises a machine to be the gateway. That is the one implicit machine in an otherwise explicit declaration, and it exists because NAT has to run somewhere. **`machines[].at`** — segment and addresses, or a **list** of them for a machine on several segments at once. Multi-homing is not exotic: it is what a border machine is, and what any node with both a LAN and a WAN interface is. Each entry carries the addresses that machine holds on that segment, one per family. Position follows from the pair, per family: on a `kind: public` segment a machine is attached; on a private one it is behind that segment's gateway, unless the gateway does not translate that family — in which case it is attached on that family and behind a gateway on the other. **`machines[].published`** — a destination-NAT rule on a named gateway, stated as an outcome rather than a port list. `{ port: 443, on: home }` means the `home` gateway forwards its own `203.0.113.50:443` to this machine's `443`. The resulting public endpoint is derivable, which is the point: a scenario never writes an endpoint down, and the mesh has to discover it. A machine may be published on **any gateway between it and the internet** — which is how *"our LAN also has a public IP"* is expressed, and why `on:` names the gateway rather than being implied. It cannot be published at all on a gateway the scenario models as foreign; attempting it is a declaration error, because that is precisely the constraint being reproduced. **`at: detached`** — on no segment. A machine that exists and can reach nothing. ## Moving a machine is a lifecycle operation `at:` states where a machine *starts*. Moving it is something a run does: ``` move laptop → { segment: elsewhere, address: 198.51.100.23 } move laptop → detached move laptop → { segment: home, address: 192.168.1.98 } ``` This is the roaming case made testable, and it is the one that finds the interesting faults. The same machine, the same identity, three positions in one run: at home where its peers can reach it directly, on a foreign network where it can only dial out and its apparent address belongs to a router it does not control, and asleep. Whether the overlay survives that, re-forms, and is noticed to have changed endpoint is **observed**, never arranged ([ADR 0031](../../02-DECISIONS/0031-the-lab-provides-the-underlay.md)). ## Why the addresses are load-bearing The internet segment uses RFC 5737 documentation space, and this is not a stylistic choice. The mesh decides *public versus private* by matching the address. A private range on the segment meant to be routable makes a would-be hub test as unreachable, and **the mesh silently never forms** — no error, no failed step, just a mesh that does not exist. Research 004 calls this the single most important fact in its analysis. RFC 5737 reserves three ranges, which is exactly enough for the topology above: | Range | Used for | |---|---| | `203.0.113.0/24` | the internet segment itself — directly attached machines, and gateway addresses | | `198.51.100.0/24` | a foreign network, so a roaming machine's apparent address is plainly not ours | | `192.0.2.0/24` | spare — a second foreign network, or a second site | Private segments use RFC 1918 and can be **byte-identical to production**, because those addresses mean the same thing everywhere. Only the public side is substituted, and only because it must be. The format should make getting this wrong hard rather than merely documented: a segment without a `gateway:` is a public segment, and an address in it — including a gateway's `address:` — that is not documentation space is a declaration error, refused before anything is raised. That is [ADR 0008](../../02-DECISIONS/0008-a-failed-step-fails-the-job.md) applied to a configuration file: the failure it prevents is silent, so the check has to be loud. ## The same declaration serves both classes The bootstrap and full scenarios differ **only in `place:`** ([ADR 0029](../../02-DECISIONS/0029-the-labs-first-scenario-has-no-pipeline.md)). Everything about the underlay is identical, which is what makes one a strict subset of the other rather than a fork. ```yaml # bootstrap — tiers 0 and 1 place: all: [host] anchor: [substrate] # full — adds a control plane, a forge, and a module under test place: all: [host] anchor: [substrate, control, forge] module: a-web-service assert: - the service answers on its published name - the certificate presented is valid for that name ``` `module:` and `assert:` are meaningless in a bootstrap scenario and absent from one. A bootstrap scenario's verdict comes from what the host reports about the state it reconciled, not from an assertion runner — which is why assertion execution is second in the build order, not first. ## What a scenario deliberately cannot say - **Overlay addresses, the hub, peer configuration.** Outcomes, not inputs ([ADR 0031](../../02-DECISIONS/0031-the-lab-provides-the-underlay.md)). - **What a machine is in mesh terms** — server or workstation, its site, its names. Mesh configuration, established by the mesh. - **A host's capability profile.** Detected, never declared. - **Steps.** A scenario is a desired state. Anything expressed as an ordered list of actions belongs in the lifecycle, not the declaration. ## Is this general? — the axes a setup can vary along The question that matters is not *does this cover our mesh*, but **can it express any mesh**. The standard applied is not "every property a network has". It is **every property that changes how the mesh behaves**. Bandwidth does not change correctness; MTU does. | Axis | Values | Expressible | | |---|---|---|---| | **Reachability** | attached · forwardable gateway · unforwardable gateway · isolated | yes | the core of the model, and **per family** | | **Address family** | IPv4 · IPv6 · dual-stack · neither | yes | `cidr:` and `address:` take both; `nat:` names which families are translated | | **Interfaces per machine** | one · several | yes | `at:` takes a list | | **Reachability policy** | symmetric · asymmetric | yes | `inbound:` — a routable machine that refuses everything | | **Gateway state** | permanent · expiring mappings | yes | `mapping_ttl:` | | **Path MTU** | standard · reduced | yes | `segments[].mtu` | | **Overlapping ranges** | distinct · two sites both on `192.168.1.0/24` | yes | segments may carry the same range | | **Segment count** | one · many · isolated island | yes | `kind:` distinguishes an island from the internet | | **Gateway depth** | direct · one gateway · nested | **partly** | `to:` chains, so nesting exists; `published:` names one gateway, so forwarding through two does not | | **Address stability** | static · dynamic · changes mid-run | **partly** | a machine can be *moved*; an address changing under it in place cannot be stated | | **Path quality** | latency · loss · bandwidth | **no**, deliberately | changes performance, not correctness — modelling it makes a network simulator, not a fixture | ### What closing the gaps changed **Address family was not a field.** It changed the position model: a machine behind a household gateway is typically *unforwardable on v4 and directly attached on v6, simultaneously*. So reachability is a property of *(machine, family)*, and *"can these two nodes reach each other"* is no longer a yes/no question — it is asked once per family, and the asymmetric answers are the interesting ones. That distinction did not exist in the model an hour ago and would have been discovered by a mesh failing over one family while reporting the other. **`inbound:` became necessary because of v6.** With NAT, unreachability was implied by the topology. With a globally routable v6 address, a machine is reachable unless something refuses — so refusing has to be sayable, or v6 addressing would silently imply reachability. **`nat:` became a list rather than a boolean** for the same reason: a real gateway translates v4 and routes v6, and a boolean cannot say that. ### What remains open, and whether it matters Two partial axes, both extensible when something needs them, neither blocking: **nested forwarding** and **an address changing in place**. A machine can already be moved, which covers the roaming case; what is missing is a lease expiring underneath a machine that stays put. One deliberate exclusion: **path quality**. Latency and loss change how fast the mesh is, not whether it is correct. If a timeout turns out to be load-bearing that judgement should be revisited — and it would be revisited by a real failure, which is the right trigger. ### The honest summary The model now covers **where a machine sits** and **what the path between machines is like**, across both address families, which together are what the mesh's reachability logic turns on. What it does not model is *change over time* beyond moving a machine, and *degradation* short of failure. Both are absences chosen rather than overlooked. ## Open - **`user` and `edge` profiles have no scenario.** A lab machine is always privileged, so the two profiles that exist for unprivileged and phone-like participation cannot be exercised. Either the lab grows a way to run the host unprivileged, or those profiles are developed against something that is not a virtual machine. This is the largest gap. - **Where `place:` gets its artifacts from.** Before the mesh is self-hosting these come from outside; afterwards from the mesh itself. The declaration should not have to care, which suggests a named source rather than a path. - **Multiple scenarios at once.** Each needs its own segments and addresses, and the shape above writes addresses absolutely. Whether a scenario carries literal addresses or a template the lab allocates from decides whether two can run side by side — and there are only three documentation ranges to go round. - **Nested forwarding** — `published:` names one gateway, so a machine behind two cannot be published through both. - **An address changing in place**, as a DHCP lease expiring under a machine that has not moved.