Files
hq/03-DESIGN/01-to-be/02-scenario-declaration.md
T
jschoubben b944904f1a Audit the scenario model for generality, and fix what it found
The question is not whether the model covers our mesh but whether it can
express any mesh. Audited against the axes a deployment varies along, with
the standard being every property that changes how the mesh BEHAVES rather
than every property a network has — bandwidth does not change correctness,
MTU does.

One real bug, now fixed. A segment with no gateway was read as the
internet, which made an isolated network inexpressible: a LAN with no route
out would have been treated as public and forced onto documentation
addresses. Segments now state kind: public or private, and a private
segment with no gateway is an island. A mesh spanning a site with no
internet is a real topology.

One modelling error, now corrected. The three positions were framed by
ownership — a gateway you control versus one you do not. The axis is
forwardability. Carrier-grade NAT is your own connection and is still
unforwardable, so it belongs with the café network. Gateways gain
forwardable:, independent of nat:, and publishing through an unforwardable
one is a declaration error because that is the constraint being reproduced.

Three genuine gaps recorded in priority order. Address family: cidr is
implicitly v4, and a v6-only node is not exotic — a mesh that assumes v4
fails there completely rather than partially, which makes this a second
world rather than a refinement. Expiring NAT mappings: without them
keepalive behaviour is hoped for rather than tested, and for a mesh mostly
behind NAT that is the fault that shows up after an idle night. MTU:
tunnels fragment, and a smaller-MTU path establishes a connection that then
silently drops large packets — the exact shape this effort exists to stop
shipping.

Latency and loss are deliberately out: they change performance, not
correctness, and modelling them makes a network simulator rather than a
fixture.

Also adds a NAT primer, because the three positions are consequences of it
and the document should not assume the reader already knows why a mesh
dials outward and never inward.
2026-08-23 22:48:46 +02:00

16 KiB

layer, status, code, updated, decisions
layer status code updated decisions
to-be designed
mesh-lab
2026-08-23
02-DECISIONS/0029-the-labs-first-scenario-has-no-pipeline.md
02-DECISIONS/0031-the-lab-provides-the-underlay.md
02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md

The scenario declaration

A scenario is a declaration of an underlay, plus what to put on it. It is the interface everything in the lab hangs off, so it is worth getting small.

It states what a hosting provider and a home router would provide, and nothing the mesh is responsible for (ADR 0031).

What NAT does, and why the design turns on it

A household or office has one address the outside world can see, and many machines behind it. Network address translation is what reconciles those.

When a machine inside dials out, the gateway rewrites the packet's source from the private address to the public one, remembers the mapping, and rewrites the replies on the way back. Four consequences follow, and every one of them shapes this design:

  1. Outbound works; inbound does not. A mapping exists only because something inside started a conversation. Nothing outside can start one — there is no mapping to look up, and no way to know which internal machine was meant.
  2. A forwarded port is a permanent mapping made by hand, in the inbound direction: anything arriving at the public address on 443 goes to this machine. That is the only way a machine behind NAT becomes reachable, and it requires control of the gateway.
  3. Mappings expire. A gateway forgets one that goes unused. This is why anything holding a connection through NAT sends keepalives, and why a mesh that does not is fine until it is idle.
  4. From outside, every machine behind the gateway looks like one address. Identity and address stop corresponding.

This is why the mesh dials outward and never inward (ADR 0001), why a hub exists at all, and why a node's endpoint is something a peer learns from arriving packets rather than something anyone configures.

Carrier-grade NAT is the same mechanism applied by an ISP: your own gateway gets a private address too, and the public one is shared with strangers. Nothing can be forwarded, because the rule would have to live on equipment you do not own. Common on mobile connections and increasingly on fixed ones.

Three positions a machine can be in

The underlay's whole job is to reproduce where a machine sits relative to the internet, because that is what the mesh has to cope with and what only production currently exercises. There are three positions, and they are genuinely different:

Position Reachable from outside Apparent address Example
Attached yes, at its own address its own a hosted server
Behind a forwardable gateway only through a forwarded port, at the gateway's address the gateway's a machine at home
Behind an unforwardable gateway no someone else's, and it changes a laptop on a café network; anything behind carrier-grade NAT

The axis is forwardability, not ownership — which is worth stating because the obvious framing gets it wrong. Carrier-grade NAT is your connection and is still unforwardable, so it belongs in the third row alongside the café. What the mesh has to cope with is whether an inbound mapping can be made, not who owns the equipment.

The third position is the hard one. A machine there can dial out and nothing more: it cannot be published, its apparent address belongs to a router it does not control, and that address changes when it moves. Every assumption a mesh makes about reachability breaks there first.

A declaration has to be able to say all three, and to move a machine between them.

The shape

scenario: roaming-and-published

segments:
  internet:
    kind: public                  # stands in for the internet — RFC 5737 addresses
    cidr: 203.0.113.0/24
  home:
    kind: private
    cidr: 192.168.1.0/24
    gateway:
      to: internet
      address: 203.0.113.50       # what the world sees this network as
      nat: true
      forwardable: true           # we control it, so ports can be opened
  elsewhere:                      # a network we do not control
    kind: private
    cidr: 198.51.100.0/24
    gateway:
      to: internet
      address: 203.0.113.80
      nat: true
      forwardable: false          # café wifi, or carrier-grade NAT

machines:
  anchor:
    at: { segment: internet, address: 203.0.113.10 }

  home-server:
    at: { segment: home, address: 192.168.1.135 }
    published:
      - { port: 443, on: home }   # DNAT: 203.0.113.50:443 → 192.168.1.135:443

  workstation:
    at: { segment: home, address: 192.168.1.250 }

  laptop:
    at: { segment: home, address: 192.168.1.98 }

place:
  all: [host]
  anchor: [substrate]

snapshot: raised

What each part means, precisely

segments — a broadcast domain with an address range, and a kind:.

kind: public marks the segment that stands in for the internet. kind: private is everything else. This is stated rather than inferred, and the earlier version inferred it — a segment with no gateway is the internet — which made an isolated network inexpressible: a LAN with no route out is a private segment with no gateway, and would have been read as the internet and forced to use documentation addresses. A mesh spanning a site with no internet access is a real topology, and the model has to be able to say it.

gateway: — how a segment reaches its parent, and this is where the previous version was too thin. It carries three facts, and all three are load-bearing:

  • to: — the parent segment.
  • address: — the address the outside world sees this network as. For a household connection this is the public address the ISP hands out. It is not decoration: it is what a peer records as the endpoint when a machine here dials out, and what a public name for a published machine here resolves to.
  • nat: — whether addresses are translated. true gives the ordinary household case: many private machines behind one public address. false describes a routed range, where machines keep their own addresses and the gateway only forwards.
  • forwardable: — whether an inbound mapping can be created. Independent of nat:, and the field that separates a home gateway from carrier-grade NAT. Publishing through a gateway with forwardable: false is a declaration error, because that is exactly the constraint being reproduced.

The lab materialises a machine to be the gateway. That is the one implicit machine in an otherwise explicit declaration, and it exists because NAT has to run somewhere.

machines[].at — segment and address. That pair alone determines which of the three positions a machine is in: on a gateway-less segment it is directly attached; on a segment with a gateway it is behind one.

machines[].published — a destination-NAT rule on a named gateway, stated as an outcome rather than a port list. { port: 443, on: home } means the home gateway forwards its own 203.0.113.50:443 to this machine's 443. The resulting public endpoint is derivable, which is the point: a scenario never writes an endpoint down, and the mesh has to discover it.

A machine may be published on any gateway between it and the internet — which is how "our LAN also has a public IP" is expressed, and why on: names the gateway rather than being implied. It cannot be published at all on a gateway the scenario models as foreign; attempting it is a declaration error, because that is precisely the constraint being reproduced.

at: detached — on no segment. A machine that exists and can reach nothing.

Moving a machine is a lifecycle operation

at: states where a machine starts. Moving it is something a run does:

move laptop → { segment: elsewhere, address: 198.51.100.23 }
move laptop → detached
move laptop → { segment: home, address: 192.168.1.98 }

This is the roaming case made testable, and it is the one that finds the interesting faults. The same machine, the same identity, three positions in one run: at home where its peers can reach it directly, on a foreign network where it can only dial out and its apparent address belongs to a router it does not control, and asleep.

Whether the overlay survives that, re-forms, and is noticed to have changed endpoint is observed, never arranged (ADR 0031).

Why the addresses are load-bearing

The internet segment uses RFC 5737 documentation space, and this is not a stylistic choice.

The mesh decides public versus private by matching the address. A private range on the segment meant to be routable makes a would-be hub test as unreachable, and the mesh silently never forms — no error, no failed step, just a mesh that does not exist. Research 004 calls this the single most important fact in its analysis.

RFC 5737 reserves three ranges, which is exactly enough for the topology above:

Range Used for
203.0.113.0/24 the internet segment itself — directly attached machines, and gateway addresses
198.51.100.0/24 a foreign network, so a roaming machine's apparent address is plainly not ours
192.0.2.0/24 spare — a second foreign network, or a second site

Private segments use RFC 1918 and can be byte-identical to production, because those addresses mean the same thing everywhere. Only the public side is substituted, and only because it must be.

The format should make getting this wrong hard rather than merely documented: a segment without a gateway: is a public segment, and an address in it — including a gateway's address: — that is not documentation space is a declaration error, refused before anything is raised. That is ADR 0008 applied to a configuration file: the failure it prevents is silent, so the check has to be loud.

The same declaration serves both classes

The bootstrap and full scenarios differ only in place: (ADR 0029). Everything about the underlay is identical, which is what makes one a strict subset of the other rather than a fork.

# bootstrap — tiers 0 and 1
place:
  all: [host]
  anchor: [substrate]

# full — adds a control plane, a forge, and a module under test
place:
  all: [host]
  anchor: [substrate, control, forge]
module: a-web-service
assert:
  - the service answers on its published name
  - the certificate presented is valid for that name

module: and assert: are meaningless in a bootstrap scenario and absent from one. A bootstrap scenario's verdict comes from what the host reports about the state it reconciled, not from an assertion runner — which is why assertion execution is second in the build order, not first.

What a scenario deliberately cannot say

  • Overlay addresses, the hub, peer configuration. Outcomes, not inputs (ADR 0031).
  • What a machine is in mesh terms — server or workstation, its site, its names. Mesh configuration, established by the mesh.
  • A host's capability profile. Detected, never declared.
  • Steps. A scenario is a desired state. Anything expressed as an ordered list of actions belongs in the lifecycle, not the declaration.

Is this general? — the axes a setup can vary along

The question that matters is not does this cover our mesh, but can it express any mesh. Audited against the axes a real deployment varies along, the answer is most, deliberately not all, and three genuine gaps.

The standard applied is not "every property a network has". It is every property that changes how the mesh behaves. Bandwidth does not change correctness; MTU does.

Axis Values Expressible
Reachability attached · forwardable gateway · unforwardable gateway · isolated yes the core of the model
Address stability static · dynamic · changes mid-run partly a machine can be moved, but an address that changes under it cannot be stated
Gateway depth direct · one gateway · nested gateways partly to: chains, so nesting exists; published: names one gateway, so forwarding through two does not
Address family IPv4 · IPv6 · dual-stack no cidr: is implicitly v4. A v6-only node is a real topology and cannot be written
Interfaces per machine one · several no at: is singular. A multi-homed node — on a LAN and a WAN at once — is inexpressible
Path properties MTU · latency · loss no MTU matters: tunnels fragment, and a lower-MTU path is a classic silent failure
Reachability policy symmetric · asymmetric no a firewall dropping inbound while outbound works is different from NAT and behaves differently
Gateway state permanent · expiring mappings no mappings time out; whether keepalives work is untestable without it
Overlapping ranges distinct · two sites both on 192.168.1.0/24 yes two segments may carry the same range — common, and it breaks routing
Segment count one · many · isolated island yes after the kind: fix above

What this says

Three gaps are real and should be closed, in this order:

  1. Address family. A v6-only or dual-stack node is not exotic, and a mesh that assumes v4 fails there completely rather than partially. This is the largest gap.
  2. Expiring NAT mappings. Without it, keepalive behaviour is hoped for rather than tested — and for a mesh where most nodes sit behind NAT, that is the failure mode most likely to appear only after everything has been idle overnight.
  3. MTU. Tunnels fragment. A path with a smaller MTU produces a connection that establishes and then silently drops large packets, which is exactly the shape of fault this whole effort exists to stop shipping.

Two are deliberately out of scope unless something argues otherwise: latency and loss. They change performance, not correctness, and a scenario that models them is a network simulator rather than a fixture.

Two are partial and probably fine for now: nested forwarding and mid-run address change. Both are expressible with small extensions when something needs them, and neither blocks the bootstrap scenario.

The honest summary

The model covers where a machine sits, which is what the mesh's reachability logic turns on, and it now covers it completely. It does not yet cover what the path between machines is like, and one of those — address family — is not a refinement but a second world the mesh would have to work in.

None of this blocks phase 0. A bootstrap scenario is one machine and a pinned bundle, and needs none of it. But the gaps should be closed before the lab is trusted to say a mesh works, because today it could only say it works over IPv4, on an unconstrained path, against gateways that never forget.

Open

  • user and edge profiles have no scenario. A lab machine is always privileged, so the two profiles that exist for unprivileged and phone-like participation cannot be exercised. Either the lab grows a way to run the host unprivileged, or those profiles are developed against something that is not a virtual machine. This is the largest gap.
  • Where place: gets its artifacts from. Before the mesh is self-hosting these come from outside; afterwards from the mesh itself. The declaration should not have to care, which suggests a named source rather than a path.
  • Multiple scenarios at once. Each needs its own segments and addresses, and the shape above writes addresses absolutely. Whether a scenario carries literal addresses or a template the lab allocates from decides whether two can run side by side — and there are only three documentation ranges to go round.
  • The three gaps from the audit above — address family, expiring NAT mappings, MTU — in that order. The first is the one that is a second world rather than a refinement.