novox/hq
3
0
Fork 0
Code Issues Pull Requests 5 Releases Wiki Activity

Close the missing axes — and address family changes the model

Browse Source 
Address family was not a field. IPv6 usually has no NAT, so a machine
behind a household gateway is typically unforwardable on v4 and DIRECTLY
ATTACHED on v6, at the same moment. The three positions therefore apply
per family, and reachability is a property of (machine, family) rather
than of a machine.

The consequence is bigger than the syntax: 'can these two nodes reach each
other' stops being a yes/no question. It is asked once per family, and the
asymmetric answers are the interesting ones. A mesh treating reachability
as one fact per node reaches a peer over one family, fails over the other,
and reports whichever it tried. That distinction did not exist in the model
and would have been found by a failure rather than by reading.

Two fields follow from it. inbound: allow|deny became necessary because
with NAT unreachability was implied by topology, while a globally routable
v6 address is reachable unless something refuses — so refusing has to be
sayable or v6 addressing silently implies reachability. And nat: became a
list of families rather than a boolean, because a real gateway translates
v4 and routes v6 and a boolean cannot say that.

mapping_ttl closes the keepalive gap: a mesh holding a connection through
NAT without refreshing it works perfectly until the far side goes quiet
for longer than the mapping lives.

segments[].mtu closes the fragmentation gap: an overlay adds a header, so
a tunnel over a reduced-MTU path establishes a connection and then
silently drops large packets.

at: takes a list, so a multi-homed machine is expressible — which the
model already implicitly required, since a border machine sits on two
segments.

v6 uses RFC 3849 documentation space, the exact counterpart of the RFC
5737 rule and load-bearing for the same reason.

Remaining: nested forwarding and an address changing in place, both
extensible when needed. Path quality stays deliberately out — it changes
performance, not correctness, and modelling it makes a network simulator
rather than a fixture.
This commit is contained in:
jschoubben
2026-08-23 22:59:31 +02:00
parent b944904f1a
commit 274bd3b304
1 changed files with 117 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
03-DESIGN/01-to-be/02-scenario-declaration.md
+117 -57
View File
@@ -71,6 +71,32 @@ changes when it moves. Every assumption a mesh makes about reachability breaks t
A declaration has to be able to say all three, and to move a machine between them. A declaration has to be able to say all three, and to move a machine between them.
## Reachability is per address family, not per machine
Adding IPv6 is not a field. It changes the position model, and the reason is worth stating
before the syntax.
**IPv6 usually has no NAT.** A machine behind a household gateway can hold a *globally routable*
v6 address while its v4 address is private and unforwardable. The same machine, at the same
moment, is in **two different positions at once**:
| | IPv4 | IPv6 |
|---|---|---|
| a typical machine at home | behind an unforwardable-or-forwardable gateway | **attached**, directly reachable |
| a machine on mobile data | behind carrier-grade NAT | often attached, sometimes absent entirely |
| a machine on an older network | attached or behind NAT | **no address at all** |
So the three positions apply **per family**, and a machine's reachability is a property of
*(machine, family)* rather than of the machine. A mesh that treats reachability as one fact per
node will reach a peer over one family, fail over the other, and report whichever it tried.
That has a direct consequence for what the lab is for: *"can these two nodes reach each other"*
stops being a yes/no question. It is asked once per family, and the interesting answers are the
asymmetric ones.
The v6 documentation prefix is `2001:db8::/32` (RFC 3849) — the exact counterpart of the RFC
5737 rule, and load-bearing for the same reason.
## The shape ## The shape
```yaml ```yaml
@@ -78,39 +104,52 @@ scenario: roaming-and-published
segments: segments:
internet: internet:
kind: public # stands in for the internet — RFC 5737 addresses kind: public # RFC 5737 for v4, RFC 3849 for v6
cidr: 203.0.113.0/24 cidr: [203.0.113.0/24, 2001:db8::/32]
home: home:
kind: private kind: private
cidr: 192.168.1.0/24 cidr: [192.168.1.0/24, 2001:db8:1::/64]
mtu: 1500
gateway: gateway:
to: internet to: internet
address: 203.0.113.50 # what the world sees this network as address: 203.0.113.50 # what the world sees this network as, on v4
nat: true nat: [v4] # v4 is translated; v6 is routed, not translated
forwardable: true # we control it, so ports can be opened forwardable: true
mapping_ttl: 120s # an unused inbound mapping is forgotten after this
elsewhere: # a network we do not control elsewhere: # a network we do not control
kind: private kind: private
cidr: 198.51.100.0/24 cidr: [198.51.100.0/24] # v4 only — no v6 offered here at all
mtu: 1400 # a tunnelled path, smaller than standard
gateway: gateway:
to: internet to: internet
address: 203.0.113.80 address: 203.0.113.80
nat: true nat: [v4]
forwardable: false # café wifi, or carrier-grade NAT forwardable: false # café wifi, or carrier-grade NAT
mapping_ttl: 30s # aggressive, as carrier NAT tends to be
machines: machines:
anchor: anchor:
at: { segment: internet, address: 203.0.113.10 } at: { segment: internet, address: [203.0.113.10, 2001:db8::10] }
home-server: home-server:
at: { segment: home, address: 192.168.1.135 } at: { segment: home, address: [192.168.1.135, 2001:db8:1::135] }
published: published:
- { port: 443, on: home } # DNAT: 203.0.113.50:443 → 192.168.1.135:443 - { port: 443, on: home } # v4 only: 203.0.113.50:443 → 192.168.1.135:443
inbound: allow # v6 is routable here, so this decides whether it is reachable
workstation: workstation:
at: { segment: home, address: 192.168.1.250 } at: { segment: home, address: [192.168.1.250, 2001:db8:1::250] }
inbound: deny # a host firewall: dials out, accepts nothing
laptop: laptop:
at: { segment: home, address: 192.168.1.98 } at: { segment: home, address: [192.168.1.98, 2001:db8:1::98] }
border: # a machine on two segments at once
at:
- { segment: home, address: [192.168.1.2] }
- { segment: internet, address: [203.0.113.60] }
place: place:
all: [host] all: [host]
@@ -138,20 +177,40 @@ too thin. It carries three facts, and all three are load-bearing:
connection this is the public address the ISP hands out. It is not decoration: it is what a connection this is the public address the ISP hands out. It is not decoration: it is what a
peer records as the endpoint when a machine here dials out, and what a public name for a peer records as the endpoint when a machine here dials out, and what a public name for a
published machine here resolves to. published machine here resolves to.
- `nat:` — whether addresses are translated. `true` gives the ordinary household case: many - `nat:` — **which families are translated**, as a list. `[v4]` is the ordinary modern case:
private machines behind one public address. `false` describes a routed range, where machines v4 translated, v6 routed. `[v4, v6]` describes a gateway that translates both, which exists
keep their own addresses and the gateway only forwards. and is worth being able to reproduce. `[]` is a routed range, where machines keep their own
addresses and the gateway only forwards.
- `forwardable:` — whether an inbound mapping can be created. Independent of `nat:`, and the - `forwardable:` — whether an inbound mapping can be created. Independent of `nat:`, and the
field that separates a home gateway from carrier-grade NAT. Publishing through a gateway with field that separates a home gateway from carrier-grade NAT. Publishing through a gateway with
`forwardable: false` is a declaration error, because that is exactly the constraint being `forwardable: false` is a declaration error, because that is exactly the constraint being
reproduced. reproduced.
- `mapping_ttl:` — how long an unused inbound mapping survives. This is what makes keepalive
behaviour testable: a mesh that holds a connection through NAT without refreshing it works
perfectly until the far side goes quiet for longer than this. Aggressive values reproduce
carrier NAT; omitting it means mappings never expire, which no real gateway does.
**`segments[].mtu`** — the largest packet the segment carries, defaulting to 1500. Lower values
reproduce tunnelled and PPPoE paths. This matters because an overlay adds its own header: a
tunnel over a 1400-byte path establishes a connection and then silently drops large packets,
which is the shape of fault this whole effort exists to stop shipping.
**`machines[].inbound`** — `allow` or `deny`, a host firewall. Distinct from NAT and behaves
differently: a machine can be perfectly routable and still refuse everything unsolicited, which
is the normal state of a v6-addressed machine. Without this, v6 addressing would imply
reachability, and it does not.
The lab materialises a machine to be the gateway. That is the one implicit machine in an The lab materialises a machine to be the gateway. That is the one implicit machine in an
otherwise explicit declaration, and it exists because NAT has to run somewhere. otherwise explicit declaration, and it exists because NAT has to run somewhere.
**`machines[].at`** — segment and address. That pair alone determines which of the three **`machines[].at`** — segment and addresses, or a **list** of them for a machine on several
positions a machine is in: on a gateway-less segment it is directly attached; on a segment with segments at once. Multi-homing is not exotic: it is what a border machine is, and what any node
a gateway it is behind one. with both a LAN and a WAN interface is. Each entry carries the addresses that machine holds on
that segment, one per family.
Position follows from the pair, per family: on a `kind: public` segment a machine is attached;
on a private one it is behind that segment's gateway, unless the gateway does not translate
that family — in which case it is attached on that family and behind a gateway on the other.
**`machines[].published`** — a destination-NAT rule on a named gateway, stated as an outcome **`machines[].published`** — a destination-NAT rule on a named gateway, stated as an outcome
rather than a port list. `{ port: 443, on: home }` means the `home` gateway forwards its own rather than a port list. `{ port: 443, on: home }` means the `home` gateway forwards its own
@@ -252,57 +311,57 @@ not first.
## Is this general? — the axes a setup can vary along ## Is this general? — the axes a setup can vary along
The question that matters is not *does this cover our mesh*, but **can it express any mesh**. The question that matters is not *does this cover our mesh*, but **can it express any mesh**.
Audited against the axes a real deployment varies along, the answer is *most, deliberately not
all, and three genuine gaps*.
The standard applied is not "every property a network has". It is **every property that changes The standard applied is not "every property a network has". It is **every property that changes
how the mesh behaves**. Bandwidth does not change correctness; MTU does. how the mesh behaves**. Bandwidth does not change correctness; MTU does.
| Axis | Values | Expressible | | | Axis | Values | Expressible | |
|---|---|---|---| |---|---|---|---|
| **Reachability** | attached · forwardable gateway · unforwardable gateway · isolated | yes | the core of the model | | **Reachability** | attached · forwardable gateway · unforwardable gateway · isolated | yes | the core of the model, and **per family** |
| **Address stability** | static · dynamic · changes mid-run | **partly** | a machine can be *moved*, but an address that changes under it cannot be stated | | **Address family** | IPv4 · IPv6 · dual-stack · neither | yes | `cidr:` and `address:` take both; `nat:` names which families are translated |
| **Gateway depth** | direct · one gateway · nested gateways | **partly** | `to:` chains, so nesting exists; `published:` names one gateway, so forwarding through two does not | | **Interfaces per machine** | one · several | yes | `at:` takes a list |
| **Address family** | IPv4 · IPv6 · dual-stack | **no** | `cidr:` is implicitly v4. A v6-only node is a real topology and cannot be written | | **Reachability policy** | symmetric · asymmetric | yes | `inbound:` — a routable machine that refuses everything |
| **Interfaces per machine** | one · several | **no** | `at:` is singular. A multi-homed node — on a LAN and a WAN at once — is inexpressible | | **Gateway state** | permanent · expiring mappings | yes | `mapping_ttl:` |
| **Path properties** | MTU · latency · loss | **no** | MTU matters: tunnels fragment, and a lower-MTU path is a classic silent failure | | **Path MTU** | standard · reduced | yes | `segments[].mtu` |
| **Reachability policy** | symmetric · asymmetric | **no** | a firewall dropping inbound while outbound works is different from NAT and behaves differently | | **Overlapping ranges** | distinct · two sites both on `192.168.1.0/24` | yes | segments may carry the same range |
| **Gateway state** | permanent · expiring mappings | **no** | mappings time out; whether keepalives work is untestable without it | | **Segment count** | one · many · isolated island | yes | `kind:` distinguishes an island from the internet |
| **Overlapping ranges** | distinct · two sites both on `192.168.1.0/24` | yes | two segments may carry the same range — common, and it breaks routing | | **Gateway depth** | direct · one gateway · nested | **partly** | `to:` chains, so nesting exists; `published:` names one gateway, so forwarding through two does not |
| **Segment count** | one · many · isolated island | yes | after the `kind:` fix above | | **Address stability** | static · dynamic · changes mid-run | **partly** | a machine can be *moved*; an address changing under it in place cannot be stated |
| **Path quality** | latency · loss · bandwidth | **no**, deliberately | changes performance, not correctness — modelling it makes a network simulator, not a fixture |
### What this says ### What closing the gaps changed
**Three gaps are real and should be closed**, in this order: **Address family was not a field.** It changed the position model: a machine behind a household
gateway is typically *unforwardable on v4 and directly attached on v6, simultaneously*. So
reachability is a property of *(machine, family)*, and *"can these two nodes reach each other"*
is no longer a yes/no question — it is asked once per family, and the asymmetric answers are the
interesting ones. That distinction did not exist in the model an hour ago and would have been
discovered by a mesh failing over one family while reporting the other.
1. **Address family.** A v6-only or dual-stack node is not exotic, and a mesh that assumes v4 **`inbound:` became necessary because of v6.** With NAT, unreachability was implied by the
fails there completely rather than partially. This is the largest gap. topology. With a globally routable v6 address, a machine is reachable unless something refuses —
2. **Expiring NAT mappings.** Without it, keepalive behaviour is hoped for rather than tested — so refusing has to be sayable, or v6 addressing would silently imply reachability.
and for a mesh where most nodes sit behind NAT, that is the failure mode most likely to
appear only after everything has been idle overnight.
3. **MTU.** Tunnels fragment. A path with a smaller MTU produces a connection that establishes
and then silently drops large packets, which is exactly the shape of fault this whole effort
exists to stop shipping.
**Two are deliberately out of scope** unless something argues otherwise: latency and loss. **`nat:` became a list rather than a boolean** for the same reason: a real gateway translates v4
They change performance, not correctness, and a scenario that models them is a network and routes v6, and a boolean cannot say that.
simulator rather than a fixture.
**Two are partial and probably fine for now:** nested forwarding and mid-run address change. ### What remains open, and whether it matters
Both are expressible with small extensions when something needs them, and neither blocks the
bootstrap scenario. Two partial axes, both extensible when something needs them, neither blocking: **nested
forwarding** and **an address changing in place**. A machine can already be moved, which covers
the roaming case; what is missing is a lease expiring underneath a machine that stays put.
One deliberate exclusion: **path quality**. Latency and loss change how fast the mesh is, not
whether it is correct. If a timeout turns out to be load-bearing that judgement should be
revisited — and it would be revisited by a real failure, which is the right trigger.
### The honest summary ### The honest summary
The model covers **where a machine sits**, which is what the mesh's reachability logic turns The model now covers **where a machine sits** and **what the path between machines is like**,
on, and it now covers it completely. It does not yet cover **what the path between machines is across both address families, which together are what the mesh's reachability logic turns on.
like**, and one of those — address family — is not a refinement but a second world the mesh
would have to work in.
None of this blocks phase 0. A bootstrap scenario is one machine and a pinned bundle, and needs What it does not model is *change over time* beyond moving a machine, and *degradation* short of
none of it. But the gaps should be closed before the lab is trusted to say a mesh *works*, failure. Both are absences chosen rather than overlooked.
because today it could only say it works over IPv4, on an unconstrained path, against gateways
that never forget.
## Open ## Open
@@ -317,5 +376,6 @@ that never forget.
writes addresses absolutely. Whether a scenario carries literal addresses or a template the writes addresses absolutely. Whether a scenario carries literal addresses or a template the
lab allocates from decides whether two can run side by side — and there are only three lab allocates from decides whether two can run side by side — and there are only three
documentation ranges to go round. documentation ranges to go round.
- **The three gaps from the audit above** — address family, expiring NAT mappings, MTU — in - **Nested forwarding** — `published:` names one gateway, so a machine behind two cannot be
that order. The first is the one that is a second world rather than a refinement. published through both.
- **An address changing in place**, as a DHCP lease expiring under a machine that has not moved.