novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Close the missing axes — and address family changes the model

Browse Source 
Address family was not a field. IPv6 usually has no NAT, so a machine
behind a household gateway is typically unforwardable on v4 and DIRECTLY
ATTACHED on v6, at the same moment. The three positions therefore apply
per family, and reachability is a property of (machine, family) rather
than of a machine.

The consequence is bigger than the syntax: 'can these two nodes reach each
other' stops being a yes/no question. It is asked once per family, and the
asymmetric answers are the interesting ones. A mesh treating reachability
as one fact per node reaches a peer over one family, fails over the other,
and reports whichever it tried. That distinction did not exist in the model
and would have been found by a failure rather than by reading.

Two fields follow from it. inbound: allow|deny became necessary because
with NAT unreachability was implied by topology, while a globally routable
v6 address is reachable unless something refuses — so refusing has to be
sayable or v6 addressing silently implies reachability. And nat: became a
list of families rather than a boolean, because a real gateway translates
v4 and routes v6 and a boolean cannot say that.

mapping_ttl closes the keepalive gap: a mesh holding a connection through
NAT without refreshing it works perfectly until the far side goes quiet
for longer than the mapping lives.

segments[].mtu closes the fragmentation gap: an overlay adds a header, so
a tunnel over a reduced-MTU path establishes a connection and then
silently drops large packets.

at: takes a list, so a multi-homed machine is expressible — which the
model already implicitly required, since a border machine sits on two
segments.

v6 uses RFC 3849 documentation space, the exact counterpart of the RFC
5737 rule and load-bearing for the same reason.

Remaining: nested forwarding and an address changing in place, both
extensible when needed. Path quality stays deliberately out — it changes
performance, not correctness, and modelling it makes a network simulator
rather than a fixture.
This commit is contained in:
jschoubben
2026-08-23 22:59:31 +02:00
parent b944904f1a
commit 274bd3b304
1 changed files with 117 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
03-DESIGN/01-to-be/02-scenario-declaration.md
+117 -57
View File
@@ -71,6 +71,32 @@ changes when it moves. Every assumption a mesh makes about reachability breaks t
A declaration has to be able to say all three, and to move a machine between them.
## Reachability is per address family, not per machine
Adding IPv6 is not a field. It changes the position model, and the reason is worth stating
before the syntax.
**IPv6 usually has no NAT.** A machine behind a household gateway can hold a *globally routable*
v6 address while its v4 address is private and unforwardable. The same machine, at the same
moment, is in **two different positions at once**:
| | IPv4 | IPv6 |
|---|---|---|
| a typical machine at home | behind an unforwardable-or-forwardable gateway | **attached**, directly reachable |
| a machine on mobile data | behind carrier-grade NAT | often attached, sometimes absent entirely |
| a machine on an older network | attached or behind NAT | **no address at all** |
So the three positions apply **per family**, and a machine's reachability is a property of
*(machine, family)* rather than of the machine. A mesh that treats reachability as one fact per
node will reach a peer over one family, fail over the other, and report whichever it tried.
That has a direct consequence for what the lab is for: *"can these two nodes reach each other"*
stops being a yes/no question. It is asked once per family, and the interesting answers are the
asymmetric ones.
The v6 documentation prefix is `2001:db8::/32` (RFC 3849) — the exact counterpart of the RFC
5737 rule, and load-bearing for the same reason.
## The shape
```yaml
@@ -78,39 +104,52 @@ scenario: roaming-and-published
segments:
internet:
kind: public # stands in for the internet — RFC 5737 addresses
cidr: 203.0.113.0/24
kind: public # RFC 5737 for v4, RFC 3849 for v6
cidr: [203.0.113.0/24, 2001:db8::/32]
home:
kind: private
cidr: 192.168.1.0/24
cidr: [192.168.1.0/24, 2001:db8:1::/64]
mtu: 1500
gateway:
to: internet
address: 203.0.113.50 # what the world sees this network as
nat: true
forwardable: true # we control it, so ports can be opened
address: 203.0.113.50 # what the world sees this network as, on v4
nat: [v4] # v4 is translated; v6 is routed, not translated
forwardable: true
mapping_ttl: 120s # an unused inbound mapping is forgotten after this
elsewhere: # a network we do not control
kind: private
cidr: 198.51.100.0/24
cidr: [198.51.100.0/24] # v4 only — no v6 offered here at all
mtu: 1400 # a tunnelled path, smaller than standard
gateway:
to: internet
address: 203.0.113.80
nat: true
nat: [v4]
forwardable: false # café wifi, or carrier-grade NAT
mapping_ttl: 30s # aggressive, as carrier NAT tends to be
machines:
anchor:
at: { segment: internet, address: 203.0.113.10 }
at: { segment: internet, address: [203.0.113.10, 2001:db8::10] }
home-server:
at: { segment: home, address: 192.168.1.135 }
at: { segment: home, address: [192.168.1.135, 2001:db8:1::135] }
published:
- { port: 443, on: home } # DNAT: 203.0.113.50:443 → 192.168.1.135:443
- { port: 443, on: home } # v4 only: 203.0.113.50:443 → 192.168.1.135:443
inbound: allow # v6 is routable here, so this decides whether it is reachable
workstation:
at: { segment: home, address: 192.168.1.250 }
at: { segment: home, address: [192.168.1.250, 2001:db8:1::250] }
inbound: deny # a host firewall: dials out, accepts nothing
laptop:
at: { segment: home, address: 192.168.1.98 }
at: { segment: home, address: [192.168.1.98, 2001:db8:1::98] }
border: # a machine on two segments at once
at:
- { segment: home, address: [192.168.1.2] }
- { segment: internet, address: [203.0.113.60] }
place:
all: [host]
@@ -138,20 +177,40 @@ too thin. It carries three facts, and all three are load-bearing:
connection this is the public address the ISP hands out. It is not decoration: it is what a
peer records as the endpoint when a machine here dials out, and what a public name for a
published machine here resolves to.
- `nat:` — whether addresses are translated. `true` gives the ordinary household case: many
private machines behind one public address. `false` describes a routed range, where machines
keep their own addresses and the gateway only forwards.
- `nat:` — **which families are translated**, as a list. `[v4]` is the ordinary modern case:
v4 translated, v6 routed. `[v4, v6]` describes a gateway that translates both, which exists
and is worth being able to reproduce. `[]` is a routed range, where machines keep their own
addresses and the gateway only forwards.
- `forwardable:` — whether an inbound mapping can be created. Independent of `nat:`, and the
field that separates a home gateway from carrier-grade NAT. Publishing through a gateway with
`forwardable: false` is a declaration error, because that is exactly the constraint being
reproduced.
- `mapping_ttl:` — how long an unused inbound mapping survives. This is what makes keepalive
behaviour testable: a mesh that holds a connection through NAT without refreshing it works
perfectly until the far side goes quiet for longer than this. Aggressive values reproduce
carrier NAT; omitting it means mappings never expire, which no real gateway does.
**`segments[].mtu`** — the largest packet the segment carries, defaulting to 1500. Lower values
reproduce tunnelled and PPPoE paths. This matters because an overlay adds its own header: a
tunnel over a 1400-byte path establishes a connection and then silently drops large packets,
which is the shape of fault this whole effort exists to stop shipping.
**`machines[].inbound`** — `allow` or `deny`, a host firewall. Distinct from NAT and behaves
differently: a machine can be perfectly routable and still refuse everything unsolicited, which
is the normal state of a v6-addressed machine. Without this, v6 addressing would imply
reachability, and it does not.
The lab materialises a machine to be the gateway. That is the one implicit machine in an
otherwise explicit declaration, and it exists because NAT has to run somewhere.
**`machines[].at`** — segment and address. That pair alone determines which of the three
positions a machine is in: on a gateway-less segment it is directly attached; on a segment with
a gateway it is behind one.
**`machines[].at`** — segment and addresses, or a **list** of them for a machine on several
segments at once. Multi-homing is not exotic: it is what a border machine is, and what any node
with both a LAN and a WAN interface is. Each entry carries the addresses that machine holds on
that segment, one per family.
Position follows from the pair, per family: on a `kind: public` segment a machine is attached;
on a private one it is behind that segment's gateway, unless the gateway does not translate
that family — in which case it is attached on that family and behind a gateway on the other.
**`machines[].published`** — a destination-NAT rule on a named gateway, stated as an outcome
rather than a port list. `{ port: 443, on: home }` means the `home` gateway forwards its own
@@ -252,57 +311,57 @@ not first.
## Is this general? — the axes a setup can vary along
The question that matters is not *does this cover our mesh*, but **can it express any mesh**.
Audited against the axes a real deployment varies along, the answer is *most, deliberately not
all, and three genuine gaps*.
The standard applied is not "every property a network has". It is **every property that changes
how the mesh behaves**. Bandwidth does not change correctness; MTU does.
| Axis | Values | Expressible | |
|---|---|---|---|
| **Reachability** | attached · forwardable gateway · unforwardable gateway · isolated | yes | the core of the model |
| **Address stability** | static · dynamic · changes mid-run | **partly** | a machine can be *moved*, but an address that changes under it cannot be stated |
| **Gateway depth** | direct · one gateway · nested gateways | **partly** | `to:` chains, so nesting exists; `published:` names one gateway, so forwarding through two does not |
| **Address family** | IPv4 · IPv6 · dual-stack | **no** | `cidr:` is implicitly v4. A v6-only node is a real topology and cannot be written |
| **Interfaces per machine** | one · several | **no** | `at:` is singular. A multi-homed node — on a LAN and a WAN at once — is inexpressible |
| **Path properties** | MTU · latency · loss | **no** | MTU matters: tunnels fragment, and a lower-MTU path is a classic silent failure |
| **Reachability policy** | symmetric · asymmetric | **no** | a firewall dropping inbound while outbound works is different from NAT and behaves differently |
| **Gateway state** | permanent · expiring mappings | **no** | mappings time out; whether keepalives work is untestable without it |
| **Overlapping ranges** | distinct · two sites both on `192.168.1.0/24` | yes | two segments may carry the same range — common, and it breaks routing |
| **Segment count** | one · many · isolated island | yes | after the `kind:` fix above |
| **Reachability** | attached · forwardable gateway · unforwardable gateway · isolated | yes | the core of the model, and **per family** |
| **Address family** | IPv4 · IPv6 · dual-stack · neither | yes | `cidr:` and `address:` take both; `nat:` names which families are translated |
| **Interfaces per machine** | one · several | yes | `at:` takes a list |
| **Reachability policy** | symmetric · asymmetric | yes | `inbound:` — a routable machine that refuses everything |
| **Gateway state** | permanent · expiring mappings | yes | `mapping_ttl:` |
| **Path MTU** | standard · reduced | yes | `segments[].mtu` |
| **Overlapping ranges** | distinct · two sites both on `192.168.1.0/24` | yes | segments may carry the same range |
| **Segment count** | one · many · isolated island | yes | `kind:` distinguishes an island from the internet |
| **Gateway depth** | direct · one gateway · nested | **partly** | `to:` chains, so nesting exists; `published:` names one gateway, so forwarding through two does not |
| **Address stability** | static · dynamic · changes mid-run | **partly** | a machine can be *moved*; an address changing under it in place cannot be stated |
| **Path quality** | latency · loss · bandwidth | **no**, deliberately | changes performance, not correctness — modelling it makes a network simulator, not a fixture |
### What this says
### What closing the gaps changed
**Three gaps are real and should be closed**, in this order:
**Address family was not a field.** It changed the position model: a machine behind a household
gateway is typically *unforwardable on v4 and directly attached on v6, simultaneously*. So
reachability is a property of *(machine, family)*, and *"can these two nodes reach each other"*
is no longer a yes/no question — it is asked once per family, and the asymmetric answers are the
interesting ones. That distinction did not exist in the model an hour ago and would have been
discovered by a mesh failing over one family while reporting the other.
1. **Address family.** A v6-only or dual-stack node is not exotic, and a mesh that assumes v4
fails there completely rather than partially. This is the largest gap.
2. **Expiring NAT mappings.** Without it, keepalive behaviour is hoped for rather than tested —
and for a mesh where most nodes sit behind NAT, that is the failure mode most likely to
appear only after everything has been idle overnight.
3. **MTU.** Tunnels fragment. A path with a smaller MTU produces a connection that establishes
and then silently drops large packets, which is exactly the shape of fault this whole effort
exists to stop shipping.
**`inbound:` became necessary because of v6.** With NAT, unreachability was implied by the
topology. With a globally routable v6 address, a machine is reachable unless something refuses —
so refusing has to be sayable, or v6 addressing would silently imply reachability.
**Two are deliberately out of scope** unless something argues otherwise: latency and loss.
They change performance, not correctness, and a scenario that models them is a network
simulator rather than a fixture.
**`nat:` became a list rather than a boolean** for the same reason: a real gateway translates v4
and routes v6, and a boolean cannot say that.
**Two are partial and probably fine for now:** nested forwarding and mid-run address change.
Both are expressible with small extensions when something needs them, and neither blocks the
bootstrap scenario.
### What remains open, and whether it matters
Two partial axes, both extensible when something needs them, neither blocking: **nested
forwarding** and **an address changing in place**. A machine can already be moved, which covers
the roaming case; what is missing is a lease expiring underneath a machine that stays put.
One deliberate exclusion: **path quality**. Latency and loss change how fast the mesh is, not
whether it is correct. If a timeout turns out to be load-bearing that judgement should be
revisited — and it would be revisited by a real failure, which is the right trigger.
### The honest summary
The model covers **where a machine sits**, which is what the mesh's reachability logic turns
on, and it now covers it completely. It does not yet cover **what the path between machines is
like**, and one of those — address family — is not a refinement but a second world the mesh
would have to work in.
The model now covers **where a machine sits** and **what the path between machines is like**,
across both address families, which together are what the mesh's reachability logic turns on.
None of this blocks phase 0. A bootstrap scenario is one machine and a pinned bundle, and needs
none of it. But the gaps should be closed before the lab is trusted to say a mesh *works*,
because today it could only say it works over IPv4, on an unconstrained path, against gateways
that never forget.
What it does not model is *change over time* beyond moving a machine, and *degradation* short of
failure. Both are absences chosen rather than overlooked.
## Open
@@ -317,5 +376,6 @@ that never forget.
writes addresses absolutely. Whether a scenario carries literal addresses or a template the
lab allocates from decides whether two can run side by side — and there are only three
documentation ranges to go round.
- **The three gaps from the audit above** — address family, expiring NAT mappings, MTU — in
that order. The first is the one that is a second world rather than a refinement.
- **Nested forwarding** — `published:` names one gateway, so a machine behind two cannot be
published through both.
- **An address changing in place**, as a DHCP lease expiring under a machine that has not moved.