jschoubben
ef5dd0751b
The 'domain grouping' item cited ADR 0017 as live guidance. 0044 superseded it -- there is no domain module to group into, so there is no domain list to settle.
120 lines
6.6 KiB
Markdown
120 lines
6.6 KiB
Markdown
---
|
|
status: accepted
|
|
date: 2026-08-27
|
|
deciders: jochen
|
|
reconstructed: false
|
|
extends: 0031-the-lab-provides-the-underlay.md
|
|
---
|
|
|
|
# 50. Reachability is declared, not inferred from an address
|
|
|
|
## Context
|
|
|
|
The overlay's peer graph is computed on each node by the `wireguard` module, which decides —
|
|
per pair — whether to write an `Endpoint` for a peer. The rule
|
|
([research 004](../01-RESEARCH/004-lab-network/analysis.md)) is a regular expression:
|
|
|
|
```js
|
|
const isPrivate = (a) => /^(10\.|127\.|192\.168\.|172\.(1[6-9]|2\d|3[01])\.)/.test(a);
|
|
```
|
|
|
|
Not private ⇒ assumed reachable ⇒ an `Endpoint` is written. Private ⇒ no endpoint, and the peer
|
|
must initiate.
|
|
|
|
The module's own comments record what this cost to arrive at: testing `profile === "server"`
|
|
was tried and was wrong, because a home-hosted node **is** a server and is not publicly
|
|
reachable — *"role does not imply reachability; the address does."*
|
|
|
|
**That lesson is right and the implementation of it is not.** The address is evidence of
|
|
reachability; it is not the fact itself, and the gap between the two has already caused failures
|
|
and will cause more:
|
|
|
|
| Address | The regex says | Actually |
|
|
|---|---|---|
|
|
| `100.64.0.0/10` — carrier-grade NAT ([RFC 6598](https://www.rfc-editor.org/rfc/rfc6598)) | **public** | **not reachable.** An endpoint is written to an address nothing can reach. |
|
|
| any IPv6 address, including `fd00::/8` unique-local | **public** | unique-local is not reachable; the regex tests v4 shapes only |
|
|
| a routable address behind a closed firewall | public | not reachable |
|
|
| `10.200.0.0/24` standing in for a public segment | private | reachable — this is the lab bug |
|
|
|
|
The CGNAT row is the serious one. A node on a carrier-grade NAT address presents exactly the
|
|
failure already recorded for the hairpin case: *1.77 MiB sent, 0 B received, no handshake.* The
|
|
mesh silently never forms, and it presents as a WireGuard fault rather than an addressing one.
|
|
|
|
The lab row is the same bug seen from the other side. [Research 004](../01-RESEARCH/004-lab-network/00-overview.md)
|
|
calls the required substitution *"the single most important fact in this document"* — a
|
|
simulated public segment must use TEST-NET-3, or nothing can ever initiate. **A test environment
|
|
having to choose its addresses to satisfy a regex is the regex telling us it is not a fact.**
|
|
|
|
There is a second inference in the same code, and it is worse because nothing records it:
|
|
|
|
> **Hub election is by convention.** The hub is the node whose `profile='server'` *and* whose
|
|
> overlay address begins `10.10.0.1`. A lab must assign that address to the node it intends as
|
|
> hub **or there will be no hub** — and nothing says so.
|
|
|
|
An election decided by the first four characters of an address is not an election. It fails
|
|
silently, it cannot be queried, and it makes a renumbering into an outage.
|
|
|
|
## Considered options
|
|
|
|
1. **Fix the regex.** Add CGNAT, add IPv6, add the ranges as they are discovered. Rejected: the
|
|
list is unbounded, and each addition is written after the outage that revealed it. The
|
|
firewall case cannot be fixed at all — no address shape encodes it.
|
|
2. **Probe for reachability and cache the answer.** Attractive, and wrong as the *primary*
|
|
source: at the moment the graph is computed a node may be legitimately down, and a probe
|
|
cannot distinguish *unreachable* from *asleep* ([ADR 0036](0036-a-node-is-a-managed-machine.md)).
|
|
Deriving topology from a liveness check makes the overlay flap with the network.
|
|
3. **Declare it, and let observation contradict it.** Chosen.
|
|
|
|
## Decision
|
|
|
|
**A node's reachability is a declared fact on its record, not an inference from its address.**
|
|
|
|
Two facts, and the mesh stores both:
|
|
|
|
- **an endpoint, or none** — where peers may reach this node, if anywhere. Absent means *this
|
|
node initiates and is never dialled*, which is the safe default and the common case.
|
|
- **its role in the overlay** — whether it is a hub. **Declared, never derived from an address.**
|
|
|
|
**The address remains evidence and stops being the fact.** When a node's observed endpoint
|
|
disagrees with its declared one, that is a **reportable condition**, not a silent correction —
|
|
the same discipline as [ADR 0035](0035-a-picture-is-read-from-what-runs.md): a picture is read
|
|
from the system, and where the reading disagrees with the intent, the disagreement is the
|
|
finding.
|
|
|
|
**The peer graph is computed by the control plane**, from these declared facts, and delivered to
|
|
each node as configuration. It is not computed on the node, which is
|
|
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) and is what removes `wireguard`'s direct
|
|
database connection ([ADR 0049](0049-a-route-is-a-grant.md) does the same for the proxy).
|
|
|
|
**What does not change** is the rule the comment was defending. Role still does not imply
|
|
reachability — a home-hosted node is still a server that cannot be dialled. This record keeps
|
|
that lesson and stops encoding it as a pattern match.
|
|
|
|
## Consequences
|
|
|
|
- **CGNAT and IPv6 nodes become expressible**, which today they are not. Neither needs a code
|
|
change to support; they need a field that says what is true.
|
|
- **The lab stops needing its substitution.** TEST-NET-3 remains the right choice for a
|
|
documentation range, but the scenario now says *this segment is reachable* rather than relying
|
|
on an address shape to imply it. The constraint research 004 calls its most important fact
|
|
becomes an ordinary declaration, and the validator's enforcement of it becomes unnecessary
|
|
rather than load-bearing.
|
|
- **Hub election becomes queryable and renumbering becomes safe.** Both follow from the same
|
|
change and neither is possible today.
|
|
- **Two facts can now disagree, and something must say so.** Declared-versus-observed is a new
|
|
reportable condition and a new way to be wrong — a node declared reachable that is not will
|
|
fail exactly as it does today until somebody looks. The gain is that *looking* is now possible;
|
|
the regex offered nothing to compare against.
|
|
- **Somebody must set these when a node joins.** [ADR 0038](0038-a-node-joins-by-linking-first.md)
|
|
has the mesh finish the job after the link, so this is one more thing it finishes — and the
|
|
honest default (no endpoint, not a hub) is correct for every node except the ones somebody
|
|
deliberately publishes.
|
|
|
|
## References
|
|
|
|
- [Research 004](../01-RESEARCH/004-lab-network/analysis.md) — the regex, the hub convention, and
|
|
the hairpin failure.
|
|
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the graph is computed centrally.
|
|
- [ADR 0035](0035-a-picture-is-read-from-what-runs.md) — declared versus observed.
|
|
- [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) — the design this serves.
|