From 974985b3d1131f49aedbf36e81fe9257483074e2 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 29 Aug 2026 18:03:39 +0200 Subject: [PATCH] Four things the lab found about the private network All on the first three machines to actually run it, and all invisible from the mesh's own state: the graph was right, the files were right, the services were up, every node reported success, and the network did not work. A running interface does not re-read its configuration, so a node joining left every existing node carrying a network that no longer existed. A hub sharing a site with a spoke was emitted twice, which WireGuard refuses. Two nodes at one site that neither can be dialled were peered directly, so nobody opened the path and the more specific route blackholed -- this document's own warning arriving in its implementation. And Docker sets the FORWARD policy to DROP, so a hub with forwarding enabled still carried nothing between its spokes. The last one is the sharpest: the substrate at tier 1 silently breaks the network at tier 2, and nothing in either tier's state says so. None of these is reachable by reasoning, and each was found within minutes of a real machine trying it. That is the argument for the lab in one line. --- 03-DESIGN/01-to-be/08-connectivity.md | 29 +++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index 385cd44..81e1084 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -129,6 +129,35 @@ two paths would mean one of them silently swallowing traffic. **What the host receives:** an interface configuration and a peer list, as files. It does not compute them, and after this it holds no credential to the mesh's database. +### Four things the lab found, none of them visible from the mesh's own state + +*Written 2026-08-29, on the first three machines to actually run this.* + +Each looked like a working network from every angle the mesh can see: the graph was right, the +files were right, the services were up, and every node reported success. + +- **A running interface does not re-read its configuration.** A node joins, every existing node's + peer list changes, each file is replaced — and the service is already running, so nothing + reloads it. Every existing node keeps a network that no longer exists. The declaration has to + say the service must *reflect* the file, which is declared state; a command to restart would be + an action, and the link may not carry one + ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). +- **A hub that shares a site with a spoke was emitted twice** — once as a direct peer and once as + the route of last resort. WireGuard takes one entry per public key, so the interface refuses the + file. The ordinary shape of a small mesh, and in none of the tests written before it ran. +- **Two nodes at one site that neither can be dialled must not peer directly.** Nobody opens the + path, and the direct route is more specific than the hub's, so it wins and blackholes. This + document's own warning, arriving in its implementation: *a more specific route to a dead + endpoint blackholes; it does not fall back to the general one.* +- **The container runtime closes the door the overlay needs.** Docker sets the FORWARD policy to + DROP, so a hub with `ip_forward` enabled still carries nothing between its spokes. The substrate + at tier 1 silently breaks the network at tier 2, and nothing in either tier's state says so. The + hub inserts its own rule above those chains and removes it on the way down. + +**The pattern in all four:** the mesh's picture of the network was correct and the network did not +work. That is the argument for the lab in one line — none of these is reachable by reasoning, and +each was found within minutes of a real machine trying it. + ## 2 — Resolution **Two name spaces, and they do not mix:**