ADR 0137: a machine says which networks it routes, and issue 137 is how that was found

2026-09-28 21:31:01 +02:00
parent b7bf601ca4
commit 05039c4f10
3 changed files with 191 additions and 0 deletions
@@ -0,0 +1,116 @@
+---
+topic: what runs on it
+status: accepted
+date: 2026-09-28
+deciders: jochen
+extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
+reconstructed: false
+---
+
+# 137. A machine says which networks it routes
+
+## Context
+
+The filter the mesh derives denies forwarding by default, because without a forward chain it says
+nothing about a container's published port
+([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md)).
+To keep a machine's own containers working it then allows two ranges: the container runtime's
+default bridge pool, and the pool its compose files are given. Those two are named in the
+controller's code, with a comment saying what the gap is:
+
+> A machine whose runtime is configured with something else needs this to say so — which is a thing
+> the mesh cannot derive and a reason this list is named here rather than computed.
+
+**There was no way to say so.** The list was a constant. A machine whose guests live anywhere else
+was filtered by a rule that looked deliberate and was a guess.
+
+**Measured, on the day a workstation was converged.** Flipping it cut egress for five of its
+container networks at once, and for every network its test beds create — the beds allocate a fresh
+range per run, from a pool neither default covers. Nothing reported a fault. The containers could
+not reach anything, the machine went on reporting that it had applied what it was told, and the
+converge preview had said nothing about it either, because the preview lists what *listens* and
+routing is not a listener.
+
+**And two questions, not one.** A guest also asks its host for an address and for names. Both arrive
+at the input chain, where nothing declared them, so denying by default left the guests of a routed
+network with no address and no resolution — which is not a closed port but a network that does not
+function, asked for by this machine's own guest.
+
+**Why the machine cannot simply be read.** A test bed creates its bridge while it runs, between one
+declaration and the next, so a filter derived from what the machine last reported would be correct
+only for the networks that already existed when it was composed. A declared range covers the ones
+that do not exist yet.
+
+## Decision
+
+**A machine says which networks it routes for what it hosts, and the filter forwards them.** A
+node-level fact, beside the node's public domain
+([ADR 0066](0066-public-routing-is-name-agnostic.md)) and for the same reason: the
+machine routes them, and the module that loads the filter holds a seat and may be replaced.
+
+**Added to the runtime's defaults, never replacing them.** A machine that names one range has not
+stopped hosting whatever was already on the runtime's own pools, and replacing would trade one
+silent breakage for another.
+
+**Their guests keep address and name service.** For a network that was named, the input chain admits
+that network's own DHCP and DNS, and nothing else: everything else a guest might want from its host
+is a port somebody declares, like every other port on this machine.
+
+**Said in CIDR form and checked when it is said.** An entry that does not parse is a line nftables
+refuses, and a refused ruleset is a machine filtering nothing while its unit reports a fault — so
+the refusal happens where a person can read it, not on the machine.
+
+**A machine that says nothing is filtered exactly as before.** Every machine already converged is
+untouched by this.
+
+## Options considered
+
+1. **Leave it constant and edit the code per installation.** Rejected: the value is a property of
+   one machine, the code is the whole mesh's, and the two ranges as they stand describe a machine
+   whose runtime was left at its defaults. It is also how this got here.
+2. **Derive it from what the machine reports.** Rejected as insufficient, not as wrong: it cannot
+   cover a network created between two declarations, which is precisely the case that was broken. It
+   would also make the filter follow whatever appeared on the machine, which is a firewall that
+   widens itself.
+3. **A per-node setting on the module that loads the filter.** Rejected: the machine routes the
+   networks. The filter module holds a node-scoped seat and is meant to be replaceable, and a
+   replacement must not lose the machine's own truth.
+4. **Replace the defaults with what is said.** Rejected: see the decision. The first machine to name
+   its bed range would lose its containers.
+5. **Admit all input from a routed network, not only address and name service.** Rejected: that is
+   every port on the machine open to anything it hosts, which is the derivation abandoned.
+
+## Consequences
+
+**The converge preview says what a machine routes**, including when it routes nothing but the
+defaults, with the command that changes it. The preview's own sentence about traffic it cannot
+preview stays, because a tunnel and the found firewall's NAT are still not previewable.
+
+**A machine whose guests are already broken by an earlier flip is fixed by saying its networks and
+pushing**, with no flip to undo.
+
+**The list is one more thing that can be wrong and stale.** A range removed from the machine and
+left here keeps forwarding for a network that no longer exists, which admits nothing, because there
+is no guest on it to admit. That is the safe direction of being out of date.
+
+## How this is checked
+
+- **What a machine says it routes is forwarded, and its guests keep address and name service.** A
+  test renders a ruleset for a machine that names one range and asserts both chains, per chain body
+  so a line in the wrong chain cannot pass it. It fails against the previous behaviour, which is how
+  it was written.
+- **The runtime's own defaults survive naming a range.** Asserted in the same test.
+- **A machine that names nothing renders byte-identically to one that names nil**, so every machine
+  already behind this filter is untouched.
+- **Each family is matched in its own syntax.** A test with one v4 and one v6 network asserts
+  `ip saddr` and `ip6 saddr`, because one set holding both is a syntax error and a ruleset that does
+  not load is a machine filtering nothing.
+- **An entry that is not a network is refused where it is said**, by the parse in the setter.
+
+## References
+
+- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) — the derived filter this completes
+- [ADR 0066](0066-public-routing-is-name-agnostic.md) — the precedent for a node-level fact
+- [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md) — why there is a forward chain at all
+- [issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md) — the measurement that produced this
+- mesh-controller `internal/catalogue/filtering.go` — the constant whose own comment named this gap
@@ -217,6 +217,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0133** — [A module owns its migrations, and the mesh owns when they run](0133-a-module-owns-its-migrations-and-the-mesh-owns-when-they-run.md)  *(superseded)*
 - **0135** — [A module version prepares its state before it runs](0135-a-module-version-prepares-its-state-before-it-runs.md)
 - **0136** — [A step gates its module, not the machine](0136-a-step-gates-its-module-not-the-machine.md)
+- **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md)

 ### How it is built

@@ -0,0 +1,74 @@
+---
+status: resolved
+opened: 2026-09-28
+located-in: [mesh-controller internal/catalogue]
+fixed-by: mesh-controller — a machine says which networks it routes and the derived filter forwards them, their guests keeping address and name service ([ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md)).
+amended-design:
+---
+
+# 137 — Converging a machine cut off its own guests, and nothing said so
+
+## What was observed
+
+A workstation was flipped from adopted to converged, so the mesh's derived filter replaced what was
+there. The flip reported success, the machine reported that it had applied its declaration, and every
+surface of the mesh read green.
+
+A container on one of that machine's networks could no longer reach anything:
+
+```
+192.168.64.2/20
+OUTBOUND BLOCKED
+```
+
+Five of the machine's container networks were affected, and every network its test beds create. The
+reason is in the filter's forward chain, which denies by default and then allows two ranges:
+
+```
+ip saddr 172.16.0.0/12 accept   # the container runtime's bridge networks
+ip saddr 192.168.128.0/17 accept # the networks its compose files are given
+```
+
+Those two are constants in the controller. The machine's guests were allocated from neither: its
+compose networks from other parts of `192.168/16`, and each test bed a fresh `10.x/24`. So the rules
+were correct for a machine whose runtime was left at its defaults, and a guess on this one.
+
+Two further things were closed by the same flip, and for the same reason nobody saw them: a guest asks
+its host for an address over DHCP and for names over DNS, both of which arrive at the input chain,
+where no module had declared them.
+
+## Why it matters beyond this instance
+
+**The preview could not have warned.** It lists what the machine reported as *listening*, and says so
+honestly: it ends with a line that traffic the machine routes is "not previewed". What it did not say
+is that routing was about to be denied by default, or which ranges would survive. An operator reading
+a 350-line preview approves what it shows.
+
+**It is the second time today that a constant stood in for something the mesh cannot know.** The
+intrusion-prevention module named a firewall front-end two machines do not have
+([issue 136](../136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md)), and the
+filter names the address ranges one runtime happens to use. Both were true where they were written and
+silently false elsewhere.
+
+**And the code already knew.** The comment above those two lines says a machine configured otherwise
+"needs this to say so — which is a thing the mesh cannot derive and a reason this list is named here
+rather than computed". The gap was documented at the point where it was introduced, and the way to say
+it was never built. A comment naming a missing mechanism is a rule that is not enforced.
+
+## What was done
+
+A machine says which networks it routes; the filter forwards them and admits their guests' address and
+name service. Added to the runtime's defaults rather than replacing them, so a machine that names one
+range keeps the others. Node-level, because the machine routes them and the module that loads the
+filter may be replaced. The converge preview now says what a machine routes, and what it will keep
+forwarding if it says nothing.
+
+## What is still true
+
+**The flip is still the moment a machine's unmanaged services close.** That is what converging means
+and the preview names each one. This issue is not about the ports that were meant to close; it is about
+the ones nothing could name.
+
+**Egress is still not previewed per network.** The preview says which ranges will be forwarded, not
+which of the machine's guests sit inside them. Deriving that would need the machine to report its
+bridges, and a bed's bridge does not exist until the bed runs.