Compare commits

..
Author SHA1 Message Date
mesh-admin 2126e7b2cb Merge pull request 'Issues 140 and 141: an endpoint's reach, and a forward chain that does not follow the modules' (#169) from issue/140-endpoint-reach-and-141-forward-chain into main 2026-09-28 20:58:03 +00:00
jschoubben dcdfcf104e Issues 140 and 141: an endpoint's reach is declared nowhere, and the forward chain follows constants instead of the modules
Found preparing the control-node's convergence. Reach is settled independently by
the filter, the proxy's names and the certificate authority, so "this must not be
public" cannot be written and a public certificate is obtained regardless. And the
forward chain allows two hardcoded ranges plus a typed list, though the mesh
already knows which networks exist because its own modules declared them — a range
wide enough to keep four of them would have forwarded two predecessor leftovers too.
2026-09-28 22:57:38 +02:00
mesh-admin d23ace1646 Merge pull request 'Issues 138 and 139' (#168) from issue/138-the-uplink-seat-and-139-an-internal-route-name into main 2026-09-28 19:46:22 +00:00
jschoubben 5dbde0b13a Issues 138 and 139: a seat with interchangeable holders that are not, and an internal route name that resolves to the wrong machine 2026-09-28 21:46:20 +02:00
mesh-admin db3868e2b0 Merge pull request 'ADR 0137: a machine says which networks it routes' (#167) from decision/0137-a-machine-says-which-networks-it-routes into main 2026-09-28 19:31:03 +00:00
jschoubben 05039c4f10 ADR 0137: a machine says which networks it routes, and issue 137 is how that was found 2026-09-28 21:31:01 +02:00
mesh-admin b7bf601ca4 Merge pull request 'Issue 136: a module may name a program the machine does not have' (#166) from issue/136-a-module-may-name-a-program-the-machine-lacks into main 2026-09-28 18:50:37 +00:00
jschoubben bff32e3370 Issue 136: a module may name a program the machine does not have, and everything reports success 2026-09-28 20:50:35 +02:00
mesh-admin d6b62387f2 Merge pull request 'Issue 135: a container's mesh names are not compared' (#165) from issue/135-a-containers-mesh-names into main 2026-09-28 15:19:37 +00:00
jschoubben 4789624857 Issue 135: a container's mesh names are not compared, so a moved address is never noticed
One container restarted 2286 times over five days while the mesh reported the machine as doing what it
was told. Its overlay address was five days out of date: the host compares a container by a digest of
its spec, and the mesh's names were not in it, so a container whose image and files never changed was
left alone holding a name that no longer resolved. Forty-eight others were current only because
something else had recreated them.

The same fault as issue 045, in the field that was left out. Resolved by putting the names in the
digest.
2026-09-28 17:19:35 +02:00
mesh-admin e00862e317 Merge pull request 'ADR 0112 is accepted, and issue 134 records what it is not yet' (#164) from decision/0112-accepted-and-issue-134 into main 2026-09-28 15:12:15 +00:00
9 changed files with 608 additions and 0 deletions
@@ -0,0 +1,116 @@
---
topic: what runs on it
status: accepted
date: 2026-09-28
deciders: jochen
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
reconstructed: false
---
# 137. A machine says which networks it routes
## Context
The filter the mesh derives denies forwarding by default, because without a forward chain it says
nothing about a container's published port
([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md)).
To keep a machine's own containers working it then allows two ranges: the container runtime's
default bridge pool, and the pool its compose files are given. Those two are named in the
controller's code, with a comment saying what the gap is:
> A machine whose runtime is configured with something else needs this to say so — which is a thing
> the mesh cannot derive and a reason this list is named here rather than computed.
**There was no way to say so.** The list was a constant. A machine whose guests live anywhere else
was filtered by a rule that looked deliberate and was a guess.
**Measured, on the day a workstation was converged.** Flipping it cut egress for five of its
container networks at once, and for every network its test beds create — the beds allocate a fresh
range per run, from a pool neither default covers. Nothing reported a fault. The containers could
not reach anything, the machine went on reporting that it had applied what it was told, and the
converge preview had said nothing about it either, because the preview lists what *listens* and
routing is not a listener.
**And two questions, not one.** A guest also asks its host for an address and for names. Both arrive
at the input chain, where nothing declared them, so denying by default left the guests of a routed
network with no address and no resolution — which is not a closed port but a network that does not
function, asked for by this machine's own guest.
**Why the machine cannot simply be read.** A test bed creates its bridge while it runs, between one
declaration and the next, so a filter derived from what the machine last reported would be correct
only for the networks that already existed when it was composed. A declared range covers the ones
that do not exist yet.
## Decision
**A machine says which networks it routes for what it hosts, and the filter forwards them.** A
node-level fact, beside the node's public domain
([ADR 0066](0066-public-routing-is-name-agnostic.md)) and for the same reason: the
machine routes them, and the module that loads the filter holds a seat and may be replaced.
**Added to the runtime's defaults, never replacing them.** A machine that names one range has not
stopped hosting whatever was already on the runtime's own pools, and replacing would trade one
silent breakage for another.
**Their guests keep address and name service.** For a network that was named, the input chain admits
that network's own DHCP and DNS, and nothing else: everything else a guest might want from its host
is a port somebody declares, like every other port on this machine.
**Said in CIDR form and checked when it is said.** An entry that does not parse is a line nftables
refuses, and a refused ruleset is a machine filtering nothing while its unit reports a fault — so
the refusal happens where a person can read it, not on the machine.
**A machine that says nothing is filtered exactly as before.** Every machine already converged is
untouched by this.
## Options considered
1. **Leave it constant and edit the code per installation.** Rejected: the value is a property of
one machine, the code is the whole mesh's, and the two ranges as they stand describe a machine
whose runtime was left at its defaults. It is also how this got here.
2. **Derive it from what the machine reports.** Rejected as insufficient, not as wrong: it cannot
cover a network created between two declarations, which is precisely the case that was broken. It
would also make the filter follow whatever appeared on the machine, which is a firewall that
widens itself.
3. **A per-node setting on the module that loads the filter.** Rejected: the machine routes the
networks. The filter module holds a node-scoped seat and is meant to be replaceable, and a
replacement must not lose the machine's own truth.
4. **Replace the defaults with what is said.** Rejected: see the decision. The first machine to name
its bed range would lose its containers.
5. **Admit all input from a routed network, not only address and name service.** Rejected: that is
every port on the machine open to anything it hosts, which is the derivation abandoned.
## Consequences
**The converge preview says what a machine routes**, including when it routes nothing but the
defaults, with the command that changes it. The preview's own sentence about traffic it cannot
preview stays, because a tunnel and the found firewall's NAT are still not previewable.
**A machine whose guests are already broken by an earlier flip is fixed by saying its networks and
pushing**, with no flip to undo.
**The list is one more thing that can be wrong and stale.** A range removed from the machine and
left here keeps forwarding for a network that no longer exists, which admits nothing, because there
is no guest on it to admit. That is the safe direction of being out of date.
## How this is checked
- **What a machine says it routes is forwarded, and its guests keep address and name service.** A
test renders a ruleset for a machine that names one range and asserts both chains, per chain body
so a line in the wrong chain cannot pass it. It fails against the previous behaviour, which is how
it was written.
- **The runtime's own defaults survive naming a range.** Asserted in the same test.
- **A machine that names nothing renders byte-identically to one that names nil**, so every machine
already behind this filter is untouched.
- **Each family is matched in its own syntax.** A test with one v4 and one v6 network asserts
`ip saddr` and `ip6 saddr`, because one set holding both is a syntax error and a ruleset that does
not load is a machine filtering nothing.
- **An entry that is not a network is refused where it is said**, by the parse in the setter.
## References
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) — the derived filter this completes
- [ADR 0066](0066-public-routing-is-name-agnostic.md) — the precedent for a node-level fact
- [issue 047](../04-ISSUES/047-the-firewall-does-not-cover-published-container-ports/00-report.md) — why there is a forward chain at all
- [issue 137](../04-ISSUES/137-converging-a-machine-cut-off-its-own-guests/00-report.md) — the measurement that produced this
- mesh-controller `internal/catalogue/filtering.go` — the constant whose own comment named this gap
+1
View File
@@ -217,6 +217,7 @@ python3 00-META/checks/index.py fail if stale
- **0133** — [A module owns its migrations, and the mesh owns when they run](0133-a-module-owns-its-migrations-and-the-mesh-owns-when-they-run.md) *(superseded)*
- **0135** — [A module version prepares its state before it runs](0135-a-module-version-prepares-its-state-before-it-runs.md)
- **0136** — [A step gates its module, not the machine](0136-a-step-gates-its-module-not-the-machine.md)
- **0137** — [A machine says which networks it routes](0137-a-machine-says-which-networks-it-routes.md)
### How it is built
@@ -0,0 +1,70 @@
---
status: resolved
opened: 2026-09-28
located-in: [mesh-host internal/apply]
fixed-by: mesh-host — a container's mesh names are part of the spec digest the host compares, sorted so the digest does not move for a reordering. A container whose names moved is now recreated like a container whose image moved, and the test fails against the previous behaviour.
amended-design:
---
# 135 — A container's mesh names are not compared, so a moved address is never noticed
## What was observed
One container on this mesh had been restarting every thirty seconds for five days — 2286 times — and
the mesh reported the machine as doing what it was told.
Its logs said its database connected and then a query timed out. The database was reachable: the same
query from the same network, with the same credential, answered in milliseconds. What differed was the
name. Inside that container, `novox.internal` resolved to `10.42.0.1`; in every other container on the
machine it resolved to `10.10.0.1`. The mesh's overlay range had moved, and this container still held
the old one:
```
umami created 2026-09-23 novox.internal:10.42.0.1
mesh-catalog created today novox.internal:10.10.0.1
```
A container resolves other machines and public names through the entries the mesh gives it when it is
created, and nothing re-reads them afterwards. The host compares a container against what was declared
by a digest of its spec — image, name, environment, ports, volumes, arguments, resolver, address, and
what it reads — and **the mesh's names were not in it**. So this container matched what was declared,
was left alone, and kept an address that had not existed for five days.
Forty-eight other containers had current names. Not because anything corrected them: each had been
recreated for some other reason — a new image, a changed file — and picked up the current roster on the
way. This one's image is an upstream release that had not moved, and nothing else about it changed, so
nothing ever recreated it.
## Why it matters beyond this instance
**It is the exact fault [issue 045](../045-a-container-keeps-the-values-it-started-with/00-report.md)
named, in the one field that was left out.** That issue is why the digest carries what a container
reads: "a container whose configuration had since been rewritten compared equal and was left alone —
running values the machine no longer holds, while every check reported success." The same sentence
describes this, with *names* in place of *files*.
**The failure is invisible in exactly the way that matters.** The container runs, so the machine
reports it applied. It restarts, but a restarting container is a normal sight during an upgrade. The
only account of the fault is inside the container's own log, in the words of the application rather
than of the mesh — and what it says is that a query timed out, which points at the database.
**And it is most likely to bite what changes least.** Every container that is rebuilt often repairs
itself by accident. The victim is the module whose image is stable — which is to say, the module that
was working fine.
## What was done
The mesh's names are part of the digest, sorted so the digest does not move for a reordering nobody
made. A container whose names moved is now recreated exactly as one whose image moved.
The first apply after this recreates every container that carries mesh names — one restart each,
already the price the mesh pays for any image update — because their recorded digests predate the
field.
## What is still true
The mesh gives a container its names at creation and has no way to change them in place. That is the
container runtime's shape, not a choice; the answer is to recreate, which is what this does. A module
that would rather re-read a roster from a file can already ask for one as a fact
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)) and restart on
it.
@@ -0,0 +1,92 @@
---
status: resolved
opened: 2026-09-28
located-in: [mesh-catalog modules/fail2ban]
fixed-by: mesh-catalog — the intrusion-prevention module bans through an action it ships itself, already in use on every machine, instead of naming a firewall front-end two of them do not have. The instance is closed; the class in "What is still true" is not.
amended-design:
---
# 136 — A module may name a program the machine does not have, and everything reports success
## What was observed
Two machines were given the intrusion-prevention module on 2026-09-28. Both refused to start it:
```
ERROR Failed during configuration: Have not found any log file for 'recidive' jail.
ERROR Async configuration of server failed
fail2ban.service: Main process exited, code=exited, status=255/EXCEPTION
```
The jail that bans whoever keeps coming back reads the service's *own* log, and the service checks
every jail's log file while it configures itself — before it has created that log. The module
declared the jail and shipped the rotation for that log, and never declared the log. On the two
machines where it had run for years the file was simply there, so nothing had ever noticed.
That failure was loud. Fixing it uncovered a second one in the same module that is not.
The module's defaults named `ufw` as the way to ban an address. Two of these four machines have no
`ufw` — they filter with nftables — and nothing checks that until an address is banned. Asked to ban
a documentation address on such a machine, the service accepted the instruction, counted it, ran the
command, and wrote this to a log nobody reads:
```
ERROR ... -- stderr: '/bin/sh: line 5: ufw: command not found'
ERROR ... -- returned 127
ERROR Failed to execute ban jail 'sshd' action 'ufw' ... Error banning 192.0.2.99
```
No rule existed afterwards. Throughout, the unit was `active`, the module was applied, and the
machine's report said so. **A machine had been added to the mesh's intrusion prevention, reported as
protected, and was banning nobody.**
## Why it matters beyond this instance
**The two faults are the same mistake with opposite symptoms.** Both are the module assuming
something about the machine — a file that happens to exist, a program that happens to be installed.
One stopped the service, which anybody notices. The other left it running and empty, which nobody
does. A mesh that only catches the loud one is a mesh whose coverage is unknown.
**"The unit is running" was taken for "the module is doing its job".** That is the only health a
service resource has. It is the right answer for most modules and it is silent for any module whose
work happens later, on an event — a ban, a renewal, a backup, a notification. The report cannot
distinguish "protecting this machine" from "installed and inert".
**And it is exactly the naming rule, one level down.**
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) says a
definition names no node, no mesh and no host path, because the same definition has to raise a
different mesh. `ufw` is not a node name, but it is the same class of assumption: a value the module
cannot know, true on some machines and false on others, written as though it were a constant. The
module already knew how to do better a few lines away — the mesh's own address range is named there
as something the machine fills in.
## What was done
The module declares the log its own jail reads, created once and never touched again, since what
grows in it is the service's and the rotation the module already ships is what keeps it small. And
it bans through the action it ships itself, which every machine here can run, which was already in
use by the other jail on all four, and which covers a container's published port as well as the
host's own.
All four machines now run it, with both jails, and a ban lands on each — verified by banning and
unbanning a documentation address on every one.
## What is still true
**Nothing would have caught either fault before it shipped.** The control plane reads a manifest, not
a machine; `ufw` and `/var/log/…` are strings in a file it has no way to evaluate. The host could in
principle be asked whether a declared program exists, but no resource says "this file names a command
that must be there", so there is nothing to check.
**Two machines' bans from before this are stale rules in the old front-end**, which the service no
longer knows about and will never lift. They reject two addresses for ever. Harmless, and a reminder
that changing how a module enforces something leaves what it already enforced behind.
## Open questions
- What does a service resource's health mean for a module whose work is event-driven? A unit being
active is the weakest claim available, and four of this mesh's modules are of that kind.
- Should a declaration be able to say that a resource depends on a program, so the machine can refuse
what it cannot carry out rather than reporting success?
- Where should the packet filter a module bans through come from — the module's own choice, as now,
or the seat that owns the machine's filtering?
@@ -0,0 +1,74 @@
---
status: resolved
opened: 2026-09-28
located-in: [mesh-controller internal/catalogue]
fixed-by: mesh-controller — a machine says which networks it routes and the derived filter forwards them, their guests keeping address and name service ([ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md)).
amended-design:
---
# 137 — Converging a machine cut off its own guests, and nothing said so
## What was observed
A workstation was flipped from adopted to converged, so the mesh's derived filter replaced what was
there. The flip reported success, the machine reported that it had applied its declaration, and every
surface of the mesh read green.
A container on one of that machine's networks could no longer reach anything:
```
192.168.64.2/20
OUTBOUND BLOCKED
```
Five of the machine's container networks were affected, and every network its test beds create. The
reason is in the filter's forward chain, which denies by default and then allows two ranges:
```
ip saddr 172.16.0.0/12 accept # the container runtime's bridge networks
ip saddr 192.168.128.0/17 accept # the networks its compose files are given
```
Those two are constants in the controller. The machine's guests were allocated from neither: its
compose networks from other parts of `192.168/16`, and each test bed a fresh `10.x/24`. So the rules
were correct for a machine whose runtime was left at its defaults, and a guess on this one.
Two further things were closed by the same flip, and for the same reason nobody saw them: a guest asks
its host for an address over DHCP and for names over DNS, both of which arrive at the input chain,
where no module had declared them.
## Why it matters beyond this instance
**The preview could not have warned.** It lists what the machine reported as *listening*, and says so
honestly: it ends with a line that traffic the machine routes is "not previewed". What it did not say
is that routing was about to be denied by default, or which ranges would survive. An operator reading
a 350-line preview approves what it shows.
**It is the second time today that a constant stood in for something the mesh cannot know.** The
intrusion-prevention module named a firewall front-end two machines do not have
([issue 136](../136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md)), and the
filter names the address ranges one runtime happens to use. Both were true where they were written and
silently false elsewhere.
**And the code already knew.** The comment above those two lines says a machine configured otherwise
"needs this to say so — which is a thing the mesh cannot derive and a reason this list is named here
rather than computed". The gap was documented at the point where it was introduced, and the way to say
it was never built. A comment naming a missing mechanism is a rule that is not enforced.
## What was done
A machine says which networks it routes; the filter forwards them and admits their guests' address and
name service. Added to the runtime's defaults rather than replacing them, so a machine that names one
range keeps the others. Node-level, because the machine routes them and the module that loads the
filter may be replaced. The converge preview now says what a machine routes, and what it will keep
forwarding if it says nothing.
## What is still true
**The flip is still the moment a machine's unmanaged services close.** That is what converging means
and the preview names each one. This issue is not about the ports that were meant to close; it is about
the ones nothing could name.
**Egress is still not previewed per network.** The preview says which ranges will be forwarded, not
which of the machine's guests sit inside them. Deriving that would need the machine to report its
bridges, and a bed's bridge does not exist until the bed runs.
@@ -0,0 +1,56 @@
---
status: open
opened: 2026-09-28
located-in: [mesh-controller internal/catalogue, mesh-catalog]
fixed-by:
amended-design:
---
# 138 — Two modules claim one seat and are not interchangeable, and nothing says so
## What was observed
Three modules claim the node-scoped uplink seat: one for each network manager a machine here might
run. [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) gives each of them the same
job — ask the manager the machine already runs to leave the resolver file alone and to leave the mesh's
interface alone — and deliberately keeps the machine's own links out of the mesh's hands.
A seat means one holder and an interchangeable holder. These are interchangeable in what they *ask*
and not in what they *do*:
- None installs, enables, starts or stops the manager. That is on purpose: stopping it takes every
link down, including the mesh's own way in.
- None carries an address, a route or a wireless credential, for the same reason.
- **Nothing checks that the module holding the seat names the manager the machine is actually
running.** Assigning the systemd-networkd holder to a machine running NetworkManager writes a file
for a daemon that is inactive and disabled, the seat reports held, and the two things the seat
exists to arrange are arranged for nobody. NetworkManager goes back to rewriting the resolver file
on every lease, which is the failure the module's own comment describes.
The machine reports which service manager and which units are active, so the fact needed to catch this
is already in the report the mesh holds.
## Why it matters beyond this instance
**A seat is the mesh's promise that a role is filled.** If the holder can be a module for software
that is not running, the seat says a role is filled while nothing fills it — and the surface that
would tell an operator says "held".
**It is the same shape as two faults found the same day.** A module named a firewall front-end the
machine does not have ([issue 136](../136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md)),
and the filter named address ranges one runtime happens to use
([issue 137](../137-converging-a-machine-cut-off-its-own-guests/00-report.md)). Each is a claim about
the machine that nothing on the machine checks.
**And it decides whether the seat is worth having.** Either the holder must match what the machine
runs, which is a condition the mesh can check from the report it already has, or the holders must be
able to switch the manager, which ADR 0117 refuses for a reason that has not changed.
## Open questions
- Should a seat's conditions of holding include a capability the machine reports, so a holder naming
absent or inactive software is refused rather than recorded?
- Is "the uplink" one seat at all, if its holders are three dialects of the same two requests? The
alternative is one module that speaks whichever dialect the machine needs, chosen from the report.
- What should happen on a machine that switches manager afterwards? The seat would then be held by the
wrong module, and the machine is the only place that knows.
@@ -0,0 +1,51 @@
---
status: open
opened: 2026-09-28
located-in: [mesh-controller internal/catalogue]
fixed-by:
amended-design:
---
# 139 — An internal route name resolves to the consumer's node, not the one that serves it
## What was observed
A module that requires a route is given two names: a public one composed under the serving node's
domain, and an internal one composed under the consumer's own machine — `<label>.<node>.internal`.
The two are published differently:
- The **public** name is written into every machine's hosts file at the address of the node whose
proxy answers it. The mesh computes that deliberately, so any container resolving a routed name
reaches the proxy.
- The **internal** name is resolved by the machine's own resolver, which answers every name under
`<node>.internal` with that node's address — the consumer's, because the name was composed from it.
Where the proxy runs beside the consumer these are the same machine, which is every case on this mesh
today, and both names work. Measured on 2026-09-28: the internal name of a service on the control node
answers with a certificate from the mesh's internal authority, and the public name with one from the
public authority.
Where the proxy is on another machine they disagree. The internal name sends the client to a machine
that runs no proxy and has nothing listening on the port, while the public name sends it to the one
that does.
## Why it matters beyond this instance
**It is latent exactly where the mesh is heading.** `route` is provided mesh-wide precisely so a
module can be routed by a proxy on another machine. The first module assigned that way gets an
internal name that does not work, and the public one that does — with no error anywhere, because both
names resolve.
**A per-machine name is what an operator will reach for.** `<service>.<machine>.internal` reads like a
promise that the service on that machine is reachable there, and the wildcard makes every such name
resolve whether or not anything answers.
## Open questions
- Should the internal name be composed under the serving node, like the public one, or should it stay
the consumer's and be published at the serving node's address like the public name is?
- Is a per-node route holder the real answer — a proxy on every machine that serves its own names —
and if so, is `route` still one mesh-wide provision or a node-scoped seat with a mesh-wide fallback?
- What certifies the name in either case? The certificate is obtained by whoever terminates TLS, and
that is the question above in another form.
@@ -0,0 +1,74 @@
---
status: open
opened: 2026-09-28
located-in: []
fixed-by:
amended-design:
---
# 140 — An endpoint's reach is not declared, so three mechanisms each decide it separately
## What was observed
Preparing to converge the mesh's control-node — the last machine still running the firewall it
had before the mesh — the question came up for one module: the forge serves git over ssh, and that
port must stay reachable from outside the private network. Where is that said?
The manifest declares the port with a source of `mesh`, so the derived filter would close it to
everything but the private network. Looking for the place an assignment says otherwise, there are
two per-node settings keys: one that gives a module's declared port a machine port, and one that
overrides a declared port's source. The second has exactly one caller — the function that builds
the node's filter rules. Nothing else in the control plane reads it.
A module's routed endpoint is declared somewhere else entirely: a route contribution naming a label
and a port. It says nothing about reach. The proxy composes a **public** name and an **internal**
name for every route it is given, and obtains a certificate for each from a different authority.
Measured on that machine the same day: an identity provider's public name signed by the public
authority for 90 days, its internal name signed by the mesh's own intermediate for 24 hours and
renewed daily. Both names exist, and both certificates, because the proxy makes every name it can.
No assignment asked for either.
So the forge's ssh endpoint has a firewall source and nothing else — no name, no certificate, and no
way to say it should be public other than a key the filter alone reads. And the forge's web endpoint
has two names and two certificates that nobody requested.
## Why it matters beyond this instance
**Reach is stated twice, in two vocabularies, in two places that cannot disagree out loud.** A port
may be exposed to anywhere while the module contributes no public route; a public route may be served
for a module whose own listen is private. Nothing reconciles the pair or refuses it. Each mechanism
is separately defensible and the combination is unstated.
**The vocabulary belongs to the filter, not to reachability.** *Public, internal, or both* cannot be
expressed. A source of `anywhere` is one rule on one chain; it says nothing about which names should
exist or which authority should sign them. So "this endpoint must not be public" has no way to be
written, and is therefore enforced by nothing — while a public certificate for that very name is
obtained automatically.
**An endpoint is not a thing in the model.** A module has ports, and separately it has routes.
Nothing binds a port to a name to a certificate, which is why three mechanisms each decide reach on
their own and none of them is wrong. This is
[ADR 0045](../../02-DECISIONS/0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md)'s fault
one level up: that record closed "a declaration that reads as a restriction and restricts nothing"
for the packet filter. Here the declaration is absent altogether and the mechanisms guess.
**It blocks the certificate work.** The open question recorded for certificates — a name that must
not be public needs either DNS-01 or the internal authority only — cannot be answered while no
assignment states whether a name should be public. Neither can expiry reporting, revocation, or what
happens to a name when a machine leaves: all of them need to know which names were *meant*.
## Open questions
- Should an assignment name each of a module's endpoints, bind it to a node-level port, and state
whether it is reachable publicly, internally or both — with the filter, the proxy's names and the
certificate authority all derived from that one statement?
- What is an endpoint that is neither routed nor certified? Git over ssh is public reach with no name
and no certificate; the model has to hold that without inventing one.
- Are the two existing settings keys the same statement, half-built? If so, is this a new declaration
or the completion of theirs?
- Does an internal-only endpoint get a certificate at all, and from which authority — and does that
settle [issue 129](../129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md), where nothing
installs the mesh's own root?
- Does declaring reach per assignment also settle
[issue 139](../139-an-internal-route-name-resolves-to-the-consumers-node/00-report.md), where an
internal name resolves to the consumer's machine instead of the one serving the endpoint?
@@ -0,0 +1,74 @@
---
status: open
opened: 2026-09-28
located-in: []
fixed-by:
amended-design:
---
# 141 — The forward chain does not follow the modules, though the modules declare their networks
## What was observed
[ADR 0137](../../02-DECISIONS/0137-a-machine-says-which-networks-it-routes.md), decided the same
week, gave a machine a way to say which networks it routes for its guests, because the derived
filter's forward chain had until then allowed two ranges named as constants in the control plane's
own source — the container runtime's default bridge pool, and half of the pool its compose files
are given.
Checking the last machine still to be converged, the same fault was found to be live there, and the
declaration needed to work around it turned out to be wrong in kind.
That machine hosts twenty-one container networks. Nine fall inside the runtime's bridge pool and
are forwarded. Twelve sit in the other private range, and **six of those fall below the lower bound
of the constant**, so the flip would have cut their guests off exactly as it did on the workstation
that produced 0137.
Naming a range to cover the six was the obvious move, and is what 0137 provides for. But of those
six networks, **four are networks the mesh's own modules declare** — they appear as network
resources in the node's plan, created by the host because a module asked for them — and **two are
leftovers of the predecessor**, compose networks of services the mesh does not run. A range wide
enough to keep the four would have forwarded the two as well: a firewall widened by hand to protect
networks that should not exist.
The mesh already knows which of the twenty-one are its own. It made them.
## Why it matters beyond this instance
**The node's configuration is supposed to follow the modules assigned to it.** That is the mesh's
founding shape — the machine runs modules, and its files, its filter and its accounts are composed
from what runs there. The forward chain is the one derived thing that does not: it consults two
constants and, since 0137, a list a person types. A module added tomorrow brings a network the filter
will not forward; a module deprecated leaves a range in the list that outlives it.
**A typed range cannot distinguish the mesh's networks from what was left behind.** It is stated in
addresses, and addresses are what the runtime allocates, so the only honest declaration is one wide
enough to include whatever else the runtime has handed out. The derivation is narrower than anything
a person can safely write, because it names networks rather than ranges.
**0137 rejected deriving this, and was right about what it rejected.** It considered deriving the
list from *what the machine reports* and refused, on two grounds: a test bed creates its bridge
between one declaration and the next, and a filter that follows whatever appeared on the machine is a
firewall that widens itself. Deriving from the **declaration** is neither. The set is known before
the network exists, because a module declared it; and it cannot widen itself, because only a network
some module asked for is ever forwarded. What remains genuinely for a machine to say is guests no
module declares — a test bed's pool — which is a much smaller residue than the list as it stands.
**The gap is invisible in the one place that should show it.** The converge preview lists what
*listens*, and routing is not a listener. It says in one line what the machine routes, and a reader
has to know the runtime's allocations to tell whether that line is sufficient. On the machine
measured here it read as though nothing needed saying.
## Open questions
- Should the forward chain be derived from the network resources the node's modules declare, with the
host resolving each declared network to its address the way it already resolves a container by name?
The controller cannot render the address itself: a module's network resource carries a name, and the
runtime allocates the subnet at creation.
- What remains of `node networks` once that exists — only guests no module declares, such as a test
bed's pool? And should it then be named for that, rather than for all routing?
- The runtime's own default bridge, which containers attach to when no module network is named, is
not a module's network. Is it derived from the machine, declared by the module that owns the
runtime, or left as the one constant?
- Should the preview say which of a machine's networks are the mesh's and which are not, so a range
that exists to protect a leftover is visible as such?