Merge pull request 'Issue 196: the hub relays the mesh only on the ports it publishes itself' (#276) from jschoubben/the-hub-relays-the-mesh into main

This commit was merged in pull request #276.
This commit is contained in:
2026-10-02 09:23:55 +00:00
2 changed files with 81 additions and 0 deletions
@@ -0,0 +1,54 @@
---
status: resolved
opened: 2026-10-02
located-in: [mesh-controller internal/catalogue/filtering.go (AsNftables: the forward chain has no rule for the mesh passing through, so a relayed packet is judged by this machine's own published ports)]
fixed-by: mesh-controller PR 209 (the forward chain relays what comes in and goes out on the tunnel), live 2026-10-02
amended-design: []
---
# 196 — The hub relays the mesh only on the ports it publishes for itself
## What was observed
A sweep of every listening port on every machine, from every other machine, on 2026-10-02. Two home
machines, neither of which can be dialled, reach a third home machine through the hub, as
[ADR 0007](../../02-DECISIONS/0007-connectivity.md) says every path between machines that are not
co-located does.
From either of the two, the third answered on **17 of its 55** listening ports over the mesh. The hub
itself, probing the same machine directly, reached all the ports that machine's rules open to the mesh.
The result was the same at 40 probes in parallel and at 4, so it was not load.
The 17 were not a property of the target. They were exactly the ports **the hub** publishes for its own
containers: ssh, the proxy's two, and the hub's own block of published ports. A capture on the target
during one probe to a port that answered and one that did not:
- the answering one: the SYN arrives on the tunnel, reaches the container, and the reply leaves by the
tunnel;
- the other: nothing arrives at all, on any interface.
## Why it matters
**ADR 0007's hub carries every path between machines that are not co-located, and the filter breaks
that path without saying so.** Whether one home machine can reach a service on another depends on
whether the hub happens to publish the same port number for something of its own. Adding or removing
a module on the hub silently opens or closes paths between two other machines that it has nothing to
do with.
It also hid behind another fault. A missing placement made the same pair look disconnected earlier the
same day, and that explanation fit well enough that the per-port pattern was not looked for.
## Open questions
- The relaying rule accepts what comes in on the tunnel and leaves on it, and leaves judging to the
machine it is for. Should the hub also restrict relayed traffic to what that machine opens to the
mesh? That would duplicate the target's rules on the hub.
- No test raises two machines behind a hub and checks a port between them that the hub does not
publish. The lab's beds have one machine per site.
## Resolved (2026-10-02)
Live on all four machines after one push each. The same sweep, from both home machines to the third
over the mesh: 45 of 55 ports answer, the same 45 the hub reaches directly. The 9 that do not are
ports the target opens to nobody on the mesh, and one is refused because it listens only on a LAN
address. Nothing answers that the target's rules do not open.
@@ -0,0 +1,27 @@
# Diagnosis
*2026-10-02.*
**Not the tunnel.** The route from either home machine to the target is the tunnel, and traffic to the
17 ports travels it in both directions. A placement fault would have stopped every port.
**Not the target's filter.** The target opens the failing ports to every address of the mesh in its
input chain and its forward chain, the hub reaches them directly, and the SYN for a failing port never
arrived at the target to be judged.
**The hub's forward chain.** A relayed packet comes in on the tunnel and leaves on it, so the hub's
forward hook judges it. The chain the controller renders (`AsNftables`) has a default of drop, accepts
established traffic, and accepts what did not arrive on an outward link or the tunnel. That last rule is
for the machine's own containers reaching outward. After that come the rules for this machine's own
published ports, each matching the **original destination port** of the connection. None of them names
an outgoing interface or a destination. So a relayed packet to another machine's port 20000 matched the
hub's own rule for its own port 20000 and passed. One to port 8080, which the hub does not publish,
matched nothing and was dropped.
**The fix.** One rule: in on the tunnel **and** out on the tunnel is accepted. That is the mesh passing
through to another of its machines, which filters it against its own rules. It does not widen anything
on the hub. A packet for the hub itself is the input chain's, and one for the hub's own containers
leaves by a bridge, not the tunnel. Both still meet their rules. WireGuard only accepts a packet from a
peer whose address that peer is allowed to use, so in-on-the-tunnel means from a machine of the mesh.
A controller test asserts the rule in the forward chain only, never in the input chain, and absent on a
machine with no tunnel. It fails without the fix, and the rendered set loads with `nft -c`.