Issue 196: the hub relays the mesh only on the ports it publishes itself #276
@@ -0,0 +1,54 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-controller internal/catalogue/filtering.go (AsNftables: the forward chain has no rule for the mesh passing through, so a relayed packet is judged by this machine's own published ports)]
|
||||
fixed-by: mesh-controller PR 209 (the forward chain relays what comes in and goes out on the tunnel), live 2026-10-02
|
||||
amended-design: []
|
||||
---
|
||||
|
||||
# 196 — The hub relays the mesh only on the ports it publishes for itself
|
||||
|
||||
## What was observed
|
||||
|
||||
A sweep of every listening port on every machine, from every other machine, on 2026-10-02. Two home
|
||||
machines, neither of which can be dialled, reach a third home machine through the hub, as
|
||||
[ADR 0007](../../02-DECISIONS/0007-connectivity.md) says every path between machines that are not
|
||||
co-located does.
|
||||
|
||||
From either of the two, the third answered on **17 of its 55** listening ports over the mesh. The hub
|
||||
itself, probing the same machine directly, reached all the ports that machine's rules open to the mesh.
|
||||
The result was the same at 40 probes in parallel and at 4, so it was not load.
|
||||
|
||||
The 17 were not a property of the target. They were exactly the ports **the hub** publishes for its own
|
||||
containers: ssh, the proxy's two, and the hub's own block of published ports. A capture on the target
|
||||
during one probe to a port that answered and one that did not:
|
||||
|
||||
- the answering one: the SYN arrives on the tunnel, reaches the container, and the reply leaves by the
|
||||
tunnel;
|
||||
- the other: nothing arrives at all, on any interface.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**ADR 0007's hub carries every path between machines that are not co-located, and the filter breaks
|
||||
that path without saying so.** Whether one home machine can reach a service on another depends on
|
||||
whether the hub happens to publish the same port number for something of its own. Adding or removing
|
||||
a module on the hub silently opens or closes paths between two other machines that it has nothing to
|
||||
do with.
|
||||
|
||||
It also hid behind another fault. A missing placement made the same pair look disconnected earlier the
|
||||
same day, and that explanation fit well enough that the per-port pattern was not looked for.
|
||||
|
||||
## Open questions
|
||||
|
||||
- The relaying rule accepts what comes in on the tunnel and leaves on it, and leaves judging to the
|
||||
machine it is for. Should the hub also restrict relayed traffic to what that machine opens to the
|
||||
mesh? That would duplicate the target's rules on the hub.
|
||||
- No test raises two machines behind a hub and checks a port between them that the hub does not
|
||||
publish. The lab's beds have one machine per site.
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Live on all four machines after one push each. The same sweep, from both home machines to the third
|
||||
over the mesh: 45 of 55 ports answer, the same 45 the hub reaches directly. The 9 that do not are
|
||||
ports the target opens to nobody on the mesh, and one is refused because it listens only on a LAN
|
||||
address. Nothing answers that the target's rules do not open.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-02.*
|
||||
|
||||
**Not the tunnel.** The route from either home machine to the target is the tunnel, and traffic to the
|
||||
17 ports travels it in both directions. A placement fault would have stopped every port.
|
||||
|
||||
**Not the target's filter.** The target opens the failing ports to every address of the mesh in its
|
||||
input chain and its forward chain, the hub reaches them directly, and the SYN for a failing port never
|
||||
arrived at the target to be judged.
|
||||
|
||||
**The hub's forward chain.** A relayed packet comes in on the tunnel and leaves on it, so the hub's
|
||||
forward hook judges it. The chain the controller renders (`AsNftables`) has a default of drop, accepts
|
||||
established traffic, and accepts what did not arrive on an outward link or the tunnel. That last rule is
|
||||
for the machine's own containers reaching outward. After that come the rules for this machine's own
|
||||
published ports, each matching the **original destination port** of the connection. None of them names
|
||||
an outgoing interface or a destination. So a relayed packet to another machine's port 20000 matched the
|
||||
hub's own rule for its own port 20000 and passed. One to port 8080, which the hub does not publish,
|
||||
matched nothing and was dropped.
|
||||
|
||||
**The fix.** One rule: in on the tunnel **and** out on the tunnel is accepted. That is the mesh passing
|
||||
through to another of its machines, which filters it against its own rules. It does not widen anything
|
||||
on the hub. A packet for the hub itself is the input chain's, and one for the hub's own containers
|
||||
leaves by a bridge, not the tunnel. Both still meet their rules. WireGuard only accepts a packet from a
|
||||
peer whose address that peer is allowed to use, so in-on-the-tunnel means from a machine of the mesh.
|
||||
A controller test asserts the rule in the forward chain only, never in the input chain, and absent on a
|
||||
machine with no tunnel. It fails without the fix, and the rendered set loads with `nft -c`.
|
||||
Reference in New Issue
Block a user