This machine's own guests are on the private network

A port declared from the mesh admitted the machines' own addresses on the private
network. A container reaching a port on the machine it runs on comes from a bridge,
matching none of them — and where the container runtime routes directly, that packet
is delivered to this machine rather than forwarded, so the forward chain's allowance
never saw it either.

ADR 0100 requires this to work: the store is reachable 'from a container on the node
itself'. It was, through a rule the predecessor left, which allowed the private
ranges wholesale. Converging the machine replaced that with the four overlay
addresses and closed it.

Measured, and it was an outage: every module reaching another by its machine's own
name timed out for eleven hours while the mesh reported the machine healthy. A web
application logged 'connection to server at novox.internal (10.10.0.1), port 6852
failed: timeout expired' throughout.

Asked for by the link it arrives on, for the reason the forward chain no longer
names an address: a range describes one machine and goes stale in silence. A port
open to everything needs no such line.
This commit is contained in:
2026-09-29 12:30:15 +02:00
parent 9c83dacfce
commit 2b20a12c4a
2 changed files with 55 additions and 0 deletions
+19
View File
@@ -374,6 +374,25 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int,
b.WriteString(fmt.Sprintf("\t\tip6 saddr { %s } %s dport %d accept\n", b.WriteString(fmt.Sprintf("\t\tip6 saddr { %s } %s dport %d accept\n",
strings.Join(six, ", "), rule.Protocol, rule.Port)) strings.Join(six, ", "), rule.Protocol, rule.Port))
} }
// **And this machine's own guests, which are part of what it hosts** (novox/hq ADR 0100:
// the store is reachable "from a container on the node itself").
//
// The addresses above are the machines' own on the private network. A container reaching a
// port on the machine it runs on comes from a bridge, so it matches none of them — and with
// the runtime routing directly, that packet is delivered to this machine rather than
// forwarded, so the forward chain's allowance never sees it either.
//
// Measured, and it was an outage: after a machine was converged, every module that reached
// another by the machine's own name timed out. A web application logged
// "connection to server at novox.internal (10.10.0.1), port 6852 failed: timeout expired"
// for eleven hours while the mesh reported the machine healthy.
//
// Asked for by the link it arrives on, for the reason §4 no longer names an address: a
// range describes one machine and goes stale in silence.
if inward != "" {
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } %s dport %d accept\n",
inward, rule.Protocol, rule.Port))
}
case FromEverywhere: case FromEverywhere:
b.WriteString(fmt.Sprintf("\t\t%s dport %d accept\n", rule.Protocol, rule.Port)) b.WriteString(fmt.Sprintf("\t\t%s dport %d accept\n", rule.Protocol, rule.Port))
} }
+36
View File
@@ -804,3 +804,39 @@ func TestSSHIsNeverLeftWithoutARule(t *testing.T) {
t.Fatalf("a machine with no mesh addresses has no ssh rule, so adopting it locks it:\n%s", nft) t.Fatalf("a machine with no mesh addresses has no ssh rule, so adopting it locks it:\n%s", nft)
} }
} }
// **This machine's own guests are part of what it hosts** (novox/hq ADR 0100: the store is reachable
// "from a container on the node itself").
//
// The addresses a mesh-scoped rule admits are the machines' own on the private network. A container
// reaching a port on the machine it runs on comes from a bridge, matching none of them — and where the
// runtime routes directly, that packet is delivered to this machine rather than forwarded, so the
// forward chain's allowance never sees it either.
//
// Measured, and it was an outage: after a machine was converged, every module reaching another by the
// machine's own name timed out for eleven hours while the mesh reported the machine healthy.
func TestAMeshScopedPortAdmitsThisMachinesOwnGuests(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}},
}}, nil), []string{"10.10.0.1", "10.10.0.2"}, false, nil, []string{"eth0"}, "mesh0")
// The private network's own addresses, as before.
if !strings.Contains(nft, "ip saddr { 10.10.0.1, 10.10.0.2 } tcp dport 5432 accept") {
t.Fatalf("the private network no longer reaches a mesh-scoped port:\n%s", nft)
}
// And this machine's guests, by the link they arrive on.
if !strings.Contains(nft, `iifname != { "eth0", "mesh0" } tcp dport 5432 accept`) {
t.Fatalf("a container on this machine cannot reach a mesh-scoped port on it, which is the "+
"outage this test exists for:\n%s", nft)
}
}
// A port open to everything needs no such line — it is already open to a guest.
func TestAPublicPortNeedsNoGuestLine(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}},
}}, nil), []string{"10.10.0.1"}, false, nil, []string{"eth0"}, "mesh0")
if strings.Count(nft, `iifname != { "eth0", "mesh0" } tcp dport 443`) != 0 {
t.Fatalf("a public port was given a guest line it does not need:\n%s", nft)
}
}