The foundation's ports are forwarded, not only accepted on input (issue 063)

The broker's amqps port is opened from anywhere so a node can enrol
before it has an overlay address — but only in the input chain. The
broker is a published container port, so a cross-node dial is DNAT'd
and forwarded, never reaching input; it survived on the first
connection's conntrack entry and no more. Adopting the foundation's own
broker restarts it, dropping that entry, after which a joined node
could never receive another declaration. The forward chain now carries
the foundation ports too, from anywhere, matching their input rule.

Intermittent in the built-store-cross-node bed: it passed whenever the
broker did not happen to restart after the joined node first connected.
This commit is contained in:
2026-09-18 02:19:34 +02:00
parent f47b6e1c31
commit 25e42b3ad6
2 changed files with 25 additions and 0 deletions
+15
View File
@@ -409,6 +409,21 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int) str
b.WriteString(fmt.Sprintf("\t\tct original proto-dst %d accept\n", rule.Port))
}
}
// The foundation ports, forwarded — for the same reason they are in the input chain, and the
// same reason the module rules above are here too: the broker is a published container port,
// so a cross-node dial to it is redirected and *forwarded*, never reaching the input chain
// (novox/hq issue 047's lesson, applied to the foundation this time). Without this a joined
// node reaches the broker only while its first connection's conntrack entry survives — and a
// broker restart, which adopting the foundation's own broker causes, drops the entry and the
// node can never receive another declaration (novox/hq issue 063). From anywhere, matching
// the input rule: a node enrolling has no overlay address yet.
if len(foundation) > 0 {
b.WriteString("\n\t\t# the mesh's own — never derived, never closed\n")
for _, port := range foundation {
b.WriteString(fmt.Sprintf("\t\tct original proto-dst %d accept\n", port))
}
}
b.WriteString("\t}\n")
b.WriteString("}\n")
return b.String()
@@ -33,6 +33,16 @@ func TestTheBrokersPortIsOpenedThoughNoModuleDeclaresIt(t *testing.T) {
"has not yet enrolled is not on:\n%s", line)
}
}
// And forwarded, not only accepted on input: the broker is a published container port, so a
// cross-node dial is redirected and forwarded and never reaches the input chain. Without this
// a joined node reaches the broker only until its first connection's conntrack entry drops —
// which a broker restart (adopting the foundation's broker) causes — and then never receives
// another declaration (novox/hq issue 063).
if !strings.Contains(out, "ct original proto-dst 5671 accept") {
t.Fatalf("the broker's port is not forwarded, so a DNAT'd broker is unreachable "+
"cross-node after it restarts:\n%s", out)
}
}
// And a mesh that was never told about a broker still gets a ruleset, rather than an empty one or