Files
jschoubben d85d74d607 059 resolved — the broker credential resolves the seat
Fixed by mesh-controller PR 30 (79a9e17), rebased onto the merged
registry-reach train and proven by a green fresh run of the no-fake
two-node bed built from that commit.
2026-09-18 01:13:03 +02:00

3.3 KiB

status, opened, located-in, fixed-by, amended-design
status opened located-in fixed-by amended-design
resolved 2026-09-17
mesh-controller
mesh-controller PR 30 (79a9e17); gated by mesh-lab built-store-cross-node

059 — The broker credential names the hub, not the node holding the broker

Symptom

An adversarial review of the issue-055 fix (brokerReachableAt) found the cross-node broker address is computed from the wrong facts. No live failure yet — every current mesh has the broker on the hub — but each is a wrong invariant that fails exactly when the topology stops being the simple one:

  1. The helper returns the hub's overlay name for any on-overlay node, but nothing ties the broker to the hub. The broker lives wherever the module holding the mesh-broker seat is assigned; hub-ness is a separate operator declaration — and the overlay design actively pushes them apart (the hub must be publicly dialable; a natural topology is hub-on-a-relay, foundation at home). In that mesh every issued credential dials a relay where no broker listens.
  2. "On the overlay" is tested as "has an overlay address." The codebase itself defines membership as resolved the networking module — "not 'has an address'… it is what resolved the module" — and excludes address-only nodes from every peer list. A node placed but not yet given networking is sealed an overlay URL it cannot route.
  3. A credential issued before overlay place keeps the public URL for ever, and nothing says so; the remedy (re-issue + push) works but is undocumented. The sibling of issue 057's ordering gap.
  4. A portless MESH_BROKER_ADDRESS silently disables the fix — SplitHostPort fails and the helper falls back to the public address without a word. The port should default to 5671, or the address should be refused at genesis.
  5. Two hubs are never refused. A second overlay place --hub leaves both flagged; the pick is then order-dependent. ErrNotOneHub exists and nothing returns it.

One review claim is recorded here as disputed: that the from-mesh forward rule could not have been the 055 failure because the broker port is opened from anywhere. The live measurements during 055 (nftables counters; a cross-node probe of the amqps port flipping CLOSED→OPEN exactly when the broker module's listens gained the port) say the forward chain was the gate for published container ports. The code-reading and the wire disagree; the wire was measured.

Why it matters

The 055 fix is correct for every mesh that exists, and wrong by construction for the first mesh whose hub is not its control-node — which the overlay design itself recommends for a home mesh behind NAT. Credentials are sealed and long-lived, so the failure arrives long after the decision that caused it, on the first module pushed to a node that joined the reshaped mesh.

Open questions

  • Should the broker address resolve from the mesh-broker seat's assignment (falling back to the hub only when nothing holds the seat — genesis), and should membership use the same resolved-networking test the peer graph uses?
  • Where does the stale-credential remedy live: a warning from overlay place, an automatic re-issue at the next push, or documentation?
  • Should a second --hub be refused outright (ErrNotOneHub finally thrown)?