1.8 KiB
status, opened, located-in, fixed-by, amended-design
| status | opened | located-in | fixed-by | amended-design | |
|---|---|---|---|---|---|
| resolved | 2026-10-01 |
|
mesh-controller PR 189 (fix/the-controller-may-publish-memberships) |
183 — The controller could not publish the memberships it issued
What was observed
The controller release that issues a membership per assignment (ADR 0160) rolled onto the control node. After its first push the controller's log said, once:
nats: permissions violation: Permissions Violation for Publish to "mesh.assignment.<node>.<module>"
No membership reached the assignments stream. Nothing else changed: every runtime kept serving the shape it derives for itself, which is what the decision says happens while no membership is issued, and so the mesh looked healthy while the whole new mechanism was inert.
Why this is here
The controller composes every account's grant, its own included, and its own grant named the control, node and JetStream subjects and not the assignments it alone issues. A grant composed by its holder is checked by nothing but the server at publish time, and a refused publish is one log line that nothing reads. The fallback that makes the roll-out safe is the same thing that makes this failure silent.
Resolved, 2026-10-01
The controller's grant names mesh.assignment.>; the broker golden changed by that one line. A
grant composed by its holder arrives late — the broker node is pushed after the controller rolls —
so the roll-out is merge, push the broker node, then any push issues memberships.
How it is checked: the broker golden carries the allow line; live, a runtime's log after the next push says it was issued a membership rather than that none exists.