3.5 KiB
status, opened, located-in, fixed-by, amended-design
| status | opened | located-in | fixed-by | amended-design | |
|---|---|---|---|---|---|
| located | 2026-10-03 |
|
mesh-controller#248 |
218 — A seat held once for the mesh is answered by a module on a machine that does not hold it
What was observed
2026-10-03. Asked which databases the mesh's store holds, mesh-store.databases answered from the
postgres on one machine with that machine's application databases; the controller's own database
lives on the postgres of the other machine, which the controller's records name as the seat's one
holder:
mesh-store scope: mesh delivers: postgres-database holders: [ {node: <the control machine>, module: postgres} ]
The discovery console, which reads what answers on the bus (ADR 0197), shows the same seat announced from both machines running postgres.
Why it matters beyond this instance
A seat held once for the mesh promises one answerer: the role's holder. A module that implements a seat's verbs on every machine it runs on, and is let serve them on each, turns "the mesh's store" into "whichever postgres replied first" — a read against the wrong database that looks like a right one, and a write would be worse. Every mesh-scoped seat whose implementing module runs on more than one machine has this shape.
Where to look
Whether the runtime serves a seat's verbs where its module merely claims the seat rather than where the mesh made it the holder (ADR 0159, ADR 0160): the membership the controller issues each assignment, and what the runtime admits from it. A check: a mesh-scoped seat's verbs are served by exactly the holder the records name, on every machine.
Root cause
The controller composed each assignment's held seats from what its module claims, once per module and not once per machine. Every machine running postgres was therefore given the store seat's grants and issued its subjects, and each runtime served the seat's verbs because it serves what it is issued (ADR 0160). The runtime behaved as designed. The fault was in what it was issued.
The seat's verbs are not the module's tools. The store's databases and query are a separate
implementation registered under the seat's name (ADR 0159).
Only that implementation should be withdrawn where the module does not hold the seat. postgres's own
tools stay served on every machine it runs on.
Fix
The controller now reads the recorded seat holdings when it composes grants and memberships. A seat held once for the mesh is issued only to the machine and module the records name as its holder. A seat held once per machine, and a mesh seat with no holder on record, are issued as before. Grants and memberships come from the same list, so they cannot disagree.
How it is checked. A controller test asserts that a claimant on another machine keeps its node
seats and loses the recorded mesh seat. Live, the discovery console's overview must show each
mesh-scoped seat announced from exactly the holder the records name. Status moves to resolved once
that holds after the fix is rolled out.