--- status: open opened: 2026-10-04 located-in: [mesh-host internal/apply, mesh-controller internal/catalogue] fixed-by: amended-design: --- # 225 — A provisioner cannot read the grant secrets since its code left the container, and every consumer of it is unserved ## What was observed On the control machine, 2026-10-04, found while looking at why one app was restarting: ``` [mongodb] [provisioner:mongodb-database] mesh_novox_photos: secret not readable yet (/var/lib/mongodb/grants/novox.photos.secret): Error: EACCES: permission denied, open '/var/lib/mongodb/grants/novox.photos.secret' ``` **4330 times, every five seconds, since 01:30:20.** The consequence is not a log line: the provisioner never reads the password, so it never creates the user, so the consumer never connects — ``` UserNotFound: Could not find user "mesh_novox_photos" for db "admin" ``` — and the app crash-loops. Two consumers on this machine are in that state. ## Why The grant secrets are what the mesh seals for each consumer and the host unseals beside the provider's contributions file. They are written `-rw------- root root`, which was right while a module's own code ran in a container as root. [ADR 0198](../../02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md) moved a module's long-running code out of its container and under the node's runtime, which runs as the operator's account. The provisioner is now that account; the secret is still root's. The timestamps say it exactly: the files are dated 2026-09-26, the first refusal is 01:30:20 on the day the runtime rolled. **Nothing reports it.** The machine applies cleanly and reads as current; the provisioner says `secret not readable yet`, whose wording is for a real and ordinary race on the first pass — the host has not written the file yet — and which is indistinguishable, in the log, from a permanent refusal. Four thousand occurrences of a message that means "wait a moment" is the shape to recognise. ## Why it matters beyond this instance This is every provider that provisions. The grant secret is the one file the sdk's harness reads for every consumer, so a provider that cannot read it serves nobody — and says so only in a line that reads like patience. It is also the general question the runtime move leaves: **what the mesh seals for a module is owned for the shape that module's code used to have.** Each module whose code moved is a module whose files may now be unreadable to it, and ownership is the mesh's to state, not the module's to work around. ## What this is not Not caused by [ADR 0202](../../02-DECISIONS/0202-a-provider-declares-what-it-derives-for-each-consumer.md) or [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md), which landed two to three hours after the first refusal. Those rebuilt the two affected consumers, which recreated their containers and made a silent fault visible as a restarting one. The dates are above. ## Open questions - Who owns a grant secret now — the module's account, as `secrets-owner` already says for a module's own secrets? Then the host writes it so, and this is a one-line statement in the declaration rather than a convention. - Should `secret not readable yet` stop saying "yet" after the first few passes? A message that is right once and wrong four thousand times is a message that hides its own meaning. - Which other modules' files did the runtime move leave behind? The sweep is the same question for every path the mesh writes for a module: directories, bundles, received files.