jschoubben
bc64c5c187
The derived-value record is renumbered a second time: the key-value-buckets record took 0201 while this waited to merge, as the bundles record took 0188 before it. Both times free when chosen, taken by the time it landed. cycle.py caught it; three repositories cite this record, so the number matters. 225 — a provisioner has not been able to read its grant secrets since 01:30, when a module's own code left its container and the files stayed root's. Four thousand refusals, each worded as patience, and two consumers unserved. Not from ADR 0202 or 0189, which landed hours later; dates in the report. 226 — the store's sweep stops at the first reference recorded with an address and collects nothing. A guard that cannot tell 'I will not ask about this' from 'it would not answer' stops the wrong amount of work. 227 — the photo app's admin client asks for the port the proxy holds. A module pinned months behind carries everything its branch gained, the first time anything makes it move.
3.5 KiB
status, opened, located-in, fixed-by, amended-design
| status | opened | located-in | fixed-by | amended-design | ||
|---|---|---|---|---|---|---|
| open | 2026-10-04 |
|
225 — A provisioner cannot read the grant secrets since its code left the container, and every consumer of it is unserved
What was observed
On the control machine, 2026-10-04, found while looking at why one app was restarting:
[mongodb] [provisioner:mongodb-database] mesh_novox_photos: secret not readable yet
(/var/lib/mongodb/grants/novox.photos.secret):
Error: EACCES: permission denied, open '/var/lib/mongodb/grants/novox.photos.secret'
4330 times, every five seconds, since 01:30:20. The consequence is not a log line: the provisioner never reads the password, so it never creates the user, so the consumer never connects —
UserNotFound: Could not find user "mesh_novox_photos" for db "admin"
— and the app crash-loops. Two consumers on this machine are in that state.
Why
The grant secrets are what the mesh seals for each consumer and the host unseals beside the
provider's contributions file. They are written -rw------- root root, which was right while a
module's own code ran in a container as root.
ADR 0198 moved a module's long-running code out of its container and under the node's runtime, which runs as the operator's account. The provisioner is now that account; the secret is still root's. The timestamps say it exactly: the files are dated 2026-09-26, the first refusal is 01:30:20 on the day the runtime rolled.
Nothing reports it. The machine applies cleanly and reads as current; the provisioner says
secret not readable yet, whose wording is for a real and ordinary race on the first pass — the
host has not written the file yet — and which is indistinguishable, in the log, from a permanent
refusal. Four thousand occurrences of a message that means "wait a moment" is the shape to
recognise.
Why it matters beyond this instance
This is every provider that provisions. The grant secret is the one file the sdk's harness reads for every consumer, so a provider that cannot read it serves nobody — and says so only in a line that reads like patience.
It is also the general question the runtime move leaves: what the mesh seals for a module is owned for the shape that module's code used to have. Each module whose code moved is a module whose files may now be unreadable to it, and ownership is the mesh's to state, not the module's to work around.
What this is not
Not caused by ADR 0202 or ADR 0189, which landed two to three hours after the first refusal. Those rebuilt the two affected consumers, which recreated their containers and made a silent fault visible as a restarting one. The dates are above.
Open questions
- Who owns a grant secret now — the module's account, as
secrets-owneralready says for a module's own secrets? Then the host writes it so, and this is a one-line statement in the declaration rather than a convention. - Should
secret not readable yetstop saying "yet" after the first few passes? A message that is right once and wrong four thousand times is a message that hides its own meaning. - Which other modules' files did the runtime move leave behind? The sweep is the same question for every path the mesh writes for a module: directories, bundles, received files.