Files
hq/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md
T
jochen f709e8e0fb Issues 119 and 122: what a host path is, and what a node's layout would replace
Not every host path names this machine. A system file the mesh owns is at that path on every machine
of the kind — the path is the fact. The operator's shared data is already answered as an access. A
path inside a container is the software's contract. What is left, and what a node's default layout
would replace, is 514: a module's own data, and what the mesh writes for that module.

Documents the reservation model as the records already have it — a root per node, one directory per
assignment, a placement for adopted data — and the three things nothing states: where the root comes
from, what sits beneath it, and the order the 514 are retired in.

Stops there deliberately. Changing where a definition looks without moving the data does not fail: the
mesh creates the directory, the container starts, the service comes up empty. Retiring these is a data
migration with a verification step, module by module, and belongs with whoever can see the machine.
2026-09-26 17:21:14 +02:00

7.7 KiB

status, opened, located-in, fixed-by, amended-design
status opened located-in fixed-by amended-design
located 2026-09-25
mesh-catalog modules
mesh-controller internal/catalogue

119 — A module definition decides where its files live on the machine

What was observed

A review of where module code reads its files turned up a cross-cutting pattern. Every module definition in the catalogue chooses, in its own manifest, where on the machine its files live. Counted on the catalogue's main, 2026-09-25:

where in the definition host-path strings
directory and file resources 257
container mounts, host side 230
own secrets 78
bindings 53
env-files 50
secrets 35
container environment 28
accesses 21
receives, grants 24
everything else 13

789 host-path strings in 70 of the 71 definitions. Mounts are checked: a container may not mount a path its module never declared (ADR 0091). Nothing checks the same path where it is retyped as a value: an environment variable, an env-file line, a literal in module code.

2026-09-26 — counted by what the path is, and what a node's layout would replace

The count above treats every host path the same. They are not the same, and only one of the categories is this issue's subject. Recounted by role across the 71 definitions:

What the path is Count Does a definition naming it name this machine?
A module's own data, /var/lib/<module>/… 289 Yes — this issue
What the mesh writes for a module, /var/lib/mesh/<module>/… 225 Yes — this issue
The operator's shared data, /services/… 166 Separately answered: an access, never created or owned by the mesh (ADR 0051); where it is belongs on the assignment
A system file the mesh owns, /etc/… and a few under /var/ 18 No. The path is the fact — that file is at that path on every machine of the kind. Naming it says nothing about this installation
A path inside a container 236 No. The software's own contract, true in any mesh that runs the image

So the category a node's default layout would replace is 514, not 789 and not 698: a module's own data, and what the mesh writes for that module. The rest is either already answered or was never the problem.

What the design already says, and what it does not

ADR 0112 (proposed) makes a directory a host provision whose contract is the owner and mode the module needs, and says where it lands on the machine is the assignment's. To-be 27 (proposed) resolves that two ways: a node's default layout — a root per node with one directory per assignment beneath it — used when the assignment says nothing, and a placement when the assignment puts one directory elsewhere, such as where an adopted machine's data already is.

That is the reservation model: assigning a module to a node gives it a directory, and the definition never names one. Three things are not written anywhere:

  1. Where the root comes from. Nothing says it is a node setting, fixed when the node is installed, and nothing says what a node that states none falls back to.
  2. What sits beneath it. To-be 27's own open list ends at exactly this: the layout beneath the root, beyond one directory per assignment. The catalogue currently keeps two subtrees per module — its data, and what the mesh writes for it — so this decides whether an assignment's directory holds both, or the mesh keeps its own.
  3. The order the existing 514 are retired in, which 0112 also leaves open.

Why this is not a manifest edit

Every one of those 514 paths names a directory that holds data a service is using. A definition that changes where it looks, without the data moving with it, does not fail: the mesh creates the new directory with the right owner and mode, the container starts, and the service comes up empty. The loudest case in this mesh is an object store consumer whose bucket holds 174.9 GiB; a mail spool and the mesh's own store are the same shape.

So the retirement of these paths is a data migration with a verification step, module by module, not a change to a definition. That is the reason this section documents and stops: the count and the categories are what someone needs to plan it, and the planning belongs with whoever can see the machine.

Where that has already gone wrong

  • A provider that would provision nobody, silently. One DNS provider mounts its grants directory at a short path inside its container, then tells its provisioner to read the contributions file at the host path, which does not exist in there. Nothing requires the provision today, so it has not failed yet. When a consumer arrives, it will get no record, and nobody will be told.
  • The mesh's own wire carries host paths into containers. Each contribution names its consumer's credential as "the file on this machine holding that consumer's credential", a host path computed from the provider's grants directory. So every provider has to mount that directory at the identical path, or it cannot read what it was given. Ten of the eleven providers with a grants directory do. It is a convention nothing states or checks, and the eleventh is the provider above.
  • The warning that would have caught it is lost in the SDK. The controller always writes the contributions file, even when empty, so a provider can tell "nothing asked" from "never written". The SDK's reconcile loop treats an unreadable file as empty, and logs nothing.
  • Code carries copies with nothing checking them. Several modules default a path in code when an environment variable is unset. Five of those defaults disagree with the value their own manifest sets. One of them is a host path used inside a container that does not mount it.

And a module cannot be assigned to one node twice

Everything that identifies a running module is keyed by the module's name: its directories, its container names, the login it presents to a provider, its broker account. Two assignments of one module to one node would share every one of them. Assigning the same application twice is an ordinary need: production beside staging, one site per customer, two instances of one service configured differently, two stores of one engine.

ADR 0112, which answers this report, declines that need rather than meeting it. A module is assigned at most once to a node, because every identity in the mesh is already a module on a node. The cases above become different modules, or the same module on different machines.

Why it matters beyond this instance

A definition that names machine paths is not portable between nodes. It cannot follow data onto a second disk, or onto a machine being adopted with its data already in place, without editing the module. It cannot run twice on one node. It keeps every path in two or three places with nothing checking that they agree. The defects above are what that allows, and each was found by reading, not by any check.

Open questions

  • Should a definition name any host path at all, or should every location come from the assignment and the mesh?
  • If a directory is something a module requires rather than declares, what is its contract: ownership, mode, whether it is kept when the module goes?
  • What identifies an assignment, if a module may be assigned to one node more than once?
  • What would the contributions file carry instead of host paths, so a provider needs no identical-path mount?