ADR 0112 and issue 118: address the review

- Secrets follow ADR 0085 as amended: a module's own secret is a provision the controller mints and the vault records. The previous commit had that backwards. Whether the vault should generate instead is recorded as an open question, not decided. - A directory's contract is owner and mode only. The persistence flag was the keep flag ADR 0030 refused; a directory is kept while it holds anything, and disposable data is a named volume (0107). - An operator's shared data stays an access (ADR 0051), which rejected an operator-owned directory. Only where its path is written moves to the assignment. - The records it changes on acceptance are named: 0051, 0091, 0046 (settings keyed by instance), 0084 (a provider is a node and an instance), and the glossary, which gains its new words only when the record is accepted. - How it is checked covers every stated rule. Container-side paths are no longer flagged by the host-path rule, and code fallbacks are covered. - Provisions are what other modules provide. A seat's occupant is not listed as one, and the vault is not described as selectable per assignment. - 'Control plane' becomes 'controller'. The provider count is ten of eleven, not eleven of twelve.
ADR 0112: a module's own secrets are a provision from the vault, not something the mesh generates
2026-09-25 22:21:39 +02:00 · 2026-09-25 22:13:37 +02:00 · 2026-09-25 22:13:02 +02:00
6 changed files with 241 additions and 153 deletions
@@ -0,0 +1,161 @@
+---
+topic: what runs on it
+status: proposed
+date: 2026-09-25
+deciders: jochen
+reconstructed: false
+extends: 0046-a-module-configuration-is-its-assignments-not-its-manifest.md
+---
+
+# 112. A module definition names no node, no mesh and no path: everything it needs is resolved at assignment
+
+## Context
+
+[Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md) found
+**789 host-path strings in 70 of the catalogue's 71 module definitions.** Each definition chooses
+where on the machine its directories, mounts, bindings, secrets, env-files and received files live,
+and often repeats that path in an environment variable or in code. Mounts are checked against what
+the definition declares ([ADR 0091](0091-a-mount-is-declared-three-ways.md)); nothing checks the
+copies. The issue records what that has already allowed:
+
+- a provider that would provision nobody without a word;
+- a contributions file that carries host paths into containers, so every provider must mount its
+  grants directory at the identical path;
+- defaults in code that disagree with their own manifests;
+- no way to assign one module to one node twice, because every identity is keyed by the module's name.
+
+**The mesh has already decided this once, for ports.** [ADR 0038](0038-the-mesh-assigns-the-port.md):
+the mesh assigns the machine-side port and the module says only what it needs, and three copies
+became one fact. [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)
+made the manifest identity and defaults, and the assignment's settings the configuration.
+[ADR 0084](0084-which-provider-serves-a-consumer.md) made which provider serves a consumer part of
+the assignment. Paths are the largest thing still left in the definition.
+
+## Considered Options
+
+**1. Keep host paths in definitions, and check that every copy agrees.** Rejected. It checks the
+agreement of something that should not be there. A definition still could not follow its data to
+another disk, be adopted onto a machine whose data is already somewhere, or run twice on one node.
+
+**2. Keep host paths in definitions, and have the mesh rewrite them per assignment.** Rejected. It is
+string surgery on paths: deciding which strings are machine paths by their shape, which is the
+inference this repository has refused elsewhere. And the definition would still read as though it
+decided where things live.
+
+**3. A definition names variables, and the assignment resolves them.** Chosen.
+
+## Decision
+
+**A module definition is node-agnostic and mesh-agnostic.** It names no node, no mesh and no host
+path. Everything that makes a running instance *this* instance is a variable.
+
+**Installing a module on a node resolves every variable, or refuses.** A refusal names each
+unresolved variable and what could answer it. Variables are answered from three sources:
+
+1. **The assignment's own configuration.** Values chosen for this module on this node, carried as
+   settings ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). An
+   endpoint binding a public name to a port is one.
+2. **Provisions, resolved by the mesh against a contract.** What other modules provide: a database,
+   a bucket, a vhost, a secret. Each comes with the contract the mesh and the module's
+   specification define. Where a provision has several providers, which one answers is part of the
+   assignment ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its
+   database from another node. Some have one provider per mesh by decision: a module's own secret
+   is a `secret` provision the controller mints and the vault records
+   ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). Whether the vault should instead
+   generate a secret against its contract is an open question, raised while reviewing this, and not
+   decided here.
+3. **What the mesh generates or knows.** The credential the controller mints for each provision a
+   module takes ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)), the ports it
+   assigns ([ADR 0038](0038-the-mesh-assigns-the-port.md)), facts about the machine.
+
+**A directory is a provision, provided by the node's host.** A module requires one by name, such as
+its configuration or its data. Its contract is the owner and mode it needs, including the owner its
+image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). It
+carries no persistence flag. A directory is kept while it holds anything
+([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), which refused a `keep` flag for good
+reason), and data that is disposable is not a directory at all but a named volume (0107).
+
+*Where* a directory is on the machine is the assignment's. A node has a default layout, and an
+assignment may place one directory elsewhere: on a second disk, or where an adopted machine's data
+already is ([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). A directory is
+always provided on the module's own node, because a host path means nothing on any other.
+
+**An operator's shared data stays an `access`** ([ADR 0051](0051-shared-data-is-the-operators.md)),
+not a directory provision. 0051 rejected giving a directory an operator owner, because the mesh
+must never create, chown or remove such data, and that stands. What changes is only where its
+location is written: the module says it needs read or read-write access, and the assignment says
+where the data is.
+
+**Inside a container, a module sees its own paths.** The definition says where the image expects
+each directory. The mesh mounts the assignment's location there. No host path is ever a value a
+process reads.
+
+**The mesh's own files carry no host path.** Bindings, secrets and contributions are named relative
+to where the module receives them, so a provider reads what it was given without mounting anything
+at a machine-identical path.
+
+**An assignment has an identity of its own: an instance name**, defaulting to the module's name.
+Everything keyed by the module's name today is keyed by the instance instead: directories,
+container names, the login a consumer presents, broker accounts, a claim's holder, the settings
+an assignment carries, and a provider's identity. So **one module may be assigned to one node more
+than once.** What must stay singular stays so by a claim, or by the assignment's own configuration
+colliding: a public name already taken is refused like any other singular thing.
+
+## What this changes in earlier records
+
+On acceptance, each of these is amended by a record of its own, not edited:
+
+- [ADR 0051](0051-shared-data-is-the-operators.md): an access keeps its shape and its semantics; its
+  path moves from the definition to the assignment.
+- [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved variable,
+  checked as resolved rather than as a path the definition declares.
+- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings are
+  addressed to an instance, not to a module on a node.
+- [ADR 0084](0084-which-provider-serves-a-consumer.md): a provider is a (node, instance) pair, not a
+  (node, module) pair, so a consumer can name one of two instances on one node.
+- The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to
+  include a directory the node's host provides, and *instance* is added. Neither lands while this
+  record is only proposed, because the glossary is the authority on the words in use, not on words
+  under review.
+
+## Consequences
+
+- **Every definition changes.** 70 of 71 name host paths today. The change is mechanical for most.
+  The design has to say how existing modules migrate without their data moving: an adopted or
+  already-running assignment is placed where its data already is.
+- The controller resolves variables at assignment and refuses unresolved ones. The host provides
+  directories. The contributions file's format changes, and so does the SDK's reconcile loop that
+  reads it.
+- Identity moves from the module to the instance, which touches logins, broker accounts, settings,
+  provider selection and every resource name.
+- **What got harder:** a definition no longer says where a module's data is on a machine. The
+  assignment does, and `plan` shows it. That is the point, and it is also a real loss of
+  at-a-glance legibility, which the overview has to give back.
+- **Not decided here:** the variable syntax; a node's default layout; whether a second instance of a
+  module is supported from the first step or after the definitions have moved; whether the vault
+  generates secrets.
+
+## How it is checked
+
+| Rule | Checked by |
+|---|---|
+| A definition names no host path | A catalogue test: the host side of every mount, and every resource location, binding, secret, receives and grants entry, is a variable rather than an absolute path. A declared list of exceptions shrinks to empty as definitions move. |
+| No host path is a value a process reads | A catalogue test: every absolute path in a container's environment or env-files lies on the container side of one of its mounts, or is declared the image's own. A second test finds literal paths in module code used as fallbacks for an environment variable. |
+| A definition names no node and no mesh | The parser has no field that names a node; a node is named only in an assignment. A catalogue test finds no domain name in any definition value. |
+| Installation resolves every variable | A resolution test with one variable unanswered: refused, naming the variable and its possible sources. |
+| A directory is provided on its module's own node | A resolution test: an assignment placing a directory on another node is refused. |
+| A provider reads what it was given without an identical mount | A provisioner test reading a contributions file whose credentials are named relative to where it is mounted. |
+| A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, separate settings, no collision. |
+| A public name already taken is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. |
+| An adopted assignment is placed where its data is | An adoption test: the directory resolves to the data's existing location, and nothing is moved. |
+
+## References
+
+- [Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md): the evidence
+- [ADR 0038](0038-the-mesh-assigns-the-port.md): the same decision, for ports
+- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): configuration is the assignment's
+- [ADR 0084](0084-which-provider-serves-a-consumer.md): which provider answers is the assignment's
+- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md):
+  secrets as a provision, and who mints what
+- [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md),
+  [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries, and what it must not
@@ -155,6 +155,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md)
 - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md)
 - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md)
+- **0112** — [A module definition names no node, no mesh and no path: everything it needs is resolved at assignment](0112-a-module-definition-names-no-node-mesh-or-path.md)  *(proposed)*

 ### How it is built

@@ -0,0 +1,79 @@
+---
+status: open
+opened: 2026-09-25
+located-in: []
+fixed-by:
+amended-design:
+---
+
+# 118 — A module definition decides where its files live on the machine
+
+## What was observed
+
+A review of where module code reads its files turned up a cross-cutting pattern. Every module
+definition in the catalogue chooses, in its own manifest, where on the machine its files live.
+Counted on the catalogue's `main`, 2026-09-25:
+
+| where in the definition | host-path strings |
+|---|---|
+| directory and file resources | 257 |
+| container mounts, host side | 230 |
+| own secrets | 78 |
+| bindings | 53 |
+| env-files | 50 |
+| secrets | 35 |
+| container environment | 28 |
+| accesses | 21 |
+| receives, grants | 24 |
+| everything else | 13 |
+
+**789 host-path strings in 70 of the 71 definitions.** Mounts are checked: a container may not
+mount a path its module never declared ([ADR 0091](../../02-DECISIONS/0091-a-mount-is-declared-three-ways.md)).
+Nothing checks the same path where it is retyped as a value: an environment variable, an env-file
+line, a literal in module code.
+
+### Where that has already gone wrong
+
+- **A provider that would provision nobody, silently.** One DNS provider mounts its grants
+  directory at a short path inside its container, then tells its provisioner to read the
+  contributions file at the host path, which does not exist in there. Nothing requires the
+  provision today, so it has not failed yet. When a consumer arrives, it will get no record, and
+  nobody will be told.
+- **The mesh's own wire carries host paths into containers.** Each contribution names its
+  consumer's credential as "the file on this machine holding that consumer's credential", a host
+  path computed from the provider's grants directory. So every provider has to mount that directory
+  at the *identical* path, or it cannot read what it was given. Ten of the eleven providers with a
+  grants directory do. It is a convention nothing states or checks, and the eleventh is the
+  provider above.
+- **The warning that would have caught it is lost in the SDK.** The controller always writes the
+  contributions file, even when empty, so a provider can tell "nothing asked" from "never written".
+  The SDK's reconcile loop treats an unreadable file as empty, and logs nothing.
+- **Code carries copies with nothing checking them.** Several modules default a path in code when an
+  environment variable is unset. Five of those defaults disagree with the value their own manifest
+  sets. One of them is a host path used inside a container that does not mount it.
+
+### And a module cannot be assigned to one node twice
+
+Everything that identifies a running module is keyed by the module's name: its directories, its
+container names, the login it presents to a provider, its broker account. Two assignments of one
+module to one node would share every one of them. Assigning the same application twice is an
+ordinary need: production beside staging, one site per customer, two instances of one service
+configured differently, two stores of one engine.
+
+## Why it matters beyond this instance
+
+A definition that names machine paths is not portable between nodes. It cannot follow data onto a
+second disk, or onto a machine being adopted with its data already in place, without editing the
+module. It cannot run twice on one node. It keeps every path in two or three places with nothing
+checking that they agree. The defects above are what that allows, and each was found by reading,
+not by any check.
+
+## Open questions
+
+- Should a definition name any host path at all, or should every location come from the
+  assignment and the mesh?
+- If a directory is something a module *requires* rather than *declares*, what is its contract:
+  ownership, mode, whether it is kept when the module goes?
+- What identifies an assignment, if a module may be assigned to one node more than once?
+- What would the contributions file carry instead of host paths, so a provider needs no
+  identical-path mount?
@@ -1,47 +0,0 @@
---
-status: located
-opened: 2026-09-25
-located-in: [mesh-catalog modules/umami]
---
-
-# 118 — umami's store answers the dial and times out the query
-
-## What was observed
-
-`umami.novox.be` has answered `502` through the whole of 2026-09-25's migration session
-(first noted mid-afternoon, still true at night). The container restart-loops on a
-timescale of about a minute. Its own log, every cycle:
-
-```
-✓ DATABASE_URL is defined.
-✓ Database connection successful.
-Invalid `prisma.$queryRaw()` invocation:
-Raw query failed. Code: `N/A`. Message: `Operation has timed out`
-```
-
-The connection is established — the dial succeeds — and the first raw query then times
-out. This is not a credentials fault and not an unreachable store.
-
-## What it is not
-
- Not the routing layer: the `502` is Traefik faithfully reporting a backend that is
-  restart-looping. The stale duplicate Traefik router for this name (a HAL-era
-  hand-authored file beside the mesh-written one) was removed the same night and changed
-  nothing, as expected.
- Not the mesh's grant machinery: the binding and sealed secret compose, and the store
-  accepts the login — a wrong credential refuses the dial, and this dial succeeds.
-
-## Where to look
-
-A dial that succeeds and a query that times out, from a container on one network to a
-store on another, has the shape of a path-MTU or conntrack fault (large response packets
-dropped after the small handshake ones pass), or of the store accepting the TCP
-connection while the backend it proxies for is wedged. Neither is proven. What is known
-to differ for umami against every working consumer of the same store tonight is nothing
-yet — that comparison is the first move.
-
-## Why it is filed rather than chased
-
-The 2026-09-25 session's scope was routing and the build chain; this fault predates the
-night's changes, survived them unchanged, and needs its own sitting with the store's own
-logs beside the consumer's.
@@ -1,59 +0,0 @@
---
-status: open
-opened: 2026-09-26
-located-in: [mesh-host internal/apply, mesh-controller]
---
-
-# 119 — a hold is not a line in the apply report, and an operator flew blind into an outage
-
-## What was observed
-
-During the route-proxy edge cutover on novox (2026-09-26): the module was assigned, the
-push reported success, `status` said the node was doing everything it was told — and the
-module's three containers did not exist. The operator stopped the predecessor's proxy on
-the strength of those reports, and every public name on the node went dark until rollback.
-
-The cause was correct behaviour, invisibly reported. The first (rolled-back) route-proxy
-attempt had left `/var/lib/route-proxy/*` on disk; on re-assign, the adopted node *found*
-those directories, held them (ADR 0100, exactly as designed), and held every container
-that mounts them — `"would mount /var/lib/route-proxy/ca, found on this adopted node;
-not run until route-proxy is taken"`. All of that lived only in `state.json`. What the
-operator saw:
-
- the push: `sent novox 346 resource(s)` — the controller's count of what it sent;
- the node's journal: `applied 330 resource(s)` — sixteen fewer, with no line saying
-  which sixteen or why;
- `status`: green — a held resource is not "wrong", so nothing was flagged;
- `node show novox`: the holds list did NOT include route-proxy's (it showed only holds
-  the *controller* knew about from take-time listings, not what the node decided at
-  apply-time).
-
-Four surfaces, none carrying the one sentence that mattered: *route-proxy is assigned
-but not taken, and its containers will not run until it is.*
-
-## Why this is a real fault and not operator error alone
-
-The operator error (an edge-flip runbook that omitted `take`) was only possible because
-every surface reported success. A system whose correct refusals are indistinguishable
-from completed work will keep converting small procedural gaps into outages. The
-`sent 346 / applied 330` discrepancy was the single visible symptom, and interpreting it
-required reading `state.json` by hand.
-
-## What would have prevented it
-
-Any one of:
-
-1. **The apply report says what it held.** `applied 330 resource(s), 16 held for
-   untaken modules (route-proxy: 13, …)` — one line in the journal.
-2. **`status` counts holds against untaken-but-assigned modules.** A module assigned,
-   pushed, and running zero of its containers is at minimum worth a "waiting on take"
-   line — it is never converged in any useful sense.
-3. **`node show <node>` shows the node's own held list**, not only what take-time
-   computed — the node already records it in `state.json` with reasons.
-
-## Precedent
-
-The photos cutover hit the same semantics benignly the same week (assign → held
-containers in `Created` state → take), and the mailu cutover documented "take is the
-verb, and ADR 0100 meant it". The semantics are consistent and right; the reporting is
-what let them be forgotten at the worst moment.
@@ -1,47 +0,0 @@
---
-status: open
-opened: 2026-09-26
-located-in: [mesh-host internal/apply]
---
-
-# 121 — a volume path is not in the spec comparison, and a roll-out raced a data move
-
-## What was observed
-
-Landing the "module data lives in /var/lib" change on novox (mesh-catalog #97), two
-distinct faults surfaced in one hour:
-
-1. **Building a module with a roll-out upgrade policy IS deploying it.** gitea's policy
-   was roll-out; the `build` that registered its repathed manifest sent it to the node
-   immediately, which recreated the container mounting the *not-yet-renamed* (empty)
-   `/var/lib/gitea/data`. The forge came back as its own install page, fresh host keys
-   and all, and every subsequent pipeline build died on `repository not found` — which
-   also blocked the fix, since re-registering the other modules needed the forge. The
-   operator narrative "build, then move data, then push" is only safe under the record
-   policy; nothing warned that one module in the batch would skip the pause.
-
-2. **Changing a container's volume paths does not recreate the container.** After the
-   final push, five of the six repathed modules kept their old containers running
-   ("Up 13–26 hours") — the new declaration's volume paths differ from the running
-   containers' mounts, and the apply judged them current. Same class as mesh-host #27
-   (`dns`/`ip` absent from the comparison): a field the comparison does not read is a
-   field that can never change a running container. Benign here only because a rename
-   on one filesystem preserves the mounted inode — the running containers keep serving
-   the same bytes the new path names, and the next natural recreation converges. A
-   cross-filesystem move, or a path change to *different* data, would have silently
-   split the module between two worlds.
-
-## What would have prevented it
-
- `build` printing the module's upgrade policy when that policy will act on the result
-  ("gitea rolls out on build — the node will receive this immediately"), or a
-  `--register-only` flag for exactly this choreography.
- Volumes (and every other container field) in the spec comparison, or the honest
-  refusal: "this field changed and I cannot apply it without recreation."
-
-## Recovery that worked
-
-Instant renames both ways broke the circular dependency (forge needed for builds,
-builds needed for the push, push needed for the forge): data back to the old path,
-old-spec forge started, artifacts rebuilt, data renamed forward, push. Nothing lost;
-the install-page junk was discarded twice.