diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md new file mode 100644 index 0000000..49eabbc --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md @@ -0,0 +1,57 @@ +--- +status: graduated +became: + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md + - 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +initiated: 2026-09-26 +touches: + - 02-DECISIONS/0113-the-vault-makes-every-secret.md + - 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md + - 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md + - 03-DESIGN/01-to-be/13-credentials-and-their-rotation.md + - 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md + - 04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md +--- + +# 016 — How a credential can be rotated + +**What.** Which rotation mechanisms the mesh's providers can actually support, measured against +their code rather than assumed. Every provider in the catalogue was read, found by listing every definition that provides something: how it names what it +makes for a consumer, what its remove destroys, whether it re-applies a password, whether its +backend can hold two secrets for one login or two logins on one resource, and how its own +administrative credential is set. The consumer side was read too: when a module reads a secret, and +what makes it read a new one. + +**Why.** [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), as first drafted, +chose *overlap*: add a second login beside the first, move every reader, then remove the old one, +"through the adapter's existing create and remove", with "no consumer changes". A review showed that +claim false. In most providers the consumer's data is named after its login, and remove drops the data +with the login. Overlap as written would have deleted every consumer's database on its first +rotation. The mechanism has to be chosen on what the providers do. + +**What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md), +which [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) decided on +these findings. The identity budget in +[ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer +gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) +describes. + +**Documents.** + +- [01 — The providers](01-the-providers.md): the survey, one row per provider, and what it shows. +- [02 — The readers](02-the-readers.md): how a secret reaches a running process, and what already + recreates it. +- [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a + recommendation. + +**Finding, in one paragraph.** All nine credential providers already re-apply a consumer's password +in place on every create, and the controller's `rotate` command relies on that. It is a working +rotation with a stated window. Eight of the nine name the consumer's resource after its login, and five +destroy the consumer's data when they remove the login. The harness, keyed by login, would do the same +on any change of login. Only one backend holds two passwords on one login, and two more hold several +tokens. Eight backends can grant two logins the same rights over one resource; the ninth can give one +login a second token. So every provider can hold **two credentials** over one resource, but only after +each adapter separates *the consumer's resource* from *the credential that reaches it*. In postgres +that also means the resource belongs to a role no login owns. Administrative credentials are a +different case. They have one party and a fixed name, and five backends take them only at first +initialisation, so changing one needs the old and the new value at once. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md new file mode 100644 index 0000000..8ce34f1 --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md @@ -0,0 +1,103 @@ +# 01 — The providers + +Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`), +the client functions they call, and each definition's own credentials. The providers were found by +listing every definition that provides something and has a provisioner, not from memory. A first pass +of this survey worked from memory and missed one, mailu. + +**The provisioner harness** in `mesh-sdk` calls `create` for a consumer when its contribution appears +or changes (its login, password or values), and after the provisioner restarts. It calls `remove` for +a login it applied earlier in the same process that is no longer contributed. Its record of what was +applied is kept in memory and keyed by login. Two things follow: + +- a consumer whose derived login changes is removed under the old login and created under the new one, + in one pass; +- a contribution that disappears while the provisioner is down is never removed, and is left behind. + +## The credential providers + +`login` is the consumer's derived identity, which the adapter receives as `as` +([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). + +| provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource | +|---|---|---|---|---|---| +| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, but only through a role that cannot log in owning the database, with each login working as it. Otherwise whatever one login creates is its own, and dropping that login means handing its objects over first. Not done today | +| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. A user owning a schema cannot be dropped, and a login with an open session cannot. Not done today | +| mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today | +| redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login | +| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key; the bucket **only if empty**. A bucket holding objects is left, and the failure logged | yes, by removing the access key and adding it again, which leaves a moment with no key | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters | +| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host, with any queued messages, and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host | +| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. The MQTT client identifier is chosen by the consumer, not tied to the login; a duplicate one takes the older session over | +| mailu | a mailbox `login@domain`, unless the consumer contributes its own account name | the mailbox with its mail, for a login-named one; a contributed name is left for an operator | yes, the password is set when the user exists | no for the password; a user can hold several authentication tokens, per the backend's documentation | **no**: a mail user *is* its mailbox | +| gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team | + +## The other providers + +| provider | answers with | credential | +|---|---|---| +| umami | a website, found by its public name | none. The site id it makes has no way back to the consumer today | +| cloudflare-dns | a public name derived from `login` | none handed to the consumer; its own API token is an operator value | +| showcase | a route | none | +| mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today | + +verdaccio provides the npm registry too, and has no provisioner. + +## Each provider's own administrative credential + +| provider | identity | how the backend takes it | +|---|---|---| +| postgres | a fixed superuser | from a file **only at first initialisation** | +| mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception | +| mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty | +| mosquitto | a fixed admin client | seeded into the broker's dynamic-security file **once**; the seeding step skips when the file exists | +| lavinmq | a fixed admin name | per its own bootstrap code, **only on a first boot** with an empty data directory. No resource in the definition runs that bootstrap; what sets it on a running mesh is outside the catalogue | +| redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts | +| minio | a fixed root user | from a file, read when the server starts | + +**In five of seven, a new administrative value takes effect only through a command run with the old +one.** The credential file is mounted directly into both the server and the provisioner. So replacing +it recreates the provisioner, which then holds only the new value while the backend still expects the +old one, and the provisioner is locked out. That is worse than changing nothing. + +Every provider module also has its own bus account, an own secret, read at start. + +## What the tables show + +1. **Every credential provider already rotates in place.** All nine re-apply the password on the + same login each time `create` runs. The controller's `rotate` command relies on that: it replaces + the credential in the inventory and sends both ends in one push. Its own comments state the window, + between the provider applying and the consumer restarting, in which the consumer cannot + authenticate. +2. **Eight of nine name the consumer's resource after its login.** Only gitea separates them, + because an organisation owns the packages. A second login therefore has no resource of its own to + reach, and cannot share the first one's without the adapter granting it. +3. **Five of nine destroy the consumer's data when they remove the login**: postgres, mssql and + mongodb drop the database, lavinmq drops the virtual host with its queued messages, and mailu + deletes the mailbox with its mail. minio drops only an empty bucket, and redis leaves the keys. In + those five, *retire a login* and *delete the consumer's data* are one call. With the harness keyed + by login, a changed login triggers it too. +4. **One backend holds two passwords on one login** (redis). Two hold several tokens beside one + password (gitea and mailu). A rotation built on two secrets per login would work for three + providers out of nine. +5. **Eight of nine can give two logins the same rights over one resource.** Group roles in postgres, + database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared + policy in minio, a shared key prefix in redis, a shared team in gitea. mailu cannot, because its + user is its mailbox, but it can give one user a second token. So every provider can hold **two + credentials** over one resource, though not every one as two logins. No adapter does either today. +6. **Ownership is a trap in two backends.** In postgres whatever a login creates is that login's, so a + second login cannot alter the first one's tables, and the first cannot be dropped while it owns + them. The one-step way out deletes them. In mssql, a login cannot be dropped with a session open, + nor its user while it owns a schema. +7. **The administrative credentials have one party and a fixed name**, and five backends take them + only at first initialisation. The provisioner needs the old and the new value at once to change + them. Today nothing can give it both. +8. **A consumer's identity is already the resource's name.** The login is derived from the + assignment, which is a module on a node, so the current login and "the consumer" are the same + string today. A second login would need a new name. The resource can keep the one it has. + +## Seen on the way + +The redis configuration names no ACL file, so a consumer's ACL user exists only in memory. A restart +of the redis server erases every consumer's user. The provisioner does not create them again until it +restarts itself, because its in-memory record says they are done. That is not a rotation finding, but +it is a live fault, and it is recorded here so it is not lost. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md new file mode 100644 index 0000000..143f53d --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md @@ -0,0 +1,46 @@ +# 02 — The readers + +How a secret reaches a running process, and what makes the process take a new one. + +## No module watches a secret + +A search of every module's code in the catalogue found no file watching of any kind, and no +re-reading of a secret while running. **Every reader reads a secret when it starts.** There is no +consumer that takes a new value live, so every rotation that changes what a consumer presents ends in +the consumer restarting. + +## The host already recreates what read a changed file + +The node host records, for every long-running container, the digest of each file it read when it +was created: its env-files, and every file bind-mounted into it directly. When a digest changes, the +host recreates the container, even though its spec is otherwise unchanged. This is the fix for +[issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md). +It is on the host's main branch, while the issue is still recorded as located, not fixed. + +Two cases are deliberately left out and need `restart-on` in the definition: + +- a file read out of a **mounted directory**, because the host cannot know whether the service reads + it once or watches it (a route proxy re-reads its routes live; a provisioner polls what it receives); +- a **process** rather than a container. + +## What the catalogue does with it + +22 definitions declare a secret they receive. In 16 of them it reaches the service through a +rendered file, usually an env-file. That case the host already covers. 14 declare `restart-on` for +something. Whether each of the 22 is fully covered depends on how its secret travels: through an +env-file or a direct mount, which the host covers, or through a directory or into a process, which +needs `restart-on`. **That was not classified module by module.** It is the check to run before a +rotation mechanism relies on it. + +The count covers only the `secrets` field. The 49 modules with their own bus account, and 54 with any +own secret, are readers too, and their bus accounts are rotated like any credential two parties hold. +Their files are mounted directly, which the host covers, but the classification has to name them. + +## What this means for rotation + +- The *read at start* half of 0113's recipient model is already true, and mostly already handled by + the host. The restart is derived from the files a container reads, not declared per secret. +- Any mechanism, in place or overlapping, ends with the reader being recreated. What differs is + whether the credential it held until then still works. +- For a single-party secret, a module's own, the reader is also the only holder. There is nobody to + overlap with, and delivering the new file recreates the reader. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md new file mode 100644 index 0000000..5bfb7e5 --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md @@ -0,0 +1,79 @@ +# 03 — The options + +Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-readers.md). + +## A. In place, as today + +The vault makes a new value. Every applier re-applies it on the same login, which all nine +providers already do. Every reader is recreated by the host. + +- **Works with:** every provider, unchanged. It is what `rotate` does now. +- **Costs:** a window per consumer, from the provider applying to the consumer being recreated. They + are on different machines, and nothing orders them. A reader whose machine is unreachable from the + mesh but still reaches its provider stays locked out until the mesh reaches it again. +- **Admin credentials:** the natural form. The provider module is the only party, and it has to + apply the new value with the old one anyway (finding 6). + +## B. Two secrets on one login + +The applier adds the new password beside the old one, readers move, and the old one is removed. + +- **Works with:** redis natively, and gitea and mailu through tokens. **Not** with the other six, whose + backends hold one password per login (finding 4). +- **Verdict:** not a mechanism, a special case. Using it where it exists and something else + elsewhere is the "this way or that way" the design is trying to remove. + +## C. Two credentials per consumer, over one resource + +The consumer has two credentials and uses one at a time. The applier ensures the other with the new +value and gives it the same rights over the consumer's resource. Readers move to it, and then the old +credential is retired, which removes the credential only, never the resource. **What a credential is, +is the adapter's**: a second login for eight providers (finding 5), a second token on the same login +for mailu. The mesh sees one mechanism. + +- **Works with:** every provider, **after** each adapter changes: + - the resource is named after the consumer, not the login. Today the two are the same string (finding + 8), so existing resources keep their names, and the current login stays one of the two; + - the resource is owned by the resource, not by a login. In postgres that is a role no one logs in + as, which each login works as, and ownership of an existing database moves to it once (finding 6); + - both credentials get the same rights, over data and structure; + - *retire a credential* and *remove the consumer* become two operations. Today they are one call, and + in five providers that call destroys data (finding 3). The harness must key by consumer, so that a + changed login is not a removal. This is the whole of the danger, and it has to be split, whatever + else is chosen. +- **Costs:** + - every credential adapter changes; + - the harness learns the alternation and a confirmation per step, and rotation state has to live + somewhere that survives a restart, which the harness's memory does not; + - the second login's name must fit the tightest backend. That is 20 characters for a minio access + key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and + a suffix spends part of it; + - retiring an mssql login has to end its sessions first. +- **Gains:** no window. A reader that cannot be reached keeps a working credential until it can. +- **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is + no second party to overlap with. + +## Independent of the choice + +- **Split remove, and key the harness by consumer.** Retiring a credential, or a login changing, must + never be able to destroy a consumer's data. That holds under A too, because A's remove is the same + call. +- **Admin credentials are applied by their own provider**, using the old value, with the new one staged + beside it. Five backends take the value only at first initialisation. Replacing the file first locks + the provisioner out (finding 7). +- **Classify the readers** ([02](02-the-readers.md)), bus-account readers included, before relying on + derived restarts. + +## Recommendation + +- **Two-party credentials, consumer credentials and bus accounts: C**, because it is the only + mechanism every provider supports, and it closes the window instead of shortening it. Its prerequisite, separating the resource from the login and + retiring a login from removing a consumer, is worth doing on its own, because it removes a + data-loss path that exists today. +- **Single-party secrets (admin credentials, a module's own): A, staged.** In place, applied by the + provider that holds them, with the new value beside the old until it has taken. +- **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is + not replaced by a mechanism the providers cannot yet carry. + +This is two mechanisms, split by a property of the secret rather than by provider: whether it has one +party or two. Every provider is treated the same way for the same kind of secret. diff --git a/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md b/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md new file mode 100644 index 0000000..e89a743 --- /dev/null +++ b/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md @@ -0,0 +1,106 @@ +--- +topic: the tiers +status: accepted +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0075-two-stores-and-which-provides-what.md +--- + +# 109. A package registry seat is one per ecosystem, not one for all of them + +## Context + +Fixing `builder`'s consumption of `package-registry` tonight surfaced the shape ADR 0075 actually +left implicit. 0075 split `artifact-store` from `package-registry` and said the second is "an +ecosystem's own registry — npm, cargo, PyPI, Go" — but it defined one provision for all four, +not one each. + +What that produces, read from the manifests as they stand: + +- `gitea`'s `module.json` declares `provides: package-registry` once, and its `serves` block + carries exactly one path: `npm-path`. Nothing names a cargo or PyPI endpoint, though gitea's own + package API serves both. +- `builder` had, until tonight, a hand-written JSON fragment standing in for a real grant — + `{"provision": "package-registry", "from": "gitea", "at": "127.0.0.1", ...}` — because nothing + in the interface gave it a real one to ask for. The fragment named `npm-path` specifically; there + was nowhere to put a second ecosystem's endpoint even if one had been wired. +- The fix applied tonight declares `requires: ["artifact-store", "package-registry"]` and lets the + mesh mint the grant properly — correct for what exists today, but it is one seat standing in for + what should be several, the same conflation 0075 itself named and did not resolve for this + provision specifically. + +**The version-skew problem 0075 wrote down for the artifact-store/package-registry split repeats +one level down, inside "package registry" itself:** npm resolves by name and range from one +namespace, cargo from another, and a single grant conflates them exactly the way one store for +both digests and ranges would have. + +## Considered Options + +**1. One `package-registry` provision, gitea answers every ecosystem it can.** What exists today. +Simplest to grant — one credential, one binding, done once per consumer. Rejected: a consumer +that only ever needs npm still receives a grant shaped to cover cargo and PyPI, and there is no way +to hand off *only* npm to a different provider (verdaccio, say) without renegotiating the whole +provision for every consumer of any ecosystem. + +**2. One provision, parameterised by ecosystem.** `requires: package-registry` plus a declared +`ecosystem: npm` alongside it, still one interface. Rejected: the `provides`/`requires` refusal +mechanism this mesh already uses (two providers of one provision is a naming conflict until +resolved) would need to become conditional on a parameter it does not otherwise carry anywhere in +the mesh's resolution — a special case for exactly one provision, rather than the mesh's existing +mechanism applied again. + +**3. One provision per ecosystem — `npm-package-registry`, `cargo-package-registry`, +`docker-package-registry`, and so on, each independently `provides`/`requires`.** Chosen. + +## Decision + +**A package registry seat is one per ecosystem.** `npm-package-registry`, `cargo-package-registry`, +`docker-package-registry` — each its own provision, resolved, granted, and refused exactly the way +`artifact-store` and today's single `package-registry` already are. Adding an ecosystem is adding a +provision, not widening one. + +**Gitea may hold several seats at once.** Nothing here says gitea answers only one; ADR 0075 +already established that a provider may answer more than one named thing on one machine ("a mesh +running gitea for git and packages alongside a registry serving artifacts is an ordinary +arrangement"). Gitea fulfilling `npm-package-registry` and `cargo-package-registry` both is the +expected shape, not an exception. + +**Each seat's grant is independent.** A consumer that only needs npm holds only the +`npm-package-registry` grant. Moving that one ecosystem to a different provider — verdaccio, +named directly as the motivating case — means assigning `npm-package-registry` to verdaccio and +leaving every other seat exactly where it was. No consumer of `cargo-package-registry` observes +the change; no manifest naming `package-registry` broadly needs to be found and re-read. + +**`builder`'s fix tonight is the interim shape, not the target.** It correctly consumes the one +seat that exists today (`package-registry`, npm in practice). Splitting it becomes, later, +replacing that one line with the ecosystems `builder` actually uses — a manifest change, not a +redesign of how `builder` asks for anything. + +## Consequences + +- `gitea`'s `module.json` gains a `provides` entry per ecosystem it actually serves, each with its + own `serves` block (`npm-path`, a cargo path, a PyPI path) in place of the one `package-registry` + entry with a single `npm-path` inside it. +- `gitea`'s provisioner (`modules/gitea/provisioner/index.ts`) currently runs one `runProvisioner` + registration for `package-registry`; each seat needs its own registration, or one provisioner + keyed by which seat's `create`/`remove` fired — the mesh's `Provision` type does not yet carry + which named provision a call is for when a module answers more than one, and that is worth + checking before assuming the harness already supports it. +- Every consumer's `requires` moves from the one name to however many ecosystems it actually uses. + `builder` is the only known consumer today; widening later is one manifest line per module, not + a migration. +- `verdaccio`'s role sharpens: not "package-registry, an alternative for npm alone" (0075's phrasing) + but a named `npm-package-registry` *provider*, a straight swap against gitea's answer to the same + seat. +- Not solved here: whether `cargo-package-registry` and `pypi-package-registry` are needed at all + before something actually consumes them. This record names the shape; building unused seats is + its own decision. + +## References + +- [ADR 0075](0075-two-stores-and-which-provides-what.md) — the record this extends; defined + `package-registry` as the second provision without splitting it per ecosystem. +- `mesh-catalog modules/builder/module.json` — tonight's fix, the interim single-seat shape. +- `mesh-catalog modules/gitea/module.json`, `modules/gitea/provisioner/index.ts` — today's + single-provision, npm-only implementation. diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md new file mode 100644 index 0000000..3101efb --- /dev/null +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -0,0 +1,219 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0009-modules-and-the-graph.md +--- + +# 110. A seat is held by one assignment, from a closed set, and it may deliver a provision + +## Context + +[ADR 0009](0009-modules-and-the-graph.md) introduced claims: a module declares something +singular, at a scope, and a second holder is refused. [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) +named the foundation's three after their servers. That mechanism is enforced and works. What it +means has drifted, and four things are now true of it that no record says. + +**Any well-formed name becomes a seat by being claimed.** The controller's manifest check +refuses a claim only for a malformed name or an unknown scope. Nothing says which seats a mesh has. +The names in use were each invented by the module that claims them: `the-showcase`, +`the-build-machine`, `the-intrusion-prevention`. + +**Nothing can say what a mesh has, or who fills it.** There is no seat table and no command that +lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. +The only way to answer "which seats does this mesh have, and which module holds each" is to read +every manifest in two repositories, because the controller's own manifest lives in its own +repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's code, +because one module it ships has its manifest composed there. While this record was being prepared, +that enumeration was done by hand, and it missed both of the last two sources: eleven claims were +reported where there are thirteen. + +**A claim in a definition makes a module singular, not a role.** The store module's definition +claims `mesh-store`, so every assignment of it claims the seat, and a second store module on any other +node is refused. What is singular is *the store the mesh itself uses*, not postgres. Any module can +run on any node whose capabilities match, which is a core principle of the module system, and a claim +written into the definition breaks it for every module that claims anything. + +**Some seats are the mesh's one of something everyone consumes, and nothing uses that fact.** A +requirement for a mesh-scoped provision with more than one provider is refused until a person pins, +**per consumer node**, which provider to use. [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) +anticipates exactly that case, gitea and verdaccio both answering npm, and under today's resolution it +would mean a pin on every machine that builds anything. + +## Considered Options + +**1. Leave seats as free-form exclusion, claimed in definitions.** Rejected. The overview stays +unanswerable, a module that claims a seat can run on only one node, and a second provider of anything +costs a pin per consumer node. + +**2. Two concepts: seats for exclusion, and a new word for the mesh's one consumable thing.** +Rejected. Both mean "this mesh's one X". Every existing claim would first have to be classified into +one or the other, and the overview a person wants is one list, not two. + +**3. A seat is held by one assignment, from a closed set, and holding it may deliver a provision.** +Chosen. + +## Decision + +**The mesh defines a closed set of seats.** Each entry has a name, a scope, what holding it delivers +(if anything), and the decision that made it a seat. A seat outside the set is refused wherever it +is named. Adding a seat is a decision, for the same reason adding a shape to the host's vocabulary is +one: the set is what a person reads to learn what a mesh can have, and a name added without an +argument is a name nobody can explain later. + +**A definition says which seats a module *can* hold. An assignment says which it *does* hold.** The +store module can hold `mesh-store`, and it may be assigned to every node. Exactly one of those +assignments holds the seat, because that assignment said so. A second assignment saying so, at the +seat's scope, is refused. So a seat makes a *role* singular, never a module, and moving the role is +changing which assignment holds it, with no definition changed and nothing unassigned. + +**What the mesh knows about a seat's holder is what it knows about that assignment**: its node, the +node's settings for it, and what it serves. Holdings are not stored separately. The seat points at an +assignment, and a second record of the same fact would be a second thing to disagree with the first. + +**Holding a seat may deliver a provision.** A seat that delivers a provision may only be held by an +assignment of a module that provides it, at the seat's scope. + +**A requirement may name a seat, and then the seat's holder answers it.** Naming the seat asks for +*the mesh's* one, not for whichever provider is nearest, so the holder answers **even when another +provider runs on the consumer's own node**, and nothing is asked of anyone. Unheld, the requirement is +refused, naming the seat. A builder asks for the mesh's npm registry this way, and is served by the +holder of `npm-package-registry` wherever it runs, with no pin on any machine. + +**A requirement that names no seat resolves as [ADR 0084](0084-which-provider-serves-a-consumer.md) +has it**: a pin, then the provider on the consumer's own node, then the only provider. Where several +remain and none is local, **a person chooses when the module is assigned**. Assignment lists the +candidates, with the holder of a seat that delivers the provision suggested first, and records the +answer on the assignment as its pin. Without an answer the module is not assigned. This keeps +[ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with several answers is never +guessed. The choice is made either by the requirement naming the seat, or by a person at assignment, +and never silently by what happens to run nearby. That is the failure +[issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) names for the vault. + +**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, +the npm registry, git and the vault are each one per mesh by their own records, so their seats +deliver them. + +**The foundation's seats deliver nothing.** `mesh-controller`, `mesh-store` and `mesh-broker` name +which assignment the mesh *itself* uses: the controller, the store holding its records, the broker +carrying its bus. The store and broker modules may run on other nodes too, and a database or `amqp` +consumer that names no seat is served by co-location from whichever runs on its own node, the seat's +holder included. Were `mesh-store` to deliver, a consumer could name it and be sent to the store the +mesh keeps its own records in. That is not a store for consumers. + +**A seat may reserve its provision.** Where a second provider would break the reason the provision +exists, only an assignment holding the seat may provide it at all: the parser refuses a definition +that provides it without being able to hold the seat, resolution refuses an assignment providing it +without holding the seat, and a pin cannot choose anyone else: a requirement for it always names the seat. `secret` is the one +reserved provision. +The vault is one per mesh because a second *"would be a second place to lose"* +([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and a second `secret` provider is exactly +that, whether a pin chose it or not. + +**Seats are also informational.** The controller lists every seat in the set, what it delivers, and +which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh has +no X", not an error. + +**The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and they +name twelve seats because two alternative modules claim `the-resolver-configuration`. This record +admits every seat the catalogue and the controller claim today, so no definition is refused by it: + +| seat | scope | delivers | can be held by | made a seat by | +|---|---|---|---|---| +| `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-store` | mesh | — | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-broker` | mesh | — | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-vault` | mesh | `secret`, reserved | `mesh-vault` | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | +| `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | +| `the-catalogue` | mesh | — | `mesh-catalog` | this record | +| `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | +| `the-build-machine` | node | — | `builder` | this record | +| `the-dns-port` | node | — | `dnsmasq` | this record | +| `the-intrusion-prevention` | node | — | `fail2ban` | this record | +| `the-packet-filter` | node | — | `nftables` | this record | +| `the-private-network` | node | — | the controller's private-network module | this record | +| `the-resolver-configuration` | node | — | `resolv-conf` or `resolved-split-dns` | this record | +| `the-showcase` | node | — | `showcase` | this record | + +There are two additions. `npm-package-registry` is [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md)'s +seat, named after the provision it delivers, as 0079 named the foundation's seats after what they are. +A gitea assignment holds it. verdaccio provides the same provision and cannot hold the seat, so it is +the second provider this record exists to make harmless. Moving npm to it would take a definition +saying it can hold the seat, and then an assignment saying it does. + +`mesh-vault` answers [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md). The vault is one +per mesh ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and until now that was enforced by +nothing. The seat is named after its server, by the 0079 convention. + +`the-dns-port` is listed as delivering nothing, although `dnsmasq` provides `wildcard-resolution`. +That provision is node-scoped and answered on the machine, so no preference between providers arises. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0009](0009-modules-and-the-graph.md): a claim in a definition says a module *can* hold a seat; + the assignment says it does. +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) and + [ADR 0078](0078-the-store-and-broker-are-modules.md): "a mesh runs one postgres and one + lavinmq" becomes one holder of `mesh-store` and one of `mesh-broker`. The store and broker modules may + run on other nodes. +- [ADR 0084](0084-which-provider-serves-a-consumer.md): a requirement may name a seat, which its holder + answers; and where several providers remain and none is local, the choice is asked when the module is + assigned and recorded as a pin, rather than refused until someone pins it. +- [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md): one provision per package + ecosystem stands. Where 0109 says *seat*, it means that provision. Only `npm-package-registry` is + also a seat in this set. A cargo or docker registry becomes one by a record, as any seat does. + "Gitea may hold several seats" reads: gitea may provide several ecosystems, and hold the seat of + each one that is a seat. Moving npm to verdaccio is not "assigning `npm-package-registry` to + verdaccio". It takes verdaccio's definition saying it can hold the seat, and then an assignment + holding it. +- [To-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): the same two changes, in the design + that describes choosing a provider. +- [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): genesis assigns the foundation's + store, broker and controller holding their seats, where their definitions claim them today. + +## Consequences + +- The controller carries the set in code. A test asserts its size, and that every entry names the + record that made it a seat, so changing the set means finding the argument rather than a number. +- An assignment gains the seats it holds. Genesis assigns the foundation's store, broker and + controller holding their seats, where today their definitions claim them. +- Manifest validation refuses an unknown seat, a seat named at the wrong scope, and a delivering seat + named by a module that does not provide the provision. Resolution refuses a second holder, and an + assignment holding a seat its module cannot hold. +- Resolution answers a requirement naming a seat with its holder. Assignment asks a person where + several providers remain, suggesting the seat's holder first, and records the answer as a pin. A + provider record gains the module it came from. +- A `seats` command lists the set with each seat's holder, derived from assignments. +- **What got harder:** a module wanting a new singular role can no longer invent a name. It needs a + record. And an assignment has one more thing to say. Both are the point. +- **Not changed:** the controller's seat placeholder stays as it is. It exists so the controller can + reach a foundation it made before any module existed. + +## How it is checked + +| Rule | Checked by | +|---|---| +| The set is closed, and every entry names its decision | A controller unit test asserts the set's size and a non-empty decision for every entry. | +| A seat outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat named by a module that does not provide it. | +| Every module in use names a seat in the set | A controller test parses every catalogue manifest and fails on any refused seat. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | +| A seat is held by one assignment, not by a module | A resolution test: the store module assigned to two nodes resolves, with one assignment holding `mesh-store`; a second assignment asking to hold it is refused. | +| A requirement naming a seat is answered by its holder | Resolution tests: two providers with the seat held; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | +| An assignment holds only a seat its module can hold | A resolution test: an assignment holding a seat its definition does not name is refused. | +| Every seat is listed with its holder | A `seats` command test: every seat in the set is listed with its scope, what it delivers and its holder, and an unheld seat is listed as unheld. | +| Several providers and none local is a person's choice | An assignment test: the candidates are listed with the delivering seat's holder first; the answer is recorded as a pin; with no answer the module is not assigned. | +| The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`; a requirement naming `mesh-store` is refused, because it delivers nothing. | +| A reserved provision has no other provider | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`; resolution refuses an assignment providing it without holding the seat, and a pin on a `secret` requirement. | + +## References + +- [ADR 0009](0009-modules-and-the-graph.md): claims, scopes, and "refused, never guessed" +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a seat named after what it is +- [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md): the per-ecosystem registry seats +- [ADR 0084](0084-which-provider-serves-a-consumer.md), [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): + pins, co-location and refusal +- `mesh-controller internal/catalogue/resolve.go` (`checkClaims`, the brokered branch of `Resolve`), + `internal/catalogue/manifest.go` (claim validation), `cmd/mesh-controller/plan.go` (holdings) diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md new file mode 100644 index 0000000..5cb4f14 --- /dev/null +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -0,0 +1,105 @@ +--- +topic: building it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0069-a-module-is-a-repository-and-a-path.md +--- + +# 111. A build source is on the mesh's git seat, or it is an external repository + +## Context + +[ADR 0069](0069-a-module-is-a-repository-and-a-path.md) made a module a repository, a path and a +ref, and the controller records all three against the module so it can rebuild it and say when +its source has moved ahead. **The repository is recorded exactly as a person typed it.** `build +` hands the string to a build machine, which runs `git clone` on it, and the same string +becomes the module's recorded source. + +**So a self-hosted forge's address is written into every module built from it.** The mesh runs its +own forge, and most of what it builds lives there. Every one of those modules carries the forge's +scheme, host and port in its recorded source. Move the forge to another machine, or change the port +it is published on, and every recorded source is stale at once. Nothing notices until a rebuild fails +to clone. + +**And nothing names the mesh's git at all.** gitea serves git over HTTP and over SSH, and the mesh's +vocabulary contains neither. No provision, no `serves`, no seat, as the forge survey +([research 013](../01-RESEARCH/013-the-forge-and-the-registries/the-survey.md)) found. The only trace +is a label on its public route, which the mesh is explicitly not meant to interpret. + +**External repositories are ordinary, and must stay so.** An application the mesh hosts may live on a +public forge. Building it from its URL works today and must keep working unchanged. + +## Considered Options + +**1. Keep recording literal URLs.** Rejected. It is the problem: the forge's address copied into +every module built from it. + +**2. Recognise a self-hosted source by matching its URL against the forge's current address.** +Rejected. It infers the kind of source from the shape of a string, and the inference fails in the +one case it exists for: after the forge moves, old URLs no longer match anything. + +**3. Two explicit forms: a repository on the holder of the `git` seat, or an external URL.** Chosen. + +## Decision + +**The mesh has a `git` seat.** It is mesh-scoped and delivers the `git` provision, per +[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md). Its holder provides `git`, +serving how a repository on it is cloned: the scheme and the port. A gitea assignment holds it. + +**A source is on the git seat, or it is external, and the mesh records which.** + +- `build --self /` builds from a repository on the seat's holder. The recorded + source is the repository's path on that holder, and the seat it is on. **It never contains an + address.** At the moment of building, the controller composes the clone URL from where the + holder runs and what it serves for `git`, so a moved forge changes nothing recorded. +- `build ` is unchanged: an external repository, recorded and cloned exactly as given. GitHub + and GitLab are the ordinary cases. + +**An unheld seat refuses self-hosted builds and nothing else.** With nobody holding `git`, `build +--self` is refused, naming the seat and saying what would hold it. External builds are unaffected. A +mesh without a forge of its own builds from external repositories only, and says so rather than +failing to clone. + +**The build machine is not told the difference.** It receives a URL either way. Composing the URL is +the controller's job, because only the controller knows where the seat's holder runs. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0069](0069-a-module-is-a-repository-and-a-path.md): a module's repository is recorded either as + a path on the `git` seat or as an external URL, never as an address of the mesh's own forge. +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): the closed set gains `git`, + mesh-scoped, delivering `git`, held by a gitea assignment. + +## Consequences + +- The controller's inventory gains a column saying which seat a source is on. It is empty for + every module recorded before this, which is correct: they were all recorded as literal URLs. +- `build`, `build --behind` and `build --dry-run` resolve a seat source before asking a builder. The + recorded source keeps the seat form; the build log keeps the URL that was actually cloned, because + that is what happened. +- gitea can hold `git` and provides it, serving HTTP clone on its web port; the forge's assignment holds the seat. +- **Not decided: credentials for private repositories.** The mesh's own repositories are public, and + clone without one. A private repository still works only if the build machine's own git + configuration authenticates, exactly as before. Delivering a clone credential through the `git` + provision's grant is the obvious next step, and it is its own decision. +- **Not changed:** modules already recorded from the forge keep their literal URLs until they are + rebuilt with `--self`. Rewriting them in place would be the URL-matching this record rejects. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A seat source records no address | A controller test resolves a seat source and asserts the recorded repository is the path alone. | +| The URL comes from the holder | A test composes the clone URL from a holder's node address and served `git` scheme and port, and a second where the port was moved on the node. | +| An unheld seat refuses self-hosted builds only | A test with no holder: `--self` is refused naming the seat; an external URL passes through unchanged. | + +## References + +- [ADR 0069](0069-a-module-is-a-repository-and-a-path.md): a module is a repository, a path and a ref +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): seats, and a seat delivering a provision +- [Research 013](../01-RESEARCH/013-the-forge-and-the-registries/the-survey.md): git is served and declared nowhere +- `mesh-controller cmd/mesh-controller/build.go`, `internal/builder/builder.go` diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md new file mode 100644 index 0000000..644268b --- /dev/null +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -0,0 +1,192 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0046-a-module-configuration-is-its-assignments-not-its-manifest.md +--- + +# 112. A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves + +## Context + +[Issue 119](../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md) found +**789 host-path strings in 70 of the catalogue's 71 module definitions.** Each definition chooses +where on the machine its directories, mounts, bindings, secrets, env-files and received files live, +and often repeats that path in an environment variable or in code. Mounts are checked against what +the definition declares ([ADR 0091](0091-a-mount-is-declared-three-ways.md)); nothing checks the +copies. The issue records what that has already allowed: + +- a provider that would provision nobody without a word; +- a contributions file that carries host paths into containers, so every provider must mount its + grants directory at the identical path; +- defaults in code that disagree with their own manifests; +- every identity keyed by the module's name, which is why one module cannot be assigned to one node + twice. This record keeps that, and says so below. + +**Paths are one case of a wider pattern.** A module gets what it needs through at least six separate +mechanisms today, each with its own syntax and its own failure modes: + +- provisions, read through bindings; +- settings on the assignment ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)); +- ports the mesh assigns ([ADR 0038](0038-the-mesh-assigns-the-port.md)); +- machine facts a manifest asks for; +- secrets, either minted or accepted from an operator; +- literals carried in the definition itself. + +The mesh has already unified parts of this. Ports became the mesh's rather than the module's (0038), +configuration became the assignment's (0046), and which provider serves a consumer became the +assignment's choice ([ADR 0084](0084-which-provider-serves-a-consumer.md)). What remains is the +concept that joins them. + +## Considered Options + +**1. Keep host paths in definitions, and check that every copy agrees.** Rejected. It checks the +agreement of something that should not be there. A definition still could not follow its data to +another disk, or be adopted onto a machine whose data is already somewhere. + +**2. Keep the separate mechanisms, and add directories as a seventh.** Rejected. It fixes paths and +keeps the pattern that produced them: each mechanism is resolved, validated and refused differently, +so a module author learns six systems and a reviewer checks six kinds of gap. + +**3. One concept: a module requires, and the mesh resolves every requirement against a contract.** +Chosen. + +## Decision + +**A module definition is node-agnostic and mesh-agnostic.** It names no node, no mesh and no host +path. **Everything a module needs is a requirement**: a name, a contract saying what the module may +read from it, and which kind of provider answers it. + +**Installing a module on a node resolves every requirement, or refuses.** A refusal names each +unresolved requirement and what could answer it, all at once. + +**There are four kinds of provider, and the set is closed:** + +| provider | answers | today's mechanism it replaces | +|---|---|---| +| **another module** | a database, a bucket, a vhost, a secret, a route | provisions and bindings | +| **the node's host** | a directory, a port, facts about the machine | resource paths, `${port:}`, `${machine:}`, facts | +| **the mesh** | the module's identity and names, and the delivery of every answer | derived logins and generated names; the controller's delivery | +| **the operator, through the assignment** | a value a person chooses that is not secret: a public name, a greeting, a number of workers | settings, carried literals | + +A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and +[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say. A requirement naming a seat is +answered by its holder. Otherwise it is a pin, then co-location, then the only provider, and where +several remain, a person chooses at assignment and the choice is recorded as a pin. +A host provider is always the module's own node, because a host path or a port means nothing on any +other. An operator value is the assignment's, or the requirement's default, or unresolved. + +**A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a +default. It needs no provider module, no grant and no credential. + +**Every shared secret is a `secret` requirement, answered by the vault**, with no exception by kind +([ADR 0113](0113-the-vault-makes-every-secret.md)). A private key is made where it is used and is not +a requirement. An external API key an operator chooses is no +different: the operator delivers it to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), +and the module requires a `secret` like any other. A provider that needs a secret for a consumer +requires it from the vault, like any consumer, and answers with resources and data. The mesh carries +every answer back. + +**A directory is a host provision.** Its contract is the owner and mode the module needs, including +the owner its image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). +It carries no persistence flag. A directory is kept while it holds anything +([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), which refused a `keep` flag for good +reason), and data that is disposable is not a directory but a named volume (0107). *Where* it is on +the machine is the assignment's. A node has a default layout, and an assignment may place a directory +elsewhere: on a second disk, or where an adopted machine's data already is +([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). + +**An operator's shared data stays an `access`** ([ADR 0051](0051-shared-data-is-the-operators.md)), +not a directory. 0051 rejected giving a directory an operator owner, because the mesh must never +create, chown or remove such data, and that stands. Only where its location is written changes: the +module requires read or read-write access, and the assignment says where the data is. + +**Inside a container, a module sees its own paths.** The definition says where the image expects each +directory. The mesh mounts the assignment's location there. No host path is ever a value a process +reads, and the mesh's own files (answers, contributions) name nothing by host path, so a provider needs +no mount at a machine-identical path. + +**A module is assigned at most once to a node.** An assignment is a module on a node, and that pair is +its identity: its directories, containers, login, broker account and settings are keyed by it, as +they are today. A module may run on many nodes, and one of those assignments may hold a seat +([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). Running the same module twice +on one machine is not supported. The cases that seemed to need it, such as two stores of one engine or +two stages of one application, are different modules, or the same module on different machines. The +line is drawn because every identity in the mesh is already a module on a node, and a second +instance would have to rename all of them. + +**What must stay singular stays so** by a seat, or by an operator value colliding: a public name +already held by another assignment is refused like any other singular thing. + +**The foundation's first secrets are delivered, then adopted.** Genesis generates them before the vault +can run and hands them to the vault once it is installed, and from then on they are answered the same +way as every other secret ([ADR 0113](0113-the-vault-makes-every-secret.md)). + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings become + operator requirements on an assignment. +- [ADR 0038](0038-the-mesh-assigns-the-port.md): a port becomes a host requirement. What 0038 decided + is unchanged; it is the first case of this rule. +- [ADR 0051](0051-shared-data-is-the-operators.md): an access keeps its shape and its semantics; its + path moves from the definition to the assignment. +- [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved requirement, + checked as resolved rather than as a path the definition declares. +- [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): the step that builds and runs a + store module as a database provider, beside the foundation's store on the same node, would run the + store module twice on one node. The adopted store module ([ADR 0078](0078-the-store-and-broker-are-modules.md)) + holds `mesh-store` and serves that node's database consumers by co-location, so there is no second + one. +- The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to a + requirement answered by any of the four providers, and *requirement* and *contract* are added. None + of it lands while this record is only proposed, because the glossary is the authority on the words + in use, not on words under review. + +## Consequences + +- **Every definition changes.** 70 of 71 name host paths today, and most use at least three of the + mechanisms this replaces. The change is mechanical for most. The design has to say how existing + modules move without their data moving: an adopted or already-running assignment is placed where + its data already is. +- The controller resolves every requirement at assignment and refuses unresolved ones. The host + answers directories and ports. The settings, placeholders, facts and bindings that exist today + are retired as separate mechanisms, once nothing uses them. +- Identity stays a module on a node. Nothing is renamed, and a login still fits the tightest backend + as [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md) arranges. +- **What got harder:** one module cannot run twice on one machine; a second stage or a second store + of one engine is a different module or a different machine. And a definition no longer says where + a module's data is on a machine, or what a setting's value is. The assignment does, and `plan` shows + it. That is the point, and it is also a real loss of at-a-glance legibility, which the overview has + to give back. +- **Not decided here:** the syntax a definition reads a requirement's fields with; a node's default + layout; the order in which the mechanisms are retired. [To-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md) + proposes all three. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A definition names no host path | A catalogue test: the host side of every mount, and every resource location, is a requirement rather than an absolute path. A declared list of exceptions shrinks to empty as definitions move. | +| No host path is a value a process reads | A catalogue test: every absolute path in a container's environment or env-files lies on the container side of one of its mounts, or is declared the image's own. A second test finds literal paths in module code used as fallbacks for an environment variable. | +| A definition names no node and no mesh | The parser has no field that names a node; a node is named only in an assignment. A catalogue test finds no domain name in any definition value. | +| Every requirement has one of the four providers | The parser refuses a requirement whose provider kind is not one of the four. | +| Installation resolves every requirement | A resolution test with one requirement unanswered: refused, naming it and what could answer it. | +| A host requirement is answered on its module's own node | A resolution test: an assignment placing a directory or a port on another node is refused. | +| A module is assigned at most once to a node | A resolution test: assigning a module to a node that already runs it is refused, naming the existing assignment. | +| A public name already taken is refused | A resolution test: a second assignment asking for a public name another holds is refused, naming the holder. | +| An adopted assignment is placed where its data is | An adoption test: the directory resolves to the data's existing location, and nothing is moved. | + +## References + +- [Issue 119](../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md): the evidence +- [ADR 0038](0038-the-mesh-assigns-the-port.md), [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), + [ADR 0084](0084-which-provider-serves-a-consumer.md): the parts already unified +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): which module provider answers +- [ADR 0113](0113-the-vault-makes-every-secret.md): the vault makes every secret, and how answers travel +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md): the operator as a provider +- [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries, and what it must not diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md new file mode 100644 index 0000000..a8ae287 --- /dev/null +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -0,0 +1,277 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +--- + +# 113. The vault makes every shared secret, a provider makes resources and data, and the mesh carries both + +## Context + +**A shared secret comes into being many different ways today**, counted across the catalogue and the +controller on 2026-09-25: + +| kind | made by | used by | +|---|---|---| +| a credential between a consumer and a provider | the controller | 16 modules, and 3 more for model access, counted below | +| a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules | +| a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder | +| an enrolment token | the controller | every node joining | +| a `secret` from the vault | the controller mints it, and the vault only records it ([ADR 0085](0085-a-secret-is-a-provision.md), as amended) | 6 modules | +| a value an operator accepts | a person ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) | where accepted | +| a licence for model access | a separate controller context with its own store | model consumers | +| the foundation's root secrets | genesis, sealed to the operator key | the foundation | + +**The vault was built to end the second row, and did not.** ADR 0085 says a module's own secret +*"stops being a generated value that nothing owns"*. 54 modules still use one, and 6 use the vault. +The replacement was added and the old path was never retired. + +**ADR 0085 considered and rejected making the vault the only maker**, because *"the controller must +mint in order to deliver any provision — the vault's own credential among them"*: the vault cannot +make the credentials that exist before it does. That objection is real, and this record has to answer +it rather than step around it. + +**Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in +which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so; +a container fed by an env-file was not recreated when that file changed +([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md), +since fixed in the host); and some secrets are read only when a service first initialises, where a restart changes nothing. + +**And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) +left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics +provider's site id and the DNS provider's record have no way back, and say so in their code. + +## Considered Options + +**1. Keep the controller minting, and tidy the paths.** Rejected. The paths are the problem: each is +made, kept, rotated and audited differently, and tidying keeps them all. + +**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. Generation +becomes uniform, but custody stays spread over every provider's machine, so rotation, audit and the +operator's break-glass copies cover only some secrets. Each SDK language needs its own implementation. + +**3. Raise the vault first at genesis, so it makes even the first secrets.** Rejected. The vault is +built on the shared runtime base, which the installation makes only after the store, the broker and +the controller exist, and the vault learns what to answer from the controller over the bus. Running +it first means reordering the whole installation and giving the vault a second way of being asked. + +**4. The vault makes every shared secret; genesis delivers the first ones to it.** Chosen. It answers +0085's objection with a mechanism the mesh already has: a value delivered to the vault. + +## Decision + +**There are two kinds of secret, and each has one rule.** + +- **A shared secret** is a value more than one party must hold: a password, a token, an API key. **The + vault makes every one.** Nothing else in the mesh generates a shared secret. +- **A private key** is made where it is used and never leaves: a node's sealing key, the operator's + key, the mesh's certificate authority. This is not a second way of making secrets. A private key any + other party ever held would no longer be private. + +**Every shared secret is a `secret` requirement, answered by the vault:** + +- a **credential between a consumer and a provider**. A provision's contract declares *for each + consumer, one secret*, and resolution expands it into one requirement per consumer. So gitea + requiring a database makes the database's provider require a secret for gitea, and the vault + answers it. The provider's own code does not change: it is handed a login and a password, as today; +- a module's **own secret**. `own-secrets` is retired; +- every **broker account** on the mesh's bus: a module's, a node's host's, the builder's, the + controller's. The broker holding `mesh-broker` carries the mesh's bus + ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates + each account from the vault's secret, like any provider. The controller no longer creates accounts, + and there is no separate command to forget; +- an **enrolment token**. The vault makes it; the operator receives the token, sealed to the operator + key, to hand to the joining machine; the controller receives only what it needs to verify it, never + the token itself; +- a **secret operator value**, such as an external API key, which the operator delivers to the vault + ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these. + What the licences context adds, refreshing a token, is provider behaviour, decided in its own record; +- a **secret a backend issues itself**, such as an API token a forge hands out exactly once when asked. + The vault cannot make that value. The module that received it delivers it to the vault, which keeps + it and provides it like any other; rotating it means asking the backend again. + +**The controller is a module, and takes the same path.** Its store logins (inventory, identity and +licences) and its bus accounts are own secrets of its definition today, and become `secret` requirements +of that definition like any module's. **A node's host is the one party with no definition.** Its bus +account is a requirement the mesh makes for each enrolled node, answered by the vault, sealed to that +node and carried like any other. It is the only requirement not written in a definition, because the +host is what runs definitions. + +**Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat. +The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a +`secret` requirement anywhere else, because there is nowhere else. + +**A secret has recipients, and the vault delivers to each.** The database credential has two: the +provider, which *applies* it by creating the login, and the consumer, which *reads* it and presents it +when it connects. The vault hands the value to the mesh sealed to each recipient's node. The controller +and the broker carry sealed values they cannot open. + +**Genesis delivers, and the vault adopts.** The vault is built on the shared runtime base, which the +installation makes only after the store, the broker and the controller are running +([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). So the vault is installed **as soon +as that base exists**, before any other module built on it, and everything needed before that moment is +generated by genesis: + +- the store's superuser, and the broker's admin in the hashed form the broker needs; +- the bus accounts of the temporary and permanent controller (its account and the broker-management + login), the control-node's host, the builder, the broker's own provisioner and the vault; +- the controller's three store logins (inventory, identity and licences), and the first enrolment + token. + +Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's +admin, as the controller does today. Genesis seals each value twice: to the control-node's key, so that when the +vault is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not +as an operator's, with nobody present; and to the operator key, as the break-glass copy +[ADR 0085](0085-a-secret-is-a-provision.md) keeps of every root secret. The first enrolment token reaches +the operator the same way. +That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), +and these are, because the vault can make their replacements. The broker's provisioner then adopts the +accounts genesis created. From then on the vault makes every shared secret, and genesis has made its +last one. + +**Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker` +seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the +values it needs are delivered, not made by a vault that is not there. They come from the operator-sealed +copies, which the operator opens. The vault keeps a copy of every secret sealed to the operator key +(0085), so nothing the mesh relies on exists only inside the vault. That is a break-glass procedure, +stated and checked, never an ordinary assignment. + +**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may +answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to +the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer +is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). + +### Rotation + +**Who asks and who makes are decided here; the mechanism is not.** A rotation is asked of the vault, +by an operator or by the vault's policy, such as a maximum age in the requirement's contract, and the +vault makes the new value. A delivered value the vault cannot replace, such as an external API key, is +not rotated by the vault: rotating it means an operator delivering a new one. A secret a backend +issued is rotated by the module that holds the backend asking it again and delivering the new value to +the vault. + +**Each recipient takes a new value one of two ways, marked per recipient:** + +| recipient takes it by | example | what happens on rotation | +|---|---|---| +| **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it | +| **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed | + +The marking is per recipient, not per secret, because one secret has recipients of both kinds. A +provision's contract marks its provider's side, which applies. A consumer's side is read at start +unless its requirement says otherwise. The broker's contract marks the host's bus account the same +way: the broker's provisioner applies it, and the host reads it. +Every module in the catalogue reads its secrets at start, and none watches them +([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a +backend takes only when it first initialises is marked applied, and its provider's provisioner makes +the change using the old value. Where no provisioner can make it, the requirement is marked **not +rotatable by the mesh**, and a rotation request is refused, saying why, rather than restarting a service +that would carry on with the old value. + +**How old and new change over is decided in [ADR +0114](0114-a-shared-credential-rotates-over-two-credentials.md).** Three mechanisms were measured against +every provider's code in [research +016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): in place, as the controller's +`rotate` does today; two secrets on one login; and two logins over one resource. Its findings bound the +choice: + +- every credential provider already re-applies a password in place, so today's rotation works, with a + window in which a consumer cannot authenticate; +- eight of nine name the consumer's resource after its login, and five destroy the consumer's data when + they remove the login. **No mechanism may retire a login through today's remove**, because in those + five it deletes the consumer's data; +- one backend holds two passwords on one login, and two more hold several tokens; +- every provider can hold two credentials over one resource, eight as two logins and one as two tokens, + once the adapter separates the resource from the credential. + +On those facts, 0114 rotates a credential two parties hold over two credentials, rotates one a single +party holds in place, staged, and separates retiring a credential from removing a consumer. + +## What this changes in earlier records + +On acceptance, each of these is superseded or amended by this record, not edited: + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: the controller + no longer mints a provider's credential; the vault makes it. That a provider is handed its + credential and seals nothing stands. +- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own + secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the + first secrets. Genesis seals its values to the control-node's key as well as to the operator key, so + the controller can deliver them unattended. "The vault stores no plaintext, ever" and the + operator-sealed break-glass copies stand. +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret + to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own, + so 0092's rule that an operator's value is never replaced does not apply to them. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker + account is created by the broker's provisioner, not the controller. Its scoping stands. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), + [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each + requirement says whether its recipient applies it or reads it at start, and the changeover is + [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s; the vault is installed as soon as the + shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker. +- [To-be 07](../03-DESIGN/01-to-be/07-the-foundation.md) is amended: genesis seals its values to + the control-node's key as well as the operator key. +- [To-be 12](../03-DESIGN/01-to-be/12-a-module-repository.md), [to-be 16](../03-DESIGN/01-to-be/16-module-coverage.md) + and [to-be 18](../03-DESIGN/01-to-be/18-building-a-module.md) are amended: `own-secrets` is retired from the + manifest they describe. +- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at + start*, and *reserved provision*, once this record is accepted. +- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + is a prerequisite, and its fix is in the host: a container is recreated when a file it read at + creation changes. The issue is to be recorded as fixed, and derived restarts rest on it. + +## Consequences + +- **The vault is on the path of every new or rotated shared secret.** Today the controller holds that + place, on the same node. A secret can no longer be made while the vault is down. +- Resolution expands per-consumer requirements from a provision's contract. The contract declares + them, never the provider's code, so what a provider requires stays predictable from the catalogue. +- A data provider's adapter gains a return value. What a credential provider's adapter must change for + rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s. +- The broker's provisioner gains every bus account, and the controller loses five separate places it + generates a secret today. +- 54 modules move from own secrets to vault requirements. Six provider clients export a password + generator nothing uses any more; it is removed, so no module can quietly start minting again. +- The installation changes order: the vault is installed as soon as the shared runtime base exists, + before any other module built on it. +- **What got harder:** a secret some services read only at first start can no longer be "rotated" by + a restart that quietly changes nothing; it is refused instead, or applied by its provisioner. And + moving the vault or the broker is a procedure, not an assignment. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls. Exempt are the vault itself, and randomness that is not a secret any other party holds, such as a password hash's salt, each named in a declared list. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. | +| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. | +| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. | +| Genesis's values reach the vault unattended, and the operator keeps a copy | An installer test: each of genesis's values is sealed to the control-node's key and to the operator key; the controller delivers the first to the vault when it is installed, with no operator step; the operator's copy opens only with the operator key. | +| Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. | +| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. | +| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. | +| A rotation never destroys a consumer's data | A provider test per credential provider: rotating a consumer's credential leaves its resource and data intact. It fails today for no provider, because rotation is in place; it guards whichever mechanism replaces it. | +| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. | +| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. | +| Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | +| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | +| An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. | +| Restarts are derived from how a secret is read | A host test: a secret read at start recreates the container that read it at creation, through an env-file or a direct mount; an applied secret restarts nothing. A catalogue test: a secret that reaches a process, or a file in a mounted directory, has `restart-on` naming it. | +| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | + +## References + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, + and the return path it left open +- [ADR 0085](0085-a-secret-is-a-provision.md): the vault, and the objection this record answers +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), + [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): delivered values, + identity, and broker accounts +- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md): + everything a module needs is a requirement +- [Issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md), + [issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): what fails today diff --git a/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md new file mode 100644 index 0000000..6cef363 --- /dev/null +++ b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md @@ -0,0 +1,245 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-26 +deciders: jochen +reconstructed: false +extends: 0113-the-vault-makes-every-secret.md +--- + +# 114. A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached + +## Context + +[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the +vault's policy) and who makes the new value (the vault). It leaves open how old and new change over. +[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every +credential provider in the catalogue against its code. There are nine: + +- **all nine re-apply a password in place**, on the same login, every time they run. The controller's + `rotate` command relies on that, and states the window it leaves: between the provider applying the + new value and the consumer restarting with it, the consumer cannot authenticate; +- **eight of nine name the consumer's resource after its login**: a database, a bucket, a virtual host, + a key prefix, a topic prefix, a mailbox. Only the forge's npm registry keeps them apart, because an + organisation owns the packages; +- **five of nine destroy the consumer's data when they remove its login**: postgres, mssql and mongodb + drop the database, lavinmq drops the virtual host with its queued messages, and mailu deletes the + mailbox with its mail. In those adapters, *retire a login* and *delete the consumer's data* are one + call. minio drops a bucket only if it is empty. The provisioner harness makes it worse: a consumer + whose derived login changed is removed under the old login and created under the new one, in one pass; +- **one backend holds two passwords on one login** (redis), and two hold several tokens beside one + password (the forge and mailu); +- **eight of nine can give two logins the same rights over one resource**. mailu cannot, because a mail + user *is* its mailbox. It can give one user several tokens. In postgres, a second login is not enough + on its own: objects belong to whichever login created them, so the resource must be owned by a role of + its own; +- **an administrative credential has one party and a fixed name.** The provider module both applies it + and reads it. Five backends take it only at first initialisation: postgres, mssql, mongodb, mosquitto + and lavinmq. Their credential file is mounted directly into both the server and the provisioner, so + replacing the file recreates the provisioner holding only the new value, which the backend does not + know yet. The provisioner is then locked out; +- **no module watches a secret.** Every reader reads at start, and the host recreates a container when + a file it read at creation changes + ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). + +A first draft of 0113 chose to overlap old and new "through the adapter's existing create and +remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on +what the providers do, and the danger has to be closed whichever mechanism is chosen. + +## Considered Options + +**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials +two parties hold. The window cannot be closed, only shortened, and the two ends are on different +machines with nothing ordering them. For an administrative credential, it locks the provisioner out. + +**2. Two secrets on one login.** Rejected as the mechanism. It works for three providers out of nine, +and using it there and something else elsewhere would put the difference in the mesh instead of in the +adapter. + +**3. Two logins over one resource.** Rejected as the mechanism. It works for eight of nine, and not for +mailu. + +**4. Two credentials over one resource, with the adapter choosing what a credential is.** A credential +is what a consumer presents, a login and a secret. The mesh alternates between two of them. Each adapter +makes the second one the way its backend can: a second login for eight providers, a second token on the +same login for mailu. A credential a single party holds is staged in place instead, and retiring a +credential is separated from removing a consumer before either is used. Chosen. + +## Decision + +### Retiring a credential never removes what it reached + +**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its +database, bucket, virtual host, key or topic prefix, mailbox) and a *credential* that reaches it. +They get separate operations: + +- **ensure the resource**, named after the consumer; +- **ensure a credential** with a value, holding the consumer's rights over its resource; +- **retire a credential**. Anything it owns moves first to the resource's owner, and any session it has + open is ended. Then the credential is removed, and nothing else; +- **remove the consumer**, which is what removes the resource, and retires every credential it has. + +**Remove the consumer runs only when the consumer no longer requires the provision from this provider.** +That happens when its assignment goes, when its definition drops the requirement, or when re-resolution +sends it to another provider. It never runs because a login or a value changed. The harness keys what it +applied by the consumer, not by the login, so a changed login is a credential change and never a removal. +What removing a resource does with the data in it stays +[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s, and re-resolving to another provider +moves no data. + +**The resource is named after the consumer, and owned by the resource, not by a login.** A consumer's +identity is derived from its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), +and today its login is that same string, so **no existing resource is renamed**. Where a backend makes +whatever a login creates the login's own, as postgres does, the resource is owned by a role that cannot +log in, and each credential works as that role. Ownership of an existing resource moves to it once. A +credential is retired by handing what it owns to that role, never by dropping what it owns. + +### A credential two parties hold rotates over two credentials + +**Two parties** means an applier and a reader that are different modules, or a module and a node's +host. The vault's custody copy does not count, because the vault holds every secret. So this covers a +credential between a consumer and a provider, and every bus account: a module's or a host's, applied by +the broker's provisioner and read by its owner. **Each consumer has two credentials, one in use at a +time**, both holding the same rights over the one resource. For eight providers the second is a second +login, derived by the mesh as the consumer's identity with a short fixed suffix. For mailu it is a +second token on the same login. + +**The vault drives each rotation and records every step durably.** A provisioner learns which +credentials to hold from what it receives: both of them, for as long as a rotation is under way. It +never learns them from its own memory, so a provisioner restarted mid-rotation resumes from the step the +vault has recorded. + +1. **The vault makes the new value.** +2. **Each applier ensures the unused credential with it**, with the consumer's rights, and leaves the + one in use untouched. It verifies that the new credential authenticates and the old one still does, + and confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so + a lost message costs one pass. +3. **Only then does the vault release the new credential to the readers.** The mesh delivers it and the + value together, and the host recreates each reader, because a file it read at creation changed. A + node's host is its own reader: it reconnects to the bus with the new login, and confirms over it. +4. **Each reader confirms by authenticating with the new credential.** It shows this through its + health check, where its definition declares one, or the applier sees the new credential in use, + where its backend reports that. A reader for which neither is possible is confirmed by an operator. + It is never assumed from the reader having restarted. +5. **Only when every reader has confirmed is the old credential retired**, as above, and verified to no + longer authenticate. + +**A reader that goes away leaves the rotation.** A reader unassigned, or re-resolved to another +provider, is no longer waited for. A consumer removed mid-rotation has both of its credentials retired +with it. + +**A rotation can be abandoned until the old credential is retired.** An operator abandons it. Readers +that moved are given the old credential back, and recreated. The new credential is retired. Nothing is +lost, because the old one was never removed. + +`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done +until the old credential is gone. A reader that cannot be reached keeps working on the old credential +until it can, and the rotation waits for it. That wait is shown, never hidden. + +**Queues and permissions belong to the consumer, not to a login.** A module's queue on the bus is named +for the module on its node, and both of its logins get the same permissions over it +([ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)). An MQTT client +identifier is chosen by the consumer and is independent of its login. A reader recreated with a new +login keeps it, and the broker hands the session over. + +### A credential a single party holds rotates in place, staged + +This covers a provider's administrative credential and a module's own secret, which only that module +reads. The vault makes the new value, and the one party takes it: + +- **applied**: the vault delivers the new value **staged, beside the current one**, and the current file + is left as it is. The party's provisioner changes the backend using the current value, verifies the + new one, and confirms. Only then does the vault make the new value current. This is the only form for + a backend that takes its administrative credential only at first initialisation. Replacing the file + first would lock the provisioner out; +- **read at start**: the vault delivers the new value as current, and the host recreates the party. + +There is no window between two parties, because there is only one. Where neither form can change the +value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why +([ADR 0113](0113-the-vault-makes-every-secret.md)). + +### One rule decides which + +**The number of parties decides, never the provider.** The resolver knows it from the requirement's +recipients, leaving out the vault's custody copy, so no definition declares it. + +### Until an adapter can + +**An adapter that cannot yet ensure a second credential says so.** The two-party credentials it applies +rotate in place, as today, and the window is stated when the rotation is asked for. So does a +two-party credential whose backend has one fixed name and no second credential for it. These are listed +by a check, and the list is meant to shrink. Separating *retire a credential* from *remove the +consumer*, and keying the harness by consumer, come first. They close a data-loss path that exists +today, whatever rotation does. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here. +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room + for the second login's suffix within the tightest backend it reaches, and both logins are checked + against it. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker + account is two logins with the same permissions over the same queue, one in use at a time. Its scoping + is unchanged. +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md), already superseded by 0113: + a provider now ensures and retires credentials over a resource it owns separately. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party + credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed. + A single-party credential is staged, not replaced. + +## Consequences + +- **Every credential provider's adapter changes**, in two steps. The first separates *retire a + credential* from *remove the consumer*. It names and owns the resource after the consumer, which + keeps the name it has but moves ownership once in postgres and mssql, and the harness is keyed by + consumer. The second ensures a second credential with the same rights. +- **The vault gains rotation state**: each rotation's step, per applier and reader, recorded durably. + Staged delivery is added for single-party secrets. The SDK harness carries the alternation and the + repeated confirmation, so no adapter implements them. +- **No consumer module changes.** It reads one credential at start, as today, and is recreated by the + host when it changes. The exception is a reader that has neither a health check nor a backend that + reports use: its rotations wait for an operator until it declares one. +- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20 + characters. +- **What got harder:** + - a provider briefly holds two credentials per consumer; + - a rotation lasts until its slowest reader moves, so an unreachable reader keeps the old credential + valid until it is reached; + - an adapter has four operations where it had two; + - retiring a login in mssql has to end its sessions first. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Retiring a credential never removes a resource | A provider test per credential provider: retiring one of a consumer's credentials leaves its resource and data intact, reachable through the other. | +| What a retired login owned survives it | A postgres and an mssql test: objects created under login A, tables included, are still there and alterable under login B after A is retired. | +| A changed login is not a removal | A harness test: changing a consumer's derived login ensures a credential and never calls remove. | +| Remove runs only when the requirement goes | Harness tests: unassigning, dropping the requirement and re-resolving each remove the consumer once; a rotation and a login change never do. | +| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, with ownership moved to the resource's own role where the backend needs one. | +| Both credentials hold the same rights | A provider test per credential provider: data and structure created under one credential are read, changed and altered under the other. | +| Readers move only after the applier confirms | A rotation test: readers receive nothing until both credentials authenticate at every applier. | +| A reader confirms by authenticating | A rotation test: a reader recreated but failing to authenticate with the new credential does not confirm, and the old credential is not retired. | +| The old credential is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old credential, the rotation shows waiting on it, and it completes when the reader returns and confirms. | +| A reader that goes away leaves the rotation | A rotation test: unassigning a waiting reader lets the rotation complete; removing the consumer mid-rotation retires both credentials. | +| A rotation can be abandoned | A rotation test: abandoning after readers moved gives them the old credential back and retires the new one. | +| Rotation state survives a restart | A test restarting the applier's provisioner, and then the vault, between steps: the rotation resumes from the recorded step. | +| A single-party applied secret is staged | A rotation test on a first-initialisation administrative credential: the provisioner receives the new value beside the current one, applies it, and only then does the new value become current. At no point does it lose its connection. | +| The number of parties decides | A resolution test: a secret with an applier and a reader in different parties is marked for two credentials, and one held by one module for in place. The vault's copy is not counted. | +| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. | +| A host rotates its bus login | A rotation test on a node's bus account: the host reconnects with the new login and confirms over the bus before the old one is retired. | +| What still rotates in place is listed | A catalogue test lists every adapter that cannot yet ensure a second credential, and every two-party credential with one fixed name. A rotation of these states its window. | + +## References + +- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this + rests on, provider by provider +- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving + its declaration +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented +- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): + why a reader's restart can be derived diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index f23c044..dd3835f 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -124,6 +124,7 @@ python3 00-META/checks/index.py fail if stale - **0095** — [The control plane is the way to ask a module](0095-the-control-plane-is-the-way-to-ask-a-module.md) - **0098** — [A fact a provider makes at first start is fetched from it, not carried in its manifest](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md) - **0108** — [A route carries the policy applied to a request, and names a secret rather than holding one](0108-a-route-carries-the-policy-applied-to-a-request.md) +- **0109** — [A package registry seat is one per ecosystem, not one for all of them](0109-a-package-registry-seat-is-one-per-ecosystem.md) ### What runs on them, and how it gets there @@ -155,6 +156,10 @@ python3 00-META/checks/index.py fail if stale - **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md) - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md) - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) +- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(proposed)* +- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* +- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* +- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)* ### How it is built @@ -174,6 +179,7 @@ python3 00-META/checks/index.py fail if stale - **0096** — [An upstream image is copied between registries, never through a machine's image store](0096-an-upstream-image-is-copied-between-registries.md) - **0097** — [A vendor image is a declared build input, and a recipe fetches nothing undeclared](0097-a-vendor-image-is-a-declared-build-input.md) - **0107** — [Persistent data is a directory bind, never a named volume](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md) +- **0111** — [A build source is on the mesh's git seat, or it is an external repository](0111-a-build-source-is-on-the-git-seat-or-external.md) *(proposed)* ### How it is checked diff --git a/03-DESIGN/01-to-be/18-building-a-module.md b/03-DESIGN/01-to-be/18-building-a-module.md index 0076a36..5c77068 100644 --- a/03-DESIGN/01-to-be/18-building-a-module.md +++ b/03-DESIGN/01-to-be/18-building-a-module.md @@ -5,8 +5,9 @@ code: - mesh-controller cmd/mesh-builder - mesh-controller internal/builder - mesh-catalog modules/builder -updated: 2026-09-21 +updated: 2026-09-25 decisions: + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md - 02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md - 02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md - 02-DECISIONS/0091-a-mount-is-declared-three-ways.md @@ -38,7 +39,7 @@ controller's again. The builder's whole responsibility is the middle. | term | is | |---|---| -| **source** | a repository, a path within it, and a ref — resolved to one commit ([ADR 0069](../../02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md)) | +| **source** | a repository, a path within it, and a ref — resolved to one commit ([ADR 0069](../../02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md)). The repository is either on the forge holding the `git` seat, recorded by its path there and cloned from wherever that forge runs at build time, or external, recorded and cloned exactly as given ([ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)) | | **recipe** | how *one* artifact is produced from that source | | **toolchain** | what a recipe runs inside — a compiler, a runtime, the SDK | | **artifact** | what a recipe produced, named by the digest of its content | diff --git a/03-DESIGN/01-to-be/22-the-work-ahead.md b/03-DESIGN/01-to-be/22-the-work-ahead.md index a9ddda4..97c3519 100644 --- a/03-DESIGN/01-to-be/22-the-work-ahead.md +++ b/03-DESIGN/01-to-be/22-the-work-ahead.md @@ -12,6 +12,7 @@ decisions: - 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md - 02-DECISIONS/0075-two-stores-and-which-provides-what.md - 02-DECISIONS/0014-no-npm-workspace.md + - 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md --- # The work ahead diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md new file mode 100644 index 0000000..3baacb3 --- /dev/null +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -0,0 +1,170 @@ +--- +layer: to-be +status: proposed +code: + - mesh-controller internal/catalogue/seats.go + - mesh-controller internal/catalogue/resolve.go + - mesh-controller cmd/mesh-controller/seats.go + - mesh-controller cmd/mesh-controller/source.go + - mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql + - mesh-catalog modules/gitea/module.json +updated: 2026-09-26 +decisions: + - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md + - 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md +--- + +# 26 — The seats + +**What a mesh can have one of, and who fills each.** A seat is a named role at a scope, held by one +module assignment. The mesh defines which seats exist. Holding one may deliver a provision, and the +list of seats with their holders is the quickest answer to "what is in this mesh". + +## What a seat is + +A seat has four properties, fixed by the mesh rather than by any module: + +| property | is | +|---|---| +| name | what a definition names and an assignment holds, and what a person reads in the list | +| scope | node, site or mesh: where its capacity applies. Every seat in the set has a capacity of one, so one holder per scope. A bench, a seat with several holders, is a word the glossary keeps and no seat uses yet | +| delivers | the provision its holder answers for, or nothing | +| decision | the record that made it a seat | + +**A definition says which seats a module can hold. An assignment says which it does hold.** The store +module can hold `mesh-store`, and it may run on every node whose capabilities match. Exactly one of +those assignments holds the seat, because that assignment says so, and a second assignment saying so +is refused. A seat makes a role singular, never a module. + +**The seat points at the assignment.** Everything the mesh knows about the holder is what it knows +about that assignment: the node, the node's settings for the module, and what the module serves. + +**The set is closed.** A seat the mesh does not define is refused wherever it is named, and so is one +named at the wrong scope. Adding a seat is a decision, recorded, for the reason every addition to the +host's vocabulary is one: the set is what a person reads to learn what a mesh can have, and an entry +nobody argued for is an entry nobody can explain. + +## The set + +| seat | scope | delivers | typically held by | +|---|---|---|---| +| `mesh-controller` | mesh | — | the controller | +| `mesh-store` | mesh | — | the store the mesh's own records live in | +| `mesh-broker` | mesh | — | the broker carrying the mesh's own bus | +| `mesh-vault` | mesh | `secret`, reserved | the vault | +| `the-artifact-store` | mesh | `artifact-store` | the artifact registry | +| `the-catalogue` | mesh | — | the catalogue | +| `npm-package-registry` | mesh | `npm-package-registry` | the forge | +| `git` | mesh | `git` | the forge | +| `the-build-machine` | node | — | a builder | +| `the-dns-port` | node | — | the local resolver | +| `the-intrusion-prevention` | node | — | an intrusion-prevention service | +| `the-packet-filter` | node | — | the packet filter | +| `the-private-network` | node | — | the private network the mesh runs over | +| `the-resolver-configuration` | node | — | whichever of the alternative resolver configurations is chosen | +| `the-showcase` | node | — | the showcase module | + +The controller holds this set in code, and a test asserts both its size and that every entry names +the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) +govern, and code that disagrees is what is wrong.** The implementation in progress predates several +things here: seats held by assignments rather than claimed by definitions, the `mesh-vault` seat and +its reservation, and the foundation's seats delivering nothing. It is brought to this table before it +merges. + +## The foundation's seats + +`mesh-controller`, `mesh-store` and `mesh-broker` name which assignment the mesh *itself* uses: the +controller, the store holding its records, the broker carrying its bus. **They route no consumer.** The +store and broker modules may run on other nodes too. A database or `amqp` consumer is served by +co-location, from whichever runs on its own node, the seat's holder included +([23 — Choosing a provider](23-choosing-a-provider.md)). A requirement cannot name one of them, +because they deliver nothing. + +## A seat that delivers a provision + +**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, +the npm registry, git and the vault are each one per mesh by decision. A seat that delivers a +provision may only be held by an assignment of a module that provides it, at the seat's scope. + +**A requirement may name the seat, and then its holder answers.** Naming the seat asks for *the +mesh's* one, so the holder answers **even when another provider runs on the consumer's own machine**, +and nobody is asked anything. With the seat unheld, the requirement is refused, naming the seat. A +second provider can run beside the holder and harm nothing. A forge assignment holds +`npm-package-registry`, and an npm proxy may provide the same provision on another machine. A builder +that names the seat is still served by the forge, without anybody pinning it. + +**A requirement that names no seat resolves as any other**: a pin, the provider on the consumer's own +machine, the only provider. If several remain and none is local, a person chooses when the module is +assigned. The candidates are listed with the seat's holder suggested first, and the answer is recorded +as the assignment's pin ([27](27-a-module-requires-the-mesh-resolves.md)). Nothing is guessed, and +nothing changes silently because a second provider happened to appear nearby. + +**Moving the role is changing which assignment holds the seat.** No definition changes and nothing is +unassigned: the forge keeps running, and keeps holding `git`, when its npm role moves. A module can +take the role only if its definition says it can hold the seat. + +**The vault's provision is reserved.** Only an assignment holding `mesh-vault` may provide `secret` at +all: a definition providing it that cannot hold the seat is refused, an assignment providing it without +holding the seat is refused, and a `secret` requirement always names the seat, because there is no +other provider. A second provider of secrets would be a second place secrets live, which is what the +vault being one per mesh exists to prevent. + +**What a consumer receives is what it required**, the same as for any provision: where the provider +answers, what it serves, and a credential. A consumer never reads the seat directly. The one exception +is the controller itself, which reaches the store and the broker through a narrow seat placeholder, +because it made them before any module existed and cannot be their consumer. One foundation module +also reads it today, to find its own server's port. [27](27-a-module-requires-the-mesh-resolves.md) +moves that to a host port requirement. + +## A seat that delivers nothing + +Most node seats deliver nothing. They say which module is this machine's packet filter, or which of +two alternative resolver configurations it runs, and a second holder is refused. That is the whole of +their job, and it is a real one: it is the mesh saying what a machine is, in words a person can read. + +## The overview + +The controller lists every seat in the set with its scope, what it delivers, and its holder as a node +and a module. A seat nobody holds is listed as unheld. That is an answer, "this mesh has no forge", +and not a fault. + +Holdings are derived from assignments whenever they are asked for, never stored. The list is always +what the mesh is running, because it is computed from the same thing that decides what the mesh runs. + +## The git seat, and where a build comes from + +A module is built from a repository, a path and a ref. The repository is one of two things, and the +mesh records which: + +| form | means | recorded as | +|---|---|---| +| on the `git` seat | a repository on the forge that holds the seat | its path on the forge, and the seat | +| external | a repository anywhere else, a public forge for instance | its URL, exactly as given | + +For a repository on the seat, the controller composes the clone URL at the moment of building, from +where the holder runs and the scheme and port it serves for `git`. The recorded source never contains +an address, so moving the forge changes nothing that was recorded. The build machine is not told the +difference: it receives a URL either way. + +With the seat unheld, a build from the seat is refused and says why. External builds carry on. + +**Not yet designed:** a credential for cloning a private repository. The mesh's own repositories are +public. The natural place for a clone credential is a `secret` from the vault, and that is a decision +still to take. + +## How it is checked + +The rules here are [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)'s +and [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)'s, and each is +checked as their tables say: + +| Rule | Checked by | +|---|---| +| The set is closed, and every entry names its decision | 0110: a unit test on the set's size and decisions; manifest tests refusing an unknown seat or the wrong scope. | +| A seat is held by one assignment, and only by one whose module can hold it | 0110: resolution tests for a second holder and for a seat the definition does not name. | +| A requirement naming a seat is answered by its holder; a foundation seat cannot be named | 0110: resolution tests with a second provider on the consumer's node, with the seat unheld, and naming `mesh-store`. | +| Several providers and none local is a person's choice | 0110: an assignment test listing candidates with the seat's holder first and recording the pin. | +| `secret` is reserved | 0110: the parser and resolution refusals for another provider and a pin. | +| Holdings are derived, and the overview lists every seat | 0110: the `seats` command test, including an unheld seat. | +| A build source on the seat records no address; an unheld seat refuses only self-hosted builds | 0111's tests. | diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md new file mode 100644 index 0000000..bd0e147 --- /dev/null +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -0,0 +1,367 @@ +--- +layer: to-be +status: proposed +code: [] +updated: 2026-09-26 +decisions: + - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md + - 02-DECISIONS/0113-the-vault-makes-every-secret.md + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md + - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md + - 02-DECISIONS/0084-which-provider-serves-a-consumer.md + - 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md + - 02-DECISIONS/0038-the-mesh-assigns-the-port.md +--- + +# 27 — A module requires, the mesh resolves + +**One concept for everything a module needs.** A module definition states what it requires. Each +requirement has a contract and a kind of provider. Installing the module on a node resolves every +requirement, or refuses and says why. Nothing else reaches a module: no path it chose, no setting +beside the model, no literal it carries. + +This replaces six mechanisms that grew separately: provisions read through bindings, settings, +assigned ports, machine facts, minted or accepted secrets, and literals in the definition. Each +resolved, validated and failed in its own way ([ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), +[issue 119](../../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md)). + +## A requirement + +A requirement has three parts: + +| part | is | +|---|---| +| name | what the module calls it, unique within the module | +| contract | the fields the module may read, and what each promises: a type, whether it is secret, and anything the provider must honour | +| provider kind | which of the four kinds of provider answers it | + +**A contract is shared, not per module.** A database's contract is the database's, whoever requires +it. **The controller holds every contract**, one per provision name, declared where the provision is +defined in the catalogue. Today contracts are implicit in each provider's served fields; the first +phase below makes them explicit, because nothing can be checked against a contract that is not +written down. A provider is checked against the contract it claims to answer. A module's own +specification may narrow a contract (a password of at least this length, a directory owned by this +user) and never widen it. + +## The four kinds of provider + +The set is closed, like the seats. A fifth kind is a decision, because each kind is a place an answer +can come from and a reviewer has to know every one. + +| provider | answers | resolved by | replaces | +|---|---|---|---| +| **a module** | a database, a bucket, a vhost, a route, a secret | the rule below | provisions and bindings | +| **the node's host** | a directory, a port, a fact about the machine | always the module's own node | resource paths, assigned ports, machine placeholders, facts | +| **the mesh** | the module's identity and names, and the delivery of every answer | the controller | derived logins and generated names; the controller's delivery | +| **the operator** | a value a person chooses | the assignment, else the requirement's default | settings, carried literals | + +### A module provider + +Which module answers, in order: + +1. **the holder of the seat the requirement names.** A requirement may name a seat instead of leaving + the provider open. It asks for *the mesh's* one, and the mesh answers with whichever assignment + holds that seat, with nothing asked of anyone. Unheld, the requirement is refused, naming the seat + ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), + [26 — The seats](26-the-seats.md)). Only a seat that delivers a provision can be named; naming a + foundation seat is refused, because it delivers nothing. A `secret` requirement always names + `mesh-vault`, because that provision is reserved; +2. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's + contents ([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md)); +3. **the provider on the consumer's own node**; +4. **the only provider** in the mesh; +5. otherwise **a person chooses, at assignment**. Assigning the module lists the candidates, with the + holder of a seat that delivers the provision suggested first, and the answer is recorded on the + assignment as its pin. Without an answer the module is not assigned, and the refusal names the + candidates. Nothing is ever guessed ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)). + +### Secrets: provisioning all the way down + +**Two kinds of secret, one rule each** ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)): + +- **a shared secret**, a value more than one party must hold (a password, a token, an API key), is + made by the vault, and by nothing else; +- **a private key**, such as a node's sealing key, the operator's key or the mesh's certificate + authority, is made where it is used and never leaves. A private key anyone else held would no + longer be private. + +**Every shared secret is a `secret` requirement, and only the vault provides `secret`.** The vault +holds the `mesh-vault` seat, and that provision is reserved to it: no other module may provide it, and +no pin can choose another provider ([26 — The seats](26-the-seats.md)). + +**A provider that needs a secret for a consumer requires one, like any consumer.** A provision's +contract declares it: *for each consumer, one secret*. Resolution expands that into one requirement +per consumer, named for that consumer: + +1. gitea requires `postgres-database`; +2. the database provider, to serve gitea, requires a `secret` named for gitea; +3. the vault makes it and hands it to the mesh; +4. the mesh delivers it to both of its **recipients**, each sealed to its own node: the database's + machine, which *applies* it by creating the login, and gitea's, which *presents* it; +5. the database provider creates the login, exactly as it does today, and gitea connects. + +Every other shared secret takes the same path: +- a module's own secret; +- every broker account's password on the mesh's bus, where the broker's own provisioner creates the + account; +- an enrolment token, which the operator receives and the controller can only verify; +- a secret operator value, which the operator delivers to the vault; +- a secret a backend issues itself, such as a forge's API token, which the module that received it + delivers to the vault. + +**The controller takes the same path, because it is a module.** Its store logins and bus accounts are +own secrets of its definition today, and become requirements of that definition. **A node's host is the +one party with no definition**, because it is what runs definitions. Its bus account is a requirement +the mesh makes for each enrolled node, answered and carried exactly as for a module. + +**A provider makes resources and data.** Beyond secrets, a provider's adapter may answer with its +contract's non-secret fields: an analytics site id, a registered public name. The mesh carries them +back to the consumer as resolved values. + +### The node's host + +The host answers what only a machine can: where a directory is, which port is free, what the machine +is. It is always the module's own node, because none of these means anything elsewhere. + +**A directory.** The contract is an owner and a mode, and the owner the image expects where it has +one. There is no persistence flag. A directory is kept while it holds anything, and data that may be +lost is a named volume, not a directory ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md), +[ADR 0107](../../02-DECISIONS/0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). + +*Where* a directory is on the machine is the assignment's: + +- **a node's default layout**, a root per node with one directory per assignment beneath it, used when + the assignment says nothing; +- **a placement**, where the assignment puts one directory elsewhere: on a second disk, or where an + adopted machine's data already is ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). + +**An operator's shared data** is an access, as before ([ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md)): +never created, owned or removed by the mesh. The module requires read or read-write access. Where +the data is, is an operator value on the assignment. + +**A port** is what [ADR 0038](../../02-DECISIONS/0038-the-mesh-assigns-the-port.md) already decided: the +module says which port its software uses, and the host answers with where the machine put it. + +**A fact** is something the machine knows: its name on the private network, the names of the mesh's +machines. Each fact has a contract like anything else. + +### The mesh + +The mesh answers who the module is: its login, its broker account, the names it is known by. These +are derived by the mesh so every party agrees by construction, and no provider may make them +([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). + +### The operator + +A value a person chooses: a public name for an endpoint, a greeting, how many workers to run. + +**It must stay cheap.** An operator requirement's contract is a type and, optionally, a default. It +needs no provider module, no grant and no credential. If asking a person for a value took more than +that, module authors would route around it, and the literals this replaces would come back. + +**A secret operator value**, such as an external API key, follows the one rule for secrets: the +vault provides it. The operator hands the value to the vault, once +([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)), and the module requires +a `secret` like any other. The only difference is that rotation never replaces it: the vault cannot +make a new external key, so rotating one means an operator handing over a new value. + +**An endpoint** is an operator value inside a route requirement: the public name is chosen on the +assignment, and the route provider answers. A public name already held by another assignment is +refused, like any other singular thing. + +## How a definition reads what was resolved + +**One form, naming a requirement and a field of its contract.** A definition that needs the database's +host in an environment variable, the directory's location on the host side of a mount, or the public +name in a configuration file writes the same thing: the requirement's name and the field. The +controller fills it at resolution. + +This one form replaces the placeholders that exist today, one per mechanism: bound values, secrets, +ports and machine facts. + +**The seat placeholder stays, for the controller alone.** The controller composes its own +declaration and reaches the store and broker it made before any module existed, so it cannot be +their consumer. One module reads the placeholder today: the store module, to find its own server's +port. That is its own port, so it becomes a host port requirement in phase 3, and after that no +module uses the seat placeholder. + +**A secret field reaches a process as a file**, as [ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md) +decided. The one exception 0086 allows is a declared env-file with its reason; a secret field as a +value in a container's environment is refused when the definition is parsed, with no exception. + +## An assignment + +**A module is assigned at most once to a node**, and that pair is the assignment's identity +([ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md)). Its directories, +containers, login, broker account and settings are keyed by it, as today, and a login still fits the +tightest backend ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). + +**A module may run on many nodes, and one assignment may hold a seat** +([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). The definition +says which seats the module can hold; the assignment says which it does. So the store module can run +on every node, one of those assignments holds `mesh-store`, and moving that role changes an +assignment, not a definition. + +What must stay singular stays so: by a seat, or by an operator value colliding, as with a public name. + +## Genesis + +**Genesis delivers, and the vault adopts.** The vault cannot run first: it is built on the shared +runtime base, which the installation makes only after the store, the broker and the controller exist +([21 — The installation in full](21-the-installation-in-full.md)), and it learns what to answer from +the controller over the bus. So the vault is installed **as soon as that base exists**, before any other +module built on it, and genesis generates what is needed until then: +- the store's superuser, and the broker's admin in the hashed form the broker needs; +- the bus accounts of the temporary and permanent controller (its account and the broker-management + login), the control-node's host, the builder, the broker's own provisioner and the vault; +- the controller's three store logins, and the first enrolment token. + +Until the broker's provisioner runs, genesis creates those bus accounts with the broker's admin, as the +controller does today; the provisioner adopts them when it starts. Genesis seals everything to the +control-node's key, and when the vault is installed the controller **delivers the values to it, +recorded as the mesh's own**, with nobody present. That distinction keeps +them rotatable: an operator's value is never replaced, and these are, because the vault can make their +replacements. + +That is the one time anything but the vault generates a shared secret, and it ends by handing them +over. It is also the answer to the objection ADR 0085 had to the vault being the only maker: the +vault cannot make what exists before it, so what exists before it is delivered to it. + +**Raising the vault or the broker again is a genesis act.** Moving either seat to a new assignment, or +recovering either after it is lost, delivers the values it needs the way genesis did. It is a stated +break-glass procedure, and an ordinary assignment attempting it is refused. + +## Rotation + +Rotating a secret is asked of the vault, by an operator or by the vault's own policy, such as a +maximum age in the requirement's contract, and the vault makes the new value +([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). An operator's external key is +not rotated by the vault, which cannot make its replacement: an operator delivers a new one. + +**Each requirement says how its recipient takes a new value.** It either *applies* it, through a +provisioner (a provider setting a login's password, the broker's provisioner updating an account, a +store's provisioner changing its own superuser), or *reads it at start*. Every module in the catalogue +reads its secrets at start, and none watches them. The host already recreates a container when a file +it read at creation changes, its env-files and files mounted into it directly +([issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). +So a reader's restart is derived, and a definition declares `restart-on` only for a secret reaching a +process, or a file in a mounted directory. A secret a service takes only at first initialisation is +applied by its provisioner or marked not rotatable by the mesh, and a rotation of it is refused rather +than reported done. + +**How old and new change over depends on how many parties hold the credential** +([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), on +[research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md)): + +- **Two parties**, a consumer and its provider, or a module or node's host and the broker: each consumer + has two credentials, both with its rights over one resource named after the consumer. For most + providers the second is a second login derived by the mesh; where a backend's user is its resource, it + is a second token. The vault drives the rotation and records each step. It makes the new value; each + applier ensures the unused credential and confirms both authenticate; only then are readers given it + and recreated; each reader confirms by authenticating with it; and only then is the old one retired. + Nobody is left without a credential that works, a rotation can be abandoned until the old one is + retired, and `status` shows who a rotation waits on. +- **One party**, a provider's administrative credential or a module's own secret: in place, staged. + An applied one is delivered beside the current value, the provisioner changes the backend with the + current one, and only then does the new value become current. One read at start is delivered, and + the host recreates the module. + +**Retiring a credential never removes what it reached.** An adapter keeps *retire a credential* and +*remove the consumer* apart, and the harness keys what it applied by consumer, so a changed login is +never a removal. The resource is removed only when the consumer no longer requires it from that +provider: unassigned, the requirement dropped, or re-resolved elsewhere. Today these are one call, and +in five providers it deletes the consumer's data, so this separation comes first. An adapter that cannot +yet ensure a second credential rotates in place, with its window stated, and is listed until it can. + +## Refusing + +Installation refuses when any requirement is unresolved, and **says everything at once**. For each +requirement it names what is missing and what would answer it: + +- an unheld seat, and which modules could hold it; +- no provider, and which modules could provide it; +- several candidates and no choice made, and which they are; +- an operator value with no default, and that the assignment must give it; +- a provider, or the vault, that has not answered yet, and which one. + +The last one is a state, not a failure. A consumer waiting for its provider or for the vault is shown +as waiting, and nothing is delivered until the answer arrives. + +## What this retires + +| mechanism | becomes | +|---|---| +| provisions read through bindings | a module requirement; its answer is the contract's fields | +| settings on an assignment ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) | operator requirements on an assignment | +| a port the mesh assigns | a host requirement | +| machine facts and machine placeholders | host requirements | +| every secret the controller mints: provider credentials, own secrets, broker passwords, enrolment tokens | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)) | +| root secrets genesis mints and keeps apart | made by genesis once, then delivered to the vault, which holds and rotates them | +| a separate command issuing a broker account | a requirement resolved on assignment | +| `restart-on` naming a secret's file | a restart the host derives | +| paths in resources, mounts, bindings, secrets and received files | host directory requirements, placed by the assignment | +| literals carried in a definition | operator requirements with defaults | + +Each is retired only once nothing uses it. Until then both are accepted, and a catalogue test lists +the definitions still using the old form. That list shrinks to empty, and then the old form is +removed from the parser. + +## Phases + +Each phase ends at a check that holds, so none of them leaves a mechanism half-replaced. + +1. **Contracts, resolution and the new form.** Every provision's contract is written down and held by + the controller. The controller resolves requirements from the four providers, refuses as above, + and fills the one form. Old mechanisms keep working beside it. *Ends when* a definition written + entirely in the new form installs on a lab machine. +2. **The vault makes every shared secret, and providers answer.** The vault holds its seat and its + reserved provision; resolution expands per-consumer secret requirements; genesis delivers the + foundation's first secrets to the vault; the broker's provisioner creates every bus account; the + mesh carries providers' data back. + [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + is fixed in the host already. *Ends when* nothing outside the vault generates a shared secret after + genesis, a lab consumer of analytics receives its site id, and a database credential rotates over + its two credentials, the consumer recreated by derivation, never without a working login, and its data intact. +3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted + and running assignments placed where their data already is, and each claim becomes a seat the + module can hold, held by the assignment that holds it today. *Ends when* the list of definitions + using an old form is empty, the old forms are removed, and the store module runs on two lab + machines with one holding `mesh-store`. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Every requirement has one of the four provider kinds | The parser refuses any other. | +| A definition names no host path, node or mesh | The catalogue tests of [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md). | +| A module provider is chosen by named seat, pin, co-location, only one, a person's choice | Resolution tests for each step: a requirement naming a seat served by its holder even with another provider on the consumer's node, and refused when the seat is unheld; several candidates and none local, where assignment lists them with the seat's holder first and records the choice as a pin, and refuses without one. | +| A provider answers within its contract | A controller test: an answer carrying a field its contract does not name, or missing one it does, is refused and not delivered. | +| A module narrows a contract and never widens it | The parser refuses a module specification that loosens a contract's field. | +| A host requirement is answered on its own node | A resolution test placing one elsewhere: refused. | +| An operator value needs no provider module | A resolution test: a requirement with a default resolves with no module assigned anywhere. | +| A secret field reaches a process as a file | The parser refuses a secret field as a container environment value, and accepts it in a file or a declared env-file with its reason. | +| A public name already held is refused | A resolution test: a second assignment asking for a public name another holds is refused, naming the holder. | +| A module is assigned at most once to a node | A resolution test: assigning a module to a node that already runs it is refused. | +| A seat is held by an assignment, not a module | A resolution test: the store module on two nodes, one holding `mesh-store`; a second assignment asking to hold it is refused. | +| A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | +| Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | +| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | +| Restarts are derived | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, and one read at start recreates its reader without a declared restart. | +| A two-party credential rotates over two credentials | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md): retiring a credential leaves the resource intact; a changed login is never a removal; readers move only after the applier confirms and confirm by authenticating; an unreachable reader keeps its old credential until it returns; rotation state survives a restart; a single-party applied secret is staged. | +| A private key is made where it is used | The per-key tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): a node's sealing key, the operator's key and the certificate authority's key never leave where they were made. | +| The controller and a node's host take the same path | 0113's tests: the controller's definition declares requirements and no own secret; a node's bus account is made by the vault and delivered sealed to that node. | +| Moving the vault or the broker is break-glass | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. | +| A secret that cannot be rotated says so | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. | +| Only the controller reads the seat placeholder | A catalogue test, from phase 3: no definition uses the seat placeholder. | +| Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | +| The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | + +## Not settled here + + +- The exact spelling of the one form. It must name a requirement and a field and nothing else. +- The layout a node's default root uses beneath it, beyond one directory per assignment. +- Whether a module provider's answer can change without the provider being asked, for example a + provider moving. The rule so far is that it cannot, and moving is re-resolving. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 5c10294..3c8f3d3 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -34,6 +34,8 @@ document is written and this one's status becomes `implemented`. | [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) | | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | +| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | +| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | ## Not yet written diff --git a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md index 7e5086d..e2aa950 100644 --- a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md +++ b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md @@ -2,7 +2,7 @@ status: resolved opened: 2026-09-23 located-in: [mesh-host internal/apply] -fixed-by: mesh-host — a container is recreated when an env file or a directly mounted file it reads changes; a pre-upgrade label is accepted once; the plan names the file +fixed-by: mesh-host PR #22 — a container records the digest of every file it reads at creation, its env-files and files mounted into it directly, and is recreated when one changes; a pre-upgrade label is accepted once, and the plan names the file. A mounted directory still needs restart-on. amended-design: --- diff --git a/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md new file mode 100644 index 0000000..ec7cd08 --- /dev/null +++ b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md @@ -0,0 +1,84 @@ +--- +status: located +opened: 2026-09-25 +located-in: [mesh-catalog modules, mesh-controller internal/catalogue] +fixed-by: +amended-design: +--- + +# 119 — A module definition decides where its files live on the machine + +## What was observed + +A review of where module code reads its files turned up a cross-cutting pattern. Every module +definition in the catalogue chooses, in its own manifest, where on the machine its files live. +Counted on the catalogue's `main`, 2026-09-25: + +| where in the definition | host-path strings | +|---|---| +| directory and file resources | 257 | +| container mounts, host side | 230 | +| own secrets | 78 | +| bindings | 53 | +| env-files | 50 | +| secrets | 35 | +| container environment | 28 | +| accesses | 21 | +| receives, grants | 24 | +| everything else | 13 | + +**789 host-path strings in 70 of the 71 definitions.** Mounts are checked: a container may not +mount a path its module never declared ([ADR 0091](../../02-DECISIONS/0091-a-mount-is-declared-three-ways.md)). +Nothing checks the same path where it is retyped as a value: an environment variable, an env-file +line, a literal in module code. + +### Where that has already gone wrong + +- **A provider that would provision nobody, silently.** One DNS provider mounts its grants + directory at a short path inside its container, then tells its provisioner to read the + contributions file at the host path, which does not exist in there. Nothing requires the + provision today, so it has not failed yet. When a consumer arrives, it will get no record, and + nobody will be told. +- **The mesh's own wire carries host paths into containers.** Each contribution names its + consumer's credential as "the file on this machine holding that consumer's credential", a host + path computed from the provider's grants directory. So every provider has to mount that directory + at the *identical* path, or it cannot read what it was given. Ten of the eleven providers with a + grants directory do. It is a convention nothing states or checks, and the eleventh is the + provider above. +- **The warning that would have caught it is lost in the SDK.** The controller always writes the + contributions file, even when empty, so a provider can tell "nothing asked" from "never written". + The SDK's reconcile loop treats an unreadable file as empty, and logs nothing. +- **Code carries copies with nothing checking them.** Several modules default a path in code when an + environment variable is unset. Five of those defaults disagree with the value their own manifest + sets. One of them is a host path used inside a container that does not mount it. + +### And a module cannot be assigned to one node twice + +Everything that identifies a running module is keyed by the module's name: its directories, its +container names, the login it presents to a provider, its broker account. Two assignments of one +module to one node would share every one of them. Assigning the same application twice is an +ordinary need: production beside staging, one site per customer, two instances of one service +configured differently, two stores of one engine. + +[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), which answers +this report, declines that need rather than meeting it. A module is assigned at most once to a node, +because every identity in the mesh is already a module on a node. The cases above become different +modules, or the same module on different machines. + +## Why it matters beyond this instance + +A definition that names machine paths is not portable between nodes. It cannot follow data onto a +second disk, or onto a machine being adopted with its data already in place, without editing the +module. It cannot run twice on one node. It keeps every path in two or three places with nothing +checking that they agree. The defects above are what that allows, and each was found by reading, +not by any check. + +## Open questions + +- Should a definition name any host path at all, or should every location come from the + assignment and the mesh? +- If a directory is something a module *requires* rather than *declares*, what is its contract: + ownership, mode, whether it is kept when the module goes? +- What identifies an assignment, if a module may be assigned to one node more than once? +- What would the contributions file carry instead of host paths, so a provider needs no + identical-path mount?