novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Apply review: two credentials, staged admin rotation, a ninth provider

Browse Source 
The fact-check found mailu, whose user is its mailbox, so 0114 rotates
over two credentials rather than two logins, the adapter choosing what a
credential is. Also: minio keeps non-empty buckets; five backends take
their admin credential only at first init, so single-party rotation is
staged; postgres ownership moves to a non-login role; the harness keys by
consumer; rotation state lives with the vault. Consistency fixes across
0110-0113, 26 and 27; issue 103 resolved by mesh-host PR #22.
This commit is contained in:
jochen
2026-09-26 00:38:06 +02:00
parent 43f63ed41c
commit e387c4bd0e
16 changed files with 472 additions and 318 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md
+14 -11
View File
@@ -1,7 +1,7 @@
---
status: graduated
became:
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
initiated: 2026-09-26
touches:
@@ -16,7 +16,7 @@ touches:
# 016 — How a credential can be rotated
**What.** Which rotation mechanisms the mesh's providers can actually support, measured against
their code rather than assumed. Every provider in the catalogue was read: how it names what it
their code rather than assumed. Every provider in the catalogue was read, found by listing every definition that provides something: how it names what it
makes for a consumer, what its remove destroys, whether it re-applies a password, whether its
backend can hold two secrets for one login or two logins on one resource, and how its own
administrative credential is set. The consumer side was read too: when a module reads a secret, and
@@ -30,7 +30,8 @@ with the login. Overlap as written would have deleted every consumer's database
rotation. The mechanism has to be chosen on what the providers do.
**What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md),
which both mark it undecided and point here. The identity budget in
which [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) decided on
these findings. The identity budget in
[ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer
gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md)
describes.
@@ -43,12 +44,14 @@ describes.
- [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a
recommendation.
**Finding, in one paragraph.** All eight credential providers already re-apply a consumer's password
**Finding, in one paragraph.** All nine credential providers already re-apply a consumer's password
in place on every create, and the controller's `rotate` command relies on that. It is a working
rotation with a stated window. Seven of the eight name the consumer's resource after its login,
and five drop the resource when they remove the login, so a second login is impossible without
changing the adapter. Only one backend holds two passwords on one login. But every backend can grant
two logins the same rights over one resource. So overlap is possible everywhere, but only after each
adapter separates *the consumer's resource* from *the login that reaches it*. Administrative
credentials are a different case. They have one party, a fixed name, and in three providers they are
taken only at first initialisation.
rotation with a stated window. Eight of the nine name the consumer's resource after its login, and five
destroy the consumer's data when they remove the login. The harness, keyed by login, would do the same
on any change of login. Only one backend holds two passwords on one login, and two more hold several
tokens. Eight backends can grant two logins the same rights over one resource; the ninth can give one
login a second token. So every provider can hold **two credentials** over one resource, but only after
each adapter separates *the consumer's resource* from *the credential that reaches it*. In postgres
that also means the resource belongs to a role no login owns. Administrative credentials are a
different case. They have one party and a fixed name, and five backends take them only at first
initialisation, so changing one needs the old and the new value at once.
01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md
+58 -26
View File
@@ -1,10 +1,18 @@
# 01 — The providers
Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`),
the client functions they call, and each definition's own credentials. The provisioner harness in
`mesh-sdk` calls `create` for a consumer when its contribution appears or changes, and after the
provisioner restarts, and `remove` when the contribution goes. Its record of what was applied is
kept in memory.
the client functions they call, and each definition's own credentials. The providers were found by
listing every definition that provides something and has a provisioner, not from memory. A first pass
of this survey worked from memory and missed one, mailu.
**The provisioner harness** in `mesh-sdk` calls `create` for a consumer when its contribution appears
or changes (its login, password or values), and after the provisioner restarts. It calls `remove` for
a login it applied earlier in the same process that is no longer contributed. Its record of what was
applied is kept in memory and keyed by login. Two things follow:
- a consumer whose derived login changes is removed under the old login and created under the new one,
in one pass;
- a contribution that disappears while the provisioner is down is never removed, and is left behind.
## The credential providers
@@ -13,13 +21,14 @@ kept in memory.
| provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource |
|---|---|---|---|---|---|
| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, both as members of one role that owns the database. Not done today |
| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. Not done today |
| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, but only through a role that cannot log in owning the database, with each login working as it. Otherwise whatever one login creates is its own, and dropping that login means handing its objects over first. Not done today |
| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. A user owning a schema cannot be dropped, and a login with an open session cannot. Not done today |
| mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today |
| redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login |
| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key and the bucket | yes, by removing the access key and adding it again | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters |
| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host |
| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. An MQTT client identifier still has to be unique per connection |
| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key; the bucket **only if empty**. A bucket holding objects is left, and the failure logged | yes, by removing the access key and adding it again, which leaves a moment with no key | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters |
| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host, with any queued messages, and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host |
| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. The MQTT client identifier is chosen by the consumer, not tied to the login; a duplicate one takes the older session over |
| mailu | a mailbox `login@domain`, unless the consumer contributes its own account name | the mailbox with its mail, for a login-named one; a contributed name is left for an operator | yes, the password is set when the user exists | no for the password; a user can hold several authentication tokens, per the backend's documentation | **no**: a mail user *is* its mailbox |
| gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team |
## The other providers
@@ -31,41 +40,64 @@ kept in memory.
| showcase | a route | none |
| mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today |
verdaccio provides the npm registry too, and has no provisioner.
## Each provider's own administrative credential
| provider | identity | how the backend takes it |
|---|---|---|
| postgres | a fixed superuser | from a file **only at first initialisation**. Afterwards the file is read by the provisioner to connect, and changing it changes nothing in the database |
| postgres | a fixed superuser | from a file **only at first initialisation** |
| mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception |
| mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty |
| mosquitto | a fixed admin client | seeded into the broker's dynamic-security file **once**; the seeding step skips when the file exists |
| lavinmq | a fixed admin name | per its own bootstrap code, **only on a first boot** with an empty data directory. No resource in the definition runs that bootstrap; what sets it on a running mesh is outside the catalogue |
| redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts |
| minio | a fixed root user | from a file, read when the server starts |
| lavinmq | a fixed admin name | from the module's own secret, which its provisioner connects with |
| mosquitto | a fixed admin client | from the module's own secret, which its provisioner connects with; the broker's dynamic-security file holds it |
**In five of seven, a new administrative value takes effect only through a command run with the old
one.** The credential file is mounted directly into both the server and the provisioner. So replacing
it recreates the provisioner, which then holds only the new value while the backend still expects the
old one, and the provisioner is locked out. That is worse than changing nothing.
Every provider module also has its own bus account, an own secret, read at start.
## What the table shows
## What the tables show
1. **Every credential provider already rotates in place.** All eight re-apply the password on the
1. **Every credential provider already rotates in place.** All nine re-apply the password on the
same login each time `create` runs. The controller's `rotate` command relies on that: it replaces
the credential in the inventory and sends both ends in one push. Its own comments state the window,
between the provider applying and the consumer restarting, in which the consumer cannot
authenticate.
2. **Seven of eight name the consumer's resource after its login.** Only gitea separates them,
2. **Eight of nine name the consumer's resource after its login.** Only gitea separates them,
because an organisation owns the packages. A second login therefore has no resource of its own to
reach, and cannot share the first one's without the adapter granting it.
3. **Five of eight destroy the resource when they remove the login.** These are postgres, mssql, mongodb,
minio and lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call.
This is the fault the first overlap draft would have triggered.
4. **Only redis holds two passwords on one login**, and gitea through tokens rather than its
password. A rotation built on two secrets per login would work for one provider out of eight.
5. **Every backend can give two logins the same rights over one resource.** Group roles in postgres,
3. **Five of nine destroy the consumer's data when they remove the login**: postgres, mssql and
mongodb drop the database, lavinmq drops the virtual host with its queued messages, and mailu
deletes the mailbox with its mail. minio drops only an empty bucket, and redis leaves the keys. In
those five, *retire a login* and *delete the consumer's data* are one call. With the harness keyed
by login, a changed login triggers it too.
4. **One backend holds two passwords on one login** (redis). Two hold several tokens beside one
password (gitea and mailu). A rotation built on two secrets per login would work for three
providers out of nine.
5. **Eight of nine can give two logins the same rights over one resource.** Group roles in postgres,
database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared
policy in minio, a shared key prefix in redis, a shared team in gitea. No adapter does it today.
6. **The administrative credentials have one party and a fixed name.** The provider module is both the
one that applies the credential and the only one that reads it. In postgres, mongodb and mssql, a new
value only takes effect through a command run with the old value. A restart changes nothing.
7. **A consumer's identity is already the resource's name.** The login is derived from the
policy in minio, a shared key prefix in redis, a shared team in gitea. mailu cannot, because its
user is its mailbox, but it can give one user a second token. So every provider can hold **two
credentials** over one resource, though not every one as two logins. No adapter does either today.
6. **Ownership is a trap in two backends.** In postgres whatever a login creates is that login's, so a
second login cannot alter the first one's tables, and the first cannot be dropped while it owns
them. The one-step way out deletes them. In mssql, a login cannot be dropped with a session open,
nor its user while it owns a schema.
7. **The administrative credentials have one party and a fixed name**, and five backends take them
only at first initialisation. The provisioner needs the old and the new value at once to change
them. Today nothing can give it both.
8. **A consumer's identity is already the resource's name.** The login is derived from the
assignment, which is a module on a node, so the current login and "the consumer" are the same
string today. A second login would need a new name. The resource can keep the one it has.
## Seen on the way
The redis configuration names no ACL file, so a consumer's ACL user exists only in memory. A restart
of the redis server erases every consumer's user. The provisioner does not create them again until it
restarts itself, because its in-memory record says they are done. That is not a rotation finding, but
it is a live fault, and it is recorded here so it is not lost.
01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md
+4
View File
@@ -32,6 +32,10 @@ env-file or a direct mount, which the host covers, or through a directory or int
needs `restart-on`. **That was not classified module by module.** It is the check to run before a
rotation mechanism relies on it.
The count covers only the `secrets` field. The 49 modules with their own bus account, and 54 with any
own secret, are readers too, and their bus accounts are rotated like any credential two parties hold.
Their files are mounted directly, which the host covers, but the classification has to name them.
## What this means for rotation
- The *read at start* half of 0113's recipient model is already true, and mostly already handled by
01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md
+35 -31
View File
@@ -4,7 +4,7 @@ Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-read
## A. In place, as today
The vault makes a new value. Every applier re-applies it on the same login, which all eight
The vault makes a new value. Every applier re-applies it on the same login, which all nine
providers already do. Every reader is recreated by the host.
- **Works with:** every provider, unchanged. It is what `rotate` does now.
@@ -18,56 +18,60 @@ providers already do. Every reader is recreated by the host.
The applier adds the new password beside the old one, readers move, and the old one is removed.
- **Works with:** redis natively, and gitea through tokens. **Not** with the other six, whose backends
hold one password per login (finding 4).
- **Works with:** redis natively, and gitea and mailu through tokens. **Not** with the other six, whose
backends hold one password per login (finding 4).
- **Verdict:** not a mechanism, a special case. Using it where it exists and something else
elsewhere is the "this way or that way" the design is trying to remove.
## C. Two logins per consumer, over one resource
## C. Two credentials per consumer, over one resource
The consumer has two logins derived by the mesh, and uses one at a time. The applier creates the other
with the new value and grants it the same rights over the consumer's resource. Readers move to it,
and then the old login is retired, which removes the login only, never the resource.
The consumer has two credentials and uses one at a time. The applier ensures the other with the new
value and gives it the same rights over the consumer's resource. Readers move to it, and then the old
credential is retired, which removes the credential only, never the resource. **What a credential is,
is the adapter's**: a second login for eight providers (finding 5), a second token on the same login
for mailu. The mesh sees one mechanism.
- **Works with:** every backend (finding 5), **after** each adapter changes:
- **Works with:** every provider, **after** each adapter changes:
- the resource is named after the consumer, not the login. Today the two are the same string (finding
7), so existing resources keep their names, the current login stays one of the two, and only the
second is new;
- both logins get the same rights, through a group role or its equivalent;
- *retire a login* and *remove the consumer* become two operations. Today they are one call, and
in five providers that call destroys data (finding 3). This is the whole of the danger, and it has
to be split, whatever else is chosen.
8), so existing resources keep their names, and the current login stays one of the two;
- the resource is owned by the resource, not by a login. In postgres that is a role no one logs in
as, which each login works as, and ownership of an existing database moves to it once (finding 6);
- both credentials get the same rights, over data and structure;
- *retire a credential* and *remove the consumer* become two operations. Today they are one call, and
in five providers that call destroys data (finding 3). The harness must key by consumer, so that a
changed login is not a removal. This is the whole of the danger, and it has to be split, whatever
else is chosen.
- **Costs:**
- seven adapters change;
- the harness learns the alternation and a confirmation per step;
- every credential adapter changes;
- the harness learns the alternation and a confirmation per step, and rotation state has to live
somewhere that survives a restart, which the harness's memory does not;
- the second login's name must fit the tightest backend. That is 20 characters for a minio access
key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and
a suffix spends part of it;
- an MQTT client identifier stays unique per connection, so mosquitto needs the client id kept apart
from the login.
- **Gains:** no window. A reader that cannot be reached keeps a working login until it can.
- retiring an mssql login has to end its sessions first.
- **Gains:** no window. A reader that cannot be reached keeps a working credential until it can.
- **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is
no second party to overlap with.
## Independent of the choice
- **Split remove.** Retiring a credential must never be able to destroy a consumer's data. That holds
under A too, because A's remove is the same call. A remove that drops a database should be a
separate, explicit operation, which [ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md)
already implies: data outlives the declaration.
- **Admin credentials are applied by their own provider**, using the old value. That happens in place
whichever mechanism consumers get. Where a backend takes the value only at first initialisation, the
file alone changes nothing, and rotation needs the provider's provisioner to run the change.
- **Classify the 22 readers** ([02](02-the-readers.md)) before relying on derived restarts.
- **Split remove, and key the harness by consumer.** Retiring a credential, or a login changing, must
never be able to destroy a consumer's data. That holds under A too, because A's remove is the same
call.
- **Admin credentials are applied by their own provider**, using the old value, with the new one staged
beside it. Five backends take the value only at first initialisation. Replacing the file first locks
the provisioner out (finding 7).
- **Classify the readers** ([02](02-the-readers.md)), bus-account readers included, before relying on
derived restarts.
## Recommendation
- **Consumer credentials: C**, because it is the only mechanism every backend supports, and it closes
the window instead of shortening it. Its prerequisite, separating the resource from the login and
- **Two-party credentials, consumer credentials and bus accounts: C**, because it is the only
mechanism every provider supports, and it closes the window instead of shortening it. Its prerequisite, separating the resource from the login and
retiring a login from removing a consumer, is worth doing on its own, because it removes a
data-loss path that exists today.
- **Single-party secrets (admin credentials, a module's own): A.** In place, applied by the provider
that holds them.
- **Single-party secrets (admin credentials, a module's own): A, staged.** In place, applied by the
provider that holds them, with the new value beside the old until it has taken.
- **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is
not replaced by a mechanism the providers cannot yet carry.
02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
+6
View File
@@ -163,6 +163,10 @@ On acceptance, each of these is amended by this record, not edited:
- [ADR 0084](0084-which-provider-serves-a-consumer.md): a requirement may name a seat, which its holder
answers; and where several providers remain and none is local, the choice is asked when the module is
assigned and recorded as a pin, rather than refused until someone pins it.
- [To-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): the same two changes, in the design
that describes choosing a provider.
- [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): genesis assigns the foundation's
store, broker and controller holding their seats, where their definitions claim them today.
## Consequences
@@ -191,6 +195,8 @@ On acceptance, each of these is amended by this record, not edited:
| Every module in use names a seat in the set | A controller test parses every catalogue manifest and fails on any refused seat. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). |
| A seat is held by one assignment, not by a module | A resolution test: the store module assigned to two nodes resolves, with one assignment holding `mesh-store`; a second assignment asking to hold it is refused. |
| A requirement naming a seat is answered by its holder | Resolution tests: two providers with the seat held; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. |
| An assignment holds only a seat its module can hold | A resolution test: an assignment holding a seat its definition does not name is refused. |
| Every seat is listed with its holder | A `seats` command test: every seat in the set is listed with its scope, what it delivers and its holder, and an unheld seat is listed as unheld. |
| Several providers and none local is a person's choice | An assignment test: the candidates are listed with the delivering seat's holder first; the answer is recorded as a pin; with no answer the module is not assigned. |
| The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`; a requirement naming `mesh-store` is refused, because it delivers nothing. |
| A reserved provision has no other provider | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`; resolution refuses an assignment providing it without holding the seat, and a pin on a `secret` requirement. |
02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
+9
View File
@@ -65,6 +65,15 @@ failing to clone.
**The build machine is not told the difference.** It receives a URL either way. Composing the URL is
the controller's job, because only the controller knows where the seat's holder runs.
## What this changes in earlier records
On acceptance, each of these is amended by this record, not edited:
- [ADR 0069](0069-a-module-is-a-repository-and-a-path.md): a module's repository is recorded either as
a path on the `git` seat or as an external URL, never as an address of the mesh's own forge.
- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): the closed set gains `git`,
mesh-scoped, delivering `git`, held by a gitea assignment.
## Consequences
- The controller's inventory gains a column saying which seat a source is on. It is empty for
02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
+6 -4
View File
@@ -22,7 +22,8 @@ copies. The issue records what that has already allowed:
- a contributions file that carries host paths into containers, so every provider must mount its
grants directory at the identical path;
- defaults in code that disagree with their own manifests;
- no way to assign one module to one node twice, because every identity is keyed by the module's name.
- every identity keyed by the module's name, which is why one module cannot be assigned to one node
twice. This record keeps that, and says so below.
**Paths are one case of a wider pattern.** A module gets what it needs through at least six separate
mechanisms today, each with its own syntax and its own failure modes:
@@ -80,8 +81,9 @@ other. An operator value is the assignment's, or the requirement's default, or u
**A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a
default. It needs no provider module, no grant and no credential.
**Every secret is a `secret` requirement, answered by the vault**, with no exception by kind
([ADR 0113](0113-the-vault-makes-every-secret.md)). An external API key an operator chooses is no
**Every shared secret is a `secret` requirement, answered by the vault**, with no exception by kind
([ADR 0113](0113-the-vault-makes-every-secret.md)). A private key is made where it is used and is not
a requirement. An external API key an operator chooses is no
different: the operator delivers it to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)),
and the module requires a `secret` like any other. A provider that needs a secret for a consumer
requires it from the vault, like any consumer, and answers with resources and data. The mesh carries
@@ -124,7 +126,7 @@ way as every other secret ([ADR 0113](0113-the-vault-makes-every-secret.md)).
## What this changes in earlier records
On acceptance, each of these is amended by a record of its own, not edited:
On acceptance, each of these is amended by this record, not edited:
- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings become
operator requirements on an assignment.
02-DECISIONS/0113-the-vault-makes-every-secret.md
+36 -22
View File
@@ -15,7 +15,7 @@ controller on 2026-09-25:
| kind | made by | used by |
|---|---|---|
| a credential between a consumer and a provider | the controller | 19 modules |
| a credential between a consumer and a provider | the controller | 16 modules, and 3 more for model access, counted below |
| a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules |
| a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules |
| a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder |
@@ -78,7 +78,7 @@ it first means reordering the whole installation and giving the vault a second w
requiring a database makes the database's provider require a secret for gitea, and the vault
answers it. The provider's own code does not change: it is handed a login and a password, as today;
- a module's **own secret**. `own-secrets` is retired;
- every **broker account** on the mesh's bus: a module's, a node agent's, the builder's, the
- every **broker account** on the mesh's bus: a module's, a node's host's, the builder's, the
controller's. The broker holding `mesh-broker` carries the mesh's bus
([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates
each account from the vault's secret, like any provider. The controller no longer creates accounts,
@@ -122,9 +122,11 @@ generated by genesis:
token.
Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's
admin, as the controller does today. Genesis seals all of it to the control-node's key, and when the vault
is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not as an
operator's. Nobody has to be present for it.
admin, as the controller does today. Genesis seals each value twice: to the control-node's key, so that when the
vault is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not
as an operator's, with nobody present; and to the operator key, as the break-glass copy
[ADR 0085](0085-a-secret-is-a-provision.md) keeps of every root secret. The first enrolment token reaches
the operator the same way.
That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)),
and these are, because the vault can make their replacements. The broker's provisioner then adopts the
accounts genesis created. From then on the vault makes every shared secret, and genesis has made its
@@ -132,7 +134,9 @@ last one.
**Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker`
seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the
values it needs are delivered, not made by a vault that is not there. That is a break-glass procedure,
values it needs are delivered, not made by a vault that is not there. They come from the operator-sealed
copies, which the operator opens. The vault keeps a copy of every secret sealed to the operator key
(0085), so nothing the mesh relies on exists only inside the vault. That is a break-glass procedure,
stated and checked, never an ordinary assignment.
**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may
@@ -149,14 +153,17 @@ not rotated by the vault: rotating it means an operator delivering a new one. A
issued is rotated by the module that holds the backend asking it again and delivering the new value to
the vault.
**Each recipient takes a new value one of two ways, marked on its requirement:**
**Each recipient takes a new value one of two ways, marked per recipient:**
| recipient takes it by | example | what happens on rotation |
|---|---|---|
| **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it |
| **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed |
The marking is on each requirement, not on the secret, because one secret has recipients of both kinds.
The marking is per recipient, not per secret, because one secret has recipients of both kinds. A
provision's contract marks its provider's side, which applies. A consumer's side is read at start
unless its requirement says otherwise. The broker's contract marks the host's bus account the same
way: the broker's provisioner applies it, and the host reads it.
Every module in the catalogue reads its secrets at start, and none watches them
([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a
backend takes only when it first initialises is marked applied, and its provider's provisioner makes
@@ -165,7 +172,7 @@ rotatable by the mesh**, and a rotation request is refused, saying why, rather t
that would carry on with the old value.
**How old and new change over is decided in [ADR
0114](0114-a-shared-credential-rotates-over-two-logins.md).** Three mechanisms were measured against
0114](0114-a-shared-credential-rotates-over-two-credentials.md).** Three mechanisms were measured against
every provider's code in [research
016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): in place, as the controller's
`rotate` does today; two secrets on one login; and two logins over one resource. Its findings bound the
@@ -173,15 +180,15 @@ choice:
- every credential provider already re-applies a password in place, so today's rotation works, with a
window in which a consumer cannot authenticate;
- seven of eight name the consumer's resource after its login, and five destroy the resource when they
remove the login. **No mechanism may retire a login through today's remove**, because in those five
it deletes the consumer's data;
- only one backend holds two passwords on one login;
- every backend can give two logins the same rights over one resource, once the adapter separates the
resource from the login.
- eight of nine name the consumer's resource after its login, and five destroy the consumer's data when
they remove the login. **No mechanism may retire a login through today's remove**, because in those
five it deletes the consumer's data;
- one backend holds two passwords on one login, and two more hold several tokens;
- every provider can hold two credentials over one resource, eight as two logins and one as two tokens,
once the adapter separates the resource from the credential.
On those facts, 0114 rotates a credential two parties hold over two logins, rotates one a single
party holds in place, and separates retiring a login from removing a consumer.
On those facts, 0114 rotates a credential two parties hold over two credentials, rotates one a single
party holds in place, staged, and separates retiring a credential from removing a consumer.
## What this changes in earlier records
@@ -192,7 +199,9 @@ On acceptance, each of these is superseded or amended by this record, not edited
credential and seals nothing stands.
- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own
secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the
first secrets. "The vault stores no plaintext, ever" stands.
first secrets. Genesis seals its values to the control-node's key as well as to the operator key, so
the controller can deliver them unattended. "The vault stores no plaintext, ever" and the
operator-sealed break-glass copies stand.
- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret
to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own,
so 0092's rule that an operator's value is never replaced does not apply to them.
@@ -201,9 +210,14 @@ On acceptance, each of these is superseded or amended by this record, not edited
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md),
[to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and
[to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each
requirement says whether its recipient applies it or reads it at start, while the changeover
mechanism stays as to-be 13 describes until its own record; the vault is installed as soon as the
requirement says whether its recipient applies it or reads it at start, and the changeover is
[ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s; the vault is installed as soon as the
shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker.
- [To-be 07](../03-DESIGN/01-to-be/07-the-foundation.md) is amended: genesis seals its values to
the control-node's key as well as the operator key.
- [To-be 12](../03-DESIGN/01-to-be/12-a-module-repository.md), [to-be 16](../03-DESIGN/01-to-be/16-module-coverage.md)
and [to-be 18](../03-DESIGN/01-to-be/18-building-a-module.md) are amended: `own-secrets` is retired from the
manifest they describe.
- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at
start*, and *reserved provision*, once this record is accepted.
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)
@@ -217,7 +231,7 @@ On acceptance, each of these is superseded or amended by this record, not edited
- Resolution expands per-consumer requirements from a provision's contract. The contract declares
them, never the provider's code, so what a provider requires stays predictable from the catalogue.
- A data provider's adapter gains a return value. What a credential provider's adapter must change for
rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-logins.md)'s.
rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s.
- The broker's provisioner gains every bus account, and the controller loses five separate places it
generates a secret today.
- 54 modules move from own secrets to vault requirements. Six provider clients export a password
@@ -236,7 +250,7 @@ On acceptance, each of these is superseded or amended by this record, not edited
| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. |
| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. |
| The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. |
| Genesis's values reach the vault unattended | An installer test: genesis's values are sealed to the control-node's key and delivered by the controller when the vault is installed, with no operator step. |
| Genesis's values reach the vault unattended, and the operator keeps a copy | An installer test: each of genesis's values is sealed to the control-node's key and to the operator key; the controller delivers the first to the vault when it is installed, with no operator step; the operator's copy opens only with the operator key. |
| Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. |
| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. |
| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. |
02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
+245
View File
@@ -0,0 +1,245 @@
---
topic: what runs on it
status: proposed
date: 2026-09-26
deciders: jochen
reconstructed: false
extends: 0113-the-vault-makes-every-secret.md
---
# 114. A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached
## Context
[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the
vault's policy) and who makes the new value (the vault). It leaves open how old and new change over.
[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every
credential provider in the catalogue against its code. There are nine:
- **all nine re-apply a password in place**, on the same login, every time they run. The controller's
`rotate` command relies on that, and states the window it leaves: between the provider applying the
new value and the consumer restarting with it, the consumer cannot authenticate;
- **eight of nine name the consumer's resource after its login**: a database, a bucket, a virtual host,
a key prefix, a topic prefix, a mailbox. Only the forge's npm registry keeps them apart, because an
organisation owns the packages;
- **five of nine destroy the consumer's data when they remove its login**: postgres, mssql and mongodb
drop the database, lavinmq drops the virtual host with its queued messages, and mailu deletes the
mailbox with its mail. In those adapters, *retire a login* and *delete the consumer's data* are one
call. minio drops a bucket only if it is empty. The provisioner harness makes it worse: a consumer
whose derived login changed is removed under the old login and created under the new one, in one pass;
- **one backend holds two passwords on one login** (redis), and two hold several tokens beside one
password (the forge and mailu);
- **eight of nine can give two logins the same rights over one resource**. mailu cannot, because a mail
user *is* its mailbox. It can give one user several tokens. In postgres, a second login is not enough
on its own: objects belong to whichever login created them, so the resource must be owned by a role of
its own;
- **an administrative credential has one party and a fixed name.** The provider module both applies it
and reads it. Five backends take it only at first initialisation: postgres, mssql, mongodb, mosquitto
and lavinmq. Their credential file is mounted directly into both the server and the provisioner, so
replacing the file recreates the provisioner holding only the new value, which the backend does not
know yet. The provisioner is then locked out;
- **no module watches a secret.** Every reader reads at start, and the host recreates a container when
a file it read at creation changes
([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)).
A first draft of 0113 chose to overlap old and new "through the adapter's existing create and
remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on
what the providers do, and the danger has to be closed whichever mechanism is chosen.
## Considered Options
**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials
two parties hold. The window cannot be closed, only shortened, and the two ends are on different
machines with nothing ordering them. For an administrative credential, it locks the provisioner out.
**2. Two secrets on one login.** Rejected as the mechanism. It works for three providers out of nine,
and using it there and something else elsewhere would put the difference in the mesh instead of in the
adapter.
**3. Two logins over one resource.** Rejected as the mechanism. It works for eight of nine, and not for
mailu.
**4. Two credentials over one resource, with the adapter choosing what a credential is.** A credential
is what a consumer presents, a login and a secret. The mesh alternates between two of them. Each adapter
makes the second one the way its backend can: a second login for eight providers, a second token on the
same login for mailu. A credential a single party holds is staged in place instead, and retiring a
credential is separated from removing a consumer before either is used. Chosen.
## Decision
### Retiring a credential never removes what it reached
**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its
database, bucket, virtual host, key or topic prefix, mailbox) and a *credential* that reaches it.
They get separate operations:
- **ensure the resource**, named after the consumer;
- **ensure a credential** with a value, holding the consumer's rights over its resource;
- **retire a credential**. Anything it owns moves first to the resource's owner, and any session it has
open is ended. Then the credential is removed, and nothing else;
- **remove the consumer**, which is what removes the resource, and retires every credential it has.
**Remove the consumer runs only when the consumer no longer requires the provision from this provider.**
That happens when its assignment goes, when its definition drops the requirement, or when re-resolution
sends it to another provider. It never runs because a login or a value changed. The harness keys what it
applied by the consumer, not by the login, so a changed login is a credential change and never a removal.
What removing a resource does with the data in it stays
[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s, and re-resolving to another provider
moves no data.
**The resource is named after the consumer, and owned by the resource, not by a login.** A consumer's
identity is derived from its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)),
and today its login is that same string, so **no existing resource is renamed**. Where a backend makes
whatever a login creates the login's own, as postgres does, the resource is owned by a role that cannot
log in, and each credential works as that role. Ownership of an existing resource moves to it once. A
credential is retired by handing what it owns to that role, never by dropping what it owns.
### A credential two parties hold rotates over two credentials
**Two parties** means an applier and a reader that are different modules, or a module and a node's
host. The vault's custody copy does not count, because the vault holds every secret. So this covers a
credential between a consumer and a provider, and every bus account: a module's or a host's, applied by
the broker's provisioner and read by its owner. **Each consumer has two credentials, one in use at a
time**, both holding the same rights over the one resource. For eight providers the second is a second
login, derived by the mesh as the consumer's identity with a short fixed suffix. For mailu it is a
second token on the same login.
**The vault drives each rotation and records every step durably.** A provisioner learns which
credentials to hold from what it receives: both of them, for as long as a rotation is under way. It
never learns them from its own memory, so a provisioner restarted mid-rotation resumes from the step the
vault has recorded.
1. **The vault makes the new value.**
2. **Each applier ensures the unused credential with it**, with the consumer's rights, and leaves the
one in use untouched. It verifies that the new credential authenticates and the old one still does,
and confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so
a lost message costs one pass.
3. **Only then does the vault release the new credential to the readers.** The mesh delivers it and the
value together, and the host recreates each reader, because a file it read at creation changed. A
node's host is its own reader: it reconnects to the bus with the new login, and confirms over it.
4. **Each reader confirms by authenticating with the new credential.** It shows this through its
health check, where its definition declares one, or the applier sees the new credential in use,
where its backend reports that. A reader for which neither is possible is confirmed by an operator.
It is never assumed from the reader having restarted.
5. **Only when every reader has confirmed is the old credential retired**, as above, and verified to no
longer authenticate.
**A reader that goes away leaves the rotation.** A reader unassigned, or re-resolved to another
provider, is no longer waited for. A consumer removed mid-rotation has both of its credentials retired
with it.
**A rotation can be abandoned until the old credential is retired.** An operator abandons it. Readers
that moved are given the old credential back, and recreated. The new credential is retired. Nothing is
lost, because the old one was never removed.
`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done
until the old credential is gone. A reader that cannot be reached keeps working on the old credential
until it can, and the rotation waits for it. That wait is shown, never hidden.
**Queues and permissions belong to the consumer, not to a login.** A module's queue on the bus is named
for the module on its node, and both of its logins get the same permissions over it
([ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)). An MQTT client
identifier is chosen by the consumer and is independent of its login. A reader recreated with a new
login keeps it, and the broker hands the session over.
### A credential a single party holds rotates in place, staged
This covers a provider's administrative credential and a module's own secret, which only that module
reads. The vault makes the new value, and the one party takes it:
- **applied**: the vault delivers the new value **staged, beside the current one**, and the current file
is left as it is. The party's provisioner changes the backend using the current value, verifies the
new one, and confirms. Only then does the vault make the new value current. This is the only form for
a backend that takes its administrative credential only at first initialisation. Replacing the file
first would lock the provisioner out;
- **read at start**: the vault delivers the new value as current, and the host recreates the party.
There is no window between two parties, because there is only one. Where neither form can change the
value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why
([ADR 0113](0113-the-vault-makes-every-secret.md)).
### One rule decides which
**The number of parties decides, never the provider.** The resolver knows it from the requirement's
recipients, leaving out the vault's custody copy, so no definition declares it.
### Until an adapter can
**An adapter that cannot yet ensure a second credential says so.** The two-party credentials it applies
rotate in place, as today, and the window is stated when the rotation is asked for. So does a
two-party credential whose backend has one fixed name and no second credential for it. These are listed
by a check, and the list is meant to shrink. Separating *retire a credential* from *remove the
consumer*, and keying the harness by consumer, come first. They close a data-loss path that exists
today, whatever rotation does.
## What this changes in earlier records
On acceptance, each of these is amended by this record, not edited:
- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here.
- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room
for the second login's suffix within the tightest backend it reaches, and both logins are checked
against it.
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker
account is two logins with the same permissions over the same queue, one in use at a time. Its scoping
is unchanged.
- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md), already superseded by 0113:
a provider now ensures and retires credentials over a resource it owns separately.
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party
credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed.
A single-party credential is staged, not replaced.
## Consequences
- **Every credential provider's adapter changes**, in two steps. The first separates *retire a
credential* from *remove the consumer*. It names and owns the resource after the consumer, which
keeps the name it has but moves ownership once in postgres and mssql, and the harness is keyed by
consumer. The second ensures a second credential with the same rights.
- **The vault gains rotation state**: each rotation's step, per applier and reader, recorded durably.
Staged delivery is added for single-party secrets. The SDK harness carries the alternation and the
repeated confirmation, so no adapter implements them.
- **No consumer module changes.** It reads one credential at start, as today, and is recreated by the
host when it changes. The exception is a reader that has neither a health check nor a backend that
reports use: its rotations wait for an operator until it declares one.
- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20
characters.
- **What got harder:**
- a provider briefly holds two credentials per consumer;
- a rotation lasts until its slowest reader moves, so an unreachable reader keeps the old credential
valid until it is reached;
- an adapter has four operations where it had two;
- retiring a login in mssql has to end its sessions first.
## How it is checked
| Rule | Checked by |
|---|---|
| Retiring a credential never removes a resource | A provider test per credential provider: retiring one of a consumer's credentials leaves its resource and data intact, reachable through the other. |
| What a retired login owned survives it | A postgres and an mssql test: objects created under login A, tables included, are still there and alterable under login B after A is retired. |
| A changed login is not a removal | A harness test: changing a consumer's derived login ensures a credential and never calls remove. |
| Remove runs only when the requirement goes | Harness tests: unassigning, dropping the requirement and re-resolving each remove the consumer once; a rotation and a login change never do. |
| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, with ownership moved to the resource's own role where the backend needs one. |
| Both credentials hold the same rights | A provider test per credential provider: data and structure created under one credential are read, changed and altered under the other. |
| Readers move only after the applier confirms | A rotation test: readers receive nothing until both credentials authenticate at every applier. |
| A reader confirms by authenticating | A rotation test: a reader recreated but failing to authenticate with the new credential does not confirm, and the old credential is not retired. |
| The old credential is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old credential, the rotation shows waiting on it, and it completes when the reader returns and confirms. |
| A reader that goes away leaves the rotation | A rotation test: unassigning a waiting reader lets the rotation complete; removing the consumer mid-rotation retires both credentials. |
| A rotation can be abandoned | A rotation test: abandoning after readers moved gives them the old credential back and retires the new one. |
| Rotation state survives a restart | A test restarting the applier's provisioner, and then the vault, between steps: the rotation resumes from the recorded step. |
| A single-party applied secret is staged | A rotation test on a first-initialisation administrative credential: the provisioner receives the new value beside the current one, applies it, and only then does the new value become current. At no point does it lose its connection. |
| The number of parties decides | A resolution test: a secret with an applier and a reader in different parties is marked for two credentials, and one held by one module for in place. The vault's copy is not counted. |
| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. |
| A host rotates its bus login | A rotation test on a node's bus account: the host reconnects with the new login and confirms over the bus before the old one is retired. |
| What still rotates in place is listed | A catalogue test lists every adapter that cannot yet ensure a second credential, and every two-party credential with one fixed name. A rotation of these states its window. |
## References
- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this
rests on, provider by provider
- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes
- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md),
[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving
its declaration
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md):
why a reader's restart can be derived
02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md
-198
View File
@@ -1,198 +0,0 @@
---
topic: what runs on it
status: proposed
date: 2026-09-26
deciders: jochen
reconstructed: false
extends: 0113-the-vault-makes-every-secret.md
---
# 114. A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached
## Context
[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the
vault's policy) and who makes the new value (the vault). It leaves open how old and new change over.
[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every provider
in the catalogue against its code:
- **all eight credential providers re-apply a password in place**, on the same login, every time they
run. The controller's `rotate` command relies on that, and states the window it leaves: between the
provider applying the new value and the consumer restarting with it, the consumer cannot authenticate;
- **seven of eight name the consumer's resource after its login**: a database, a bucket, a virtual host,
a key prefix, a topic prefix. Only the forge's npm registry keeps them apart, because an organisation
owns the packages;
- **five of eight destroy the resource when they remove the login**: postgres, mssql, mongodb, minio and
lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call;
- **only one backend holds two passwords on one login**, redis. A second, the forge, holds several
tokens beside one password;
- **every backend can give two logins the same rights over one resource**: a group role, a database
role, a shared policy, shared permissions, a shared role, a shared key prefix, a shared team. No
adapter does it today;
- **administrative credentials have one party and a fixed name.** The provider module is both the one
that applies the value and the only one that reads it. Three backends take it only at first
initialisation, so it can only be changed through a command run with the old value;
- **no module watches a secret.** Every reader reads at start, and the host already recreates a
container when a file it read at creation changes
([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)).
A first draft of 0113 chose to overlap old and new "through the adapter's existing create and
remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on
what the providers do, and the danger has to be closed whichever mechanism is chosen.
## Considered Options
**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials
two parties hold. The window cannot be closed, only shortened, and the two ends are on different
machines with nothing ordering them. A reader the mesh cannot reach, but which still reaches its
provider, is locked out until the mesh reaches it again.
**2. Two secrets on one login.** The applier adds the new password beside the old one. Rejected. It
works for one provider out of eight. Using it where it exists and something else elsewhere is a
mechanism per provider, which is what the mesh is trying to stop having.
**3. Two logins over one resource for everything.** Rejected for credentials a single party holds.
The applier and the reader are the same module, so there is no second party to keep working while the
other moves. A fixed administrative name has no second name to alternate with.
**4. Split by the secret, not by the provider.** A credential two parties hold rotates over two logins;
one a single party holds rotates in place; and retiring a login is separated from removing a consumer
before either is used. Chosen.
## Decision
### Retiring a login never removes what it reached
**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its
database, bucket, virtual host, key or topic prefix) and the *login* that reaches it. They get separate
operations:
- **ensure the resource**, named after the consumer;
- **ensure a login** with a value, holding the consumer's rights over its resource;
- **retire a login**, which removes the login and nothing else;
- **remove the consumer**, which is what removes the resource. It is run only when the consumer is
unassigned, as today, and never by a rotation. Whether the resource's data is kept beyond that stays
[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s.
**The resource is named after the consumer, not after a login.** A consumer's identity is derived from
its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), and today its login
is that same string. So **no existing resource is renamed**. The login every consumer holds today
becomes the first of its two, under the name it already has.
### A credential two parties hold rotates over two logins
This covers a credential between a consumer and a provider, and a bus account, which the broker's
provisioner applies and its module reads. **Each consumer has two logins, derived by the mesh**: its
identity, and its identity with a short fixed suffix. It uses one at a time, and both hold the same
rights over the one resource.
1. **The vault makes the new value.**
2. **Each applier ensures the unused login with it**, with the consumer's rights, and leaves the login
in use untouched. It verifies that the new login authenticates and the old one still does, and
confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so a
lost message costs one pass.
3. **Only then are the readers given the new login and value, together.** The host recreates each
reader, because a file it read at creation changed.
4. **Each reader confirms** by being recreated with the new login and passing its health check, where
its definition declares one.
5. **Only when every reader has confirmed is the old login retired.** Each applier retires it, the
login and nothing else, and verifies that it no longer authenticates.
`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done
until the old login is gone. A reader that cannot be reached keeps working on the old login until it
can, and the rotation waits for it. That wait is shown, never hidden.
**Where a backend identifies a connection separately from a login, the two are kept apart.** An MQTT
client identifier must be unique per connection, so the mosquitto adapter derives the connection's
identifier from the consumer and the login in use, and two logins never collide.
### A credential a single party holds rotates in place
This covers a provider's administrative credential and a module's own secret, which only that module
reads. **The vault makes the new value, and the one party takes it:**
- **applied**: the party's provisioner changes it using the old value, then confirms. This is the only
form for a backend that takes its administrative credential only at first initialisation, where a
restart would change nothing;
- **read at start**: the host recreates the party.
There is no window between two parties, because there is only one party. Where neither form can change
the value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why
([ADR 0113](0113-the-vault-makes-every-secret.md)).
### One rule decides which
**The number of parties that hold the credential decides, never the provider.** A requirement whose
secret has an applier and a reader in different modules rotates over two logins. A secret held by one
module rotates in place. The resolver knows which from the requirement's recipients, so no definition
declares it.
### Until an adapter can
**An adapter that cannot yet ensure a second login says so**, and the credentials it applies rotate in
place, as today, with the window stated when the rotation is asked for. That is a migration state, not
a second mechanism. It is listed by a check, and the list shrinks to empty. Separating *retire a
login* from *remove the consumer* comes first in every adapter, because it closes a data-loss path that
exists today, whatever rotation does.
## What this changes in earlier records
On acceptance, each of these is amended by this record, not edited:
- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here.
- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room
for the second login's suffix within the tightest backend it reaches, and both logins are checked
against it.
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker
account is two logins with the same permissions over the same queue, one in use at a time. Its scoping
is unchanged.
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party
credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed.
Single-party rotation keeps the form to-be 13 describes.
## Consequences
- **Every credential provider's adapter changes**, in two steps. The first separates *retire a login*
from *remove the consumer*, and names the resource after the consumer, which is the name it already
has. The second ensures a second login with the same rights. That is seven adapters for the second
step, since the forge's already holds its packages apart from the user.
- **The SDK's provider harness** carries the alternation, the verification of both logins, and the
repeated confirmation, so no adapter implements them. Its record of what was applied has to survive a
restart of the provisioner mid-rotation. Today it is kept in memory.
- **No consumer module changes.** It reads one login and a value at start, as today, and is recreated
by the host when they change.
- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20
characters.
- **What got harder:** a provider briefly holds two logins per consumer. A rotation of a two-party
credential lasts until its slowest reader moves, so an unreachable reader keeps the old login valid
until it is reached. And an adapter has four operations where it had two.
## How it is checked
| Rule | Checked by |
|---|---|
| Retiring a login never removes a resource | A provider test per credential provider: retiring one of a consumer's logins leaves its resource and data intact, reachable through the other. |
| Removing a consumer is not a rotation | A harness test: no rotation step calls remove; remove runs only when a contribution goes. |
| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, and its existing login becomes the first of its two. |
| Both logins hold the same rights | A provider test per credential provider: data written under one login is read and changed under the other. |
| Readers move only after the applier confirms | A rotation test: readers receive nothing until both logins authenticate at every applier. |
| The old login is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old login, the rotation shows waiting on it, and it completes when the reader returns and confirms. |
| A lost confirmation costs one pass | A harness test dropping the first confirmation: the next pass repeats it. |
| A restarted provisioner resumes a rotation | A harness test restarting the provisioner between steps: it resumes from the step it reached. |
| A single-party secret rotates in place | A vault test: a provider's administrative credential is applied by its own provisioner with the old value; a module's own secret recreates the module; neither has a second login. |
| The number of parties decides | A resolution test: a secret with an applier and a reader in different modules is marked for two logins, and one held by one module for in place, with nothing declared. |
| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. |
| A connection identifier never collides | A mosquitto provider test: two connections under a consumer's two logins are both accepted. |
| Adapters still in place are listed | A catalogue test lists every credential provider that cannot yet ensure a second login. The list shrinks to empty, and a rotation of their credentials states its window. |
## References
- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this
rests on, provider by provider
- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes
- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md),
[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving
its declaration
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md):
why a reader's restart can be derived
02-DECISIONS/README.md
+1 -1
View File
@@ -159,7 +159,7 @@ python3 00-META/checks/index.py fail if stale
- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(proposed)*
- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)*
- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)*
- **0114** — [A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached](0114-a-shared-credential-rotates-over-two-logins.md) *(proposed)*
- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)*
### How it is built
03-DESIGN/01-to-be/26-the-seats.md
+16
View File
@@ -152,3 +152,19 @@ With the seat unheld, a build from the seat is refused and says why. External bu
**Not yet designed:** a credential for cloning a private repository. The mesh's own repositories are
public. The natural place for a clone credential is a `secret` from the vault, and that is a decision
still to take.
## How it is checked
The rules here are [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)'s
and [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)'s, and each is
checked as their tables say:
| Rule | Checked by |
|---|---|
| The set is closed, and every entry names its decision | 0110: a unit test on the set's size and decisions; manifest tests refusing an unknown seat or the wrong scope. |
| A seat is held by one assignment, and only by one whose module can hold it | 0110: resolution tests for a second holder and for a seat the definition does not name. |
| A requirement naming a seat is answered by its holder; a foundation seat cannot be named | 0110: resolution tests with a second provider on the consumer's node, with the seat unheld, and naming `mesh-store`. |
| Several providers and none local is a person's choice | 0110: an assignment test listing candidates with the seat's holder first and recording the pin. |
| `secret` is reserved | 0110: the parser and resolution refusals for another provider and a pin. |
| Holdings are derived, and the overview lists every seat | 0110: the `seats` command test, including an unheld seat. |
| A build source on the seat records no address; an unheld seat refuses only self-hosted builds | 0111's tests. |
03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
+31 -19
View File
@@ -6,7 +6,7 @@ updated: 2026-09-26
decisions:
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
- 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
- 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md
- 02-DECISIONS/0084-which-provider-serves-a-consumer.md
@@ -64,8 +64,9 @@ Which module answers, in order:
the provider open. It asks for *the mesh's* one, and the mesh answers with whichever assignment
holds that seat, with nothing asked of anyone. Unheld, the requirement is refused, naming the seat
([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md),
[26 — The seats](26-the-seats.md)). A `secret` requirement always names `mesh-vault`, because
that provision is reserved;
[26 — The seats](26-the-seats.md)). Only a seat that delivers a provision can be named; naming a
foundation seat is refused, because it delivers nothing. A `secret` requirement always names
`mesh-vault`, because that provision is reserved;
2. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's
contents ([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md));
3. **the provider on the consumer's own node**;
@@ -250,22 +251,28 @@ applied by its provisioner or marked not rotatable by the mesh, and a rotation o
than reported done.
**How old and new change over depends on how many parties hold the credential**
([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md), on
([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), on
[research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md)):
- **Two parties**, a consumer and its provider, or a module and the broker: each consumer has two logins
derived by the mesh, both with its rights over one resource, named after the consumer. The vault makes
the new value; each applier ensures the unused login with it and confirms both authenticate; only then
are readers given it and recreated; once every reader has confirmed, the old login is retired. Nobody
is left without a credential that works, and `status` shows who a rotation waits on.
- **One party**, a provider's administrative credential or a module's own secret: in place. Its own
provisioner applies it with the old value, or the host recreates it.
- **Two parties**, a consumer and its provider, or a module or node's host and the broker: each consumer
has two credentials, both with its rights over one resource named after the consumer. For most
providers the second is a second login derived by the mesh; where a backend's user is its resource, it
is a second token. The vault drives the rotation and records each step. It makes the new value; each
applier ensures the unused credential and confirms both authenticate; only then are readers given it
and recreated; each reader confirms by authenticating with it; and only then is the old one retired.
Nobody is left without a credential that works, a rotation can be abandoned until the old one is
retired, and `status` shows who a rotation waits on.
- **One party**, a provider's administrative credential or a module's own secret: in place, staged.
An applied one is delivered beside the current value, the provisioner changes the backend with the
current one, and only then does the new value become current. One read at start is delivered, and
the host recreates the module.
**Retiring a login never removes what it reached.** An adapter keeps *retire a login* and *remove the
consumer* apart. Only unassigning removes the resource, and never a rotation. Today the two are one
call, and in five providers it deletes the consumer's data, so this separation comes first. An adapter
that cannot yet ensure a second login rotates in place, with its window stated, and is listed until it
can.
**Retiring a credential never removes what it reached.** An adapter keeps *retire a credential* and
*remove the consumer* apart, and the harness keys what it applied by consumer, so a changed login is
never a removal. The resource is removed only when the consumer no longer requires it from that
provider: unassigned, the requirement dropped, or re-resolved elsewhere. Today these are one call, and
in five providers it deletes the consumer's data, so this separation comes first. An adapter that cannot
yet ensure a second credential rotates in place, with its window stated, and is listed until it can.
## Refusing
@@ -314,8 +321,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r
mesh carries providers' data back.
[Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)
is fixed in the host already. *Ends when* nothing outside the vault generates a shared secret after
genesis, a lab consumer of analytics receives its site id, and a database credential rotates over its two
logins, the consumer recreated by derivation, never without a working login, and its data intact.
genesis, a lab consumer of analytics receives its site id, and a database credential rotates over
its two credentials, the consumer recreated by derivation, never without a working login, and its data intact.
3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted
and running assignments placed where their data already is, and each claim becomes a seat the
module can hold, held by the assignment that holds it today. *Ends when* the list of definitions
@@ -342,7 +349,12 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r
| Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. |
| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. |
| Restarts are derived | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, and one read at start recreates its reader without a declared restart. |
| A two-party credential rotates over two logins | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md): retiring a login leaves the resource intact; readers move only after the applier confirms; an unreachable reader keeps its old login until it returns; a single-party secret rotates in place. |
| A two-party credential rotates over two credentials | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md): retiring a credential leaves the resource intact; a changed login is never a removal; readers move only after the applier confirms and confirm by authenticating; an unreachable reader keeps its old credential until it returns; rotation state survives a restart; a single-party applied secret is staged. |
| A private key is made where it is used | The per-key tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): a node's sealing key, the operator's key and the certificate authority's key never leave where they were made. |
| The controller and a node's host take the same path | 0113's tests: the controller's definition declares requirements and no own secret; a node's bus account is made by the vault and delivered sealed to that node. |
| Moving the vault or the broker is break-glass | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. |
| A secret that cannot be rotated says so | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. |
| Only the controller reads the seat placeholder | A catalogue test, from phase 3: no definition uses the seat placeholder. |
| Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. |
| The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. |
03-DESIGN/01-to-be/README.md
+2 -2
View File
@@ -34,8 +34,8 @@ document is written and this one's status becomes `implemented`.
| [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) |
| [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) |
| [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) |
| [`26-the-seats.md`](26-the-seats.md) | What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) |
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) |
## Not yet written
04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md
+2 -2
View File
@@ -1,8 +1,8 @@
---
status: located
status: resolved
opened: 2026-09-23
located-in: [mesh-host internal/apply]
fixed-by:
fixed-by: mesh-host PR #22 — a container records the digest of every file it reads at creation, its env-files and files mounted into it directly, and is recreated when one changes; a mounted directory still needs restart-on
amended-design:
---
04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md
+7 -2
View File
@@ -1,7 +1,7 @@
---
status: open
status: located
opened: 2026-09-25
located-in: []
located-in: [mesh-catalog modules, mesh-controller internal/catalogue]
fixed-by:
amended-design:
---
@@ -60,6 +60,11 @@ module to one node would share every one of them. Assigning the same application
ordinary need: production beside staging, one site per customer, two instances of one service
configured differently, two stores of one engine.
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), which answers
this report, declines that need rather than meeting it. A module is assigned at most once to a node,
because every identity in the mesh is already a module on a node. The cases above become different
modules, or the same module on different machines.
## Why it matters beyond this instance
A definition that names machine paths is not portable between nodes. It cannot follow data onto a