novox/hq
3
0
Fork 0
Code Issues 3 Pull Requests 5 Releases Wiki Activity

Research 016: survey how each provider can rotate a credential

Browse Source 
Overlap as drafted in 0113 would have deleted consumer data: seven of
eight providers name the resource after the login and five drop it on
remove. Rotation is now undecided in 0113 and to-be 27, pending the
survey. Also: a requirement naming a seat resolves to its holder, a
person chooses among remaining candidates at assignment, the controller's
secrets are requirements of its definition, genesis seals to the
control-node key, and moving the vault or broker is break-glass.
This commit is contained in:
jochen
2026-09-26 00:14:23 +02:00
parent 805df3f81e
commit 942ebe350f
9 changed files with 422 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0113-the-vault-makes-every-secret.md
+69 -99
View File
@@ -36,9 +36,9 @@ it rather than step around it.
**Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in
which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so;
a container fed by an env-file is not recreated when that file changes
([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md));
and some secrets are read only when a service first initialises, where a restart changes nothing.
a container fed by an env-file was not recreated when that file changed
([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md),
since fixed in the host); and some secrets are read only when a service first initialises, where a restart changes nothing.
**And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)
left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics
@@ -93,11 +93,12 @@ it first means reordering the whole installation and giving the vault a second w
The vault cannot make that value. The module that received it delivers it to the vault, which keeps
it and provides it like any other; rotating it means asking the backend again.
**Parties that are not modules take the same path.** The controller's own store login and bus account,
and each node agent's bus account, have no definition to require them. The controller asks the vault
on its own behalf, or a node's, and the vault answers the way it answers any requirement: made by the
vault, sealed to the recipient, carried by the mesh. The requirement is not written in a definition,
because the controller and a node agent are the mesh itself, but it is answered no differently.
**The controller is a module, and takes the same path.** Its store logins (inventory, identity and
licences) and its bus accounts are own secrets of its definition today, and become `secret` requirements
of that definition like any module's. **A node's host is the one party with no definition.** Its bus
account is a requirement the mesh makes for each enrolled node, answered by the vault, sealed to that
node and carried like any other. It is the only requirement not written in a definition, because the
host is what runs definitions.
**Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat.
The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a
@@ -115,18 +116,25 @@ as that base exists**, before any other module built on it, and everything neede
generated by genesis:
- the store's superuser, and the broker's admin in the hashed form the broker needs;
- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder,
the broker's own provisioner and the vault;
- the controller's store login, and the first enrolment token.
- the bus accounts of the temporary and permanent controller (its account and the broker-management
login), the control-node's host, the builder, the broker's own provisioner and the vault;
- the controller's three store logins (inventory, identity and licences), and the first enrolment
token.
Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's
admin, as the controller does today. Genesis seals all of it to the operator key, and when the vault is
installed it **delivers the values to the vault, recorded as the mesh's own**, not as an operator's.
admin, as the controller does today. Genesis seals all of it to the control-node's key, and when the vault
is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not as an
operator's. Nobody has to be present for it.
That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)),
and these are, because the vault can make their replacements. The broker's provisioner then adopts the
accounts genesis created. From then on the vault makes every shared secret, and genesis has made its
last one.
**Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker`
seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the
values it needs are delivered, not made by a vault that is not there. That is a break-glass procedure,
stated and checked, never an ordinary assignment.
**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may
answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to
the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer
@@ -134,74 +142,44 @@ is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits
### Rotation
**It is asked of the vault**, by an operator or by the vault's policy, such as a maximum age in the
secret's contract. A delivered value the vault cannot replace, such as an external API key, is not
rotated by the vault: rotating it means an operator delivering a new one.
**Who asks and who makes are decided here; the mechanism is not.** A rotation is asked of the vault,
by an operator or by the vault's policy, such as a maximum age in the requirement's contract, and the
vault makes the new value. A delivered value the vault cannot replace, such as an external API key, is
not rotated by the vault: rotating it means an operator delivering a new one. A secret a backend
issued is rotated by the module that holds the backend asking it again and delivering the new value to
the vault.
**A secret's contract says how each recipient takes a new value:**
**Each recipient takes a new value one of two ways, marked on its requirement:**
| recipient takes it by | example | what happens on rotation |
|---|---|---|
| **applying** it | a provider creating the login; the broker's provisioner updating an account; the store's own provisioner changing its superuser | its provisioner applies the new value; it is never restarted for it |
| **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it |
| **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it |
| **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed |
A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks
it applied, and a provisioner makes the change. Where no provisioner exists to make it, such as a
module's own bootstrap password, the contract marks the secret **not rotatable by the mesh**, and a
rotation request is refused, saying why, rather than restarting a service that would carry on with
the old value. The host derives which recipients read a secret at start from the requirement their
definition reads it through, so no definition declares a restart for a secret. A provider's
per-consumer secrets are applied, never read at start, so the host never restarts a provider for one.
The marking is on each requirement, not on the secret, because one secret has recipients of both kinds.
Every module in the catalogue reads its secrets at start, and none watches them
([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a
backend takes only when it first initialises is marked applied, and its provider's provisioner makes
the change using the old value. Where no provisioner can make it, the requirement is marked **not
rotatable by the mesh**, and a rotation request is refused, saying why, rather than restarting a service
that would carry on with the old value.
**Old and new overlap: nobody is ever without a credential that works.** A credential is never
changed in place. The new one is added beside the old, every reader moves to it, and only then is the
old one removed. There is one mechanism, the same for every provider:
**How old and new change over is not decided here.** Three mechanisms were measured against every
provider's code in [research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md):
in place, as the controller's `rotate` does today; two secrets on one login; and two logins over one
resource. Its findings bound the choice:
1. **The vault makes the new value.**
2. **Each applier adds it beside the old.** A consumer has two logins, both derived by the mesh, and it
uses one at a time. The provider's loop creates the other with the new value, through the adapter's
existing create, and leaves the one in use untouched. It verifies that the new login works and the
old one still does, and confirms. It repeats that confirmation on every reconcile pass until the
vault acknowledges it, so a lost message costs one pass.
3. **Only then is the new login released to the readers.** A reader receives the new login and its
value together. The host restarts it, or recreates a container whose env-file carries it.
4. **Each reader confirms**, by restarting with the new login and passing its health check, where its
definition declares one.
5. **Only when every reader has confirmed is the old login retired.** Each applier removes it, through
the adapter's existing remove, and verifies that it no longer authenticates.
- every credential provider already re-applies a password in place, so today's rotation works, with a
window in which a consumer cannot authenticate;
- seven of eight name the consumer's resource after its login, and five destroy the resource when they
remove the login. **No mechanism may retire a login through today's remove**, because in those five
it deletes the consumer's data;
- only one backend holds two passwords on one login;
- every backend can give two logins the same rights over one resource, once the adapter separates the
resource from the login.
**What overlap closes:**
- a reader whose machine is offline keeps the old login, which still works, until it returns and
moves; the rotation shows as waiting on that reader, and nobody is locked out;
- a bus account's owner keeps its old account until it has confirmed the new one over the bus it still
has, so no party can lose the bus it would hear the new value on;
- a provisioner restarted mid-rotation is still delivered both values until the old is retired, so it
can verify either.
**What overlap costs.**
- **Consumer modules: nothing.** A consumer reads one login at a time and changes it when it restarts.
- **Providers: one duty.** Both of a consumer's logins must have the same rights over its data,
because the consumer's data was written under one login and is read under the other. In postgres,
both are members of one role that owns the data. That is the adapter's part, and the only place
overlap touches provider code. The alternation itself is the provider loop's, so every provider gets
it by using the harness.
- **The mesh:** it derives two logins per consumer, and both must still fit the tightest backend
([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)).
- **Secrets with no applier**, such as a module's own secret read only by itself, have no second party
to overlap with. They are delivered and the reader restarted, where their contract allows rotation at
all.
| step | who |
|---|---|
| asks | an operator, or the vault's policy |
| makes the value | the vault |
| adds the new login beside the old | each applier's provisioner, confirming on every pass until acknowledged |
| moves each reader | the host, restarting or recreating what reads the secret |
| confirms each reader | its restart and health check |
| retires the old login | each applier's provisioner, once every reader has confirmed |
| shows progress | `status`: waiting on which applier or reader, never done until the old is retired |
The mechanism is decided in its own record, on those facts. Until then rotation stays as the
controller implements it, in place, with its window stated.
## What this changes in earlier records
@@ -218,19 +196,17 @@ On acceptance, each of these is superseded or amended by this record, not edited
so 0092's rule that an operator's value is never replaced does not apply to them.
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker
account is created by the broker's provisioner, not the controller. Its scoping stands.
- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md) is amended: the mesh derives two
logins per consumer, and both fit the tightest backend.
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md),
[to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and
[to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation overlaps old and new
instead of being all-or-nothing, with restarts derived and each step confirmed; the vault is
installed as soon as the shared runtime base exists, and genesis delivers its secrets to it; the vault
is the only maker.
[to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each
requirement says whether its recipient applies it or reads it at start, while the changeover
mechanism stays as to-be 13 describes until its own record; the vault is installed as soon as the
shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker.
- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at
start*, and *reserved provision*, once this record is accepted.
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)
becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on
its old value.
is a prerequisite, and its fix is in the host: a container is recreated when a file it read at
creation changes. The issue is to be recorded as fixed, and derived restarts rest on it.
## Consequences
@@ -238,43 +214,37 @@ On acceptance, each of these is superseded or amended by this record, not edited
place, on the same node. A secret can no longer be made while the vault is down.
- Resolution expands per-consumer requirements from a provision's contract. The contract declares
them, never the provider's code, so what a provider requires stays predictable from the catalogue.
- The SDK's provider loop gains the alternation of two logins per consumer and repeated confirmation
of each step. A credential provider's adapter gains one duty, giving both logins the same rights over
the consumer's data. A data provider's adapter gains a return value. No consumer module changes.
- A data provider's adapter gains a return value. What a credential provider's adapter must change for
rotation is decided with the mechanism ([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md)).
- The broker's provisioner gains every bus account, and the controller loses five separate places it
generates a secret today.
- 54 modules move from own secrets to vault requirements. Six provider clients export a password
generator nothing uses any more; it is removed, so no module can quietly start minting again.
- The installation changes order: the vault is installed as soon as the shared runtime base exists,
before any other module built on it.
- **What got harder:** a rotation lasts until its slowest reader has moved, so a reader offline for a
week keeps the old login valid for a week. That is shown, and it is the price of never locking anyone
out. A provider briefly holds two logins per consumer. A secret some services read only at first
start can no longer be "rotated" by a restart that quietly changes nothing; it is refused instead.
- **What got harder:** a secret some services read only at first start can no longer be "rotated" by
a restart that quietly changes nothing; it is refused instead, or applied by its provisioner. And
moving the vault or the broker is a procedure, not an assignment.
## How it is checked
| Rule | Checked by |
|---|---|
| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls, with none exempt. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. |
| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls. Exempt are the vault itself, and randomness that is not a secret any other party holds, such as a password hash's salt, each named in a declared list. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. |
| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. |
| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. |
| Parties that are not modules take the same path | Controller tests: its own store login, its bus account and a node agent's bus account are each made by the vault and delivered sealed; an enrolment token reaches the controller only as what verifies it. |
| The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. |
| Genesis's values reach the vault unattended | An installer test: genesis's values are sealed to the control-node's key and delivered by the controller when the vault is installed, with no operator step. |
| Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. |
| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. |
| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. |
| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. |
| A rotation never destroys a consumer's data | A provider test per credential provider: rotating a consumer's credential leaves its resource and data intact. It fails today for no provider, because rotation is in place; it guards whichever mechanism replaces it. |
| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. |
| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. |
| Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. |
| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. |
| An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. |
| Old and new overlap | A rotation test: after an applier adds the new login, both authenticate; readers are released only after it confirms; the old login is removed only after every reader confirms, and then no longer authenticates. |
| An offline reader is never locked out | A rotation test with one reader's node offline: it keeps authenticating with the old login throughout, the rotation shows waiting on it, and completes when it returns. |
| A bus account's owner keeps the bus | A rotation test on a node agent's bus account: the agent stays connected on the old account until it has confirmed the new one. |
| A restarted provisioner can still verify | A rotation test restarting the applier's provisioner mid-rotation: it is delivered both values and confirms. |
| Both logins have the same rights | A provider test per credential provider: data written under one of a consumer's logins is read and changed under the other. |
| Both logins fit the tightest backend | A controller test: the two derived logins for the longest node and module names fit the limit ADR 0049 sets. |
| Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. |
| Rotation is confirmed | A rotation test: the rotation shows unconfirmed until every reader has restarted with the new login and passed its health check, and the old login is retired. |
| Restarts are derived from how a secret is read | A host test: a secret read at start recreates the container that read it at creation, through an env-file or a direct mount; an applied secret restarts nothing. A catalogue test: a secret that reaches a process, or a file in a mounted directory, has `restart-on` naming it. |
| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. |
## References