novox/hq
3
0
Fork 0
Code Issues 5 Pull Requests 5 Releases Wiki Activity

Seats held by assignments, one assignment per module per node, and 0113's bottom of the stack

Browse Source 
Decided with the author:
- A seat is held by one assignment, not claimed by a definition. A definition says which seats a module
  can hold; an assignment says which it does. The store module can run on every node and one assignment
  holds mesh-store; moving a role changes an assignment, never a definition. The foundation's seats name
  what the mesh itself uses and route no consumer — database and amqp consumers use co-location, the
  holder included. This replaces the wrong rationale that the foundation's store is "provider to nobody",
  which contradicted ADR 0078 and to-be 21. 0079's one-postgres rule becomes one mesh-store holder.
- A module is assigned at most once to a node. The instance identity in 0112 and 27 is withdrawn, and the
  login-length problem with it.

Review fixes to 0113:
- The bottom of the stack: the vault is installed as soon as the shared runtime base exists, and genesis
  generates everything needed until then — including the permanent controller's, the control-node
  agent's, the builder's and the broker provisioner's bus accounts, and the controller's store login.
  Genesis creates those accounts until the broker's provisioner runs and adopts them.
- Genesis's values are delivered recorded as the mesh's own, so 0092's never-replace rule for operator
  values does not make them unrotatable.
- Backend-issued secrets (a forge's once-only API token) enter through the vault. Non-module parties
  (the controller's logins, node agents' accounts) are answered the same way, the controller asking on
  their behalf; an enrolment token reaches the controller only as what verifies it.
- A secret with no provisioner to apply it is marked not rotatable by the mesh and refused, instead of
  a restart reported as done. Unused password generators in six provider clients are removed, and a
  catalogue scan checks no module mints.
- Rotation's lock-out cases (offline reader, bus account owner, restarted provisioner) are recorded as
  open, with overlap and re-confirm-with-safeguards as the two answers, to be chosen before acceptance.

0110, 0111 and 26 are marked proposed: they changed in meaning and are under review, and an accepted
record must not rest on proposed ones. To-be 23 and the glossary are restored to main; they change when
these records are accepted.
This commit is contained in:
jochen
2026-09-25 23:47:26 +02:00
parent 1b5f2c2c1a
commit 4a1b218706
9 changed files with 346 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0113-the-vault-makes-every-secret.md
+83 -39
View File
@@ -78,32 +78,54 @@ it first means reordering the whole installation and giving the vault a second w
requiring a database makes the database's provider require a secret for gitea, and the vault
answers it. The provider's own code does not change: it is handed a login and a password, as today;
- a module's **own secret**. `own-secrets` is retired;
- every **broker account**: a module's, a node's, the builder's. The broker delivers `amqp` through
its seat ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and the broker's own
provisioner creates each account from the vault's secret, like any provider. The controller no
longer creates accounts, and there is no separate command to forget;
- an **enrolment token**;
- every **broker account** on the mesh's bus: a module's, a node agent's, the builder's, the
controller's. The broker holding `mesh-broker` carries the mesh's bus
([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates
each account from the vault's secret, like any provider. The controller no longer creates accounts,
and there is no separate command to forget;
- an **enrolment token**. The vault makes it; the operator receives the token, sealed to the operator
key, to hand to the joining machine; the controller receives only what it needs to verify it, never
the token itself;
- a **secret operator value**, such as an external API key, which the operator delivers to the vault
([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these.
What the licences context adds, refreshing a token, is provider behaviour, decided in its own record.
What the licences context adds, refreshing a token, is provider behaviour, decided in its own record;
- a **secret a backend issues itself**, such as an API token a forge hands out exactly once when asked.
The vault cannot make that value. The module that received it delivers it to the vault, which keeps
it and provides it like any other; rotating it means asking the backend again.
**Only the vault may provide `secret`.** A module providing it must hold the `mesh-vault` seat, and
the parser refuses one that does not. A pin cannot route a `secret` requirement anywhere else, because
there is nowhere else.
**Parties that are not modules take the same path.** The controller's own store login and bus account,
and each node agent's bus account, have no definition to require them. The controller asks the vault
on its own behalf, or a node's, and the vault answers the way it answers any requirement: made by the
vault, sealed to the recipient, carried by the mesh. The requirement is not written in a definition,
because the controller and a node agent are the mesh itself, but it is answered no differently.
**Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat.
The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a
`secret` requirement anywhere else, because there is nowhere else.
**A secret has recipients, and the vault delivers to each.** The database credential has two: the
provider, which *applies* it by creating the login, and the consumer, which *presents* it. The vault
hands the value to the mesh sealed to each recipient's node. The controller and the broker carry sealed
values they cannot open.
provider, which *applies* it by creating the login, and the consumer, which *reads* it and presents it
when it connects. The vault hands the value to the mesh sealed to each recipient's node. The controller
and the broker carry sealed values they cannot open.
**Genesis delivers, and the vault adopts.** The foundation's first shared secrets exist before the
vault can run: the store's superuser, the broker's admin in the hashed form the broker needs, the bus
accounts of the temporary controller and of the vault itself, and the first enrolment token. Genesis
generates these, seals them to the operator key as today, and **delivers them to the vault when the
vault is installed**, through the same path an operator's value takes. From then on the vault holds,
audits and rotates them. Unlike an operator's external key, the vault can make their replacements, so
they are delivered but replaceable. Genesis is the only thing besides the vault that ever generates a
shared secret, once, before the vault exists, and it hands them over.
**Genesis delivers, and the vault adopts.** The vault is built on the shared runtime base, which the
installation makes only after the store, the broker and the controller are running
([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). So the vault is installed **as soon
as that base exists**, before any other module built on it, and everything needed before that moment is
generated by genesis:
- the store's superuser, and the broker's admin in the hashed form the broker needs;
- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder,
the broker's own provisioner and the vault;
- the controller's store login, and the first enrolment token.
Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's
admin, as the controller does today. Genesis seals all of it to the operator key, and when the vault is
installed it **delivers the values to the vault, recorded as the mesh's own**, not as an operator's.
That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)),
and these are, because the vault can make their replacements. The broker's provisioner then adopts the
accounts genesis created. From then on the vault makes every shared secret, and genesis has made its
last one.
**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may
answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to
@@ -124,9 +146,12 @@ rotated by the vault: rotating it means an operator delivering a new one.
| **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it |
A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks
it applied, and a provisioner makes the change. The host derives which recipients read a secret at
start from the requirement their definition reads it through, so no definition declares a restart for
a secret.
it applied, and a provisioner makes the change. Where no provisioner exists to make it, such as a
module's own bootstrap password, the contract marks the secret **not rotatable by the mesh**, and a
rotation request is refused, saying why, rather than restarting a service that would carry on with
the old value. The host derives which recipients read a secret at start from the requirement their
definition reads it through, so no definition declares a restart for a secret. A provider's
per-consumer secrets are applied, never read at start, so the host never restarts a provider for one.
**It is applier-first.** The vault delivers the new value first to the recipients that apply it. Each
applies it, verifies that the new value authenticates and the old one no longer does, and confirms.
@@ -134,11 +159,20 @@ applies it, verifies that the new value authenticates and the old one no longer
message costs one pass. Only after every applier has confirmed does the vault release the value to the
recipients that read it at start.
**The remaining window is stated.** If an applier applies the new value and its provisioner stops
before confirming, the recipients that present the secret are locked out until the provisioner runs
again, because the old value no longer works and they have not been sent the new one. A provisioner is
supervised and restarted when it exits, so the window is bounded by that restart. The mesh shows the
rotation as waiting on that applier for as long as it lasts, never as done.
**Open: keeping readers from being locked out.** Review found three cases this rule does not survive:
- a reader whose machine is offline when an applier has already applied the new value is locked out
until it returns, where to-be 13 would have refused the rotation and kept the old value working;
- a bus account's owner can be locked out permanently, because the confirmation and the new value
travel over the bus it has just lost;
- a provisioner restarted mid-rotation no longer knows the old value, so it cannot verify that the old
value has stopped working.
Two answers are recorded, and one must be chosen before this record is accepted. **Overlap:** an
applier keeps the old and new credential valid together until every reader has confirmed the new one,
for instance by alternating between two derived logins with the adapter's existing create and remove,
which needs no change on the consumer's side. Or **re-confirm with safeguards:** a pre-check that every
reader is reachable before any applier starts, plus a special path for bus accounts.
**It is confirmed.** A rotation is shown as unconfirmed until every applier has confirmed, and every
recipient that reads at start has restarted with the new value and passed its health check, where its
@@ -164,13 +198,17 @@ On acceptance, each of these is superseded or amended by this record, not edited
secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the
first secrets. "The vault stores no plaintext, ever" stands.
- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret
to the vault, and genesis delivers the foundation's first secrets the same way.
to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own,
so 0092's rule that an operator's value is never replaced does not apply to them.
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker
account is created by the broker's provisioner, not the controller. Its scoping stands.
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md),
[to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and
[to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is applier-first,
derived and confirmed; genesis delivers its secrets to the vault; the vault is the only maker.
derived and confirmed; the vault is installed as soon as the shared runtime base exists, and genesis
delivers its secrets to it; the vault is the only maker.
- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at
start*, and *reserved provision*, once this record is accepted.
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)
becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on
its old value.
@@ -185,24 +223,30 @@ On acceptance, each of these is superseded or amended by this record, not edited
adapter is unchanged. A data provider's adapter gains a return value.
- The broker's provisioner gains every bus account, and the controller loses five separate places it
generates a secret today.
- 54 modules move from own secrets to vault requirements.
- **What got harder:** a rotation waits for its appliers, and an applier that stops mid-rotation locks
presenters out until it restarts. Both are shown, not hidden, and the second is bounded by a
supervised restart rather than by someone noticing.
- 54 modules move from own secrets to vault requirements. Six provider clients export a password
generator nothing uses any more; it is removed, so no module can quietly start minting again.
- The installation changes order: the vault is installed as soon as the shared runtime base exists,
before any other module built on it.
- **What got harder:** a rotation waits for its appliers, and how a reader is kept from being locked
out while it does is still open (above). A secret some services read only at first start can no
longer be "rotated" by a restart that quietly changes nothing; it is refused instead.
## How it is checked
| Rule | Checked by |
|---|---|
| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. |
| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, and the operator's private key never enters the mesh. |
| Only the vault provides `secret` | The parser refuses a module providing `secret` without holding `mesh-vault`, and resolution refuses a pin on a `secret` requirement. |
| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls, with none exempt. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. |
| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. |
| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. |
| Parties that are not modules take the same path | Controller tests: its own store login, its bus account and a node agent's bus account are each made by the vault and delivered sealed; an enrolment token reaches the controller only as what verifies it. |
| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. |
| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. |
| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. |
| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. |
| Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. |
| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. |
| An irreplaceable delivered value is not rotated by the vault | A vault test: a rotation request on an operator's external key is refused, naming the operator as its source. |
| Rotation is applier-first and re-confirmed | A rotation test: presenters are not sent the new value until every applier confirms, and a confirmation lost in transit is repeated on the next pass. |
| An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. |
| Rotation is applier-first | A rotation test: readers are not sent the new value until every applier confirms. How lock-out is prevented is open, and its check is written when that is decided. |
| Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. |
| Rotation is confirmed | A rotation test: an applier confirms only after the new value authenticates and the old one does not, and the rotation shows unconfirmed until every reader restarted and passed its health check. |
| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. |