hq/02-DECISIONS/0113-the-vault-makes-every-secret.md

---
topic: what runs on it
status: proposed
date: 2026-09-25
deciders: jochen
reconstructed: false
---

# 113. The vault makes every shared secret, a provider makes resources and data, and the mesh carries both

## Context

**A shared secret comes into being many different ways today**, counted across the catalogue and the
controller on 2026-09-25:

| kind | made by | used by |
|---|---|---|
| a credential between a consumer and a provider | the controller | 16 modules, and 3 more for model access, counted below |
| a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules |
| a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules |
| a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder |
| an enrolment token | the controller | every node joining |
| a `secret` from the vault | the controller mints it, and the vault only records it ([ADR 0085](0085-a-secret-is-a-provision.md), as amended) | 6 modules |
| a value an operator accepts | a person ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) | where accepted |
| a licence for model access | a separate controller context with its own store | model consumers |
| the foundation's root secrets | genesis, sealed to the operator key | the foundation |

**The vault was built to end the second row, and did not.** ADR 0085 says a module's own secret
*"stops being a generated value that nothing owns"*. 54 modules still use one, and 6 use the vault.
The replacement was added and the old path was never retired.

**ADR 0085 considered and rejected making the vault the only maker**, because *"the controller must
mint in order to deliver any provision — the vault's own credential among them"*: the vault cannot
make the credentials that exist before it does. That objection is real, and this record has to answer
it rather than step around it.

**Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in
which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so;
a container fed by an env-file was not recreated when that file changed
([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md),
since fixed in the host); and some secrets are read only when a service first initialises, where a restart changes nothing.

**And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)
left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics
provider's site id and the DNS provider's record have no way back, and say so in their code.

## Considered Options

**1. Keep the controller minting, and tidy the paths.** Rejected. The paths are the problem: each is
made, kept, rotated and audited differently, and tidying keeps them all.

**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. Generation
becomes uniform, but custody stays spread over every provider's machine, so rotation, audit and the
operator's break-glass copies cover only some secrets. Each SDK language needs its own implementation.

**3. Raise the vault first at genesis, so it makes even the first secrets.** Rejected. The vault is
built on the shared runtime base, which the installation makes only after the store, the broker and
the controller exist, and the vault learns what to answer from the controller over the bus. Running
it first means reordering the whole installation and giving the vault a second way of being asked.

**4. The vault makes every shared secret; genesis delivers the first ones to it.** Chosen. It answers
0085's objection with a mechanism the mesh already has: a value delivered to the vault.

## Decision

**There are two kinds of secret, and each has one rule.**

- **A shared secret** is a value more than one party must hold: a password, a token, an API key. **The
  vault makes every one.** Nothing else in the mesh generates a shared secret.
- **A private key** is made where it is used and never leaves: a node's sealing key, the operator's
  key, the mesh's certificate authority. This is not a second way of making secrets. A private key any
  other party ever held would no longer be private.

**Every shared secret is a `secret` requirement, answered by the vault:**

- a **credential between a consumer and a provider**. A provision's contract declares *for each
  consumer, one secret*, and resolution expands it into one requirement per consumer. So gitea
  requiring a database makes the database's provider require a secret for gitea, and the vault
  answers it. The provider's own code does not change: it is handed a login and a password, as today;
- a module's **own secret**. `own-secrets` is retired;
- every **broker account** on the mesh's bus: a module's, a node's host's, the builder's, the
  controller's. The broker holding `mesh-broker` carries the mesh's bus
  ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates
  each account from the vault's secret, like any provider. The controller no longer creates accounts,
  and there is no separate command to forget;
- an **enrolment token**. The vault makes it; the operator receives the token, sealed to the operator
  key, to hand to the joining machine; the controller receives only what it needs to verify it, never
  the token itself;
- a **secret operator value**, such as an external API key, which the operator delivers to the vault
  ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these.
  What the licences context adds, refreshing a token, is provider behaviour, decided in its own record;
- a **secret a backend issues itself**, such as an API token a forge hands out exactly once when asked.
  The vault cannot make that value. The module that received it delivers it to the vault, which keeps
  it and provides it like any other; rotating it means asking the backend again.

**The controller is a module, and takes the same path.** Its store logins (inventory, identity and
licences) and its bus accounts are own secrets of its definition today, and become `secret` requirements
of that definition like any module's. **A node's host is the one party with no definition.** Its bus
account is a requirement the mesh makes for each enrolled node, answered by the vault, sealed to that
node and carried like any other. It is the only requirement not written in a definition, because the
host is what runs definitions.

**Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat.
The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a
`secret` requirement anywhere else, because there is nowhere else.

**A secret has recipients, and the vault delivers to each.** The database credential has two: the
provider, which *applies* it by creating the login, and the consumer, which *reads* it and presents it
when it connects. The vault hands the value to the mesh sealed to each recipient's node. The controller
and the broker carry sealed values they cannot open.

**Genesis delivers, and the vault adopts.** The vault is built on the shared runtime base, which the
installation makes only after the store, the broker and the controller are running
([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). So the vault is installed **as soon
as that base exists**, before any other module built on it, and everything needed before that moment is
generated by genesis:

- the store's superuser, and the broker's admin in the hashed form the broker needs;
- the bus accounts of the temporary and permanent controller (its account and the broker-management
  login), the control-node's host, the builder, the broker's own provisioner and the vault;
- the controller's three store logins (inventory, identity and licences), and the first enrolment
  token.

Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's
admin, as the controller does today. Genesis seals each value twice: to the control-node's key, so that when the
vault is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not
as an operator's, with nobody present; and to the operator key, as the break-glass copy
[ADR 0085](0085-a-secret-is-a-provision.md) keeps of every root secret. The first enrolment token reaches
the operator the same way.
That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)),
and these are, because the vault can make their replacements. The broker's provisioner then adopts the
accounts genesis created. From then on the vault makes every shared secret, and genesis has made its
last one.

**Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker`
seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the
values it needs are delivered, not made by a vault that is not there. They come from the operator-sealed
copies, which the operator opens. The vault keeps a copy of every secret sealed to the operator key
(0085), so nothing the mesh relies on exists only inside the vault. That is a break-glass procedure,
stated and checked, never an ordinary assignment.

**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may
answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to
the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer
is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)).

### Rotation

**Who asks and who makes are decided here; the mechanism is not.** A rotation is asked of the vault,
by an operator or by the vault's policy, such as a maximum age in the requirement's contract, and the
vault makes the new value. A delivered value the vault cannot replace, such as an external API key, is
not rotated by the vault: rotating it means an operator delivering a new one. A secret a backend
issued is rotated by the module that holds the backend asking it again and delivering the new value to
the vault.

**Each recipient takes a new value one of two ways, marked per recipient:**

| recipient takes it by | example | what happens on rotation |
|---|---|---|
| **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it |
| **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed |

The marking is per recipient, not per secret, because one secret has recipients of both kinds. A
provision's contract marks its provider's side, which applies. A consumer's side is read at start
unless its requirement says otherwise. The broker's contract marks the host's bus account the same
way: the broker's provisioner applies it, and the host reads it.
Every module in the catalogue reads its secrets at start, and none watches them
([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a
backend takes only when it first initialises is marked applied, and its provider's provisioner makes
the change using the old value. Where no provisioner can make it, the requirement is marked **not
rotatable by the mesh**, and a rotation request is refused, saying why, rather than restarting a service
that would carry on with the old value.

**How old and new change over is decided in [ADR
0114](0114-a-shared-credential-rotates-over-two-credentials.md).** Three mechanisms were measured against
every provider's code in [research
016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): in place, as the controller's
`rotate` does today; two secrets on one login; and two logins over one resource. Its findings bound the
choice:

- every credential provider already re-applies a password in place, so today's rotation works, with a
  window in which a consumer cannot authenticate;
- eight of nine name the consumer's resource after its login, and five destroy the consumer's data when
  they remove the login. **No mechanism may retire a login through today's remove**, because in those
  five it deletes the consumer's data;
- one backend holds two passwords on one login, and two more hold several tokens;
- every provider can hold two credentials over one resource, eight as two logins and one as two tokens,
  once the adapter separates the resource from the credential.

On those facts, 0114 rotates a credential two parties hold over two credentials, rotates one a single
party holds in place, staged, and separates retiring a credential from removing a consumer.

## What this changes in earlier records

On acceptance, each of these is superseded or amended by this record, not edited:

- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: the controller
  no longer mints a provider's credential; the vault makes it. That a provider is handed its
  credential and seals nothing stands.
- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own
  secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the
  first secrets. Genesis seals its values to the control-node's key as well as to the operator key, so
  the controller can deliver them unattended. "The vault stores no plaintext, ever" and the
  operator-sealed break-glass copies stand.
- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret
  to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own,
  so 0092's rule that an operator's value is never replaced does not apply to them.
- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker
  account is created by the broker's provisioner, not the controller. Its scoping stands.
- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md),
  [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and
  [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each
  requirement says whether its recipient applies it or reads it at start, and the changeover is
  [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s; the vault is installed as soon as the
  shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker.
- [To-be 07](../03-DESIGN/01-to-be/07-the-foundation.md) is amended: genesis seals its values to
  the control-node's key as well as the operator key.
- [To-be 12](../03-DESIGN/01-to-be/12-a-module-repository.md), [to-be 16](../03-DESIGN/01-to-be/16-module-coverage.md)
  and [to-be 18](../03-DESIGN/01-to-be/18-building-a-module.md) are amended: `own-secrets` is retired from the
  manifest they describe.
- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at
  start*, and *reserved provision*, once this record is accepted.
- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)
  is a prerequisite, and its fix is in the host: a container is recreated when a file it read at
  creation changes. The issue is to be recorded as fixed, and derived restarts rest on it.

## Consequences

- **The vault is on the path of every new or rotated shared secret.** Today the controller holds that
  place, on the same node. A secret can no longer be made while the vault is down.
- Resolution expands per-consumer requirements from a provision's contract. The contract declares
  them, never the provider's code, so what a provider requires stays predictable from the catalogue.
- A data provider's adapter gains a return value. What a credential provider's adapter must change for
  rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s.
- The broker's provisioner gains every bus account, and the controller loses five separate places it
  generates a secret today.
- 54 modules move from own secrets to vault requirements. Six provider clients export a password
  generator nothing uses any more; it is removed, so no module can quietly start minting again.
- The installation changes order: the vault is installed as soon as the shared runtime base exists,
  before any other module built on it.
- **What got harder:** a secret some services read only at first start can no longer be "rotated" by
  a restart that quietly changes nothing; it is refused instead, or applied by its provisioner. And
  moving the vault or the broker is a procedure, not an assignment.

## How it is checked

| Rule | Checked by |
|---|---|
| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls. Exempt are the vault itself, and randomness that is not a secret any other party holds, such as a password hash's salt, each named in a declared list. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. |
| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. |
| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. |
| The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. |
| Genesis's values reach the vault unattended, and the operator keeps a copy | An installer test: each of genesis's values is sealed to the control-node's key and to the operator key; the controller delivers the first to the vault when it is installed, with no operator step; the operator's copy opens only with the operator key. |
| Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. |
| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. |
| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. |
| A rotation never destroys a consumer's data | A provider test per credential provider: rotating a consumer's credential leaves its resource and data intact. It fails today for no provider, because rotation is in place; it guards whichever mechanism replaces it. |
| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. |
| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. |
| Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. |
| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. |
| An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. |
| Restarts are derived from how a secret is read | A host test: a secret read at start recreates the container that read it at creation, through an env-file or a direct mount; an applied secret restarts nothing. A catalogue test: a secret that reaches a process, or a file in a mounted directory, has `restart-on` naming it. |
| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. |

## References

- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes,
  and the return path it left open
- [ADR 0085](0085-a-secret-is-a-provision.md): the vault, and the objection this record answers
- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md),
  [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): delivered values,
  identity, and broker accounts
- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md):
  everything a module needs is a requirement
- [Issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md),
  [issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): what fails today