diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index 6800228..b55badc 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -92,13 +92,21 @@ and which assignment holds it, including seats nobody holds. An unheld seat is a has no X", not an error. **A seat delivers a provision only where the mesh has one answer for everyone.** That is a design -decision about the provision, not about the seat. The artifact store, the npm registry, git and the -vault are each one per mesh by their own records, so their seats deliver them. The store and the -broker are not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its -own stores, with a consumer served by the one on its own machine. So `mesh-store` and `mesh-broker` -keep guarding that the foundation's own server is singular, and deliver nothing. Were they to +decision about the provision, not about the seat. The broker, the artifact store, the npm registry, +git and the vault are each one per mesh by their own records, so their seats deliver them. The store +is not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its own +stores, with a consumer served by the one on its own machine, and the foundation's store is the +controller's own memory, provider to nobody ([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). +So `mesh-store` guards that the foundation's store is singular, and delivers nothing. Were it to deliver, every database consumer on every node would be sent to the control-node's store. +**A seat may reserve its provision.** Where a second provider would break a rule the provision exists +for, only the seat's holder may provide it at all: the parser refuses anyone else, and a pin cannot +choose anyone else. `secret` is the one reserved provision. The vault is one per mesh because a second +one *"would be a second place to lose"* ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and +a second `secret` provider is exactly that, whether a pin chose it or not. Every other delivered +provision may have second providers, which a pin can choose. + **The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and they name twelve seats because two alternative modules claim `the-resolver-configuration`. This record admits every seat the catalogue and the controller claim today, so no module is refused by @@ -108,8 +116,8 @@ it: |---|---|---|---|---| | `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | | `mesh-store` | mesh | — | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-broker` | mesh | — | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-vault` | mesh | `secret` | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | +| `mesh-broker` | mesh | `amqp` | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-vault` | mesh | `secret`, reserved | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | | `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | | `the-catalogue` | mesh | — | `mesh-catalog` | this record | | `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | @@ -166,7 +174,8 @@ question real. | A claim outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat whose claimant does not provide. | | Every module in use claims a seat in the set | A controller test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | | The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own machine, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | -| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` and `mesh-broker` deliver nothing, so a database consumer is still served by co-location. | +| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` delivers nothing, so a database consumer is still served by co-location, and `mesh-broker` delivers `amqp`. | +| A reserved provision has no other provider | The parser refuses a module providing `secret` without claiming `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | ## References diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index fa771d0..56449e6 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -68,7 +68,7 @@ unresolved requirement and what could answer it, all at once. | **another module** | a database, a bucket, a vhost, a secret, a route | provisions and bindings | | **the node's host** | a directory, a port, facts about the machine | resource paths, `${port:}`, `${machine:}`, facts | | **the mesh** | the module's identity and names, and the delivery of every answer | derived logins and generated names; the controller's delivery | -| **the operator, through the assignment** | a value a person chooses: a public name, a greeting, an external key | settings, carried literals | +| **the operator, through the assignment** | a value a person chooses that is not secret: a public name, a greeting, a number of workers | settings, carried literals | A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say: a pin, then the holder of a @@ -77,14 +77,14 @@ A host provider is always the module's own node, because a host path or a port m other. An operator value is the assignment's, or the requirement's default, or unresolved. **A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a -default. It needs no provider module, no grant and no credential. An operator value that is secret, -like an external API key, is still the operator's: the vault is where it is *kept*, as an -operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), not who -provides it. +default. It needs no provider module, no grant and no credential. -**Every secret is made by the vault, and a provider answers with resources and data** -([ADR 0113](0113-the-vault-makes-every-secret.md)). A provider that needs a secret for a consumer -requires it from the vault, like any consumer. The mesh carries every answer back. +**Every secret is a `secret` requirement, answered by the vault**, with no exception by kind +([ADR 0113](0113-the-vault-makes-every-secret.md)). An external API key an operator chooses is no +different: the operator delivers it to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), +and the module requires a `secret` like any other. A provider that needs a secret for a consumer +requires it from the vault, like any consumer, and answers with resources and data. The mesh carries +every answer back. **A directory is a host provision.** Its contract is the owner and mode the module needs, including the owner its image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index 7dd624d..fe69170 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -6,34 +6,39 @@ deciders: jochen reconstructed: false --- -# 113. The vault makes every secret, a provider makes resources and data, and the mesh carries both +# 113. The vault makes every shared secret, a provider makes resources and data, and the mesh carries both ## Context -**A secret comes into being seven different ways today**, counted across the catalogue and the +**A shared secret comes into being many different ways today**, counted across the catalogue and the controller on 2026-09-25: | kind | made by | used by | |---|---|---| | a credential between a consumer and a provider | the controller | 19 modules | | a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules | -| a broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder | +| an enrolment token | the controller | every node joining | | a `secret` from the vault | the controller mints it, and the vault only records it ([ADR 0085](0085-a-secret-is-a-provision.md), as amended) | 6 modules | | a value an operator accepts | a person ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) | where accepted | | a licence for model access | a separate controller context with its own store | model consumers | -| the mesh's root secrets | genesis, sealed to the operator key | the foundation | +| the foundation's root secrets | genesis, sealed to the operator key | the foundation | **The vault was built to end the second row, and did not.** ADR 0085 says a module's own secret *"stops being a generated value that nothing owns"*. 54 modules still use one, and 6 use the vault. -The replacement was added and the old path was never retired. The same has happened to the broker -account, a special case of the second row that fails silently when the separate command is forgotten. +The replacement was added and the old path was never retired. -**Rotation has its own gaps.** To-be 13 makes rotation one command, all-or-nothing, and states the -window in which a consumer cannot authenticate: the provider has taken the new password, and the -consumer has not yet restarted with it. A consumer restarts only if its definition remembered to say -so, and a container fed by an env-file is not recreated when that file changes, so it keeps the old -value ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). -Nothing confirms that the new secret works. +**ADR 0085 considered and rejected making the vault the only maker**, because *"the controller must +mint in order to deliver any provision — the vault's own credential among them"*: the vault cannot +make the credentials that exist before it does. That objection is real, and this record has to answer +it rather than step around it. + +**Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in +which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so; +a container fed by an env-file is not recreated when that file changes +([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)); +and some secrets are read only when a service first initialises, where a restart changes nothing. **And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics @@ -41,86 +46,112 @@ provider's site id and the DNS provider's record have no way back, and say so in ## Considered Options -**1. Keep the controller minting, and tidy the seven paths.** Rejected. The paths are the problem: -each is made, kept, rotated and audited differently, and tidying keeps all seven. +**1. Keep the controller minting, and tidy the paths.** Rejected. The paths are the problem: each is +made, kept, rotated and audited differently, and tidying keeps them all. -**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. It makes -generation uniform and leaves custody scattered: every provider's machine holds secrets the vault -never sees, so rotation, audit and the operator's break-glass copies cover only some of them. The -function would also need a conforming implementation in every language a provider is written in. +**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. Generation +becomes uniform, but custody stays spread over every provider's machine, so rotation, audit and the +operator's break-glass copies cover only some secrets. Each SDK language needs its own implementation. -**3. The vault makes every secret, and a provider that needs one requires it, like any consumer.** -Chosen. A provider serving a consumer requires a secret for that consumer from the vault. Secrets -become provisioning all the way down, with one maker at the bottom. +**3. Raise the vault first at genesis, so it makes even the first secrets.** Rejected. The vault is +built on the shared runtime base, which the installation makes only after the store, the broker and +the controller exist, and the vault learns what to answer from the controller over the bus. Running +it first means reordering the whole installation and giving the vault a second way of being asked. + +**4. The vault makes every shared secret; genesis delivers the first ones to it.** Chosen. It answers +0085's objection with a mechanism the mesh already has: a value delivered to the vault. ## Decision -**The vault makes every secret in the mesh.** Generation, to a secret's contract, exists in the vault -and nowhere else. The controller mints nothing. +**There are two kinds of secret, and each has one rule.** -**A provider that needs a secret for a consumer requires it from the vault.** A provision's contract -declares it: *for each consumer, one secret*. Resolution expands that into one requirement per -consumer, named for the consumer. So gitea requiring a database makes the database's provider require -a secret named for gitea, and the vault answers it. The provider's own code does not change. It is -handed a login and a password, as it is today. +- **A shared secret** is a value more than one party must hold: a password, a token, an API key. **The + vault makes every one.** Nothing else in the mesh generates a shared secret. +- **A private key** is made where it is used and never leaves: a node's sealing key, the operator's + key, the mesh's certificate authority. This is not a second way of making secrets. A private key any + other party ever held would no longer be private. -**A secret has holders, and the vault delivers to each.** The database credential has two: the -provider, which creates the login with it, and the consumer, which presents it. The vault hands it to -the mesh, which delivers it to each holder sealed to that holder's node. Plaintext exists in the -vault while it is made and on each holder's machine, and nowhere else. The controller carries sealed -values it cannot open. +**Every shared secret is a `secret` requirement, answered by the vault:** -**Every other secret takes the same path:** +- a **credential between a consumer and a provider**. A provision's contract declares *for each + consumer, one secret*, and resolution expands it into one requirement per consumer. So gitea + requiring a database makes the database's provider require a secret for gitea, and the vault + answers it. The provider's own code does not change: it is handed a login and a password, as today; +- a module's **own secret**. `own-secrets` is retired; +- every **broker account**: a module's, a node's, the builder's. The broker delivers `amqp` through + its seat ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and the broker's own + provisioner creates each account from the vault's secret, like any provider. The controller no + longer creates accounts, and there is no separate command to forget; +- an **enrolment token**; +- a **secret operator value**, such as an external API key, which the operator delivers to the vault + ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these. + What the licences context adds, refreshing a token, is provider behaviour, decided in its own record. -- a module's **own secret** is a `secret` requirement the vault answers. `own-secrets` is retired; -- a **broker account** is the module's identity on the bus. Its name is the mesh's, its password is a - secret the vault makes, and the controller creates the account with it, as it creates accounts - today. There is no separate command to forget; -- an **operator's value** that is secret is handed to the vault, which provides it like any other - secret. It is never replaced by rotation ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)); -- a **licence's** credential is an operator's value the vault keeps. What the licences context adds, - refreshing a token and a manager holding the refresh credential, is provider behaviour, decided in - its own record. +**Only the vault may provide `secret`.** A module providing it must hold the `mesh-vault` seat, and +the parser refuses one that does not. A pin cannot route a `secret` requirement anywhere else, because +there is nowhere else. -**Genesis is the vault's first answer, not an exception.** Genesis raises the vault before anything -else and asks it for the foundation's secrets: the store's superuser, the broker's admin and its hashed -form, and the vault's own broker account. The vault answers with the same code it always uses, before -the bus exists. Genesis mints nothing itself. +**A secret has recipients, and the vault delivers to each.** The database credential has two: the +provider, which *applies* it by creating the login, and the consumer, which *presents* it. The vault +hands the value to the mesh sealed to each recipient's node. The controller and the broker carry sealed +values they cannot open. -**A provider makes resources and data, and the mesh carries data back.** A provider answers with its -contract's non-secret fields: a site id, a registered name. The mesh delivers them to the consumer as -resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer is *given*, -never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). +**Genesis delivers, and the vault adopts.** The foundation's first shared secrets exist before the +vault can run: the store's superuser, the broker's admin in the hashed form the broker needs, the bus +accounts of the temporary controller and of the vault itself, and the first enrolment token. Genesis +generates these, seals them to the operator key as today, and **delivers them to the vault when the +vault is installed**, through the same path an operator's value takes. From then on the vault holds, +audits and rotates them. Unlike an operator's external key, the vault can make their replacements, so +they are delivered but replaceable. Genesis is the only thing besides the vault that ever generates a +shared secret, once, before the vault exists, and it hands them over. + +**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may +answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to +the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer +is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). ### Rotation -**It is asked of the vault**, by an operator or by the vault's own policy, such as the maximum age a -secret's contract sets. +**It is asked of the vault**, by an operator or by the vault's policy, such as a maximum age in the +secret's contract. A delivered value the vault cannot replace, such as an external API key, is not +rotated by the vault: rotating it means an operator delivering a new one. -**It is provider-first.** The vault delivers the new value first to the holders that *accept* it, -such as the database, and waits for each to confirm it has applied it. Only then does it release the -value to the holders that *present* it, such as gitea. A consumer is never sent a value its provider -has not accepted, so the window shrinks to the consumer's own restart. A provider that does not -confirm holds the rotation: the consumer keeps the old value, which still works, and `status` shows the -rotation as waiting on that provider. It is never half-done and never silently abandoned. +**A secret's contract says how each recipient takes a new value:** -**Restarts are derived, not declared.** The mesh knows which process reads which secret, because the -definition reads it through its requirement ([to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md)). -When a secret changes, the host restarts every process that reads it, and recreates a container whose -env-file carries it. No definition has to remember `restart-on` for a secret. +| recipient takes it by | example | what happens on rotation | +|---|---|---| +| **applying** it | a provider creating the login; the broker's provisioner updating an account; the store's own provisioner changing its superuser | its provisioner applies the new value; it is never restarted for it | +| **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it | -**It is confirmed.** A rotated secret is shown as unconfirmed until each consumer has restarted with -it and, where its definition declares a health check, passed it. Delivered is not the same as working, -and the mesh says which one it knows. +A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks +it applied, and a provisioner makes the change. The host derives which recipients read a secret at +start from the requirement their definition reads it through, so no definition declares a restart for +a secret. + +**It is applier-first.** The vault delivers the new value first to the recipients that apply it. Each +applies it, verifies that the new value authenticates and the old one no longer does, and confirms. +**It repeats that confirmation on every reconcile pass until the vault acknowledges it**, so a lost +message costs one pass. Only after every applier has confirmed does the vault release the value to the +recipients that read it at start. + +**The remaining window is stated.** If an applier applies the new value and its provisioner stops +before confirming, the recipients that present the secret are locked out until the provisioner runs +again, because the old value no longer works and they have not been sent the new one. A provisioner is +supervised and restarted when it exits, so the window is bounded by that restart. The mesh shows the +rotation as waiting on that applier for as long as it lasts, never as done. + +**It is confirmed.** A rotation is shown as unconfirmed until every applier has confirmed, and every +recipient that reads at start has restarted with the new value and passed its health check, where its +definition declares one. | step | who | |---|---| | asks | an operator, or the vault's policy | | makes the value | the vault | -| carries it | the controller, sealed, provider first | -| applies it on the provider | the provider's provisioner, which confirms | -| applies it on the consumer | the host, restarting or recreating what reads it | -| confirms it works | the consumer's restart and health check, shown in `status` | +| carries it | the controller, sealed, appliers first | +| applies it | each applier's provisioner, which verifies and confirms on every pass until acknowledged | +| takes it at start | the host, restarting or recreating what reads it | +| confirms it | applier confirmations, then restarts and health checks, shown in `status` | ## What this changes in earlier records @@ -129,53 +160,61 @@ On acceptance, each of these is superseded or amended by this record, not edited - [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: the controller no longer mints a provider's credential; the vault makes it. That a provider is handed its credential and seals nothing stands. -- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes a module's own secret rather - than recording one the controller minted, own secrets are retired, and genesis asks the vault for - the root secrets. "The vault stores no plaintext, ever" stands. +- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own + secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the + first secrets. "The vault stores no plaintext, ever" stands. - [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret - to the vault. -- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) and - [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is provider-first, - derived and confirmed, and the vault is the only maker. + to the vault, and genesis delivers the foundation's first secrets the same way. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker + account is created by the broker's provisioner, not the controller. Its scoping stands. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), + [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is applier-first, + derived and confirmed; genesis delivers its secrets to the vault; the vault is the only maker. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on its old value. ## Consequences -- **The vault is on the path of every new or rotated secret.** Today the controller is, and both run - on the control-node, so no new single point of failure appears. It is stated rather than implied. +- **The vault is on the path of every new or rotated shared secret.** Today the controller holds that + place, on the same node. A secret can no longer be made while the vault is down. - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. -- The SDK's provider loop gains one thing: confirming that a rotation was applied. Adapters are - unchanged. -- The mesh gains the way back from provider to consumer, for confirmations and for data. -- 54 modules move from own secrets to vault requirements, and the broker account stops needing a - separate command. -- **What got harder:** a secret can no longer be made when the vault is down, where today the - controller makes one regardless. And a rotation waits for its provider. Both move failures from - late and silent to early and visible, which is the trade this record makes throughout. +- The SDK's provider loop gains repeated confirmation of an applied rotation. A credential provider's + adapter is unchanged. A data provider's adapter gains a return value. +- The broker's provisioner gains every bus account, and the controller loses five separate places it + generates a secret today. +- 54 modules move from own secrets to vault requirements. +- **What got harder:** a rotation waits for its appliers, and an applier that stops mid-rotation locks + presenters out until it restarts. Both are shown, not hidden, and the second is bounded by a + supervised restart rather than by someone noticing. ## How it is checked | Rule | Checked by | |---|---| -| Only the vault makes secrets | A controller test: no code path mints a secret. A vault test: the one generation function is the only one, and genesis reaches it through the vault. | -| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement named for it, answered by the vault. | -| A secret reaches each holder sealed to its node | A controller test: each holder's copy opens with that holder's node key and with no other, the controller's included. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | +| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, and the operator's private key never enters the mesh. | +| Only the vault provides `secret` | The parser refuses a module providing `secret` without holding `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. | +| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. | | Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | -| A broker account needs no separate command | A resolution test: assigning a module that speaks on the bus yields its account. | -| Rotation is provider-first | A rotation test: the consumer is not sent the new value until the provider confirms; a provider that never confirms leaves the consumer on the old value, shown as waiting. | -| Restarts are derived | A host test: changing a secret restarts every process that reads it, and recreates a container whose env-file carries it, with no `restart-on` declared. | -| Rotation is confirmed | A rotation test: the secret shows as unconfirmed until the consumer restarted and passed its health check. | -| A provider answers data back | A resolution test: the analytics provider's site id reaches its consumer as a resolved value. | +| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | +| An irreplaceable delivered value is not rotated by the vault | A vault test: a rotation request on an operator's external key is refused, naming the operator as its source. | +| Rotation is applier-first and re-confirmed | A rotation test: presenters are not sent the new value until every applier confirms, and a confirmation lost in transit is repeated on the next pass. | +| Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. | +| Rotation is confirmed | A rotation test: an applier confirms only after the new value authenticates and the old one does not, and the rotation shows unconfirmed until every reader restarted and passed its health check. | +| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | ## References - [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, and the return path it left open -- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), - [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): the vault, the operator, and identity +- [ADR 0085](0085-a-secret-is-a-provision.md): the vault, and the objection this record answers +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), + [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): delivered values, + identity, and broker accounts - [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md): everything a module needs is a requirement - [Issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md), diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 984208e..bd8635a 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -158,7 +158,7 @@ python3 00-META/checks/index.py fail if stale - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) - **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* -- **0113** — [The vault makes every secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* +- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/23-choosing-a-provider.md b/03-DESIGN/01-to-be/23-choosing-a-provider.md index a16b953..ddb6670 100644 --- a/03-DESIGN/01-to-be/23-choosing-a-provider.md +++ b/03-DESIGN/01-to-be/23-choosing-a-provider.md @@ -57,9 +57,10 @@ consumer's own machine does not take over for that consumer. That is not picking once, mesh-wide, by assigning the holder, rather than once per consumer by naming it ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer -coupled to particular contents has said so. Only provisions the design makes one-per-mesh are -delivered by a seat: the artifact store, a package registry, git and the vault. A database is not. -Node-local stores, served by co-location, are the rule above. +coupled to particular contents has said so, except for `secret`, which only the vault may provide. +Only provisions the design makes one-per-mesh are delivered by a seat: the broker, the artifact store, +a package registry, git and the vault. A database is not. Node-local stores, served by co-location, +are the rule above. **Ambiguity is refused, never resolved by picking.** If several providers of a kind exist, none is named, none is co-located, and no seat delivers it, the requirement is unsatisfiable and is refused diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 719442f..d90dad0 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -48,8 +48,8 @@ argued for is an entry nobody can explain. |---|---|---|---| | `mesh-controller` | mesh | — | the controller | | `mesh-store` | mesh | — | the foundation's store | -| `mesh-broker` | mesh | — | the foundation's broker | -| `mesh-vault` | mesh | `secret` | the vault | +| `mesh-broker` | mesh | `amqp` | the broker | +| `mesh-vault` | mesh | `secret`, reserved | the vault | | `the-artifact-store` | mesh | `artifact-store` | the artifact registry | | `the-catalogue` | mesh | — | the catalogue | | `npm-package-registry` | mesh | `npm-package-registry` | the forge | @@ -64,8 +64,8 @@ argued for is an entry nobody can explain. The controller holds this set in code, and a test asserts both its size and that every entry names the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) -govern, and code that disagrees is what is wrong.** The implementation in progress predates two -things here: the `mesh-vault` seat, and the rule that `mesh-store` and `mesh-broker` deliver +govern, and code that disagrees is what is wrong.** The implementation in progress predates three +things here: the `mesh-vault` seat and its reservation, and the rule that `mesh-store` delivers nothing. It is brought to this table before it merges. ## A seat that delivers a provision @@ -73,11 +73,18 @@ nothing. It is brought to this table before it merges. A seat that delivers a provision may only be held by a module that provides it, at the seat's scope. A mesh seat delivers a mesh-scoped provision. -**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, -the npm registry, git and the vault are each one per mesh by decision. The store and the broker are +**A seat delivers a provision only where the mesh has one answer for everyone.** The broker, the +artifact store, the npm registry, git and the vault are each one per mesh by decision. The store is not: nodes run their own stores and a consumer uses the one on its machine -([23 — Choosing a provider](23-choosing-a-provider.md)). So their seats guard that the foundation's -own server is singular, and route nobody. +([23 — Choosing a provider](23-choosing-a-provider.md)), and the foundation's store is the controller's +own memory, provider to nobody. So `mesh-store` guards that the foundation's store is singular, and +routes nobody. + +**The vault's provision is reserved.** Only the holder of `mesh-vault` may provide `secret` at all: a +module providing it without the seat is refused, and a pin cannot choose another provider, because +there is none. A second provider of secrets would be a second place secrets live, which is what the +vault being one per mesh exists to prevent. Every other delivered provision may have second +providers, which a pin can choose. **Its holder answers for that provision.** A requirement for it resolves, in order, to: diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index b9fc2ff..a76658c 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -71,9 +71,17 @@ Which module answers, in order: ### Secrets: provisioning all the way down -**The vault makes every secret, and it is the only thing that does** -([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). It holds the `mesh-vault` seat, -so every `secret` requirement resolves to it. +**Two kinds of secret, one rule each** ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)): + +- **a shared secret**, a value more than one party must hold (a password, a token, an API key), is + made by the vault, and by nothing else; +- **a private key**, such as a node's sealing key, the operator's key or the mesh's certificate + authority, is made where it is used and never leaves. A private key anyone else held would no + longer be private. + +**Every shared secret is a `secret` requirement, and only the vault provides `secret`.** The vault +holds the `mesh-vault` seat, and that provision is reserved to it: no other module may provide it, and +no pin can choose another provider ([26 — The seats](26-the-seats.md)). **A provider that needs a secret for a consumer requires one, like any consumer.** A provision's contract declares it: *for each consumer, one secret*. Resolution expands that into one requirement @@ -82,16 +90,19 @@ per consumer, named for that consumer: 1. gitea requires `postgres-database`; 2. the database provider, to serve gitea, requires a `secret` named for gitea; 3. the vault makes it and hands it to the mesh; -4. the mesh delivers it to both holders, each sealed to its own node: the database's machine, to - create the login, and gitea's, to present it; +4. the mesh delivers it to both of its **recipients**, each sealed to its own node: the database's + machine, which *applies* it by creating the login, and gitea's, which *presents* it; 5. the database provider creates the login, exactly as it does today, and gitea connects. -A module's own secret, a broker account's password and a secret operator value take the same path. -Nothing in the mesh makes a secret except the vault. +Every other shared secret takes the same path: +- a module's own secret; +- every broker account's password, where the broker's own provisioner creates the account; +- an enrolment token; +- a secret operator value, which the operator delivers to the vault. -**A provider makes resources and data.** Beyond secrets, a provider answers with its contract's -non-secret fields: an analytics site id, a registered public name. The mesh carries them back to the -consumer as resolved values. +**A provider makes resources and data.** Beyond secrets, a provider's adapter may answer with its +contract's non-secret fields: an analytics site id, a registered public name. The mesh carries them +back to the consumer as resolved values. ### The node's host @@ -180,36 +191,54 @@ slug, under the same rules as a slug. This has to be settled before a second ins ## Genesis -**Genesis is the vault's first answer, not an exception.** It raises the vault before anything else -and asks it for the foundation's secrets: the store's superuser, the broker's admin in the hashed form -the broker needs, and the vault's own broker account. The vault answers with the same code it always -uses, before the bus exists, and seals the root secrets to the operator key as today -([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). Genesis makes no secret itself, so -there is one way a secret is made, from the first one onwards. +**Genesis delivers, and the vault adopts.** The vault cannot run first: it is built on the shared +runtime base, which the installation makes only after the store, the broker and the controller exist +([21 — The installation in full](21-the-installation-in-full.md)), and it learns what to answer from +the controller over the bus. So genesis generates the foundation's first shared secrets itself: +- the store's superuser; +- the broker's admin, in the hashed form the broker needs; +- the bus accounts of the temporary controller and of the vault; +- the first enrolment token. -The vault can sit at the bottom because it requires nothing but a broker account: it keeps its data -on its own disk, not in the store. So the waterfall ends at the vault. +It seals them to the operator key as today ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). +When the vault is installed, genesis **delivers them to it**, through the same path an operator's value +takes. From then on the vault holds, audits and rotates them. It can make their replacements, unlike +an operator's external key. + +That is the one time anything but the vault generates a shared secret, and it ends by handing them +over. It is also the answer to the objection ADR 0085 had to the vault being the only maker: the +vault cannot make what exists before it, so what exists before it is delivered to it. ## Rotation Rotating a secret is asked of the vault, by an operator or by the vault's own policy, such as a maximum age in the secret's contract ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). +An operator's external key is not rotated by the vault, which cannot make its replacement: an +operator delivers a new one. + +**A secret's contract says how each recipient takes a new value.** A recipient either *applies* it, +through a provisioner (a provider creating the login, the broker's provisioner updating an account, +the store's own provisioner changing its superuser), or *reads it at start*. A secret a service reads +only when it first initialises is marked applied, because a restart would change nothing. 1. **The vault makes the new value.** -2. **It is delivered to the holders that accept it first.** For a database credential that is the - database's provider, whose provisioner applies it and confirms it did. -3. **Only then is it released to the holders that present it**, such as gitea. A consumer is never - sent a value its provider has not accepted, so the window in which it cannot log in shrinks to its - own restart. A provider that does not confirm holds the rotation: the consumer keeps the old value, - which still works, and the rotation shows as waiting on that provider. -4. **The host restarts every process that reads the secret**, and recreates a container whose - env-file carries it. It knows which, because a definition reads a secret only through its - requirement. No definition declares a restart for a secret. This needs - [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) +2. **It goes first to the recipients that apply it.** Each applies it, verifies that the new value + authenticates and the old one no longer does, and confirms. It repeats that confirmation on every + reconcile pass until the vault acknowledges it, so a lost message costs one pass. +3. **Only then is it released to the recipients that read it at start**, such as gitea. +4. **The host restarts every such recipient**, and recreates a container whose env-file carries the + secret. It knows which, because a definition reads a secret only through its requirement, so no + definition declares a restart for a secret. An applying recipient is never restarted for it. + This needs [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) fixed, or a container fed by an env-file keeps the old value. -5. **It is confirmed.** The rotation shows as unconfirmed until each consumer has restarted with the - new value and passed its health check, where its definition declares one. Delivered and working - are shown as different things. +5. **It is confirmed.** The rotation shows as unconfirmed until every applier has confirmed, and every + recipient that reads at start has restarted and passed its health check, where its definition + declares one. Delivered and working are shown as different things. + +**The remaining window is stated.** An applier whose provisioner stops after applying and before +confirming leaves the recipients that read at start locked out: the old value no longer works, and +they have not been sent the new one. A provisioner is supervised and restarted when it exits, so the +window lasts until that restart. The rotation shows as waiting on that applier throughout, never as done. ## Refusing @@ -250,12 +279,14 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r the controller. The controller resolves requirements from the four providers, refuses as above, and fills the one form. Old mechanisms keep working beside it. *Ends when* a definition written entirely in the new form installs on a lab machine. -2. **The vault makes every secret, and providers answer.** The vault holds its seat and is the only - maker; resolution expands per-consumer secret requirements; genesis asks the vault first; the mesh - carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - is fixed first. *Ends when* no code path outside the vault makes a secret, the analytics and DNS - providers answer their consumers, and a database credential rotates provider-first, with the - consumer restarted by derivation and the rotation confirmed. +2. **The vault makes every shared secret, and providers answer.** The vault holds its seat and its + reserved provision; resolution expands per-consumer secret requirements; genesis delivers the + foundation's first secrets to the vault; the broker's provisioner creates every bus account; the + mesh carries providers' data back. + [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + is fixed first. *Ends when* nothing outside the vault generates a shared secret after genesis, a lab + consumer of analytics receives its site id, and a database credential rotates applier-first, with + the consumer restarted by derivation and the rotation confirmed. 3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is. *Ends when* the list of definitions using an old form is empty, and the old forms are removed. @@ -277,9 +308,10 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | A public name already held is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | | Everything keyed by a module is keyed by its instance | A resolution test: two instances of one module on one node get two directories, two containers, two logins and separate settings. | | A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | -| Only the vault makes secrets | A controller test: no code path mints a secret, genesis included. | -| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both holders. | -| Rotation is provider-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): the consumer waits for the provider's confirmation, is restarted without a declared restart, and shows unconfirmed until healthy. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | +| Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | +| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | +| Rotation is applier-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): readers wait for every applier's repeated confirmation; an applied secret restarts nothing, and one read at start restarts its reader without a declared restart; the rotation shows unconfirmed until the new value authenticates, the old does not, and readers are healthy. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. |