From 0c433d51ad12f20a71d1d4aaf4293351702571fb Mon Sep 17 00:00:00 2001 From: jochen Date: Wed, 23 Sep 2026 23:42:42 +0200 Subject: [PATCH 01/26] =?UTF-8?q?Design=2025:=20the=20bus=20on=20NATS=20?= =?UTF-8?q?=E2=80=94=20subjects,=20streams,=20accounts=20as=20configuratio?= =?UTF-8?q?n,=20enrolment,=20a=20person's=20client,=20the=20cutover,=20the?= =?UTF-8?q?=20beds?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The architecture ADR 0106 asks for, proposed for review before any code. --- 03-DESIGN/01-to-be/25-the-bus-on-nats.md | 223 +++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 03-DESIGN/01-to-be/25-the-bus-on-nats.md diff --git a/03-DESIGN/01-to-be/25-the-bus-on-nats.md b/03-DESIGN/01-to-be/25-the-bus-on-nats.md new file mode 100644 index 0000000..8ad276e --- /dev/null +++ b/03-DESIGN/01-to-be/25-the-bus-on-nats.md @@ -0,0 +1,223 @@ +--- +layer: to-be +status: proposed +code: + - mesh-controller internal/link (to be replaced) + - mesh-host internal/link (to be replaced) + - mesh-tools src/broker-amqp.ts (to be replaced) + - mesh-catalog modules/nats (to be written) +updated: 2026-09-23 +decisions: + - 02-DECISIONS/0106-the-bus-is-nats.md + - 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md + - 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md + - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md +--- + +# 25. The bus on NATS + +**Status: proposed — a design to be reviewed before any code.** This is the architecture +[ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md) asks for. It says what rides the bus, +under which subject, with which guarantee, under whose account; how a node joins; how a person +reaches a tool; how the mesh moves from the bus it has to this one; and how each claim is checked. +Prose and diagrams only; no configuration is pasted. + +## 1. What the bus is for + +The bus carries five kinds of traffic today, and this design keeps the five, renaming nothing a +module can see: + +| Traffic | Today | Guarantee it needs | +|---|---|---| +| **control** — a node's report, its heartbeat, a build's outcome, an enrolment | queues `control`, `.upgrades`, `.catchup` | nothing lost while the store restarts; retried; in order per node | +| **declarations** — the controller tells a node what to be | queue `node.` | the node gets the newest; a stale one is never applied | +| **builds** — the controller asks the build machine to build | queue `builds` | at least once, one builder at a time | +| **events** — a module says something happened | topic exchange `mesh.events`, keys `.` | delivered to every consumer that declared it; dead-lettered when it cannot be | +| **tools** — one module or person asks another's tool a question | exchange `mesh.rpc`, per-tool service queues `serve..` | one answer, from one server, or a timeout | + +The sdk's contract — `request`, `handle`, `publish`, `subscribe`, `close` — is the whole surface a +module sees, and it does not change ([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)). + +## 2. Subjects + +NATS addresses everything by subject. The mesh's subject space is one tree, and every account's +permissions are expressed as which branches of it that account may publish to and subscribe from. + +``` +mesh.control..report a node's report (JetStream: CONTROL) +mesh.control..alive heartbeat (core, no persistence) +mesh.control.enrol an enrolment request (JetStream: CONTROL) +mesh.control.built a build's outcome (JetStream: CONTROL) +mesh.node..declare a declaration for a node (JetStream: NODES, last-per-subject) +mesh.build.request work for the build machine (JetStream: BUILDS, work queue) +mesh.events.. an event (JetStream: EVENTS) +mesh.tools.. a tool invocation (core request/reply) +mesh.ask.. the controller's command api (core request/reply) +``` + +Two things this buys over the exchanges: **request/reply is native** — a tool call is one +`request` on `mesh.tools..` answered by whichever runtime serves it (a queue group per +tool, so several nodes may serve one tool); and **a declaration is last-per-subject** — the NODES +stream keeps only the newest message on `mesh.node..declare`, so a node that was away gets +exactly the current declaration and nothing older. That is the wire-level answer to +[issue 107](../../04-ISSUES/107-a-declaration-carries-no-order/00-report.md): the stream's sequence +*is* the order, and a node that sees sequence n refuses n−1 by construction. + +## 3. Streams, and the guarantees they carry + +Core NATS is at-most-once. Everything the mesh must not lose lives in a JetStream stream: + +| Stream | Subjects | Retention | Why | +|---|---|---|---| +| CONTROL | `mesh.control.>` except `alive` | work queue, one consumer (the controller), explicit ack | the store-window guarantee ([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)): the controller `nak`s with a delay while its store is away and the message is redelivered; nothing is dropped | +| NODES | `mesh.node.>` | last per subject | one declaration per node, always the newest | +| BUILDS | `mesh.build.>` | work queue, explicit ack | at least once; a builder that dies mid-build has its message redelivered | +| EVENTS | `mesh.events.>` | limits (age, size), durable consumer per subscribing module | a subscriber that was down catches up; after `max-deliver` attempts the advisory feeds `mesh.events.dead` (its own small stream) | + +Tool calls and heartbeats stay on core NATS: a lost heartbeat is the next heartbeat; a lost tool +call is a timeout the caller already handles. + +Streams and consumers are objects the controller creates at genesis and asserts on start; a module +declares nothing about them. The controller is the only writer of stream definitions. + +## 4. Accounts + +[ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) says a +module's account may publish only what it `emits` and consume only what it `consumes`. NATS +expresses this exactly, per subject, and better than a vhost could: + +- **One NATS account for the mesh.** Accounts in NATS isolate subject spaces entirely; the mesh + is one space, so it is one account. The predecessor's compatibility broker is not on this bus at + all. +- **One user per module per node**, as today, with publish permissions + `mesh.events..` for each emit, `mesh.tools..>` to serve its tools, and + its reply inbox; subscribe permissions for each consumed event's subject and its tool subjects. + Nothing else. A module that tries to publish outside its emits is refused by the server, not by + convention. +- **The controller's user** owns `mesh.control.>`, `mesh.node.>`, `mesh.build.>` and the streams. + **A host's user** may publish its own `mesh.control..>` and subscribe its own + `mesh.node..declare` — and nothing of any other node's. +- **A person's user** (§7) is a module-shaped user with permissions on the tool subjects it may + invoke, issued and revoked by the controller like any account. + +**Accounts are configuration, not API calls.** The controller composes the server's user list and +permissions into a file the host declares; the server reloads on change (`reload-on`, as the mesh +already does for the container runtime's trust). No management API, no credential travelling +through a management call, and the [issue 102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md) +discipline from the first day: an address or a permission is read where it is used, never stored +with a port. Passwords are minted and sealed exactly as today; the file holds bcrypt hashes. + +Alternative considered and not taken: the operator/JWT model (`nsc`), where accounts are signed +tokens resolved by the server. It is the right model for a multi-tenant NATS; the mesh is one +tenant, already has a sealing key and a controller that writes files, and would gain a second +signing hierarchy for nothing. + +## 5. The broker as a module + +`nats` is a catalogue module claiming the seat `mesh-broker` +([ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md): the seat +is the server, and the server changes). It declares one container (a single binary; JetStream on a +named volume), its listening ports — client, TLS, and the monitoring endpoint on loopback — a +configuration file the controller composes (accounts, permissions, TLS, JetStream), and a +`reload-on` for that file. Its guard is the same rule as the AMQP broker's: the monitoring port is +refused from anything but the private network. It is raised at genesis like the store, adopted as a +module in the same phase. The predecessor's AMQP broker remains a module of its own, +`lavinmq-compat`, with a single purpose and a retirement condition: no client connected for a +period the operator sets. + +## 6. Joining: the enrolment handshake + +Unchanged in shape, changed in transport. A node that has a token connects to the bus over TLS +with the **enrolment user** — a user that may publish `mesh.control.enrol` and subscribe one reply +inbox and nothing else — publishes its request (the claim of the token, its keys, its proof, and +the found tunnel from [ADR 0105](../../02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md)), +and waits on the inbox. The controller spends the token, records the node, composes the node's +own user into the server's configuration, and answers with the credentials sealed to the node's +sealing key. The node reconnects as itself. The enrolment user's permissions are what make a +leaked token useless for anything but enrolling: it cannot read a declaration or hear an event. + +## 7. A person's client + +The operator asked for the mesh's tools from a workstation, and for it designed here rather than +bridged. It is three things: + +1. **A person's account**: `operator issue ` on the controller creates a user whose + permissions are the tool subjects it may invoke — `mesh.tools.>` for an administrator, a list + for anyone else — and nothing on control, nodes or builds. It is issued, sealed to the person's + own key, and revoked, like a module's. +2. **A client that speaks the bus**: a small program on the workstation that connects as that user + over TLS, lists tools by asking the catalogue (`mesh.tools.mesh-catalog.catalog_tools`), and + turns each tool into a call — as an MCP server for an agent, and as a command line for a person. + It uses the sdk's `Broker` contract on the NATS runtime, so it is the same code path a module's + tools use, not a second protocol. +3. **Reachability**: the workstation reaches the bus over the private network once it is a node, + or over the predecessor's tunnel before that, on the bus's port; the guard and the openings + treat the bus as they do today. + +Nothing is built of this before §10's bed passes; the MCP surface is a thin adapter over (2). + +## 8. What a module sees + +Nothing new. `publish` on an envelope becomes a publish on `mesh.events..`; +`subscribe` with a pattern becomes a durable JetStream consumer on the matching subject filter; +`request`/`handle` become a NATS request and a queue-group subscription on +`mesh.tools..`. The envelope's shape ([ADR 0042](../../02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md)) +is unchanged; it is the message body. A module built today runs on the new runtime without a +rebuild — that is the test of ADR 0039, and it is in §10. + +## 9. Moving from the bus the mesh has + +Per ADR 0106: built beside, cut over once, after the core. + +1. The `nats` module, the controller's and host's link on NATS, the runtime's client — built and + proven in the lab (§10) while the migration continues on AMQP. Modules converted meanwhile + target the sdk contract and are untouched by this. +2. The cutover is one rollout, previewed: the controller assigns `nats` to the hub (raised beside + the AMQP broker on its own ports), composes every node's and module's account into it, then + rolls out the controller, every host and every runtime built for NATS. Each node's host connects + to the new bus as it comes up and reports; the controller confirms every node heard before it + stops listening on AMQP. The predecessor's clients never notice: their broker is the + compatibility module and stays. +3. The AMQP-side mesh accounts are removed from the compatibility broker; it keeps only the + predecessor's users. The bus's port settings follow ADR 0100 like any port. +4. The compatibility broker retires when its retirement condition holds. + +What is not done: no dual-bus period for the mesh's own traffic, no bridge, no module rebuilt. + +## 10. How it is checked + +Two lab beds, both required green before any node's bus moves. + +**The bus bed** — a mesh raised on NATS from genesis: +- a node enrols over TLS with a claimed token, and the enrolment user cannot read a declaration; +- a push composes; the store is stopped; the push is held (nak with delay), the store returns, the + push applies, nothing was lost or duplicated; +- a node that was away gets exactly the newest declaration, and a replayed older one is refused + by sequence; +- an upgrade rolls out to two nodes; +- a module's tool is invoked from another node and from a person's client, each with an account + that can invoke it, and refused from one that cannot; +- a module's account cannot publish outside its `emits` nor subscribe outside its `consumes` — + refused by the server; +- an event whose consumer keeps failing dead-letters after `max-deliver`; +- a module built before this design serves its tools unchanged on the new runtime. + +**The cutover bed** — a mesh on AMQP with a predecessor stand-in on the compatibility broker moves +its bus in one rollout; every node reports on NATS afterwards; the stand-in's client on AMQP is +still connected throughout. + +Unit tests hold the controller to composing accounts from `emits`/`consumes` and nothing else, to +creating the four streams and asserting them idempotently, and to spending a token exactly once; +the host to connecting as the enrolment user with nothing but enrolment permissions; the runtime +to mapping the sdk contract onto subjects exactly as §8 says. + +## 11. Open, for the review + +- Whether EVENTS should be one stream or one per emitting module (retention per module vs. one + policy). One stream is proposed; the review may disagree. +- The heartbeat interval and the controller's "quiet" threshold on core NATS without persistence — + the same numbers as today are proposed. +- Whether the person's client is a catalogue module (runs on an enrolled workstation node) or a + standalone program (runs anywhere with credentials). Both, in that order, is proposed. +- Leaf nodes: a NATS leaf per machine would make every module's connection local and survive the + hub's restart. Deliberately out of scope; noted so it is not forgotten. From f3ad60b98cce3f8cfe3fe7178e2e0a30d734c115 Mon Sep 17 00:00:00 2001 From: jochen Date: Wed, 23 Sep 2026 23:44:31 +0200 Subject: [PATCH 02/26] Issue 103 resolved by mesh-host #22 --- .../00-report.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md index f34684d..7e5086d 100644 --- a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md +++ b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md @@ -1,8 +1,8 @@ --- -status: located +status: resolved opened: 2026-09-23 located-in: [mesh-host internal/apply] -fixed-by: +fixed-by: mesh-host — a container is recreated when an env file or a directly mounted file it reads changes; a pre-upgrade label is accepted once; the plan names the file amended-design: --- From 6abfec74336718816703985417cb9ee1be676b71 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 24 Sep 2026 13:53:53 +0200 Subject: [PATCH 03/26] Design 25: address first review's four findings before any code Fixes, each named where it was wrong: - reload-on is a service field; a container only has restart-on, which recreates. Cited precedent (registry-trust-reload) is a service resource, not a container. Fix: nats-server's own SIGHUP reload, triggered by an in-image entrypoint watching a directory-mounted config file (issue 103's recreate-on-change applies to a directly-mounted file, not a directory's contents) -- asks nothing new of the host. - A JetStream-delivered message's Reply field is already claimed by the consumer's own ack address, so a responder using it answers nobody. Fix: every CONTROL message needing a reply carries its reply subject in its own payload; the controller publishes there explicitly, never via Respond(). Enrolment is the case this design actually depends on, so it's fixed there too, not just noted. - The listed permissions never granted publish on a durable consumer's own ack-reply subject -- a module could receive but never ack, so every message redelivers forever. Fixed with a scoped grant per module's own consumer. - One account (a deliberate choice, kept) means inbox privacy is the permission list or nothing. The design granted 'its reply inbox' without scoping it, which read as any user reaching any inbox. Fixed: each user's inbox prefix is derived from its own identity and its permissions name only that prefix. New open question from this revision, not closed: whether the in-image watch-and-SIGHUP shape belongs in mesh-sdk if a second module ever needs it. Checks pass (records.py, cycle.py, index.py). --- 03-DESIGN/01-to-be/25-the-bus-on-nats.md | 129 +++++++++++++++++++---- 1 file changed, 109 insertions(+), 20 deletions(-) diff --git a/03-DESIGN/01-to-be/25-the-bus-on-nats.md b/03-DESIGN/01-to-be/25-the-bus-on-nats.md index 8ad276e..3ffe4f7 100644 --- a/03-DESIGN/01-to-be/25-the-bus-on-nats.md +++ b/03-DESIGN/01-to-be/25-the-bus-on-nats.md @@ -6,7 +6,7 @@ code: - mesh-host internal/link (to be replaced) - mesh-tools src/broker-amqp.ts (to be replaced) - mesh-catalog modules/nats (to be written) -updated: 2026-09-23 +updated: 2026-09-24 decisions: - 02-DECISIONS/0106-the-bus-is-nats.md - 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md @@ -63,6 +63,21 @@ exactly the current declaration and nothing older. That is the wire-level answer [issue 107](../../04-ISSUES/107-a-declaration-carries-no-order/00-report.md): the stream's sequence *is* the order, and a node that sees sequence n refuses n−1 by construction. +**A reply-to travelling through a JetStream stream is carried in the payload, never in the +transport `Reply` field.** Revision, first review: core NATS request/reply sets the requester's +ephemeral inbox as the message's `Reply` field, and a plain responder answers it directly — but a +message a JetStream consumer delivers has already had that field claimed for the consumer's own +ack address (`$JS.ACK.....`), so by the time the controller (§3's CONTROL +consumer) sees the message, `Reply` names where *it* must ack, not where the original caller is +waiting. `mesh.control.enrol` is the case that matters: a synchronous-feeling caller waiting on an +ephemeral inbox, over a subject the store-window guarantee may legitimately delay by several +`nak` cycles — exactly the combination that would otherwise deliver the answer to a caller who +has long since timed out and unsubscribed. So every CONTROL message that expects an answer states +its reply subject as an ordinary field of its own payload; the controller reads it from there and +publishes the answer to it explicitly, never via `Respond()`. Nothing else in this design routes +a reply through a stream — tools and heartbeats stay on core NATS, where `Reply` means what it has +always meant. + ## 3. Streams, and the guarantees they carry Core NATS is at-most-once. Everything the mesh must not lose lives in a JetStream stream: @@ -89,9 +104,29 @@ expresses this exactly, per subject, and better than a vhost could: - **One NATS account for the mesh.** Accounts in NATS isolate subject spaces entirely; the mesh is one space, so it is one account. The predecessor's compatibility broker is not on this bus at all. + + **One account is a choice with a cost, stated plainly on revision:** none of NATS's own + isolation is free here, because there is only the one subject space, for everyone. Two + consequences that a single-account mesh must therefore grant on purpose, not by omission: + + - **A durable consumer needs permission to ack, or it never really consumes.** Acking a + JetStream delivery is a publish to that consumer's own ack-reply address + (`$JS.ACK...>`), a different subject from anything the consumer subscribes. + A module's user is therefore granted publish on `$JS.ACK.EVENTS..>` as well as its + emits — scoped to the one consumer name the controller derives for that module, so a module + can ack only its own deliveries. Without this, first review found, every message it receives + would be redelivered forever: refused by the permission list it already has. + - **A reply inbox needs a subject nothing else can guess or enumerate.** With one account, + inbox privacy is the permission list or it is nothing — there is no second account backing + it up. So no user is ever granted a bare `_INBOX.>`. Each user's inbox subject is derived + from its own identity (`_INBOX...>`, or `_INBOX.person..>`), and its + permissions name only that one prefix, for the reply to any request it makes and nothing + wider. First review found the account note without this and read it as "any user may + subscribe any inbox" — which was accurate against the text as it stood. - **One user per module per node**, as today, with publish permissions - `mesh.events..` for each emit, `mesh.tools..>` to serve its tools, and - its reply inbox; subscribe permissions for each consumed event's subject and its tool subjects. + `mesh.events..` for each emit, `mesh.tools..>` to serve its tools, its + own ack-reply subject for each durable consumer it holds, and its own inbox prefix; subscribe + permissions for each consumed event's subject, its tool subjects, and that same inbox prefix. Nothing else. A module that tries to publish outside its emits is refused by the server, not by convention. - **The controller's user** owns `mesh.control.>`, `mesh.node.>`, `mesh.build.>` and the streams. @@ -101,9 +136,10 @@ expresses this exactly, per subject, and better than a vhost could: invoke, issued and revoked by the controller like any account. **Accounts are configuration, not API calls.** The controller composes the server's user list and -permissions into a file the host declares; the server reloads on change (`reload-on`, as the mesh -already does for the container runtime's trust). No management API, no credential travelling -through a management call, and the [issue 102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md) +permissions into a file the host declares. **How that file reaches the running server is §5's, +not this one's** — revision, first review: an earlier draft said "reloads" and cited a precedent +that does not apply to a container (see §5). No management API, no credential travelling through a +management call, and the [issue 102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md) discipline from the first day: an address or a permission is read where it is used, never stored with a port. Passwords are minted and sealed exactly as today; the file holds bcrypt hashes. @@ -117,24 +153,53 @@ signing hierarchy for nothing. `nats` is a catalogue module claiming the seat `mesh-broker` ([ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md): the seat is the server, and the server changes). It declares one container (a single binary; JetStream on a -named volume), its listening ports — client, TLS, and the monitoring endpoint on loopback — a -configuration file the controller composes (accounts, permissions, TLS, JetStream), and a -`reload-on` for that file. Its guard is the same rule as the AMQP broker's: the monitoring port is -refused from anything but the private network. It is raised at genesis like the store, adopted as a -module in the same phase. The predecessor's AMQP broker remains a module of its own, -`lavinmq-compat`, with a single purpose and a retirement condition: no client connected for a -period the operator sets. +named volume), its listening ports — client, TLS, and the monitoring endpoint on loopback — and a +configuration file the controller composes (accounts, permissions, TLS, JetStream). + +**How that file's changes reach the running server, corrected on revision.** First review: the +earlier draft named `reload-on` as the mechanism, citing the container runtime's own trust file as +precedent. `reload-on` is real, but it is a **service** field +([mesh-host declaration.go](https://git.novox.be/novox/mesh-host), `Service.ReloadOn` — +`docker.service` is reloaded via systemd, which is what the cited precedent actually does). A +**container** resource has no reload field at all — only `restart-on`, and a container's +`restart-on` is documented, exactly, to mean *recreate*. Declared as the earlier draft had it, +either the field is silently meaningless on a container resource or — if read as the nearest real +equivalent — every account, permission, or key change recreates the bus's own server: every +connection dropped, every in-flight JetStream ack lost, mid-flight the moment a module is added, +reassigned, or a person's access changes. For the one resource everything else depends on, that +is not an edge case; it is the common case. + +**The fix asks nothing new of the host.** `nats-server` already reloads its own configuration +live on `SIGHUP` — accounts, permissions, everything in §4 — without dropping a connection; this +is the server's own documented capability, not something built for the mesh. So the composed +configuration file is mounted into a **directory** resource, not directly — a directory's contents +are not compared for change the way [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)'s +fix made a directly-mounted file's content, so a rewritten file inside it is not, on its own, a +reason to recreate the container. The image's own entrypoint watches that one file and sends +`nats-server` its own process `SIGHUP` when it changes — self-contained, inside the module, the +same place `modules/gitea/token.ts` keeps its own state rather than asking the host to model it. +The host's only job is what it already does for any directory resource: keep the file's content +current. Nothing is declared as `reload-on` or `restart-on` for this resource at all. + +Its guard is the same rule as the AMQP broker's: the monitoring port is refused from anything but +the private network. It is raised at genesis like the store, adopted as a module in the same +phase. The predecessor's AMQP broker remains a module of its own, `lavinmq-compat`, with a single +purpose and a retirement condition: no client connected for a period the operator sets. ## 6. Joining: the enrolment handshake Unchanged in shape, changed in transport. A node that has a token connects to the bus over TLS -with the **enrolment user** — a user that may publish `mesh.control.enrol` and subscribe one reply -inbox and nothing else — publishes its request (the claim of the token, its keys, its proof, and -the found tunnel from [ADR 0105](../../02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md)), -and waits on the inbox. The controller spends the token, records the node, composes the node's -own user into the server's configuration, and answers with the credentials sealed to the node's -sealing key. The node reconnects as itself. The enrolment user's permissions are what make a -leaked token useless for anything but enrolling: it cannot read a declaration or hear an event. +with the **enrolment user** — a user that may publish `mesh.control.enrol` and subscribe its own +`_INBOX.enrol..>` and nothing else — publishes its request (the claim of the token, its +keys, its proof, its own reply subject as §2 now requires, and the found tunnel from +[ADR 0105](../../02-DECISIONS/0105-the-mesh-adopts-the-predecessors-tunnel-in-place.md)), and waits +on that inbox. The controller spends the token, records the node, composes the node's own user +into the server's configuration, and — reading the reply subject from the request's payload, never +from the transport `Reply` field the CONTROL consumer has already claimed for its own ack — +answers with the credentials sealed to the node's sealing key. The node reconnects as itself. The +enrolment user's permissions are what make a leaked token useless for anything but enrolling: it +cannot read a declaration or hear an event, and it cannot subscribe any inbox but the one its own +token derives. ## 7. A person's client @@ -199,7 +264,16 @@ Two lab beds, both required green before any node's bus moves. that can invoke it, and refused from one that cannot; - a module's account cannot publish outside its `emits` nor subscribe outside its `consumes` — refused by the server; +- a module acks a delivery from its own durable consumer, and is refused acking another module's; +- a user subscribes another module's or person's inbox prefix and is refused by the server, not + by the client's own good behaviour; - an event whose consumer keeps failing dead-letters after `max-deliver`; +- an enrolment request held by a `nak`-with-delay cycle still reaches the enrolling node's inbox + once the controller answers — proving the reply travels in the payload and not the transport + field a consumer's ack has already claimed; +- the `nats` container is not recreated when only its composed configuration file changes, and + a change to that file is live (a new user can connect, a revoked one cannot) within one + watcher-poll interval, without a restart; - a module built before this design serves its tools unchanged on the new runtime. **The cutover bed** — a mesh on AMQP with a predecessor stand-in on the compatibility broker moves @@ -213,6 +287,14 @@ to mapping the sdk contract onto subjects exactly as §8 says. ## 11. Open, for the review +**Closed by this revision** (first review, recorded in `MIGRATION-LOG.md`, 2026-09-24): the +`reload-on`/container mismatch (§5), the eaten reply subject on a CONTROL-stream message (§2, §6), +the missing ack permission (§4), and the un-scoped reply inbox under one account (§4). Each is +named where it was wrong, not silently fixed, so a reader comparing against the first version can +find what changed and why. + +**Still open:** + - Whether EVENTS should be one stream or one per emitting module (retention per module vs. one policy). One stream is proposed; the review may disagree. - The heartbeat interval and the controller's "quiet" threshold on core NATS without persistence — @@ -221,3 +303,10 @@ to mapping the sdk contract onto subjects exactly as §8 says. standalone program (runs anywhere with credentials). Both, in that order, is proposed. - Leaf nodes: a NATS leaf per machine would make every module's connection local and survive the hub's restart. Deliberately out of scope; noted so it is not forgotten. +- **New, from this revision:** the `nats` image's own entrypoint now carries logic (watch a file, + signal a process) that no other module's container needed before. Is a one-file-watcher-and- + `SIGHUP` helper common enough across future modules with the same shape (a service that reloads + on `SIGHUP` but runs in a container) to belong in `mesh-sdk` rather than written once per module + that needs it? [ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)'s test — + *does editing it recompile unrelated modules, and does it change often* — probably says no for + one instance; worth asking again if a second module needs the same shape. From ab7d216fc6dd3df5080e9afd4e0cf778bb7b6eb0 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 00:55:13 +0200 Subject: [PATCH 04/26] Issue 113 resolved by the repin, and what issue 064 did not cover MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The object-store module was repinned to a maintained fork of the withdrawn server image, its runtime sidecar built rather than pulled, and its data moved off the predecessor's live directory. The instance is closed; the three general points the report makes are not, and What was done says so rather than letting a resolved status imply otherwise. Folds in the one thing a duplicate report of this symptom had that this one did not: issue 064 asked whether the build environment can reach a declared vendor image and assumed that, once declared, it stays fetchable. Withdrawal is the case that assumption does not cover. The duplicate is not merged — it carried the reading this report's diagnosis retracts. --- .../00-report.md | 26 +++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md b/04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md index 09a951e..8bfbab6 100644 --- a/04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md +++ b/04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md @@ -1,8 +1,8 @@ --- -status: located +status: resolved opened: 2026-09-24 located-in: [mesh-catalog modules/minio] -fixed-by: +fixed-by: mesh-catalog — the object-store module repinned to a maintained fork of the withdrawn server image, its runtime sidecar built from source rather than pulled, and its data moved off the predecessor's live directory. The standing condition this report names is not closed by it — see What was done. amended-design: --- @@ -109,6 +109,28 @@ a registry the mesh does not control — **by tag or by digest, it makes no diff dependency with no guarantee behind it, and the mesh currently learns it has lost one only by trying to use it. +[Issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md) is the +nearest precedent, and it does not cover this. That issue asked whether the mesh's build +environment can **reach** a declared vendor image — a network-policy question, answered by +requiring the image be declared as a build input — and it assumed that an image, once declared, +stays fetchable. Withdrawal is the case the assumption does not cover: no network policy and no +declaration makes a deleted repository resolvable, so a module can satisfy 064 in full and still +be unbuildable on a node that holds nothing. + +## What was done + +The module was repinned to a maintained fork of the server image, published to a registry that +still serves it; its runtime sidecar is now built from source rather than pulled; and its data was +moved off the predecessor's live directory. The object store runs on the control-node from that +pin, and a node holding nothing can obtain it again. + +That answers the instance and none of the three points above. The mesh still cannot say which of +its other pinned third-party images are still obtainable, and it would still learn of a withdrawal +only when a node without the image tried to deploy. The replacement question — S3 the protocol +rather than this product — is carried by +[research 015](../../01-RESEARCH/015-the-object-store-after-minio/00-overview.md); the detection +question is carried by nothing, and is the first of the open questions below. + ## Open questions - Should the mesh **hold** the images it depends on — mirroring third-party images into its own From 35db2aaa41555226495332aa1528f06509994d13 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 16:15:00 +0200 Subject: [PATCH 05/26] Issue 117: a module's own code is a container in one record and a process in another MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Asked what the "sidecar" is and whether a supervised process would do instead. The repository answers both ways. ADR 0047 (accepted, unsuperseded) says a module with tools or events runs a container carrying its compiled code. To-be 18 and 20 (both proposed) define a `process` resource type — the module's own code, a unit the machine's supervisor keeps up — and the worked guide says plainly "it is why these are `process` rather than four containers." Neither design doc names 0047, and no decision record mentions a `process` shape at all. Diagnosed rather than left open, because the ground truth settles what the report could not. The shape is real: mesh-host defines TypeProcess, applies it, and tests it, and the host's vocabulary is twelve shapes rather than the nine ADR 0029 counted. So the alternative the report offered — that two proposed documents describe a type that does not exist — is disproven. ADR 0029's mechanism is intact and was not enough. The vocabulary-count test names the decision behind each addition: network 0029, access 0051, opening 0100. The eleventh names a *proposed design document*, and TypeProcess is the only shape in the vocabulary whose doc comment cites no ADR. Requiring every addition to name something does not require it to name a decision. The argument this issue asked for already exists — as a Go test comment. "It is a full-host shape rather than a portable one: it needs a process supervisor to install into. It does NOT need a container runtime, which is the point — only software that genuinely needs isolation asks for a container." That is a decision's context and consequences, in another repository. What the catalogue does is a third thing: 115 container declarations against 3 process, all three in showcase — the module to-be 20 documents. There the tools resource is a container running `sleep infinity` on a bare upstream base with the broker credential mounted, and the tools and provisioner entrypoints are run by nothing. That is the condition 0047 was written to end, back in a new shape. Where the isolation argument leaks is narrower than expected and worth having precisely: the serving key and the credential shape both conform. But serveTools serves every registered module over one broker connection, the runtime takes its modules from a comma-separated list, and x-source is stamped from the single credential — so two modules in one runtime means the second's events are attributed to the first. Nothing refuses it and no test asserts against it. Located on hq rather than on a code repository: the implementation and the design layer agree, and the missing thing is the record. Which shape is right is left open, deliberately — this establishes that the question was answered in practice and never written down, not which answer is correct. One correction kept in the trail: the first search here was for len(Vocabulary()), found nothing, and was two steps from being written up as "the mechanism ADR 0029 relied on is gone." The test binds the slice to a local first. A negative search result read as a fact about the world is the same error issue 113 recorded. --- .../00-report.md | 101 +++++++++ .../01-diagnosis.md | 193 ++++++++++++++++++ 2 files changed, 294 insertions(+) create mode 100644 04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/00-report.md create mode 100644 04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md diff --git a/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/00-report.md b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/00-report.md new file mode 100644 index 0000000..1ce0e9c --- /dev/null +++ b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/00-report.md @@ -0,0 +1,101 @@ +--- +status: located +opened: 2026-09-25 +located-in: [hq, mesh-catalog modules/showcase, mesh-sdk src/tools/index.ts, mesh-tools] +fixed-by: +amended-design: +--- + +# 117 — A module's own code is a container in one record and a process in another + +## What was observed + +Asked what the "sidecar" is — the second container a code-carrying module runs beside its +service — and whether a supervised process would do instead. Reading the records to answer it, +the repository answers both ways, and nothing reconciles them. + +| record | status | what runs a module's own code | +|---|---|---| +| [ADR 0047](../../02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md) | **accepted**, 2026-09-04 | "a **container**, the tool runtime carrying that module's compiled code" — one module, one process, one account; events and tools in that same process, "not a second one to scope and seal" | +| [`01-to-be/18-building-a-module.md`](../../03-DESIGN/01-to-be/18-building-a-module.md) | proposed, 2026-09-21 | a resource type table in which `container` is "an image" and **`process`** is "**its own code**, in three modes", whose default mode is "a unit restarted when it exits", supervised by the machine | +| [`01-to-be/20-writing-a-module.md`](../../03-DESIGN/01-to-be/20-writing-a-module.md) | proposed, 2026-09-21 | one module declaring **four** `process` resources — events, tools, provisioner, a scheduled ingest — each with its own `run` argv, and the sentence "it is why these are `process` rather than four containers" | + +Three disagreements, not one: + +1. **Container or unit.** ADR 0047 chose a container and said why: a node-wide runtime loading + every module's code could not hold a per-module account, so the runtime is per-module. The + design docs choose a supervised unit running an argv and give no reason, because they do not + record that they are choosing. +2. **One process or several.** ADR 0047's "one module, one process, one account" is the whole + content of its second and third sections. The worked guide declares four for one module and + presents four as the point. +3. **Whether the record was consulted at all.** Neither design doc names ADR 0047 in + `decisions:`. No record supersedes or extends it on this. **The string `process` as a resource + type appears in no decision record** — the shape exists only in two `proposed` design docs. + +Meanwhile the thing as built is the container. [ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md) +records that "anything that is a service plus a sidecar currently has to publish a port to talk +to itself," which is one of the things the host's `network` shape was added for. +[Issue 113's diagnosis](../113-the-object-stores-images-were-withdrawn-upstream/01-diagnosis.md) +found a catalogue module declaring "two container resources," the second a runtime sidecar +"pinned at an all-zeros digest, meaning nothing was ever published for it." +[Issue 095](../095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md) is a +sidecar crash-looping on a credential while its service served correctly. +[ADR 0093](../../02-DECISIONS/0093-a-fixture-that-runs-a-modules-runtime-carries-its-name.md) +records that a bed wanting "a sidecar without its server raises the server." + +### And the word is in no glossary + +"Sidecar" appears sixteen times across five records — two decisions and three issues. It is +absent from [`00-META/glossary.md`](../../00-META/glossary.md), and absent from every document +under [`03-DESIGN/`](../../03-DESIGN/), in both layers. ADR 0047, which creates the thing, never +uses the word; it says "runtime process" and "runtime container". The glossary's own rule is that +"a new name for an existing thing lands here first, in the same change that introduces it in +code," and the page exists because "the terms kept drifting in conversation." A reader asking +what the sidecar is has nowhere in the design layer to look, which is how this was found. + +## Why it matters beyond this instance + +- **A module author reading the current guide writes a `process`; the catalogue as built declares + a `container`.** [`20-writing-a-module.md`](../../03-DESIGN/01-to-be/20-writing-a-module.md) is + a worked guide with a manifest in it. Whichever of the two is wrong, somebody follows it. +- **The cost of the container shape is paid in four places and totalled in none.** A published + image per code-carrying module, a network so a module can reach itself, a bed that cannot run a + runtime without raising the server it manages, and a credential failure that presents as the + module's own bug. Each record argues its own piece is worth paying. No record puts them beside + the alternative. +- **Both shapes carry a cost the other does not, and neither is written down.** A container + carries its own interpreter; a `process` declaring `run: ["node", "index.js"]` needs an + interpreter present on the machine, which is the machine dependency the statically linked host + ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) exists to avoid. And `run` is an argv, + where [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) refuses `action` because the link may + not carry a command — a refusal [`18-building-a-module.md`](../../03-DESIGN/01-to-be/18-building-a-module.md) + restates on the same page that it introduces `process`. +- **This is the repository's own named failure mode, in its own records.** `cycle.py` enforces + that a to-be doc names *at least one* decision. Both docs do, so both pass, while introducing a + resource type no decision records and contradicting an accepted one. The rule is "no design + without a decision"; the check is "no design without *a* decision." An unenforced rule is + indistinguishable from a wrong one, and these two documents are what that gap looks like when + something walks through it. + +## Open questions + +- Which is the decision — container or supervised unit? If the design docs are right, ADR 0047 + needs superseding rather than quietly outliving. If ADR 0047 is right, two proposed documents + and a worked manifest describe a resource type that does not exist. +- Is one account per module satisfied by a per-module *unit* as well as a per-module *container*? + ADR 0047's argument rules out a node-wide runtime sharing one account. It does not appear to + rule out a unit holding one scoped credential, and nothing has said so either way. +- If several processes for one module are right, what holds the accounts? ADR 0047 refused "a + second one to scope and seal" for events beside tools. Four processes are four somethings. +- How does a `process` get its interpreter, and does declaring one reintroduce the machine + dependency the host is built to avoid? +- Is `run` an argv the link may carry, given `action` is refused for being one? If the answer is + that a `process` reconciles and an `action` does not, that distinction is not written down. +- What is the thing called, and where does the design layer describe it? Whichever shape wins, no + document in either layer currently says a code-carrying module runs a second thing beside its + service. +- **How would this have been caught?** A decision and a design doc disagreeing on a resource type + is mechanically checkable: the resource types a design doc names are a closed set, and every + member of it either appears in a decision or does not. Whether that check is worth writing is + part of this issue, not settled by it. diff --git a/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md new file mode 100644 index 0000000..c6f6b3a --- /dev/null +++ b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md @@ -0,0 +1,193 @@ +# Diagnosis — 117 + +## Which trees were searched, 2026-09-25 + +Named first, because [issue 113](../113-the-object-stores-images-were-withdrawn-upstream/01-diagnosis.md) +is the record of reporting absence in one repository as absence in the mesh. + +| Searched | At | +|---|---| +| `mesh-host`, `mesh-catalog`, `mesh-tools`, `mesh-sdk`, `mesh-controller` | `main`, fresh shallow clones | +| `hq` | `main`, and the two branches named under finding 7 | + +**Not searched:** the private migration repository; the open pull requests on the catalogue and +the controller; any branch of a code repository other than `main`. A statement below about "the +catalogue" is a statement about its `main`. + +## The report's central question is answered: the shape exists + +`mesh-host` `internal/declaration/declaration.go` defines `TypeProcess Type = "process"`. +`internal/apply/process.go` applies it — it writes the unit, writes the timer for a scheduled one, +and gates what follows a run-once one. It has tests of its own in both packages. The resource +carries a bundle `source` with a `digest`, a `run` argv, `env` and `env-file`, a `user`, +`restart-on`, and the `run-once` and `schedule` modifiers. + +So the report's alternative — "if ADR 0047 is right, two proposed documents and a worked manifest +describe a resource type that does not exist" — is **disproven**. It exists, it is implemented, it +is tested, and the host's vocabulary is now **twelve** shapes rather than the nine +[ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md) +counted. + +## The enforcement ADR 0029 asked for is intact, and it recorded this gap rather than closing it + +ADR 0029 said "the vocabulary is nine, and the count moves with a record. The test that asserts it +names this one." That test exists — `internal/declaration/declaration_test.go` asserts the count is +twelve and fails with the reason rather than a number. Above the assertion, a paragraph per +addition names what made it one: + +| shape | the test names | +|---|---| +| `network`, ninth | ADR 0029 | +| `access`, tenth | ADR 0051 | +| **the eleventh** | **`03-DESIGN/01-to-be/18-building-a-module.md`** — a design document, `status: proposed` | +| `opening`, twelfth | ADR 0100 | + +The eleventh is this one. The test still calls it `daemon`, the code calls it `TypeProcess`, and +its paragraph is the only one that names a design document where the others name a decision. +Independently: in `declaration.go`, `TypeProcess` is the **only** shape in the vocabulary whose doc +comment cites no ADR — `network` cites 0029, `access` 0051, `opening` 0100, `user` and the refusal +of `action` cite 0005. + +**So ADR 0029's mechanism worked exactly as designed and was not enough.** It requires every +addition to name something. It does not require that something to be a decision, and the one +addition that named a proposed design document instead is the one this issue is about. + +### A correction to this trail, recorded because it was one grep from being a finding + +The first search here was for `len(Vocabulary())` and found nothing, and the working conclusion for +two steps was that no count assertion existed any more — which would have been written up as "the +mechanism ADR 0029 relied on is gone." It is not gone. The test binds the slice to a local variable +first, so the assertion reads `len(speaks) != 12`. The claim was wrong, it was caught by reading the +file rather than by grepping it, and the shape of the error is the same one issue 113 recorded: a +negative search result read as a fact about the world. + +## The argument the report asked for already exists, in a test comment + +The report asked why a container rather than a supervised process, and said the reasoning was not +written down. It is — in `declaration_test.go`, as the eleventh shape's paragraph: + +> Running code of one's own meant a `container` and therefore an image; running a script meant a +> `service` and a unit somebody else had to install. One intent — run this and keep it running — +> expressed two unrelated ways, with the hosting chosen before anything could be declared. […] It +> is a full-host shape rather than a portable one: it needs a process supervisor to install into. +> It does NOT need a container runtime, which is the point — only software that genuinely needs +> isolation asks for a container. + +That is a decision's Context and Consequences, in a Go comment, in another repository. Nothing in +`02-DECISIONS/` contains it. `TypeProcess`'s own doc comment adds the rest — that three modes beat +three kinds, and that a first draft added a `daemon` for the long-running case alone. + +## The catalogue is containers, and the one exception is the reference module + +71 modules on `main`. Counting the `type` of every declared resource: + +| `container` | `process` | +|---|---| +| 115 | **3** | + +All three `process` resources are in **one** module: `showcase` — the module +[`20-writing-a-module.md`](../../03-DESIGN/01-to-be/20-writing-a-module.md) is a worked guide for. + +### And in that module, the tools do not run + +`showcase` declares its migrate, server and reporting steps as `process`. Its fourth resource, the +one for tools, is a **`container`** — and its image is the module's `helper` artifact, which the +same manifest declares as `kind: upstream` from a bare distribution base. Its command is +`sleep infinity`. It mounts the broker credential and sets the variable naming it, and runs nothing. + +Meanwhile the module's `code` bundle declares six entrypoints. Three are run by the three `process` +resources. The tools entrypoint and the provisioner entrypoint are **run by no resource in the +manifest.** + +Two consequences worth stating separately: + +- **The worked guide does not match the module it documents.** The guide shows four `process` + resources, the fourth being `{"id": "tools", "type": "process"}`. The module has three and a + container. +- **This is the condition ADR 0047 was written to end, in a new shape.** That record's Context says + the conversion "produced tools and events that, as it stands, never execute," and its first + Consequence is that they become runnable. In the reference module they do not execute again — + not for want of a runtime this time, but because nothing declares one that runs them. + +## The harness has no per-module boundary, and nothing refuses a second module + +This is where ADR 0047's isolation argument is load-bearing, so it was checked rather than assumed. + +- `mesh-sdk` `src/tools/index.ts`: `serveTools` iterates `collectTools()` over a module-level + registration array and serves **every registered module's** tools over the **one** `broker` it + was handed. +- `mesh-tools` `src/main.ts`: the modules to load come from one variable as a **comma-separated + list**, and the runtime sets its module and node identity from the **single** credential. +- `mesh-sdk` `src/events/index.ts`: an emitted event's `x-source` is stamped from that single + module identity. + +Put together: load two modules into one runtime and everything the second emits is attributed to +the first, because there is one credential and the identity comes from it. That is precisely the +failure ADR 0047 predicted — "able to emit as any of them" — reached by a different route, since +the credential is correct and there is only one of it for two modules. **Nothing in either +repository refuses the second module**, and no test asserts that a runtime serves one. + +### Ruled out, in fairness to the implementation + +- **The serving key conforms.** ADR 0047 replaced a single `tools.invoke` dispatch with a per-tool + key, and the SDK does that: a tool is served on `.` with the account scoped + `serve..*`. The superseded `tools.invoke` survives only in **prose** — the doc comment + directly above the conforming code, and the `mesh-tools` README, which also describes the runtime + as per-node. The code is ahead of its own documentation. +- **The credential shape conforms.** The sealed per-module credential file is preferred in code, and + the plain URL is documented as the bootstrap case before a module has an account — not the + ordinary path. + +So the account is the right shape and the key is the right shape. It is the **process boundary** +that is declared nowhere and enforced by nothing. + +## An unmerged report already asks the narrow version of this + +Branch `issue/113-controller-container-or-process`, one commit, 2026-09-24, adds a report titled +**"Should the controller run as a container, or as a process the host supervises directly?"** with +`located-in: [mesh-controller module.json, mesh-host internal/apply]`. Its observation is that the +controller is declared a `container` with `network: host` — so container network isolation, the +property that resource type usually buys, is not in use — and it asks what `type: container` buys +that `type: process` would not. + +It is unmerged and numbered 113, which is taken. A sibling branch, +`issue/113-record-the-repin-and-fold-114`, is why `114` is free. + +**That report and this one are the instance and the general condition**, and they do not conflict: +it asks about one module that is not a code-carrying sidecar at all, and reaches the same question +from the opposite end. + +## What is located, and what is not + +**Located — and it is not a code defect.** The implementation and the design layer agree with each +other; the **decision record is what is missing**, and the accepted record that occupies its place +says the other thing. ADR 0047 is `accepted`, cited by the module protocol, and unsuperseded, while +the host it describes has had a purpose-built shape for a module's own code since the eleventh +vocabulary entry. + +| Owner | What is theirs | +|---|---| +| `hq` | the missing record for the `process` shape; ADR 0047 left standing; the worked guide that does not match the module | +| `mesh-catalog modules/showcase` | tools and provisioner entrypoints that no resource runs; a tools container that sleeps | +| `mesh-sdk src/tools/index.ts` | several modules served over one credential, unrefused and untested; a doc comment describing a superseded dispatch | +| `mesh-tools` | a README describing a per-node multi-module runtime the code no longer prefers | + +**Not located, and deliberately open:** whether `process` or `container` is *right* for a module's +own code. This diagnosis establishes that the question was answered in practice and never recorded +— not which answer is correct. The arguments on both sides now exist in writing; they exist in a +test comment and a proposed design document, and one of them contradicts an accepted decision. + +## What would close it + +1. A decision record for the `process` shape, carrying the argument currently in + `declaration_test.go`, and saying what becomes of ADR 0047 — superseded in whole, or in the part + that names a container. +2. `18-building-a-module.md` and `20-writing-a-module.md` naming that record in `decisions:`, and + the worked manifest agreeing with the module. +3. The eleventh shape's paragraph in the vocabulary test naming a decision, like the other three. +4. **How the rule is checked, since a rule states how it is checked:** every shape in the host's + vocabulary names a decision, asserted where the count is already asserted — which turns "no + design without a decision" into something stronger than "no design without *a* decision" for + the one vocabulary where each entry is a security decision. +5. Whether a runtime may serve more than one module answered either way, and asserted — a refusal + if not, a test that two modules' events keep their own source if so. From 10a2b706c6ac8247d9ee47475464c4deff45f8e6 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 24 Sep 2026 11:41:55 +0200 Subject: [PATCH 06/26] Issue 113: should the controller be a container or a process the host supervises MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Filed after a session where every mesh-controller interaction went through docker exec — its manifest runs it as a container with network: host, using none of the isolation that resource type usually buys, while ADR 0006 makes it the mesh's single point of coordination. Open question, not a claimed defect: does type: container get the controller anything type: process (supervised the way the host supervises its own unit, per ADR 0005) would not. --- .../00-report.md | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md diff --git a/04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md b/04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md new file mode 100644 index 0000000..ce0e98c --- /dev/null +++ b/04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md @@ -0,0 +1,73 @@ +--- +status: open +opened: 2026-09-24 +located-in: [mesh-controller module.json, mesh-host internal/apply] +fixed-by: +amended-design: +--- + +# 113 — Should the controller run as a container, or as a process the host supervises directly? + +## What was observed + +On the control-node, 2026-09-24, over a long session of operating the mesh through +`mesh-controller`'s CLI (build, push, plan, status, module moved). Every mutating step reached the +binary the same way: `docker exec mesh-controller /mesh-controller ` — because +`mesh-controller`'s own manifest declares its one resource as: + +```json +{ "id": "server", "type": "container", "name": "mesh-controller", "network": "host", "args": ["serve"] } +``` + +Two things about that declaration are worth naming together, because neither is a problem on its +own and the combination is what raises the question: + +- **`network: host`.** The controller does not use container network isolation, which is the + property a `container` resource type usually buys over a `process` one. It runs with the node's + own network namespace either way. +- **It is the mesh's single point of coordination.** [`03-DESIGN/01-to-be/06-the-controller.md`](../../03-DESIGN/01-to-be/06-the-controller.md) + is explicit: "one node runs it, and nothing takes over" — no election, no quorum, no failover; + recovery is restore, not failover. + +[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) gives the host — the one thing tier 0 requires +to be a real system daemon — exactly this reasoning for refusing to run in a container: *"installing +the container runtime is a step of the bootstrap, so a host inside a container would need the thing +it exists to install."* The controller is one tier up and does not install the runtime, but it +shares the profile that argument turns on: something the rest of the mesh's operation depends on, +sharing fate with a runtime that is not itself. + +## Why it matters beyond this instance + +Practically, tonight: every controller interaction was raw shell into a container (`docker exec`), +not a first-class surface — no logs command beyond `docker logs`, no `systemctl status`, and a +session permission classifier that (correctly) treats arbitrary shell into a container as needing +sign-off every time, unlike an ordinary supervised process. That friction is a symptom, not the +issue itself. + +The actual question is whether `type: container` is buying the controller anything here besides +image-based delivery and a restart policy — both of which [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)'s +launcher pattern already describes as buildable directly into the host's own supervision (restart on +exit, count consecutive failures, roll back after too many, halt after that), for the host's own +unit. If the controller were declared `type: process` instead — still built and versioned through +the same delivery pipeline, just executed on the node and supervised by the host the way the host +supervises itself — it would stop sharing fate with the container runtime's health (restarts, +upgrades, disk pressure evicting containers) for the one piece of software whose absence the rest of +the mesh is designed to tolerate but nothing is designed to *want*. + +This is squarely a question, not a claim that today's shape is wrong: [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) +already tolerates the controller being down by construction (nodes reconcile from their own +last-applied state), which may make the container-runtime coupling moot in practice. Nobody has +checked. + +## Open questions + +- Does `mesh-host`'s `process` resource type already support the restart/failure-counting semantics + [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) describes for the host's own launcher well + enough for something this central — or would this need host-side work first? +- With `network: host` already in use, what does `type: container` provide the controller today that + `type: process` would not? +- Is there a real circularity risk — the controller's own health depending on the container runtime + it (indirectly, via the host) manages — or does "one node runs it, nothing takes over" already make + a controller outage tolerable regardless of which resource type it is? +- If the answer is "keep it a container," what does that answer, precisely, that this issue asked — + so the next person who notices the same asymmetry finds it answered rather than open again? From 82a6badc7c3eb62dd0690bdea10e8b4908a37d95 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 16:16:21 +0200 Subject: [PATCH 07/26] Issue 114: land the controller's container-or-process question, renumbered MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Filed 2026-09-24 on a branch of its own and never merged, numbered 113, which is taken. 114 is free because a sibling branch folded it, so it takes that number and keeps its commit. Kept separate from issue 117 rather than folded into it. 117 asks the same question of every module and locates the missing decision; this asks it of the controller, where `network: host` means container network isolation — the property that resource type usually buys — is not in use. That observation is this report's own and is nowhere in 117, and folding would lose it. Its first open question is answered by 117's diagnosis and now says so: the host's `process` shape is built, applied and tested, restart and run-to-completion semantics included, so deciding this does not wait on host-side work. --- .../00-report.md | 15 ++++++++++++++- .../01-diagnosis.md | 14 +++++++++++--- 2 files changed, 25 insertions(+), 4 deletions(-) rename 04-ISSUES/{113-should-the-controller-be-a-container-or-a-process-the-host-supervises => 114-should-the-controller-be-a-container-or-a-process}/00-report.md (83%) diff --git a/04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md b/04-ISSUES/114-should-the-controller-be-a-container-or-a-process/00-report.md similarity index 83% rename from 04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md rename to 04-ISSUES/114-should-the-controller-be-a-container-or-a-process/00-report.md index ce0e98c..6b03b96 100644 --- a/04-ISSUES/113-should-the-controller-be-a-container-or-a-process-the-host-supervises/00-report.md +++ b/04-ISSUES/114-should-the-controller-be-a-container-or-a-process/00-report.md @@ -6,7 +6,7 @@ fixed-by: amended-design: --- -# 113 — Should the controller run as a container, or as a process the host supervises directly? +# 114 — Should the controller run as a container, or as a process the host supervises directly? ## What was observed @@ -71,3 +71,16 @@ checked. a controller outage tolerable regardless of which resource type it is? - If the answer is "keep it a container," what does that answer, precisely, that this issue asked — so the next person who notices the same asymmetry finds it answered rather than open again? + +## The general case + +[Issue 117](../117-a-modules-own-code-is-a-container-and-a-process/00-report.md) is the same +question asked of every module rather than of the controller: a module's own code is a `container` +in [ADR 0047](../../02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md) +and a `process` in the to-be design, and no record moves it. Its +[diagnosis](../117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md) answers the +first open question above: the host's `process` shape is built, applied and tested, including the +restart and run-to-completion semantics — so this would not need host-side work first. + +The two do not collapse into one. The controller is not a code-carrying sidecar, and `network: host` +is what makes the asymmetry visible here and nowhere else. diff --git a/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md index c6f6b3a..3ec972e 100644 --- a/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md +++ b/04-ISSUES/117-a-modules-own-code-is-a-container-and-a-process/01-diagnosis.md @@ -150,12 +150,20 @@ controller is declared a `container` with `network: host` — so container netwo property that resource type usually buys, is not in use — and it asks what `type: container` buys that `type: process` would not. -It is unmerged and numbered 113, which is taken. A sibling branch, -`issue/113-record-the-repin-and-fold-114`, is why `114` is free. +It was unmerged and numbered 113, which is taken. A sibling branch, +`issue/113-record-the-repin-and-fold-114`, is why `114` was free. **That report and this one are the instance and the general condition**, and they do not conflict: it asks about one module that is not a code-carrying sidecar at all, and reaches the same question -from the opposite end. +from the opposite end. So it lands in this change as +[issue 114](../114-should-the-controller-be-a-container-or-a-process/00-report.md), its commit and +authorship intact, with a section pointing here — rather than being folded in and losing the +`network: host` observation, which is its own and is not reproduced above. + +This diagnosis answers its first open question. The host's `process` shape does support what +[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) describes for the host's own launcher — the +unit, the timer, restart, and run-to-completion gating are implemented and tested — so that report +does not need host-side work before it can be decided. ## What is located, and what is not From 59c93dcfe45d7aff451f15fccb99503678e544f0 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 16:47:53 +0200 Subject: [PATCH 08/26] 109: a package registry seat is one per ecosystem, not one for all of them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extends ADR 0075. Surfaced fixing builder's hand-faked package-registry binding tonight: gitea's manifest declares the provision once with a single npm-path, conflating what should be independently assignable per ecosystem (npm/cargo/docker/...) the same way artifact-store and package-registry were themselves split. Cited in 22-the-work-ahead.md's Phase 2, where the target state this decision points at was already described a week ago. Numbered 109, not 108: route-proxy's policy feature (mesh-controller PR still-unwritten decision record — reserved but never committed. Renumbered around it rather than colliding. --- ...kage-registry-seat-is-one-per-ecosystem.md | 106 ++++++++++++++++++ 02-DECISIONS/README.md | 1 + 03-DESIGN/01-to-be/22-the-work-ahead.md | 1 + 3 files changed, 108 insertions(+) create mode 100644 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md diff --git a/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md b/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md new file mode 100644 index 0000000..e89a743 --- /dev/null +++ b/02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md @@ -0,0 +1,106 @@ +--- +topic: the tiers +status: accepted +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0075-two-stores-and-which-provides-what.md +--- + +# 109. A package registry seat is one per ecosystem, not one for all of them + +## Context + +Fixing `builder`'s consumption of `package-registry` tonight surfaced the shape ADR 0075 actually +left implicit. 0075 split `artifact-store` from `package-registry` and said the second is "an +ecosystem's own registry — npm, cargo, PyPI, Go" — but it defined one provision for all four, +not one each. + +What that produces, read from the manifests as they stand: + +- `gitea`'s `module.json` declares `provides: package-registry` once, and its `serves` block + carries exactly one path: `npm-path`. Nothing names a cargo or PyPI endpoint, though gitea's own + package API serves both. +- `builder` had, until tonight, a hand-written JSON fragment standing in for a real grant — + `{"provision": "package-registry", "from": "gitea", "at": "127.0.0.1", ...}` — because nothing + in the interface gave it a real one to ask for. The fragment named `npm-path` specifically; there + was nowhere to put a second ecosystem's endpoint even if one had been wired. +- The fix applied tonight declares `requires: ["artifact-store", "package-registry"]` and lets the + mesh mint the grant properly — correct for what exists today, but it is one seat standing in for + what should be several, the same conflation 0075 itself named and did not resolve for this + provision specifically. + +**The version-skew problem 0075 wrote down for the artifact-store/package-registry split repeats +one level down, inside "package registry" itself:** npm resolves by name and range from one +namespace, cargo from another, and a single grant conflates them exactly the way one store for +both digests and ranges would have. + +## Considered Options + +**1. One `package-registry` provision, gitea answers every ecosystem it can.** What exists today. +Simplest to grant — one credential, one binding, done once per consumer. Rejected: a consumer +that only ever needs npm still receives a grant shaped to cover cargo and PyPI, and there is no way +to hand off *only* npm to a different provider (verdaccio, say) without renegotiating the whole +provision for every consumer of any ecosystem. + +**2. One provision, parameterised by ecosystem.** `requires: package-registry` plus a declared +`ecosystem: npm` alongside it, still one interface. Rejected: the `provides`/`requires` refusal +mechanism this mesh already uses (two providers of one provision is a naming conflict until +resolved) would need to become conditional on a parameter it does not otherwise carry anywhere in +the mesh's resolution — a special case for exactly one provision, rather than the mesh's existing +mechanism applied again. + +**3. One provision per ecosystem — `npm-package-registry`, `cargo-package-registry`, +`docker-package-registry`, and so on, each independently `provides`/`requires`.** Chosen. + +## Decision + +**A package registry seat is one per ecosystem.** `npm-package-registry`, `cargo-package-registry`, +`docker-package-registry` — each its own provision, resolved, granted, and refused exactly the way +`artifact-store` and today's single `package-registry` already are. Adding an ecosystem is adding a +provision, not widening one. + +**Gitea may hold several seats at once.** Nothing here says gitea answers only one; ADR 0075 +already established that a provider may answer more than one named thing on one machine ("a mesh +running gitea for git and packages alongside a registry serving artifacts is an ordinary +arrangement"). Gitea fulfilling `npm-package-registry` and `cargo-package-registry` both is the +expected shape, not an exception. + +**Each seat's grant is independent.** A consumer that only needs npm holds only the +`npm-package-registry` grant. Moving that one ecosystem to a different provider — verdaccio, +named directly as the motivating case — means assigning `npm-package-registry` to verdaccio and +leaving every other seat exactly where it was. No consumer of `cargo-package-registry` observes +the change; no manifest naming `package-registry` broadly needs to be found and re-read. + +**`builder`'s fix tonight is the interim shape, not the target.** It correctly consumes the one +seat that exists today (`package-registry`, npm in practice). Splitting it becomes, later, +replacing that one line with the ecosystems `builder` actually uses — a manifest change, not a +redesign of how `builder` asks for anything. + +## Consequences + +- `gitea`'s `module.json` gains a `provides` entry per ecosystem it actually serves, each with its + own `serves` block (`npm-path`, a cargo path, a PyPI path) in place of the one `package-registry` + entry with a single `npm-path` inside it. +- `gitea`'s provisioner (`modules/gitea/provisioner/index.ts`) currently runs one `runProvisioner` + registration for `package-registry`; each seat needs its own registration, or one provisioner + keyed by which seat's `create`/`remove` fired — the mesh's `Provision` type does not yet carry + which named provision a call is for when a module answers more than one, and that is worth + checking before assuming the harness already supports it. +- Every consumer's `requires` moves from the one name to however many ecosystems it actually uses. + `builder` is the only known consumer today; widening later is one manifest line per module, not + a migration. +- `verdaccio`'s role sharpens: not "package-registry, an alternative for npm alone" (0075's phrasing) + but a named `npm-package-registry` *provider*, a straight swap against gitea's answer to the same + seat. +- Not solved here: whether `cargo-package-registry` and `pypi-package-registry` are needed at all + before something actually consumes them. This record names the shape; building unused seats is + its own decision. + +## References + +- [ADR 0075](0075-two-stores-and-which-provides-what.md) — the record this extends; defined + `package-registry` as the second provision without splitting it per ecosystem. +- `mesh-catalog modules/builder/module.json` — tonight's fix, the interim single-seat shape. +- `mesh-catalog modules/gitea/module.json`, `modules/gitea/provisioner/index.ts` — today's + single-provision, npm-only implementation. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index f23c044..bf98713 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -124,6 +124,7 @@ python3 00-META/checks/index.py fail if stale - **0095** — [The control plane is the way to ask a module](0095-the-control-plane-is-the-way-to-ask-a-module.md) - **0098** — [A fact a provider makes at first start is fetched from it, not carried in its manifest](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md) - **0108** — [A route carries the policy applied to a request, and names a secret rather than holding one](0108-a-route-carries-the-policy-applied-to-a-request.md) +- **0109** — [A package registry seat is one per ecosystem, not one for all of them](0109-a-package-registry-seat-is-one-per-ecosystem.md) ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/22-the-work-ahead.md b/03-DESIGN/01-to-be/22-the-work-ahead.md index a9ddda4..97c3519 100644 --- a/03-DESIGN/01-to-be/22-the-work-ahead.md +++ b/03-DESIGN/01-to-be/22-the-work-ahead.md @@ -12,6 +12,7 @@ decisions: - 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md - 02-DECISIONS/0075-two-stores-and-which-provides-what.md - 02-DECISIONS/0014-no-npm-workspace.md + - 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md --- # The work ahead From dbe100ca96b664a6c5a50bb7544fafd94c24791a Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 20:33:14 +0200 Subject: [PATCH 09/26] ADR 0110 and 0111: a seat is a module assignment from a closed set, and a build source may live on the git seat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Seats have been doing two jobs and neither is written down. The mechanism ADR 0009 introduced is enforced — a second holder is refused — but any well-formed name becomes a seat by being claimed, and nothing can say which seats a mesh has or who holds them: holdings are assembled while planning and discarded. The enumeration done while preparing this missed the control plane's own manifest, because core modules' manifests live in its repository rather than the catalogue. 0110 closes the set. Each seat has a name, a scope, what occupying it delivers, and the record that made it one; a claim outside the set is refused. A seat is held by a module assignment, and what the mesh knows about the holder is what it knows about that assignment — nothing is stored beside it. A seat may deliver a provision, and then its holder answers for it among several providers: pin, then the holder, then the only provider, then refused. That keeps 0009's "refused, never guessed": the seat is the choice made once, mesh-wide, instead of a pin per consumer node. The first set is the eleven seats already claimed plus 0109's npm-package-registry, so nothing in use is refused. Two concepts — seats for exclusion, a new word for consumable singulars — was rejected: both mean "this mesh's one X", and the overview a person wants is one list. 0111 gives the mesh a git seat and makes a build source one of two explicit forms: a repository on the seat's holder, recorded by its path and cloned from wherever the holder runs at build time; or an external URL, recorded and cloned exactly as given. Recognising self-hosted sources by matching URLs against the forge's address was rejected — it fails in the one case it exists for, after the forge moves. Credentials for private repositories are left undecided and said so. Design: new to-be 26 (the seats); 23 gains the seat step in resolution; 18's source entry names the two forms; the glossary's seat and provision entries say where they meet. 0109 is carried from its own branch so every link here resolves. --- 00-META/glossary.md | 15 +- ...s-a-module-assignment-from-a-closed-set.md | 154 ++++++++++++++++++ ...d-source-is-on-the-git-seat-or-external.md | 96 +++++++++++ 02-DECISIONS/README.md | 2 + 03-DESIGN/01-to-be/18-building-a-module.md | 5 +- 03-DESIGN/01-to-be/23-choosing-a-provider.md | 14 +- 03-DESIGN/01-to-be/26-the-seats.md | 125 ++++++++++++++ 03-DESIGN/01-to-be/README.md | 1 + 8 files changed, 403 insertions(+), 9 deletions(-) create mode 100644 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md create mode 100644 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md create mode 100644 03-DESIGN/01-to-be/26-the-seats.md diff --git a/00-META/glossary.md b/00-META/glossary.md index 7d38f6d..4ffe66e 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -44,15 +44,22 @@ another — and a mesh you cannot name precisely is a mesh two people describe d ## How modules relate to the mesh -- **seat** — a named position at a scope (node / site / mesh) with a **capacity**. A capacity-1 seat - is exclusive (one holder); a higher-capacity seat is a **bench** (several holders coexist). +- **seat** — a named role at a scope (node / site / mesh), held by a module assignment, from a + **closed set** the mesh defines: a claim naming a seat outside the set is refused. A seat may + **deliver a provision**, and its holder is then the mesh's answer for it when several modules + provide it ([ADR 0110](../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). + The set, with who holds each seat, is the overview of what a mesh has + ([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)). A seat has a **capacity**: a + capacity-1 seat is exclusive (one holder); a higher-capacity seat is a **bench** (several holders + coexist). - **claim** — a module taking a spot on a seat. `claims: [{name, scope}]` in a manifest. A mesh-scoped exclusive claim is how the mesh says "there is one of me". A foundation seat is named after the server it guards: the `mesh-controller`, `postgres` and `lavinmq` modules claim the `mesh-controller`, `mesh-store` and `mesh-broker` seats ([ADR 0079](../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)). - **provision** — a service one module `provides` and others `require`; the mesh resolves a provider - and wires the two with an endpoint and a credential. This is separate from seats: a provision is - a service you offer, a seat is a slot you occupy. + and wires the two with an endpoint and a credential. A provision is a service you offer, a seat + is a role you occupy, and the two meet where a seat delivers a provision: occupying the seat is + what makes a module *the* provider of it. ## How this page is kept diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md new file mode 100644 index 0000000..0d9e313 --- /dev/null +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -0,0 +1,154 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0009-modules-and-the-graph.md +--- + +# 110. A seat is a module assignment from a closed set, and it may deliver a provision + +## Context + +[ADR 0009](0009-modules-and-the-graph.md) introduced claims: a module declares something +singular, at a scope, and a second holder is refused. [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) +named the foundation's three after their servers. That mechanism is enforced and works. What it +means has drifted, and three things are now true of it that no record says. + +**Any well-formed name becomes a seat by being claimed.** The control plane's manifest check +refuses a claim only for a malformed name or an unknown scope. Nothing says which seats a mesh has. +The names in use were each invented by the module that claims them: `the-showcase`, +`the-build-machine`, `the-intrusion-prevention`. + +**Nothing can say what a mesh has, or who fills it.** There is no seat table and no command that +lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. +The only way to answer "which seats does this mesh have, and which module holds each" is to read +every manifest in two repositories, because the core modules' manifests moved into the control +plane's own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)). While this record +was being prepared, that enumeration was done once by hand, and it missed the control plane's own +manifest: eleven claims were reported where there are twelve. + +**Some seats are the mesh's one of something that others consume, and nothing uses that fact.** +Of the twelve claims, four are held by a module that provides something consumers require: +`mesh-store` (`postgres-database`, eleven consumers), `mesh-broker` (`amqp`), `the-artifact-store` +(`artifact-store`) and `the-dns-port`. The rest deliver nothing to anybody, and are still +meaningful: they say which module is this mesh's packet filter, or resolver configuration. +Meanwhile a requirement for a mesh-scoped provision with more than one provider is refused until a +person pins, **per consumer node**, which provider to use. [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) +anticipates exactly that case — gitea and verdaccio both answering npm — and under today's +resolution it would mean a pin on every machine that builds anything. + +## Considered Options + +**1. Leave seats as free-form exclusion, and keep provisions as the only thing that delivers.** +Rejected. The overview stays unanswerable, and a second provider of anything costs a pin per +consumer node — the decision "gitea is our npm registry" made again on every machine. + +**2. Two concepts: seats for exclusion, and a new word for the mesh's one consumable thing.** +Rejected. Both mean "this mesh's one X". Every existing claim would first have to be classified +into one or the other, and the overview a person wants is one list, not two. + +**3. A seat is a module assignment from a closed set, and occupying it may deliver a provision.** +Chosen. + +## Decision + +**The mesh defines a closed set of seats.** Each entry has a name, a scope, what occupying it +delivers (if anything), and the decision that made it a seat. A claim naming a seat outside the set +is refused, and so is a claim at a scope other than the seat's. Adding a seat is a decision, for the +same reason adding a shape to the host's vocabulary is one: the set is what a person reads to learn +what a mesh can have, and a name added without an argument is a name nobody can explain later. + +**A seat is held by a module assignment.** What the mesh knows about a seat's holder is what it knows +about that assignment: its node, the node's settings for it, and what it serves. Holdings are not +stored separately. The seat points at an assignment, and a second record of the same fact would be a +second thing to disagree with the first. + +**A seat may deliver a provision, and then its holder answers for it.** A seat that delivers a +provision can only be held by a module that provides it, at the seat's scope, and a claim that does +not is refused. When several providers answer a requirement for that provision, resolution takes, in +order: + +1. a provider the consumer's node was **pinned** to — a consumer coupled to one provider's + contents, as [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) already allows; +2. **the holder of the seat** that delivers it; +3. the **only** provider, when there is one; +4. otherwise, refused with the candidates named, as now. + +This keeps [ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with several answers is +never guessed. The seat is not a guess. It is the choice made once, mesh-wide, by assigning the holder, +instead of once per consumer by pinning. A second provider may run beside the holder, and whatever +requires the provision still resolves to the holder without anybody naming it. + +**Seats are also informational.** The control plane lists every seat in the set, what it delivers, +and which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh +has no X", not an error. + +**The first set is the eleven seats already claimed, plus one.** Twelve claims are in use, and they +name eleven seats because two alternative modules claim `the-resolver-configuration`. This record +admits every seat the catalogue and the control plane claim today, so no module is refused by it: + +| seat | scope | delivers | held today by | made a seat by | +|---|---|---|---|---| +| `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-store` | mesh | `postgres-database` | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-broker` | mesh | `amqp` | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | +| `the-catalogue` | mesh | — | `mesh-catalog` | this record | +| `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | +| `the-build-machine` | node | — | `builder` | this record | +| `the-dns-port` | node | — | `dnsmasq` | this record | +| `the-intrusion-prevention` | node | — | `fail2ban` | this record | +| `the-packet-filter` | node | — | `nftables` | this record | +| `the-resolver-configuration` | node | — | `resolv-conf` or `resolved-split-dns` | this record | +| `the-showcase` | node | — | `showcase` | this record | + +`npm-package-registry` is the one addition. It is [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md)'s +seat, named after the provision it delivers, as 0079 named the foundation's seats after what they are. +gitea claims it. verdaccio provides the same provision and claims nothing, so it is the second +provider this record exists to make harmless. + +`the-dns-port` is listed as delivering nothing, although `dnsmasq` provides `wildcard-resolution`. +That provision is node-scoped and answered on the machine, so no preference between providers +arises. Whether the seat should say it delivers it is left for when a second resolver makes the +question real. + +## Consequences + +- The control plane carries the set in code. A test asserts its size, and that every entry names the + record that made it a seat, so changing the set means finding the argument rather than a number. + This is the pattern the host's vocabulary test already follows. +- Manifest validation refuses an unknown seat, a seat claimed at the wrong scope, and a + provision-delivering seat claimed by a module that does not provide the provision. The three + refusals name the seat and the set. +- Resolution prefers the seat's holder among several providers, after a pin. A provider record + gains the module it came from, because two modules on one node could otherwise not be told apart + as holder and non-holder. +- A `seats` command lists the set with each seat's holders, derived from assignments. +- gitea claims `npm-package-registry`. The catalogue's `package-registry` becomes + `npm-package-registry` throughout, per [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md). +- **What got harder:** a module wanting a new singular role can no longer invent a name. It needs a + record. That is the point, and it costs one record per seat. +- **Not changed:** `${seat::}` stays as it is. It exists so the control plane can reach a + foundation it made before any module existed, and it cannot be a consumer. A module that needs + something from a seat's holder requires the provision the seat delivers, and receives it the way + any provision is received: through a grant. + +## How it is checked + +| Rule | Checked by | +|---|---| +| The set is closed, and every entry names its decision | A control-plane unit test asserts the set's size and a non-empty decision for every entry. | +| A claim outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat whose claimant does not provide. | +| Every module in use claims a seat in the set | A control-plane test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | +| The holder answers among several providers | Resolution tests: two providers with the seat held, two with a pin overriding the seat, two with the seat unheld (refused). | + +## References + +- [ADR 0009](0009-modules-and-the-graph.md): claims, scopes, and "refused, never guessed" +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a seat named after what it is +- [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md): the per-ecosystem registry seats +- [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): pins, co-location and refusal +- `mesh-controller internal/catalogue/resolve.go` (`checkClaims`, the brokered branch of `Resolve`), + `internal/catalogue/manifest.go` (claim validation), `cmd/mesh-controller/plan.go` (holdings) diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md new file mode 100644 index 0000000..ed7441a --- /dev/null +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -0,0 +1,96 @@ +--- +topic: building it +status: accepted +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0069-a-module-is-a-repository-and-a-path.md +--- + +# 111. A build source is on the mesh's git seat, or it is an external repository + +## Context + +[ADR 0069](0069-a-module-is-a-repository-and-a-path.md) made a module a repository, a path and a +ref, and the control plane records all three against the module so it can rebuild it and say when +its source has moved ahead. **The repository is recorded exactly as a person typed it.** `build +` hands the string to a build machine, which runs `git clone` on it, and the same string +becomes the module's recorded source. + +**So a self-hosted forge's address is written into every module built from it.** The mesh runs its +own forge, and most of what it builds lives there. Every one of those modules carries the forge's +scheme, host and port in its recorded source. Move the forge to another machine, or change the port +it is published on, and every recorded source is stale at once. Nothing notices until a rebuild fails +to clone. + +**And nothing names the mesh's git at all.** gitea serves git over HTTP and over SSH, and the mesh's +vocabulary contains neither. No provision, no `serves`, no seat, as the forge survey +([research 013](../01-RESEARCH/013-the-forge-and-the-registries/the-survey.md)) found. The only trace +is a label on its public route, which the mesh is explicitly not meant to interpret. + +**External repositories are ordinary, and must stay so.** An application the mesh hosts may live on a +public forge. Building it from its URL works today and must keep working unchanged. + +## Considered Options + +**1. Keep recording literal URLs.** Rejected. It is the problem: the forge's address copied into +every module built from it. + +**2. Recognise a self-hosted source by matching its URL against the forge's current address.** +Rejected. It infers the kind of source from the shape of a string, and the inference fails in the +one case it exists for: after the forge moves, old URLs no longer match anything. + +**3. Two explicit forms: a repository on the holder of the `git` seat, or an external URL.** Chosen. + +## Decision + +**The mesh has a `git` seat.** It is mesh-scoped and delivers the `git` provision, per +[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md). Its holder provides `git`, +serving how a repository on it is cloned: the scheme and the port. gitea claims it. + +**A source is on the git seat, or it is external, and the mesh records which.** + +- `build --self /` builds from a repository on the seat's holder. The recorded + source is the repository's path on that holder, and the seat it is on. **It never contains an + address.** At the moment of building, the control plane composes the clone URL from where the + holder runs and what it serves for `git`, so a moved forge changes nothing recorded. +- `build ` is unchanged: an external repository, recorded and cloned exactly as given. GitHub + and GitLab are the ordinary cases. + +**An unheld seat refuses self-hosted builds and nothing else.** With nobody holding `git`, `build +--self` is refused, naming the seat and saying what would hold it. External builds are unaffected. A +mesh without a forge of its own builds from external repositories only, and says so rather than +failing to clone. + +**The build machine is not told the difference.** It receives a URL either way. Composing the URL is +the control plane's job, because only the control plane knows where the seat's holder runs. + +## Consequences + +- The control plane's inventory gains a column saying which seat a source is on. It is empty for + every module recorded before this, which is correct: they were all recorded as literal URLs. +- `build`, `build --behind` and `build --dry-run` resolve a seat source before asking a builder. The + recorded source keeps the seat form; the build log keeps the URL that was actually cloned, because + that is what happened. +- gitea claims `git` and provides it, serving HTTP clone on its web port. +- **Not decided: credentials for private repositories.** The mesh's own repositories are public, and + clone without one. A private repository still works only if the build machine's own git + configuration authenticates, exactly as before. Delivering a clone credential through the `git` + provision's grant is the obvious next step, and it is its own decision. +- **Not changed:** modules already recorded from the forge keep their literal URLs until they are + rebuilt with `--self`. Rewriting them in place would be the URL-matching this record rejects. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A seat source records no address | A control-plane test resolves a seat source and asserts the recorded repository is the path alone. | +| The URL comes from the holder | A test composes the clone URL from a holder's node address and served `git` scheme and port, and a second where the port was moved on the node. | +| An unheld seat refuses self-hosted builds only | A test with no holder: `--self` is refused naming the seat; an external URL passes through unchanged. | + +## References + +- [ADR 0069](0069-a-module-is-a-repository-and-a-path.md): a module is a repository, a path and a ref +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): seats, and a seat delivering a provision +- [Research 013](../01-RESEARCH/013-the-forge-and-the-registries/the-survey.md): git is served and declared nowhere +- `mesh-controller cmd/mesh-controller/build.go`, `internal/builder/builder.go` diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index bf98713..964231e 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -156,6 +156,7 @@ python3 00-META/checks/index.py fail if stale - **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md) - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md) - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) +- **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) ### How it is built @@ -175,6 +176,7 @@ python3 00-META/checks/index.py fail if stale - **0096** — [An upstream image is copied between registries, never through a machine's image store](0096-an-upstream-image-is-copied-between-registries.md) - **0097** — [A vendor image is a declared build input, and a recipe fetches nothing undeclared](0097-a-vendor-image-is-a-declared-build-input.md) - **0107** — [Persistent data is a directory bind, never a named volume](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md) +- **0111** — [A build source is on the mesh's git seat, or it is an external repository](0111-a-build-source-is-on-the-git-seat-or-external.md) ### How it is checked diff --git a/03-DESIGN/01-to-be/18-building-a-module.md b/03-DESIGN/01-to-be/18-building-a-module.md index 0076a36..5c77068 100644 --- a/03-DESIGN/01-to-be/18-building-a-module.md +++ b/03-DESIGN/01-to-be/18-building-a-module.md @@ -5,8 +5,9 @@ code: - mesh-controller cmd/mesh-builder - mesh-controller internal/builder - mesh-catalog modules/builder -updated: 2026-09-21 +updated: 2026-09-25 decisions: + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md - 02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md - 02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md - 02-DECISIONS/0091-a-mount-is-declared-three-ways.md @@ -38,7 +39,7 @@ controller's again. The builder's whole responsibility is the middle. | term | is | |---|---| -| **source** | a repository, a path within it, and a ref — resolved to one commit ([ADR 0069](../../02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md)) | +| **source** | a repository, a path within it, and a ref — resolved to one commit ([ADR 0069](../../02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md)). The repository is either on the forge holding the `git` seat, recorded by its path there and cloned from wherever that forge runs at build time, or external, recorded and cloned exactly as given ([ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)) | | **recipe** | how *one* artifact is produced from that source | | **toolchain** | what a recipe runs inside — a compiler, a runtime, the SDK | | **artifact** | what a recipe produced, named by the digest of its content | diff --git a/03-DESIGN/01-to-be/23-choosing-a-provider.md b/03-DESIGN/01-to-be/23-choosing-a-provider.md index 50b3ed3..e3f36c0 100644 --- a/03-DESIGN/01-to-be/23-choosing-a-provider.md +++ b/03-DESIGN/01-to-be/23-choosing-a-provider.md @@ -2,10 +2,11 @@ layer: to-be status: designed code: [] -updated: 2026-09-20 +updated: 2026-09-25 decisions: - 02-DECISIONS/0084-which-provider-serves-a-consumer.md - 02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md + - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md --- # 23 — Choosing a provider @@ -50,9 +51,16 @@ provider on a different node. That coupling is exactly what may not be guessed, names the provider. Naming it is also what makes a later move safe — the mesh knows the binding is to that provider and not to whichever one is nearest. +**A seat names the mesh's one provider of a kind.** Where a seat delivers the provision, its holder +answers for it when several providers exist and the consumer named none. That is not picking: the +choice was made once, mesh-wide, by assigning the holder, rather than once per consumer by naming it +([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), +[26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer +coupled to particular contents has said so. + **Ambiguity is refused, never resolved by picking.** If several providers of a kind exist, none is -named, and none is co-located, the requirement is unsatisfiable and is refused with the candidates -shown — the same stance +named, none is co-located, and no seat delivers it, the requirement is unsatisfiable and is refused +with the candidates shown — the same stance [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) took against a confidently-wrong match, applied to the instance rather than the dialect. A wrong answer delivered quietly costs more than a refusal. diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md new file mode 100644 index 0000000..b809256 --- /dev/null +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -0,0 +1,125 @@ +--- +layer: to-be +status: in-progress +code: + - mesh-controller internal/catalogue/seats.go + - mesh-controller internal/catalogue/resolve.go + - mesh-controller cmd/mesh-controller/seats.go + - mesh-controller cmd/mesh-controller/build.go + - mesh-catalog modules/gitea/module.json +updated: 2026-09-25 +decisions: + - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md + - 02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md +--- + +# 26 — The seats + +**What a mesh can have one of, and who fills each.** A seat is a named role at a scope, taken by a +module assignment. The mesh defines which seats exist. Occupying one may deliver a provision, and the +list of seats with their holders is the quickest answer to "what is in this mesh". + +## What a seat is + +A seat has four properties, fixed by the mesh rather than by any module: + +| property | is | +|---|---| +| name | what a manifest claims, and what a person reads in the list | +| scope | node, site or mesh: where there may be only one holder | +| delivers | the provision its holder answers for, or nothing | +| decision | the record that made it a seat | + +**A module assignment holds a seat by claiming it.** The claim is the manifest's `claims`, and it is +satisfied by assigning the module somewhere. The seat is not a second record beside the assignment. +It points at the assignment, and everything the mesh knows about the holder is what it knows about +that assignment: the node, the node's settings for the module, and what the module serves. + +**The set is closed.** A claim naming a seat the mesh does not define is refused, and so is a claim at +the wrong scope. Adding a seat is a decision, recorded, for the reason every addition to the host's +vocabulary is one: the set is what a person reads to learn what a mesh can have, and an entry nobody +argued for is an entry nobody can explain. + +## The set + +| seat | scope | delivers | typically held by | +|---|---|---|---| +| `mesh-controller` | mesh | — | the controller | +| `mesh-store` | mesh | `postgres-database` | the store | +| `mesh-broker` | mesh | `amqp` | the broker | +| `the-artifact-store` | mesh | `artifact-store` | the artifact registry | +| `the-catalogue` | mesh | — | the catalogue | +| `npm-package-registry` | mesh | `npm-package-registry` | the forge | +| `git` | mesh | `git` | the forge | +| `the-build-machine` | node | — | a builder | +| `the-dns-port` | node | — | the local resolver | +| `the-intrusion-prevention` | node | — | an intrusion-prevention service | +| `the-packet-filter` | node | — | the packet filter | +| `the-resolver-configuration` | node | — | whichever of the alternative resolver configurations is chosen | +| `the-showcase` | node | — | the showcase module | + +The control plane holds this set in code, and a test asserts both its size and that every entry names +the record that made it a seat. This document follows the code, not the reverse. If the two disagree, +the test has been changed without this table, and the table is what is wrong. + +## A seat that delivers a provision + +A seat that delivers a provision may only be held by a module that provides it, at the seat's scope. +A mesh seat delivers a mesh-scoped provision. + +**Its holder answers for that provision.** When a requirement for it has more than one provider in the +mesh, the control plane takes, in order: + +1. the provider the consumer's node was pinned to, because a consumer coupled to one provider's + contents has said so ([23 — Choosing a provider](23-choosing-a-provider.md)); +2. the holder of the seat; +3. the only provider, when there is one; +4. otherwise nothing, and the requirement is refused with the candidates named. + +So a second provider can run beside the holder and harm nothing. The forge holds +`npm-package-registry`. An npm proxy may provide the same provision on another machine, and a module +requiring an npm registry is still served by the forge, without anybody pinning it. Moving the role +to the proxy is moving the seat: unassign the claim from one, assign it to the other, and every +consumer follows. + +**What a consumer receives is a grant**, the same as for any provision: where the provider answers, +what it serves, and a credential where one is minted. A consumer never reads the seat directly. The +one exception is the control plane itself, which reaches the store and the broker through a narrow +seat placeholder because it made them before any module existed and cannot be their consumer. + +## A seat that delivers nothing + +Most node seats deliver nothing. They say which module is this machine's packet filter, or which of +two alternative resolver configurations it runs, and a second holder is refused. That is the whole of +their job, and it is a real one: it is the mesh saying what a machine is, in words a person can read. + +## The overview + +The control plane lists every seat in the set with its scope, what it delivers, and each holder as a +node and a module. A seat nobody holds is listed as unheld. That is an answer, "this mesh has no +forge", and not a fault. + +Holdings are derived from assignments whenever they are asked for, never stored. The list is always +what the mesh is running, because it is computed from the same thing that decides what the mesh runs. + +## The git seat, and where a build comes from + +A module is built from a repository, a path and a ref. The repository is one of two things, and the +mesh records which: + +| form | means | recorded as | +|---|---|---| +| on the `git` seat | a repository on the forge that holds the seat | its path on the forge, and the seat | +| external | a repository anywhere else, a public forge for instance | its URL, exactly as given | + +For a repository on the seat, the control plane composes the clone URL at the moment of building, +from where the holder runs and the scheme and port it serves for `git`. The recorded source never +contains an address, so moving the forge changes nothing that was recorded. The build machine is not +told the difference: it receives a URL either way. + +With the seat unheld, a build from the seat is refused and says why. External builds carry on. + +**Not yet designed:** a credential for cloning a private repository. The mesh's own repositories are +public. The natural place for a clone credential is the `git` provision's grant, and that is a +decision still to take. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 5c10294..bc88ea9 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -34,6 +34,7 @@ document is written and this one's status becomes `implemented`. | [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) | | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | +| [`26-the-seats.md`](26-the-seats.md) | What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | ## Not yet written From c4cac767f8063ab13a531ff1562fd657b5168a94 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 20:35:31 +0200 Subject: [PATCH 10/26] ADR 0110: admit the-private-network, claimed by a manifest the control plane composes in code The enumeration behind the first set read manifests in two repositories and missed a claim made in the control plane's own code: the private-network module it ships claims the-private-network at node scope. A closed set without it would refuse the control plane's own module. Thirteen claims in use, naming twelve seats. --- ...-is-a-module-assignment-from-a-closed-set.md | 17 ++++++++++------- 03-DESIGN/01-to-be/26-the-seats.md | 1 + 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index 0d9e313..fb77c64 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -25,12 +25,13 @@ The names in use were each invented by the module that claims them: `the-showcas lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. The only way to answer "which seats does this mesh have, and which module holds each" is to read every manifest in two repositories, because the core modules' manifests moved into the control -plane's own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)). While this record -was being prepared, that enumeration was done once by hand, and it missed the control plane's own -manifest: eleven claims were reported where there are twelve. +plane's own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the +control plane's code, because one module it ships has its manifest composed there. While this +record was being prepared, that enumeration was done by hand, and it missed both of the last two +sources: eleven claims were reported where there are thirteen. **Some seats are the mesh's one of something that others consume, and nothing uses that fact.** -Of the twelve claims, four are held by a module that provides something consumers require: +Of the thirteen claims, four are held by a module that provides something consumers require: `mesh-store` (`postgres-database`, eleven consumers), `mesh-broker` (`amqp`), `the-artifact-store` (`artifact-store`) and `the-dns-port`. The rest deliver nothing to anybody, and are still meaningful: they say which module is this mesh's packet filter, or resolver configuration. @@ -85,9 +86,10 @@ requires the provision still resolves to the holder without anybody naming it. and which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh has no X", not an error. -**The first set is the eleven seats already claimed, plus one.** Twelve claims are in use, and they -name eleven seats because two alternative modules claim `the-resolver-configuration`. This record -admits every seat the catalogue and the control plane claim today, so no module is refused by it: +**The first set is the twelve seats already claimed, plus one.** Thirteen claims are in use, and +they name twelve seats because two alternative modules claim `the-resolver-configuration`. This +record admits every seat the catalogue and the control plane claim today, so no module is refused +by it: | seat | scope | delivers | held today by | made a seat by | |---|---|---|---|---| @@ -101,6 +103,7 @@ admits every seat the catalogue and the control plane claim today, so no module | `the-dns-port` | node | — | `dnsmasq` | this record | | `the-intrusion-prevention` | node | — | `fail2ban` | this record | | `the-packet-filter` | node | — | `nftables` | this record | +| `the-private-network` | node | — | the control plane's private-network module | this record | | `the-resolver-configuration` | node | — | `resolv-conf` or `resolved-split-dns` | this record | | `the-showcase` | node | — | `showcase` | this record | diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index b809256..1cda28f 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -56,6 +56,7 @@ argued for is an entry nobody can explain. | `the-dns-port` | node | — | the local resolver | | `the-intrusion-prevention` | node | — | an intrusion-prevention service | | `the-packet-filter` | node | — | the packet filter | +| `the-private-network` | node | — | the private network the mesh runs over | | `the-resolver-configuration` | node | — | whichever of the alternative resolver configurations is chosen | | `the-showcase` | node | — | the showcase module | From d94fe8f63869d612cf5bafd2ccfb259f035e7d02 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 20:47:48 +0200 Subject: [PATCH 11/26] To-be 26: name the files that implement the seats and the build source --- 03-DESIGN/01-to-be/26-the-seats.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 1cda28f..00c7a29 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -5,7 +5,8 @@ code: - mesh-controller internal/catalogue/seats.go - mesh-controller internal/catalogue/resolve.go - mesh-controller cmd/mesh-controller/seats.go - - mesh-controller cmd/mesh-controller/build.go + - mesh-controller cmd/mesh-controller/source.go + - mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql - mesh-catalog modules/gitea/module.json updated: 2026-09-25 decisions: From ca235e775fb5a9406d5acded7936f12a15f4136f Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 22:13:02 +0200 Subject: [PATCH 12/26] Issue 118 and ADR 0112 (proposed): a module definition names no node, no mesh and no path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Issue 118 records what a review of where module code reads its files found: 789 host-path strings in 70 of the catalogue's 71 definitions, every one a decision the definition makes about a machine. Mounts are checked (ADR 0091); the same paths retyped as values are not. It records what that has already allowed — a DNS provider that would provision nobody silently, a contributions file that names credentials by host path and so forces every provider to mount at the identical path, an SDK loop that treats an unwritten contributions file as empty without a word, defaults in code that disagree with their own manifests — and that no module can be assigned to one node twice, because every identity is keyed by the module's name. ADR 0112, proposed for review, answers it the way ADR 0038 answered ports: a definition names variables, and installing it resolves every one or refuses, from three sources — the assignment's own configuration, provisions the mesh resolves against a contract, and what the mesh generates or knows. A directory becomes a provision: the module requires one by name with its owner, mode and persistence, and where it lands is the assignment's. The mesh's own files stop carrying host paths. An assignment gets an identity of its own, so a module may run twice on one node. Checking copies for agreement was rejected as checking something that should not exist; rewriting paths per assignment was rejected as inferring which strings are paths by their shape. Syntax, a node's default layout, and when a second instance becomes possible are left to the design. --- ...e-definition-names-no-node-mesh-or-path.md | 129 ++++++++++++++++++ 02-DECISIONS/README.md | 1 + .../00-report.md | 78 +++++++++++ 3 files changed, 208 insertions(+) create mode 100644 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md create mode 100644 04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md new file mode 100644 index 0000000..0d664f9 --- /dev/null +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -0,0 +1,129 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +extends: 0046-a-module-configuration-is-its-assignments-not-its-manifest.md +--- + +# 112. A module definition names no node, no mesh and no path: everything it needs is resolved at assignment + +## Context + +[Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md) found +**789 host-path strings in 70 of the catalogue's 71 module definitions.** Each definition chooses +where on the machine its directories, mounts, bindings, secrets, env-files and received files live, +and often repeats that path in an environment variable or in code. Mounts are checked against what +the definition declares ([ADR 0091](0091-a-mount-is-declared-three-ways.md)); nothing checks the +copies. The issue records what that has already allowed: + +- a provider that would provision nobody without a word; +- a contributions file that carries host paths into containers, so every provider must mount its + grants directory at the identical path; +- defaults in code that disagree with their own manifests; +- no way to assign one module to one node twice, because every identity is keyed by the module's name. + +**The mesh has already decided this once, for ports.** [ADR 0038](0038-the-mesh-assigns-the-port.md): +the mesh assigns the machine-side port and the module says only what it needs, and three copies +became one fact. [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) +made the manifest identity and defaults, and the assignment's settings the configuration. +[ADR 0084](0084-which-provider-serves-a-consumer.md) made which provider serves a consumer part of +the assignment. Paths are the largest thing still left in the definition. + +## Considered Options + +**1. Keep host paths in definitions, and check that every copy agrees.** Rejected. It checks the +agreement of something that should not be there. A definition still could not follow its data to +another disk, be adopted onto a machine whose data is already somewhere, or run twice on one node. + +**2. Keep host paths in definitions, and have the mesh rewrite them per assignment.** Rejected. It is +string surgery on paths: deciding which strings are machine paths by their shape, which is the +inference this repository has refused elsewhere. And the definition would still read as though it +decided where things live. + +**3. A definition names variables, and the assignment resolves them.** Chosen. + +## Decision + +**A module definition is node-agnostic and mesh-agnostic.** It names no node, no mesh and no host +path. Everything that makes a running instance *this* instance is a variable. + +**Installing a module on a node resolves every variable, or refuses.** A refusal names each +unresolved variable and what could answer it. Variables are answered from three sources: + +1. **The assignment's own configuration.** Values chosen for this module on this node, carried as + settings ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). An + endpoint binding a public name to a port is one. +2. **Provisions, resolved by the mesh against a contract.** A database, a bucket, a vhost, the + occupant of a seat, another assignment: each with the contract the mesh and the module's + specification define. Which node answers one is part of the assignment + ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its database from + another node. +3. **What the mesh generates or knows.** Minted secrets, the ports it assigns + ([ADR 0038](0038-the-mesh-assigns-the-port.md)), facts about the machine. + +**A directory is a provision.** A module requires one by name, such as its configuration or its +data, and the host on the node where the assignment runs answers it. Its contract is what the +module needs from it: + +- the owner and mode, including the owner the image expects + ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)); +- whether it holds data that outlives the module ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)) + or is disposable. + +*Where* it is on the machine is the assignment's. A node has a default layout, and an assignment +may place one directory elsewhere, on a second disk or where an adopted machine's data already is +([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). An operator's shared data +([ADR 0051](0051-shared-data-is-the-operators.md)) is the same provision with the operator as owner: +the module says it needs read or read-write access, and the assignment says where the data is. A +directory is always answered on the module's own node, because a host path means nothing on any +other. + +**Inside a container, a module sees its own paths.** The definition says where the image expects +each directory. The mesh mounts the assignment's location there. No host path is ever a value a +process reads. + +**The mesh's own files carry no host path.** Bindings, secrets and contributions are named relative +to where the module receives them, so a provider reads what it was given without mounting anything +at a machine-identical path. + +**An assignment has an identity of its own**, an instance name defaulting to the module's name. +Everything keyed by the module's name today is keyed by it instead: directories, container names, +the login a consumer presents, broker accounts, a seat's holder. So **one module may be assigned to +one node more than once.** What must stay singular stays so by a seat, or by the assignment's own +configuration colliding: a public name already taken is refused like any other singular thing. + +## Consequences + +- **Every definition changes.** 70 of 71 name host paths today. The change is mechanical for most + and needs the design to say how existing modules migrate without their data moving: an adopted + or already-running assignment is placed where its data already is. +- The control plane resolves variables at assignment and refuses unresolved ones. The host answers + directory provisions. The contributions file's format changes, and so does the SDK's reconcile + loop that reads it. +- Identity moves from the module to the assignment, which touches logins, broker accounts and + every resource name. +- **What got harder:** a definition no longer says where a module's data is on a machine. The + assignment does, and `plan` shows it. That is the point, and it is also a real loss of + at-a-glance legibility, which the overview has to give back. +- **Not decided here:** the variable syntax; a node's default layout; whether a second instance of a + module is supported from the first step or after the definitions have moved. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A definition names no host path | A catalogue test fails on any absolute path outside the image side of a mount, with a declared list of exceptions that shrinks to empty as definitions move. | +| Installation resolves every variable | A resolution test with one variable unanswered: refused, naming the variable and its possible sources. | +| A provider reads what it was given without an identical mount | A provisioner test reading a contributions file whose credentials are named relative to where it is mounted. | +| A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, no collision. | + +## References + +- [Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md): the evidence +- [ADR 0038](0038-the-mesh-assigns-the-port.md): the same decision, for ports +- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): configuration is the assignment's +- [ADR 0084](0084-which-provider-serves-a-consumer.md): which node answers is the assignment's +- [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 964231e..a17b486 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -157,6 +157,7 @@ python3 00-META/checks/index.py fail if stale - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md) - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) - **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) +- **0112** — [A module definition names no node, no mesh and no path: everything it needs is resolved at assignment](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* ### How it is built diff --git a/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md b/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md new file mode 100644 index 0000000..89a0924 --- /dev/null +++ b/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md @@ -0,0 +1,78 @@ +--- +status: open +opened: 2026-09-25 +located-in: [] +fixed-by: +amended-design: +--- + +# 118 — A module definition decides where its files live on the machine + +## What was observed + +A review of where module code reads its files turned up a cross-cutting pattern. Every module +definition in the catalogue chooses, in its own manifest, where on the machine its files live. +Counted on the catalogue's `main`, 2026-09-25: + +| where in the definition | host-path strings | +|---|---| +| directory and file resources | 257 | +| container mounts, host side | 230 | +| own secrets | 78 | +| bindings | 53 | +| env-files | 50 | +| secrets | 35 | +| container environment | 28 | +| accesses | 21 | +| receives, grants | 24 | +| everything else | 13 | + +**789 host-path strings in 70 of the 71 definitions.** Mounts are checked: a container may not +mount a path its module never declared ([ADR 0091](../../02-DECISIONS/0091-a-mount-is-declared-three-ways.md)). +Nothing checks the same path where it is retyped as a value: an environment variable, an env-file +line, a literal in module code. + +### Where that has already gone wrong + +- **A provider that would provision nobody, silently.** One DNS provider mounts its grants + directory at a short path inside its container, then tells its provisioner to read the + contributions file at the host path, which does not exist in there. Nothing requires the + provision today, so it has not failed yet. When a consumer arrives, it will get no record, and + nobody will be told. +- **The mesh's own wire carries host paths into containers.** Each contribution names its + consumer's credential as "the file on this machine holding that consumer's credential", a host + path computed from the provider's grants directory. So every provider has to mount that directory + at the *identical* path, or it cannot read what it was given. Eleven of the twelve providers do. + It is a convention nothing states or checks, and the twelfth is the provider above. +- **The warning that would have caught it is lost in the SDK.** The control plane always writes the + contributions file, even when empty, so a provider can tell "nothing asked" from "never written". + The SDK's reconcile loop treats an unreadable file as empty, and logs nothing. +- **Code carries copies with nothing checking them.** Several modules default a path in code when an + environment variable is unset. Five of those defaults disagree with the value their own manifest + sets. One of them is a host path used inside a container that does not mount it. + +### And a module cannot be assigned to one node twice + +Everything that identifies a running module is keyed by the module's name: its directories, its +container names, the login it presents to a provider, its broker account. Two assignments of one +module to one node would share every one of them. Assigning the same application twice is an +ordinary need: production beside staging, one site per customer, two instances of one service +configured differently, two stores of one engine. + +## Why it matters beyond this instance + +A definition that names machine paths is not portable between nodes. It cannot follow data onto a +second disk, or onto a machine being adopted with its data already in place, without editing the +module. It cannot run twice on one node. It keeps every path in two or three places with nothing +checking that they agree. The defects above are what that allows, and each was found by reading, +not by any check. + +## Open questions + +- Should a definition name any host path at all, or should every location come from the + assignment and the mesh? +- If a directory is something a module *requires* rather than *declares*, what is its contract: + ownership, mode, whether it is kept when the module goes? +- What identifies an assignment, if a module may be assigned to one node more than once? +- What would the contributions file carry instead of host paths, so a provider needs no + identical-path mount? From 90fb7ae4ada35d38229c1bd34db66c682c704615 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 22:13:37 +0200 Subject: [PATCH 13/26] ADR 0112: a module's own secrets are a provision from the vault, not something the mesh generates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first draft listed minted secrets under what the mesh generates. ADR 0085 made a module's own secret — a password, an internal token, an external key it was handed — a secret provision answered by the vault, like a database by the store. What the mesh still mints is the delivery credential for each provision a module takes (ADR 0048), the vault's own included. --- ...module-definition-names-no-node-mesh-or-path.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index 0d664f9..da9bed4 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -56,12 +56,16 @@ unresolved variable and what could answer it. Variables are answered from three settings ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). An endpoint binding a public name to a port is one. 2. **Provisions, resolved by the mesh against a contract.** A database, a bucket, a vhost, the - occupant of a seat, another assignment: each with the contract the mesh and the module's - specification define. Which node answers one is part of the assignment + occupant of a seat, another assignment, and **a secret from the vault**: a module's own + password, internal token or external key is a `secret` provision like any other + ([ADR 0085](0085-a-secret-is-a-provision.md)). Each comes with the contract the mesh and the + module's specification define. Which node answers one is part of the assignment ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its database from another node. -3. **What the mesh generates or knows.** Minted secrets, the ports it assigns - ([ADR 0038](0038-the-mesh-assigns-the-port.md)), facts about the machine. +3. **What the mesh generates or knows.** The credential it mints for each provision a module + takes, which is how every provision is delivered, the vault's included + ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)); the ports it assigns + ([ADR 0038](0038-the-mesh-assigns-the-port.md)); facts about the machine. **A directory is a provision.** A module requires one by name, such as its configuration or its data, and the host on the node where the assignment runs answers it. Its contract is what the @@ -125,5 +129,7 @@ configuration colliding: a public name already taken is refused like any other s - [ADR 0038](0038-the-mesh-assigns-the-port.md): the same decision, for ports - [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): configuration is the assignment's - [ADR 0084](0084-which-provider-serves-a-consumer.md): which node answers is the assignment's +- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): + a module's own secrets come from the vault; the mesh mints only the delivery credential - [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries From 766819015412e44e54ffb6af06cc455eea96be06 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 22:21:39 +0200 Subject: [PATCH 14/26] ADR 0112 and issue 118: address the review - Secrets follow ADR 0085 as amended: a module's own secret is a provision the controller mints and the vault records. The previous commit had that backwards. Whether the vault should generate instead is recorded as an open question, not decided. - A directory's contract is owner and mode only. The persistence flag was the keep flag ADR 0030 refused; a directory is kept while it holds anything, and disposable data is a named volume (0107). - An operator's shared data stays an access (ADR 0051), which rejected an operator-owned directory. Only where its path is written moves to the assignment. - The records it changes on acceptance are named: 0051, 0091, 0046 (settings keyed by instance), 0084 (a provider is a node and an instance), and the glossary, which gains its new words only when the record is accepted. - How it is checked covers every stated rule. Container-side paths are no longer flagged by the host-path rule, and code fallbacks are covered. - Provisions are what other modules provide. A seat's occupant is not listed as one, and the vault is not described as selectable per assignment. - 'Control plane' becomes 'controller'. The provider count is ten of eleven, not eleven of twelve. --- ...e-definition-names-no-node-mesh-or-path.md | 114 +++++++++++------- .../00-report.md | 7 +- 2 files changed, 74 insertions(+), 47 deletions(-) diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index da9bed4..a22f532 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -55,34 +55,36 @@ unresolved variable and what could answer it. Variables are answered from three 1. **The assignment's own configuration.** Values chosen for this module on this node, carried as settings ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). An endpoint binding a public name to a port is one. -2. **Provisions, resolved by the mesh against a contract.** A database, a bucket, a vhost, the - occupant of a seat, another assignment, and **a secret from the vault**: a module's own - password, internal token or external key is a `secret` provision like any other - ([ADR 0085](0085-a-secret-is-a-provision.md)). Each comes with the contract the mesh and the - module's specification define. Which node answers one is part of the assignment - ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its database from - another node. -3. **What the mesh generates or knows.** The credential it mints for each provision a module - takes, which is how every provision is delivered, the vault's included - ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)); the ports it assigns - ([ADR 0038](0038-the-mesh-assigns-the-port.md)); facts about the machine. +2. **Provisions, resolved by the mesh against a contract.** What other modules provide: a database, + a bucket, a vhost, a secret. Each comes with the contract the mesh and the module's + specification define. Where a provision has several providers, which one answers is part of the + assignment ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its + database from another node. Some have one provider per mesh by decision: a module's own secret + is a `secret` provision the controller mints and the vault records + ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). Whether the vault should instead + generate a secret against its contract is an open question, raised while reviewing this, and not + decided here. +3. **What the mesh generates or knows.** The credential the controller mints for each provision a + module takes ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)), the ports it + assigns ([ADR 0038](0038-the-mesh-assigns-the-port.md)), facts about the machine. -**A directory is a provision.** A module requires one by name, such as its configuration or its -data, and the host on the node where the assignment runs answers it. Its contract is what the -module needs from it: +**A directory is a provision, provided by the node's host.** A module requires one by name, such as +its configuration or its data. Its contract is the owner and mode it needs, including the owner its +image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). It +carries no persistence flag. A directory is kept while it holds anything +([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), which refused a `keep` flag for good +reason), and data that is disposable is not a directory at all but a named volume (0107). -- the owner and mode, including the owner the image expects - ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)); -- whether it holds data that outlives the module ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)) - or is disposable. +*Where* a directory is on the machine is the assignment's. A node has a default layout, and an +assignment may place one directory elsewhere: on a second disk, or where an adopted machine's data +already is ([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). A directory is +always provided on the module's own node, because a host path means nothing on any other. -*Where* it is on the machine is the assignment's. A node has a default layout, and an assignment -may place one directory elsewhere, on a second disk or where an adopted machine's data already is -([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). An operator's shared data -([ADR 0051](0051-shared-data-is-the-operators.md)) is the same provision with the operator as owner: -the module says it needs read or read-write access, and the assignment says where the data is. A -directory is always answered on the module's own node, because a host path means nothing on any -other. +**An operator's shared data stays an `access`** ([ADR 0051](0051-shared-data-is-the-operators.md)), +not a directory provision. 0051 rejected giving a directory an operator owner, because the mesh +must never create, chown or remove such data, and that stands. What changes is only where its +location is written: the module says it needs read or read-write access, and the assignment says +where the data is. **Inside a container, a module sees its own paths.** The definition says where the image expects each directory. The mesh mounts the assignment's location there. No host path is ever a value a @@ -92,44 +94,68 @@ process reads. to where the module receives them, so a provider reads what it was given without mounting anything at a machine-identical path. -**An assignment has an identity of its own**, an instance name defaulting to the module's name. -Everything keyed by the module's name today is keyed by it instead: directories, container names, -the login a consumer presents, broker accounts, a seat's holder. So **one module may be assigned to -one node more than once.** What must stay singular stays so by a seat, or by the assignment's own -configuration colliding: a public name already taken is refused like any other singular thing. +**An assignment has an identity of its own: an instance name**, defaulting to the module's name. +Everything keyed by the module's name today is keyed by the instance instead: directories, +container names, the login a consumer presents, broker accounts, a claim's holder, the settings +an assignment carries, and a provider's identity. So **one module may be assigned to one node more +than once.** What must stay singular stays so by a claim, or by the assignment's own configuration +colliding: a public name already taken is refused like any other singular thing. + +## What this changes in earlier records + +On acceptance, each of these is amended by a record of its own, not edited: + +- [ADR 0051](0051-shared-data-is-the-operators.md): an access keeps its shape and its semantics; its + path moves from the definition to the assignment. +- [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved variable, + checked as resolved rather than as a path the definition declares. +- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings are + addressed to an instance, not to a module on a node. +- [ADR 0084](0084-which-provider-serves-a-consumer.md): a provider is a (node, instance) pair, not a + (node, module) pair, so a consumer can name one of two instances on one node. +- The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to + include a directory the node's host provides, and *instance* is added. Neither lands while this + record is only proposed, because the glossary is the authority on the words in use, not on words + under review. ## Consequences -- **Every definition changes.** 70 of 71 name host paths today. The change is mechanical for most - and needs the design to say how existing modules migrate without their data moving: an adopted - or already-running assignment is placed where its data already is. -- The control plane resolves variables at assignment and refuses unresolved ones. The host answers - directory provisions. The contributions file's format changes, and so does the SDK's reconcile - loop that reads it. -- Identity moves from the module to the assignment, which touches logins, broker accounts and - every resource name. +- **Every definition changes.** 70 of 71 name host paths today. The change is mechanical for most. + The design has to say how existing modules migrate without their data moving: an adopted or + already-running assignment is placed where its data already is. +- The controller resolves variables at assignment and refuses unresolved ones. The host provides + directories. The contributions file's format changes, and so does the SDK's reconcile loop that + reads it. +- Identity moves from the module to the instance, which touches logins, broker accounts, settings, + provider selection and every resource name. - **What got harder:** a definition no longer says where a module's data is on a machine. The assignment does, and `plan` shows it. That is the point, and it is also a real loss of at-a-glance legibility, which the overview has to give back. - **Not decided here:** the variable syntax; a node's default layout; whether a second instance of a - module is supported from the first step or after the definitions have moved. + module is supported from the first step or after the definitions have moved; whether the vault + generates secrets. ## How it is checked | Rule | Checked by | |---|---| -| A definition names no host path | A catalogue test fails on any absolute path outside the image side of a mount, with a declared list of exceptions that shrinks to empty as definitions move. | +| A definition names no host path | A catalogue test: the host side of every mount, and every resource location, binding, secret, receives and grants entry, is a variable rather than an absolute path. A declared list of exceptions shrinks to empty as definitions move. | +| No host path is a value a process reads | A catalogue test: every absolute path in a container's environment or env-files lies on the container side of one of its mounts, or is declared the image's own. A second test finds literal paths in module code used as fallbacks for an environment variable. | +| A definition names no node and no mesh | The parser has no field that names a node; a node is named only in an assignment. A catalogue test finds no domain name in any definition value. | | Installation resolves every variable | A resolution test with one variable unanswered: refused, naming the variable and its possible sources. | +| A directory is provided on its module's own node | A resolution test: an assignment placing a directory on another node is refused. | | A provider reads what it was given without an identical mount | A provisioner test reading a contributions file whose credentials are named relative to where it is mounted. | -| A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, no collision. | +| A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, separate settings, no collision. | +| A public name already taken is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | +| An adopted assignment is placed where its data is | An adoption test: the directory resolves to the data's existing location, and nothing is moved. | ## References - [Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md): the evidence - [ADR 0038](0038-the-mesh-assigns-the-port.md): the same decision, for ports - [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): configuration is the assignment's -- [ADR 0084](0084-which-provider-serves-a-consumer.md): which node answers is the assignment's +- [ADR 0084](0084-which-provider-serves-a-consumer.md): which provider answers is the assignment's - [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): - a module's own secrets come from the vault; the mesh mints only the delivery credential + secrets as a provision, and who mints what - [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), - [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries, and what it must not diff --git a/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md b/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md index 89a0924..7a8e416 100644 --- a/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md +++ b/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md @@ -42,9 +42,10 @@ line, a literal in module code. - **The mesh's own wire carries host paths into containers.** Each contribution names its consumer's credential as "the file on this machine holding that consumer's credential", a host path computed from the provider's grants directory. So every provider has to mount that directory - at the *identical* path, or it cannot read what it was given. Eleven of the twelve providers do. - It is a convention nothing states or checks, and the twelfth is the provider above. -- **The warning that would have caught it is lost in the SDK.** The control plane always writes the + at the *identical* path, or it cannot read what it was given. Ten of the eleven providers with a + grants directory do. It is a convention nothing states or checks, and the eleventh is the + provider above. +- **The warning that would have caught it is lost in the SDK.** The controller always writes the contributions file, even when empty, so a provider can tell "nothing asked" from "never written". The SDK's reconcile loop treats an unreadable file as empty, and logs nothing. - **Code carries copies with nothing checking them.** Several modules default a path in code when an From aad92ea8fec15c1f665629ebcfd0ab23b215ec1c Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 22:34:46 +0200 Subject: [PATCH 15/26] To-be 27 and ADR 0113 (proposed): a module requires, the mesh resolves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The design pass. Everything a module needs is a requirement: a name, a contract, and one of four kinds of provider — a module, the node's host, the mesh, the operator. Installing a module resolves every requirement or refuses, naming everything missing at once. It retires six mechanisms that grew separately: provisions through bindings, settings, assigned ports, machine facts, minted secrets and literals in the definition. ADR 0113, proposed: a provider makes what it provides, and the mesh carries it back sealed to the consumer's node. It is the return path ADR 0048 left "to a separate decision", now needed three ways: data provisions with nothing to answer with, contracts needing a value the controller cannot make, and a vault that generates nothing. Who a consumer is stays the mesh's (ADR 0049). Genesis is the one exception. On acceptance it supersedes 0048 and amends 0085. ADR 0112 is revised from three sources to that single concept. ADR 0110 is amended for two points raised in review. The vault gets the mesh-vault seat (issue 106). A seat's holder outranks co-location for a provision it delivers. Writing that down exposed an inconsistency: mesh-store delivering postgres-database would have sent every database consumer to the control-node, against to-be 23's node-local stores. So a seat delivers a provision only where the mesh has one answer for everyone — artifact store, npm registry, git, vault — and mesh-store and mesh-broker deliver nothing. 23 and 26 follow. 'Control plane' becomes 'controller' in the records written today. --- ...s-a-module-assignment-from-a-closed-set.md | 59 +++-- ...d-source-is-on-the-git-seat-or-external.md | 8 +- ...e-definition-names-no-node-mesh-or-path.md | 183 +++++++------- ...hat-it-provides-and-the-mesh-carries-it.md | 125 ++++++++++ 02-DECISIONS/README.md | 3 +- 03-DESIGN/01-to-be/23-choosing-a-provider.md | 11 +- 03-DESIGN/01-to-be/26-the-seats.md | 32 ++- .../27-a-module-requires-the-mesh-resolves.md | 225 ++++++++++++++++++ 03-DESIGN/01-to-be/README.md | 1 + 9 files changed, 524 insertions(+), 123 deletions(-) create mode 100644 02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md create mode 100644 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index fb77c64..a0d18cc 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -16,7 +16,7 @@ singular, at a scope, and a second holder is refused. [ADR 0079](0079-the-founda named the foundation's three after their servers. That mechanism is enforced and works. What it means has drifted, and three things are now true of it that no record says. -**Any well-formed name becomes a seat by being claimed.** The control plane's manifest check +**Any well-formed name becomes a seat by being claimed.** The controller's manifest check refuses a claim only for a malformed name or an unknown scope. Nothing says which seats a mesh has. The names in use were each invented by the module that claims them: `the-showcase`, `the-build-machine`, `the-intrusion-prevention`. @@ -24,9 +24,9 @@ The names in use were each invented by the module that claims them: `the-showcas **Nothing can say what a mesh has, or who fills it.** There is no seat table and no command that lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. The only way to answer "which seats does this mesh have, and which module holds each" is to read -every manifest in two repositories, because the core modules' manifests moved into the control -plane's own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the -control plane's code, because one module it ships has its manifest composed there. While this +every manifest in two repositories, because the core modules' manifests moved into the controller's +own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's +code, because one module it ships has its manifest composed there. While this record was being prepared, that enumeration was done by hand, and it missed both of the last two sources: eleven claims were reported where there are thirteen. @@ -68,34 +68,49 @@ second thing to disagree with the first. **A seat may deliver a provision, and then its holder answers for it.** A seat that delivers a provision can only be held by a module that provides it, at the seat's scope, and a claim that does -not is refused. When several providers answer a requirement for that provision, resolution takes, in -order: +not is refused. A requirement for that provision resolves, in order, to: 1. a provider the consumer's node was **pinned** to — a consumer coupled to one provider's contents, as [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) already allows; -2. **the holder of the seat** that delivers it; -3. the **only** provider, when there is one; -4. otherwise, refused with the candidates named, as now. +2. **the holder of the seat** that delivers it, **even when another provider runs on the consumer's + own node**; +3. otherwise refused, naming the unheld seat. + +**Co-location does not apply to a provision a seat delivers.** For every other provision, a provider +on the consumer's own node answers first, then the only provider, then refusal +([to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md)). A seat exists to say *which one is the +mesh's*, and co-location answering first would let any second provider on a consumer's machine take +over silently for that consumer. That is the failure [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) +names for the vault. This keeps [ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with several answers is never guessed. The seat is not a guess. It is the choice made once, mesh-wide, by assigning the holder, instead of once per consumer by pinning. A second provider may run beside the holder, and whatever requires the provision still resolves to the holder without anybody naming it. -**Seats are also informational.** The control plane lists every seat in the set, what it delivers, +**Seats are also informational.** The controller lists every seat in the set, what it delivers, and which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh has no X", not an error. -**The first set is the twelve seats already claimed, plus one.** Thirteen claims are in use, and +**A seat delivers a provision only where the mesh has one answer for everyone.** That is a design +decision about the provision, not about the seat. The artifact store, the npm registry, git and the +vault are each one per mesh by their own records, so their seats deliver them. The store and the +broker are not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its +own stores, with a consumer served by the one on its own machine. So `mesh-store` and `mesh-broker` +keep guarding that the foundation's own server is singular, and deliver nothing. Were they to +deliver, every database consumer on every node would be sent to the control-node's store. + +**The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and they name twelve seats because two alternative modules claim `the-resolver-configuration`. This -record admits every seat the catalogue and the control plane claim today, so no module is refused -by it: +record admits every seat the catalogue and the controller claim today, so no module is refused by +it: | seat | scope | delivers | held today by | made a seat by | |---|---|---|---|---| | `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-store` | mesh | `postgres-database` | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-broker` | mesh | `amqp` | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-store` | mesh | — | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-broker` | mesh | — | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-vault` | mesh | `secret` | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | | `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | | `the-catalogue` | mesh | — | `mesh-catalog` | this record | | `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | @@ -103,15 +118,21 @@ by it: | `the-dns-port` | node | — | `dnsmasq` | this record | | `the-intrusion-prevention` | node | — | `fail2ban` | this record | | `the-packet-filter` | node | — | `nftables` | this record | -| `the-private-network` | node | — | the control plane's private-network module | this record | +| `the-private-network` | node | — | the controller's private-network module | this record | | `the-resolver-configuration` | node | — | `resolv-conf` or `resolved-split-dns` | this record | | `the-showcase` | node | — | `showcase` | this record | -`npm-package-registry` is the one addition. It is [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md)'s +There are two additions. `npm-package-registry` is [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md)'s seat, named after the provision it delivers, as 0079 named the foundation's seats after what they are. gitea claims it. verdaccio provides the same provision and claims nothing, so it is the second provider this record exists to make harmless. +`mesh-vault` answers [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md). The vault +is one per mesh ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and until now that was +enforced by nothing. A second vault would have answered requirements silently, and any consumer on +its machine would have been served by it through co-location. The seat is named after its server, by +the 0079 convention, and the vault module claims it. + `the-dns-port` is listed as delivering nothing, although `dnsmasq` provides `wildcard-resolution`. That provision is node-scoped and answered on the machine, so no preference between providers arises. Whether the seat should say it delivers it is left for when a second resolver makes the @@ -119,7 +140,7 @@ question real. ## Consequences -- The control plane carries the set in code. A test asserts its size, and that every entry names the +- The controller carries the set in code. A test asserts its size, and that every entry names the record that made it a seat, so changing the set means finding the argument rather than a number. This is the pattern the host's vocabulary test already follows. - Manifest validation refuses an unknown seat, a seat claimed at the wrong scope, and a @@ -133,7 +154,7 @@ question real. `npm-package-registry` throughout, per [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md). - **What got harder:** a module wanting a new singular role can no longer invent a name. It needs a record. That is the point, and it costs one record per seat. -- **Not changed:** `${seat::}` stays as it is. It exists so the control plane can reach a +- **Not changed:** `${seat::}` stays as it is. It exists so the controller can reach a foundation it made before any module existed, and it cannot be a consumer. A module that needs something from a seat's holder requires the provision the seat delivers, and receives it the way any provision is received: through a grant. diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md index ed7441a..0f087af 100644 --- a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -12,7 +12,7 @@ extends: 0069-a-module-is-a-repository-and-a-path.md ## Context [ADR 0069](0069-a-module-is-a-repository-and-a-path.md) made a module a repository, a path and a -ref, and the control plane records all three against the module so it can rebuild it and say when +ref, and the controller records all three against the module so it can rebuild it and say when its source has moved ahead. **The repository is recorded exactly as a person typed it.** `build ` hands the string to a build machine, which runs `git clone` on it, and the same string becomes the module's recorded source. @@ -52,7 +52,7 @@ serving how a repository on it is cloned: the scheme and the port. gitea claims - `build --self /` builds from a repository on the seat's holder. The recorded source is the repository's path on that holder, and the seat it is on. **It never contains an - address.** At the moment of building, the control plane composes the clone URL from where the + address.** At the moment of building, the controller composes the clone URL from where the holder runs and what it serves for `git`, so a moved forge changes nothing recorded. - `build ` is unchanged: an external repository, recorded and cloned exactly as given. GitHub and GitLab are the ordinary cases. @@ -63,11 +63,11 @@ mesh without a forge of its own builds from external repositories only, and says failing to clone. **The build machine is not told the difference.** It receives a URL either way. Composing the URL is -the control plane's job, because only the control plane knows where the seat's holder runs. +the controller's job, because only the controller knows where the seat's holder runs. ## Consequences -- The control plane's inventory gains a column saying which seat a source is on. It is empty for +- The controller's inventory gains a column saying which seat a source is on. It is empty for every module recorded before this, which is correct: they were all recorded as literal URLs. - `build`, `build --behind` and `build --dry-run` resolve a seat source before asking a builder. The recorded source keeps the seat form; the build log keeps the URL that was actually cloned, because diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index a22f532..afd4469 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -7,7 +7,7 @@ reconstructed: false extends: 0046-a-module-configuration-is-its-assignments-not-its-manifest.md --- -# 112. A module definition names no node, no mesh and no path: everything it needs is resolved at assignment +# 112. A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves ## Context @@ -24,12 +24,20 @@ copies. The issue records what that has already allowed: - defaults in code that disagree with their own manifests; - no way to assign one module to one node twice, because every identity is keyed by the module's name. -**The mesh has already decided this once, for ports.** [ADR 0038](0038-the-mesh-assigns-the-port.md): -the mesh assigns the machine-side port and the module says only what it needs, and three copies -became one fact. [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md) -made the manifest identity and defaults, and the assignment's settings the configuration. -[ADR 0084](0084-which-provider-serves-a-consumer.md) made which provider serves a consumer part of -the assignment. Paths are the largest thing still left in the definition. +**Paths are one case of a wider pattern.** A module gets what it needs through at least six separate +mechanisms today, each with its own syntax and its own failure modes: + +- provisions, read through bindings; +- settings on the assignment ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)); +- ports the mesh assigns ([ADR 0038](0038-the-mesh-assigns-the-port.md)); +- machine facts a manifest asks for; +- secrets, either minted or accepted from an operator; +- literals carried in the definition itself. + +The mesh has already unified parts of this. Ports became the mesh's rather than the module's (0038), +configuration became the assignment's (0046), and which provider serves a consumer became the +assignment's choice ([ADR 0084](0084-which-provider-serves-a-consumer.md)). What remains is the +concept that joins them. ## Considered Options @@ -37,114 +45,123 @@ the assignment. Paths are the largest thing still left in the definition. agreement of something that should not be there. A definition still could not follow its data to another disk, be adopted onto a machine whose data is already somewhere, or run twice on one node. -**2. Keep host paths in definitions, and have the mesh rewrite them per assignment.** Rejected. It is -string surgery on paths: deciding which strings are machine paths by their shape, which is the -inference this repository has refused elsewhere. And the definition would still read as though it -decided where things live. +**2. Keep the separate mechanisms, and add directories as a seventh.** Rejected. It fixes paths and +keeps the pattern that produced them: each mechanism is resolved, validated and refused differently, +so a module author learns six systems and a reviewer checks six kinds of gap. -**3. A definition names variables, and the assignment resolves them.** Chosen. +**3. One concept: a module requires, and the mesh resolves every requirement against a contract.** +Chosen. ## Decision **A module definition is node-agnostic and mesh-agnostic.** It names no node, no mesh and no host -path. Everything that makes a running instance *this* instance is a variable. +path. **Everything a module needs is a requirement**: a name, a contract saying what the module may +read from it, and which kind of provider answers it. -**Installing a module on a node resolves every variable, or refuses.** A refusal names each -unresolved variable and what could answer it. Variables are answered from three sources: +**Installing a module on a node resolves every requirement, or refuses.** A refusal names each +unresolved requirement and what could answer it, all at once. -1. **The assignment's own configuration.** Values chosen for this module on this node, carried as - settings ([ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md)). An - endpoint binding a public name to a port is one. -2. **Provisions, resolved by the mesh against a contract.** What other modules provide: a database, - a bucket, a vhost, a secret. Each comes with the contract the mesh and the module's - specification define. Where a provision has several providers, which one answers is part of the - assignment ([ADR 0084](0084-which-provider-serves-a-consumer.md)), so a module may take its - database from another node. Some have one provider per mesh by decision: a module's own secret - is a `secret` provision the controller mints and the vault records - ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). Whether the vault should instead - generate a secret against its contract is an open question, raised while reviewing this, and not - decided here. -3. **What the mesh generates or knows.** The credential the controller mints for each provision a - module takes ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)), the ports it - assigns ([ADR 0038](0038-the-mesh-assigns-the-port.md)), facts about the machine. +**There are four kinds of provider, and the set is closed:** -**A directory is a provision, provided by the node's host.** A module requires one by name, such as -its configuration or its data. Its contract is the owner and mode it needs, including the owner its -image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). It -carries no persistence flag. A directory is kept while it holds anything +| provider | answers | today's mechanism it replaces | +|---|---|---| +| **another module** | a database, a bucket, a vhost, a secret, a route | provisions and bindings | +| **the node's host** | a directory, a port, facts about the machine | resource paths, `${port:}`, `${machine:}`, facts | +| **the mesh** | the module's identity and names, and the delivery of every answer | derived logins, minted delivery | +| **the operator, through the assignment** | a value a person chooses: a public name, a greeting, an external key | settings, carried literals | + +A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and +[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say: a pin, then the holder of a +seat that delivers the provision, then co-location, then the only one. A host provider is always the +module's own node, because a host path or a port means nothing on any other. An operator value is the +assignment's, or the requirement's default, or unresolved. + +**A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a +default. It needs no provider module, no grant and no credential. An operator value that is secret, +like an external API key, is kept by the vault as an operator-delivered value +([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). + +**What a provider answers with is its contract's fields**, made by the provider and carried back to +the consumer by the mesh ([ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). + +**A directory is a host provision.** Its contract is the owner and mode the module needs, including +the owner its image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). +It carries no persistence flag. A directory is kept while it holds anything ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), which refused a `keep` flag for good -reason), and data that is disposable is not a directory at all but a named volume (0107). - -*Where* a directory is on the machine is the assignment's. A node has a default layout, and an -assignment may place one directory elsewhere: on a second disk, or where an adopted machine's data -already is ([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). A directory is -always provided on the module's own node, because a host path means nothing on any other. +reason), and data that is disposable is not a directory but a named volume (0107). *Where* it is on +the machine is the assignment's. A node has a default layout, and an assignment may place a directory +elsewhere: on a second disk, or where an adopted machine's data already is +([ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)). **An operator's shared data stays an `access`** ([ADR 0051](0051-shared-data-is-the-operators.md)), -not a directory provision. 0051 rejected giving a directory an operator owner, because the mesh -must never create, chown or remove such data, and that stands. What changes is only where its -location is written: the module says it needs read or read-write access, and the assignment says -where the data is. +not a directory. 0051 rejected giving a directory an operator owner, because the mesh must never +create, chown or remove such data, and that stands. Only where its location is written changes: the +module requires read or read-write access, and the assignment says where the data is. -**Inside a container, a module sees its own paths.** The definition says where the image expects -each directory. The mesh mounts the assignment's location there. No host path is ever a value a -process reads. - -**The mesh's own files carry no host path.** Bindings, secrets and contributions are named relative -to where the module receives them, so a provider reads what it was given without mounting anything -at a machine-identical path. +**Inside a container, a module sees its own paths.** The definition says where the image expects each +directory. The mesh mounts the assignment's location there. No host path is ever a value a process +reads, and the mesh's own files (answers, contributions) name nothing by host path, so a provider needs +no mount at a machine-identical path. **An assignment has an identity of its own: an instance name**, defaulting to the module's name. -Everything keyed by the module's name today is keyed by the instance instead: directories, -container names, the login a consumer presents, broker accounts, a claim's holder, the settings -an assignment carries, and a provider's identity. So **one module may be assigned to one node more -than once.** What must stay singular stays so by a claim, or by the assignment's own configuration -colliding: a public name already taken is refused like any other singular thing. +Everything keyed by the module's name today is keyed by the instance instead: directories, container +names, the login a consumer presents, broker accounts, a claim's holder, the settings an assignment +carries, and a provider's identity. So **one module may be assigned to one node more than once.** What +must stay singular stays so by a claim, or by an operator value colliding: a public name already taken +is refused like any other singular thing. + +**Genesis is the one exception**, as [ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md) +states: the foundation's requirements are answered by genesis itself, before any provider exists. ## What this changes in earlier records On acceptance, each of these is amended by a record of its own, not edited: +- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings become + operator requirements, addressed to an instance rather than to a module on a node. +- [ADR 0038](0038-the-mesh-assigns-the-port.md): a port becomes a host requirement. What 0038 decided + is unchanged; it is the first case of this rule. - [ADR 0051](0051-shared-data-is-the-operators.md): an access keeps its shape and its semantics; its path moves from the definition to the assignment. -- [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved variable, +- [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved requirement, checked as resolved rather than as a path the definition declares. -- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings are - addressed to an instance, not to a module on a node. - [ADR 0084](0084-which-provider-serves-a-consumer.md): a provider is a (node, instance) pair, not a (node, module) pair, so a consumer can name one of two instances on one node. -- The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to - include a directory the node's host provides, and *instance* is added. Neither lands while this +- The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to a + requirement answered by any of the four providers, and *instance* is added. Neither lands while this record is only proposed, because the glossary is the authority on the words in use, not on words under review. ## Consequences -- **Every definition changes.** 70 of 71 name host paths today. The change is mechanical for most. - The design has to say how existing modules migrate without their data moving: an adopted or - already-running assignment is placed where its data already is. -- The controller resolves variables at assignment and refuses unresolved ones. The host provides - directories. The contributions file's format changes, and so does the SDK's reconcile loop that - reads it. +- **Every definition changes.** 70 of 71 name host paths today, and most use at least three of the + mechanisms this replaces. The change is mechanical for most. The design has to say how existing + modules move without their data moving: an adopted or already-running assignment is placed where + its data already is. +- The controller resolves every requirement at assignment and refuses unresolved ones. The host + answers directories and ports. The settings, placeholders, facts and bindings that exist today + are retired as separate mechanisms, once nothing uses them. - Identity moves from the module to the instance, which touches logins, broker accounts, settings, - provider selection and every resource name. -- **What got harder:** a definition no longer says where a module's data is on a machine. The - assignment does, and `plan` shows it. That is the point, and it is also a real loss of - at-a-glance legibility, which the overview has to give back. -- **Not decided here:** the variable syntax; a node's default layout; whether a second instance of a - module is supported from the first step or after the definitions have moved; whether the vault - generates secrets. + provider selection and every resource name. A login already has a 20-character limit + ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), which a node and an instance + name will strain. The design must answer that before a second instance is possible. +- **What got harder:** a definition no longer says where a module's data is on a machine, or what a + setting's value is. The assignment does, and `plan` shows it. That is the point, and it is also a + real loss of at-a-glance legibility, which the overview has to give back. +- **Not decided here:** the syntax a definition reads a requirement's fields with; a node's default + layout; the order in which the mechanisms are retired. [To-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md) + proposes all three. ## How it is checked | Rule | Checked by | |---|---| -| A definition names no host path | A catalogue test: the host side of every mount, and every resource location, binding, secret, receives and grants entry, is a variable rather than an absolute path. A declared list of exceptions shrinks to empty as definitions move. | +| A definition names no host path | A catalogue test: the host side of every mount, and every resource location, is a requirement rather than an absolute path. A declared list of exceptions shrinks to empty as definitions move. | | No host path is a value a process reads | A catalogue test: every absolute path in a container's environment or env-files lies on the container side of one of its mounts, or is declared the image's own. A second test finds literal paths in module code used as fallbacks for an environment variable. | | A definition names no node and no mesh | The parser has no field that names a node; a node is named only in an assignment. A catalogue test finds no domain name in any definition value. | -| Installation resolves every variable | A resolution test with one variable unanswered: refused, naming the variable and its possible sources. | -| A directory is provided on its module's own node | A resolution test: an assignment placing a directory on another node is refused. | -| A provider reads what it was given without an identical mount | A provisioner test reading a contributions file whose credentials are named relative to where it is mounted. | +| Every requirement has one of the four providers | The parser refuses a requirement whose provider kind is not one of the four. | +| Installation resolves every requirement | A resolution test with one requirement unanswered: refused, naming it and what could answer it. | +| A host requirement is answered on its module's own node | A resolution test: an assignment placing a directory or a port on another node is refused. | | A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, separate settings, no collision. | | A public name already taken is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | | An adopted assignment is placed where its data is | An adoption test: the directory resolves to the data's existing location, and nothing is moved. | @@ -152,10 +169,10 @@ On acceptance, each of these is amended by a record of its own, not edited: ## References - [Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md): the evidence -- [ADR 0038](0038-the-mesh-assigns-the-port.md): the same decision, for ports -- [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): configuration is the assignment's -- [ADR 0084](0084-which-provider-serves-a-consumer.md): which provider answers is the assignment's -- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): - secrets as a provision, and who mints what +- [ADR 0038](0038-the-mesh-assigns-the-port.md), [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), + [ADR 0084](0084-which-provider-serves-a-consumer.md): the parts already unified +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): which module provider answers +- [ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md): who makes an answer, and how it travels +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md): the operator as a provider - [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries, and what it must not diff --git a/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md b/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md new file mode 100644 index 0000000..b31d10a --- /dev/null +++ b/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md @@ -0,0 +1,125 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +--- + +# 113. A provider makes what it provides, and the mesh carries it back to the consumer + +## Context + +[ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) decided that a provider is +handed the credential and makes none: the controller mints one per consumer and provider pair, seals +it to both nodes, and the provider creates the login under it. It fixed a real fault. The provider +harness of the time generated its own password and sealed it with a symmetric key nobody held, so a +consumer could never receive what the provider made. Controller-minting worked because it needed no +way back from provider to consumer. + +**It left that way back undecided, deliberately.** 0048 says so: *"delivering provider-generated data +back to a consumer is a return path the mesh does not have and this decision does not build — a +separate shape, left to a separate decision."* Since then, the missing return path has come up +repeatedly: + +- **Data provisions have nothing to answer with.** The analytics provider assigns a site id the + consumer needs. The DNS provider registers a name the consumer should be told. Both have no path + back, and say so in their code. +- **Some contracts need a value the controller cannot make.** The broker needs its admin password in + a hashed form the mesh's plain secret delivery cannot produce, so a module-specific bootstrap step + was written to derive it. A provider making the value to its own contract would not need one. +- **The vault is a ledger.** A module's own secret is "a `secret` provision the controller mints and + the vault records" ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). The one module whose + job is secrets generates none, and cannot apply a policy (length, form, lifetime) because it never + makes one. Every other provider creates what it provides. The vault is the exception. + +[ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) proposes that everything a module +needs is a requirement answered by a provider against a contract. Under that, "the controller mints +this one kind of answer on the provider's behalf" is a special case the model would carry forever. + +## Considered Options + +**1. Keep 0048: the controller mints credentials, and data provisions stay without a way back.** +Rejected. The vault stays a ledger, contracts needing a derived form keep needing bespoke steps, and +a data provision stays unable to answer at all. + +**2. The provider makes the value and hands it to the consumer itself.** Rejected. This is the fault +0048 fixed. The provider cannot seal to the consumer's node, the two may be on different machines, +and a key both ends hold is the distribution problem one level down. + +**3. The provider makes the value and gives it to the mesh, and the mesh carries it to the consumer.** +Chosen. The provider answers over its own scoped account. The controller, which already seals to +every node, seals each secret field to the consumer's node and delivers it the way it delivers +everything else. + +## Decision + +**A provider makes what it provides.** Given a consumer, it creates the resource and answers with +the fields its contract names: a password, an access key, a site id, a registered name, a hashed +admin secret. The vault generates the secrets it provides, to their contract, and rotates them. + +**The mesh carries the answer back.** The provider hands its answer to the controller over its own +scoped broker account. The controller seals every field the contract marks secret to the consumer's +node, and delivers the answer as the consumer's resolved values. A provider never reaches a consumer +directly. Plaintext exists on the provider's machine, as it does today, and on the consumer's, and +nowhere between. + +**Who a consumer is stays the mesh's.** The login a consumer presents is the mesh's derivation, which +both ends agree on by construction ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), +issue 023). A provider makes what a consumer is *given*, never what it is *called*. + +**Rotation is the provider's act.** Asked to rotate, a provider makes a new value and answers again, +and the mesh redelivers it. An operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) +is still never replaced by the mesh: its provider is the operator. + +**Genesis is the one exception.** The foundation's own credentials and the vault's own access exist +before any provider can answer. Genesis mints those itself, seals them to the operator key as today, +and hands them to their holders. Nothing else is minted by the controller. + +## What this changes in earlier records + +On acceptance, each of these is superseded or amended by this record, not edited: + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: a provider no + longer receives a minted credential. Its refusal of provider-held symmetric keys stands, and is why + option 2 is rejected here. +- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault generates a module's own secret + rather than recording one the controller minted. "The vault stores no plaintext, ever" stands. It + makes a value, hands it to the mesh and keeps only what it keeps today. + +## Consequences + +- **A consumer waits for its provider.** Its requirement is not resolved until the provider has + answered, so resolution gains a state, *waiting on a provider*, that is shown rather than silent. + Today a consumer can receive a credential before the resource behind it exists. After this it + cannot. +- The provider harness in the SDK changes: an adapter's create answers with its contract's fields + instead of returning nothing, and every provider in the catalogue moves to it. This is one + migration per provider, the cost 0048 named for changing the contract, and it is paid once. +- The controller gains the return path: receiving an answer on a provider's account, sealing its + secret fields, and delivering them. Data provisions gain the same path, so the analytics and DNS + providers can finally answer. +- Bespoke derivation steps, like the broker's admin-hash bootstrap, become the provider's own + answer and can be removed. +- **What got harder:** a provider that is down cannot hand out credentials, where today the controller + could mint one in its absence. That is honest, because a credential for a resource that does not + exist yet was never usable, but it moves a failure from later and silent to earlier and visible. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A provider's answer reaches the consumer sealed to its node | A controller test delivering a provider's answer: each secret field is sealed to the consumer's node key and to nothing else. | +| A consumer's identity is still the mesh's | A resolution test: the login a consumer presents is the mesh's derivation, whatever the provider answers. | +| A consumer waits for its provider | A resolution test with no answer yet: the requirement shows as waiting on a provider, and nothing is delivered. | +| The controller mints nothing outside genesis | A controller test: outside genesis, no code path mints a credential. The minting function is reachable only from genesis. | +| The vault generates and rotates | A vault test: a requested secret is generated to its contract, and a rotation answers with a new value. | + +## References + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, + and the return path it left open +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): identity stays the mesh's +- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0092](0092-an-operator-delivers-a-pair-credential.md): + the vault, and the operator as a provider +- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): everything a module needs is a requirement diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index a17b486..4083388 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -157,7 +157,8 @@ python3 00-META/checks/index.py fail if stale - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md) - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) - **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) -- **0112** — [A module definition names no node, no mesh and no path: everything it needs is resolved at assignment](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* +- **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* +- **0113** — [A provider makes what it provides, and the mesh carries it back to the consumer](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/23-choosing-a-provider.md b/03-DESIGN/01-to-be/23-choosing-a-provider.md index e3f36c0..a16b953 100644 --- a/03-DESIGN/01-to-be/23-choosing-a-provider.md +++ b/03-DESIGN/01-to-be/23-choosing-a-provider.md @@ -51,12 +51,15 @@ provider on a different node. That coupling is exactly what may not be guessed, names the provider. Naming it is also what makes a later move safe — the mesh knows the binding is to that provider and not to whichever one is nearest. -**A seat names the mesh's one provider of a kind.** Where a seat delivers the provision, its holder -answers for it when several providers exist and the consumer named none. That is not picking: the -choice was made once, mesh-wide, by assigning the holder, rather than once per consumer by naming it +**Some provisions have one provider for the whole mesh, and a seat names it.** Where a seat delivers +the provision, its holder answers for it, **and co-location does not apply**: a second provider on the +consumer's own machine does not take over for that consumer. That is not picking: the choice was made +once, mesh-wide, by assigning the holder, rather than once per consumer by naming it ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer -coupled to particular contents has said so. +coupled to particular contents has said so. Only provisions the design makes one-per-mesh are +delivered by a seat: the artifact store, a package registry, git and the vault. A database is not. +Node-local stores, served by co-location, are the rule above. **Ambiguity is refused, never resolved by picking.** If several providers of a kind exist, none is named, none is co-located, and no seat delivers it, the requirement is unsatisfiable and is refused diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 00c7a29..215ae4b 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -47,8 +47,9 @@ argued for is an entry nobody can explain. | seat | scope | delivers | typically held by | |---|---|---|---| | `mesh-controller` | mesh | — | the controller | -| `mesh-store` | mesh | `postgres-database` | the store | -| `mesh-broker` | mesh | `amqp` | the broker | +| `mesh-store` | mesh | — | the foundation's store | +| `mesh-broker` | mesh | — | the foundation's broker | +| `mesh-vault` | mesh | `secret` | the vault | | `the-artifact-store` | mesh | `artifact-store` | the artifact registry | | `the-catalogue` | mesh | — | the catalogue | | `npm-package-registry` | mesh | `npm-package-registry` | the forge | @@ -61,7 +62,7 @@ argued for is an entry nobody can explain. | `the-resolver-configuration` | node | — | whichever of the alternative resolver configurations is chosen | | `the-showcase` | node | — | the showcase module | -The control plane holds this set in code, and a test asserts both its size and that every entry names +The controller holds this set in code, and a test asserts both its size and that every entry names the record that made it a seat. This document follows the code, not the reverse. If the two disagree, the test has been changed without this table, and the table is what is wrong. @@ -70,16 +71,23 @@ the test has been changed without this table, and the table is what is wrong. A seat that delivers a provision may only be held by a module that provides it, at the seat's scope. A mesh seat delivers a mesh-scoped provision. -**Its holder answers for that provision.** When a requirement for it has more than one provider in the -mesh, the control plane takes, in order: +**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, +the npm registry, git and the vault are each one per mesh by decision. The store and the broker are +not: nodes run their own stores and a consumer uses the one on its machine +([23 — Choosing a provider](23-choosing-a-provider.md)). So their seats guard that the foundation's +own server is singular, and route nobody. + +**Its holder answers for that provision.** A requirement for it resolves, in order, to: 1. the provider the consumer's node was pinned to, because a consumer coupled to one provider's contents has said so ([23 — Choosing a provider](23-choosing-a-provider.md)); -2. the holder of the seat; -3. the only provider, when there is one; -4. otherwise nothing, and the requirement is refused with the candidates named. +2. the holder of the seat, **even when another provider runs on the consumer's own machine**; +3. otherwise nothing, and the requirement is refused, naming the unheld seat. -So a second provider can run beside the holder and harm nothing. The forge holds +Co-location, which answers first for every other provision, does not apply here: a seat says which +one is the mesh's, and co-location answering first would let any second provider on a consumer's +machine take over for that consumer, silently. So a second provider can run beside the holder and +harm nothing. The forge holds `npm-package-registry`. An npm proxy may provide the same provision on another machine, and a module requiring an npm registry is still served by the forge, without anybody pinning it. Moving the role to the proxy is moving the seat: unassign the claim from one, assign it to the other, and every @@ -87,7 +95,7 @@ consumer follows. **What a consumer receives is a grant**, the same as for any provision: where the provider answers, what it serves, and a credential where one is minted. A consumer never reads the seat directly. The -one exception is the control plane itself, which reaches the store and the broker through a narrow +one exception is the controller itself, which reaches the store and the broker through a narrow seat placeholder because it made them before any module existed and cannot be their consumer. ## A seat that delivers nothing @@ -98,7 +106,7 @@ their job, and it is a real one: it is the mesh saying what a machine is, in wor ## The overview -The control plane lists every seat in the set with its scope, what it delivers, and each holder as a +The controller lists every seat in the set with its scope, what it delivers, and each holder as a node and a module. A seat nobody holds is listed as unheld. That is an answer, "this mesh has no forge", and not a fault. @@ -115,7 +123,7 @@ mesh records which: | on the `git` seat | a repository on the forge that holds the seat | its path on the forge, and the seat | | external | a repository anywhere else, a public forge for instance | its URL, exactly as given | -For a repository on the seat, the control plane composes the clone URL at the moment of building, +For a repository on the seat, the controller composes the clone URL at the moment of building, from where the holder runs and the scheme and port it serves for `git`. The recorded source never contains an address, so moving the forge changes nothing that was recorded. The build machine is not told the difference: it receives a URL either way. diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md new file mode 100644 index 0000000..606c5ef --- /dev/null +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -0,0 +1,225 @@ +--- +layer: to-be +status: proposed +code: [] +updated: 2026-09-25 +decisions: + - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md + - 02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md + - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md + - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md + - 02-DECISIONS/0084-which-provider-serves-a-consumer.md + - 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md + - 02-DECISIONS/0038-the-mesh-assigns-the-port.md +--- + +# 27 — A module requires, the mesh resolves + +**One concept for everything a module needs.** A module definition states what it requires. Each +requirement has a contract and a kind of provider. Installing the module on a node resolves every +requirement, or refuses and says why. Nothing else reaches a module: no path it chose, no setting +beside the model, no literal it carries. + +This replaces six mechanisms that grew separately: provisions read through bindings, settings, +assigned ports, machine facts, minted or accepted secrets, and literals in the definition. Each +resolved, validated and failed in its own way ([ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), +[issue 118](../../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md)). + +## A requirement + +A requirement has three parts: + +| part | is | +|---|---| +| name | what the module calls it, unique within the module | +| contract | the fields the module may read, and what each promises: a type, whether it is secret, and anything the provider must honour | +| provider kind | which of the four kinds of provider answers it | + +**A contract is shared, not per module.** A database's contract is the database's, whoever requires +it. The mesh knows each contract, and a provider is checked against the one it claims to answer. A +module's own specification may narrow a contract (a password of at least this length, a directory +owned by this user) and never widen it. + +## The four kinds of provider + +The set is closed, like the seats. A fifth kind is a decision, because each kind is a place an answer +can come from and a reviewer has to know every one. + +| provider | answers | resolved by | replaces | +|---|---|---|---| +| **a module** | a database, a bucket, a vhost, a route, a secret | the rule below | provisions and bindings | +| **the node's host** | a directory, a port, a fact about the machine | always the module's own node | resource paths, assigned ports, machine placeholders, facts | +| **the mesh** | the module's identity and names | the controller | derived logins, generated names | +| **the operator** | a value a person chooses | the assignment, else the requirement's default | settings, carried literals | + +### A module provider + +Which module answers, in order: + +1. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's + contents ([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md)); +2. **the holder of a seat** that delivers the provision, where one does. Co-location does not apply + to these: the seat is the mesh's one answer for everyone ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), + [26 — The seats](26-the-seats.md)); +3. **the provider on the consumer's own node**, for a provision no seat delivers; +4. **the only provider** in the mesh; +5. otherwise **refused**, naming the candidates, or the unheld seat. + +The provider makes what it provides and answers with its contract's fields. The mesh carries the +answer back to the consumer, sealing every secret field to the consumer's node +([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). The vault +is a module provider like any other: it holds the `mesh-vault` seat and generates the secrets it +provides. + +### The node's host + +The host answers what only a machine can: where a directory is, which port is free, what the machine +is. It is always the module's own node, because none of these means anything elsewhere. + +**A directory.** The contract is an owner and a mode, and the owner the image expects where it has +one. There is no persistence flag. A directory is kept while it holds anything, and data that may be +lost is a named volume, not a directory ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md), +[ADR 0107](../../02-DECISIONS/0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). + +*Where* a directory is on the machine is the assignment's: + +- **a node's default layout**, a root per node with one directory per instance beneath it, used when + the assignment says nothing; +- **a placement**, where the assignment puts one directory elsewhere: on a second disk, or where an + adopted machine's data already is ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). + +**An operator's shared data** is an access, as before ([ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md)): +never created, owned or removed by the mesh. The module requires read or read-write access. Where +the data is, is an operator value on the assignment. + +**A port** is what [ADR 0038](../../02-DECISIONS/0038-the-mesh-assigns-the-port.md) already decided: the +module says which port its software uses, and the host answers with where the machine put it. + +**A fact** is something the machine knows: its name on the private network, the names of the mesh's +machines. Each fact has a contract like anything else. + +### The mesh + +The mesh answers who the module is: its login, its broker account, the names it is known by. These +are derived by the mesh so every party agrees by construction, and no provider may make them +([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). + +### The operator + +A value a person chooses: a public name for an endpoint, a greeting, how many workers to run. + +**It must stay cheap.** An operator requirement's contract is a type and, optionally, a default. It +needs no provider module, no grant and no credential. If asking a person for a value took more than +that, module authors would route around it, and the literals this replaces would come back. + +**A secret operator value**, such as an external API key, is held by the vault as an +operator-delivered value ([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)), +never stored as a setting. + +**An endpoint** is an operator value inside a route requirement: the public name is chosen on the +assignment, and the route provider answers. A public name already held by another assignment is +refused, like any other singular thing. + +## How a definition reads what was resolved + +**One form, naming a requirement and a field of its contract.** A definition that needs the database's +host in an environment variable, the directory's location on the host side of a mount, or the public +name in a configuration file writes the same thing: the requirement's name and the field. The +controller fills it at resolution. + +This one form replaces the placeholders that exist today, one per mechanism: bound values, secrets, +ports, machine facts and seats. The seat placeholder the controller uses to reach its own foundation +stays, because the controller cannot be a consumer of a foundation it made before any module +existed. It is the controller's own and no module uses it. + +**A secret field reaches a process as a file**, as today ([ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md)). +Reading one into a plain value, such as an environment variable, is refused when the definition is +parsed, unless the definition declares the exception 0086 allows, with its reason. + +## An instance + +An assignment has an identity: an **instance name**, which defaults to the module's name. Everything +keyed by the module's name today is keyed by the instance: directories, containers, the login it +presents, its broker account, the seats it holds, its settings and its identity as a provider. + +So a module may run twice on one node, under two instance names. What must stay singular stays so: +by a seat, or by an operator value colliding, as with a public name. + +**A login still has to fit the tightest backend**, which is twenty characters today +([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). A node name and +an instance name will not fit in full. The instance therefore gets a short form alongside the module's +slug, under the same rules as a slug. This has to be settled before a second instance is allowed. + +## Genesis + +The one exception. Before any provider exists, genesis answers the foundation's own requirements +itself: the store's and broker's credentials, the vault's own access, and the root secrets. It seals +them to the operator key as it does now ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). +After genesis, nothing is answered except by a provider. + +## Refusing + +Installation refuses when any requirement is unresolved, and **says everything at once**. For each +requirement it names what is missing and what would answer it: + +- an unheld seat, and which modules could hold it; +- no provider, and which modules could provide it; +- an operator value with no default, and that the assignment must give it; +- a provider that has not answered yet, and which one. + +The last one is a state, not a failure. A consumer waiting for its provider is shown as waiting, and +nothing is delivered until the answer arrives ([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). + +## What this retires + +| mechanism | becomes | +|---|---| +| provisions read through bindings | a module requirement; its answer is the contract's fields | +| settings on an assignment ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) | operator requirements, addressed to an instance | +| a port the mesh assigns | a host requirement | +| machine facts and machine placeholders | host requirements | +| a secret the controller mints | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)) | +| paths in resources, mounts, bindings, secrets and received files | host directory requirements, placed by the assignment | +| literals carried in a definition | operator requirements with defaults | + +Each is retired only once nothing uses it. Until then both are accepted, and a catalogue test lists +the definitions still using the old form. That list shrinks to empty, and then the old form is +removed from the parser. + +## Phases + +Each phase ends at a check that holds, so none of them leaves a mechanism half-replaced. + +1. **Resolution and the new form.** The controller resolves requirements from the four providers, + refuses as above, and fills the one form. Old mechanisms keep working beside it. *Ends when* a + definition written entirely in the new form installs on a lab machine. +2. **Providers answer.** The SDK harness answers with contract fields, the controller carries answers + back and seals them, and the vault holds its seat and generates. *Ends when* the analytics and DNS + providers answer their consumers and the broker's bootstrap step is removed. +3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments + placed where their data already is. *Ends when* the list of definitions using an old form is + empty, and the old forms are removed. +4. **Instances.** The instance name and its short form, and everything keyed by it. *Ends when* one + module runs twice on one lab machine with two public names. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Every requirement has one of the four provider kinds | The parser refuses any other. | +| A definition names no host path, node or mesh | The catalogue tests of [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md). | +| A module provider is chosen by pin, seat, co-location, only one, refusal | Resolution tests for each step, and for a second provider on a consumer's own machine when a seat delivers the provision. | +| A host requirement is answered on its own node | A resolution test placing one elsewhere: refused. | +| An operator value needs no provider | A resolution test: a requirement with a default resolves with no module assigned anywhere. | +| A secret field reaches a process as a file | The parser refuses a secret field read into a plain value, unless the definition declares the 0086 exception with a reason. | +| Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | +| The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | + +## Not settled here + +- The exact spelling of the one form. It must name a requirement and a field and nothing else. +- The layout a node's default root uses beneath it, beyond one directory per instance. +- Whether a module provider's answer can change without the provider being asked, for example a + provider moving. The rule so far is that it cannot, and moving is re-resolving. +- A contract registry: where contracts live, and how a new provision gets one. Today contracts are + implicit in each provider's served fields. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index bc88ea9..fdbfce4 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -35,6 +35,7 @@ document is written and this one's status becomes `implemented`. | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | | [`26-the-seats.md`](26-the-seats.md) | What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | +| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | ## Not yet written From 6e3373c879cd9391d5992e9a8f8774a35a49f04b Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 22:46:10 +0200 Subject: [PATCH 16/26] Design pass: address the review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 0113 — the plaintext claim was false under its own mechanism: handing a provider's answer to the controller puts every secret on the broker and in the controller in the clear. The provider now seals each secret field itself, to the consumer node's public key the mesh hands it, and the controller carries sealed fields it cannot open. That is stricter than today, where the controller holds every minted credential in the clear. Option 3 (plaintext to the controller) is recorded and rejected. The foundation exception now covers root-secret rotation (0085) and forms like the broker admin's hash, so no phase claims to remove the broker's bootstrap step. To-be 24 and 13 are named among what it amends. 27 — resolution is consistent with 0110: co-location and the only provider apply only where no seat delivers the provision, so an unheld seat is refused even with one provider. The secret-field rule now matches 0086 exactly (a declared env-file, never a container environment value). The seat placeholder is the controller's, and the one module reading it moves to a host port. Contracts are held by the controller and written down in phase 1, so they can be checked; every rule has a check. An operator's secret is still the operator's, with the vault as custodian. Which seats a module holds is listed as not settled. 0110 — the unheld-seat-with-one-provider case and the one-answer-for-everyone rule have checks; the claim about moved manifests is corrected. 26 — the table governs and the code catches up, not the reverse; scope and capacity agree with the glossary; moving a seat is described as it really is today. 0112 — aligned with 27, and lists 0049 and 26 among what it changes. Issue 118 is renumbered 119: another branch took 118 first. 'Control-plane' is gone from 0110 and 0111. --- ...s-a-module-assignment-from-a-closed-set.md | 10 +-- ...d-source-is-on-the-git-seat-or-external.md | 2 +- ...e-definition-names-no-node-mesh-or-path.md | 21 ++++-- ...hat-it-provides-and-the-mesh-carries-it.md | 66 +++++++++++------ 03-DESIGN/01-to-be/26-the-seats.md | 28 ++++--- .../27-a-module-requires-the-mesh-resolves.md | 74 ++++++++++++------- .../00-report.md | 2 +- 7 files changed, 128 insertions(+), 75 deletions(-) rename 04-ISSUES/{118-a-module-definition-decides-where-its-files-live => 119-a-module-definition-decides-where-its-files-live}/00-report.md (98%) diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index a0d18cc..6800228 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -24,8 +24,7 @@ The names in use were each invented by the module that claims them: `the-showcas **Nothing can say what a mesh has, or who fills it.** There is no seat table and no command that lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. The only way to answer "which seats does this mesh have, and which module holds each" is to read -every manifest in two repositories, because the core modules' manifests moved into the controller's -own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's +every manifest in two repositories, because the controller's own manifest lives in its own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's code, because one module it ships has its manifest composed there. While this record was being prepared, that enumeration was done by hand, and it missed both of the last two sources: eleven claims were reported where there are thirteen. @@ -163,10 +162,11 @@ question real. | Rule | Checked by | |---|---| -| The set is closed, and every entry names its decision | A control-plane unit test asserts the set's size and a non-empty decision for every entry. | +| The set is closed, and every entry names its decision | A controller unit test asserts the set's size and a non-empty decision for every entry. | | A claim outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat whose claimant does not provide. | -| Every module in use claims a seat in the set | A control-plane test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | -| The holder answers among several providers | Resolution tests: two providers with the seat held, two with a pin overriding the seat, two with the seat unheld (refused). | +| Every module in use claims a seat in the set | A controller test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | +| The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own machine, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | +| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` and `mesh-broker` deliver nothing, so a database consumer is still served by co-location. | ## References diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md index 0f087af..b021ea5 100644 --- a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -84,7 +84,7 @@ the controller's job, because only the controller knows where the seat's holder | Rule | Checked by | |---|---| -| A seat source records no address | A control-plane test resolves a seat source and asserts the recorded repository is the path alone. | +| A seat source records no address | A controller test resolves a seat source and asserts the recorded repository is the path alone. | | The URL comes from the holder | A test composes the clone URL from a holder's node address and served `git` scheme and port, and a second where the port was moved on the node. | | An unheld seat refuses self-hosted builds only | A test with no holder: `--self` is refused naming the seat; an external URL passes through unchanged. | diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index afd4469..f2743c3 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -11,7 +11,7 @@ extends: 0046-a-module-configuration-is-its-assignments-not-its-manifest.md ## Context -[Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md) found +[Issue 119](../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md) found **789 host-path strings in 70 of the catalogue's 71 module definitions.** Each definition chooses where on the machine its directories, mounts, bindings, secrets, env-files and received files live, and often repeats that path in an environment variable or in code. Mounts are checked against what @@ -67,19 +67,20 @@ unresolved requirement and what could answer it, all at once. |---|---|---| | **another module** | a database, a bucket, a vhost, a secret, a route | provisions and bindings | | **the node's host** | a directory, a port, facts about the machine | resource paths, `${port:}`, `${machine:}`, facts | -| **the mesh** | the module's identity and names, and the delivery of every answer | derived logins, minted delivery | +| **the mesh** | the module's identity and names, and the delivery of every answer | derived logins and generated names; the controller's delivery | | **the operator, through the assignment** | a value a person chooses: a public name, a greeting, an external key | settings, carried literals | A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say: a pin, then the holder of a -seat that delivers the provision, then co-location, then the only one. A host provider is always the -module's own node, because a host path or a port means nothing on any other. An operator value is the -assignment's, or the requirement's default, or unresolved. +seat that delivers the provision; for a provision no seat delivers, co-location and then the only one. +A host provider is always the module's own node, because a host path or a port means nothing on any +other. An operator value is the assignment's, or the requirement's default, or unresolved. **A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a default. It needs no provider module, no grant and no credential. An operator value that is secret, -like an external API key, is kept by the vault as an operator-delivered value -([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). +like an external API key, is still the operator's: the vault is where it is *kept*, as an +operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), not who +provides it. **What a provider answers with is its contract's fields**, made by the provider and carried back to the consumer by the mesh ([ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). @@ -127,6 +128,10 @@ On acceptance, each of these is amended by a record of its own, not edited: checked as resolved rather than as a path the definition declares. - [ADR 0084](0084-which-provider-serves-a-consumer.md): a provider is a (node, instance) pair, not a (node, module) pair, so a consumer can name one of two instances on one node. +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a login is built from the + instance, which gets a short form under the same rules as a slug, so it still fits the tightest + backend. +- [To-be 26](../03-DESIGN/01-to-be/26-the-seats.md): a seat's holder is an instance. - The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to a requirement answered by any of the four providers, and *instance* is added. Neither lands while this record is only proposed, because the glossary is the authority on the words in use, not on words @@ -168,7 +173,7 @@ On acceptance, each of these is amended by a record of its own, not edited: ## References -- [Issue 118](../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md): the evidence +- [Issue 119](../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md): the evidence - [ADR 0038](0038-the-mesh-assigns-the-port.md), [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), [ADR 0084](0084-which-provider-serves-a-consumer.md): the parts already unified - [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): which module provider answers diff --git a/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md b/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md index b31d10a..fe81eb3 100644 --- a/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md +++ b/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md @@ -27,7 +27,9 @@ repeatedly: back, and say so in their code. - **Some contracts need a value the controller cannot make.** The broker needs its admin password in a hashed form the mesh's plain secret delivery cannot produce, so a module-specific bootstrap step - was written to derive it. A provider making the value to its own contract would not need one. + was written to derive it. That admin is a foundation credential, which genesis makes, so this + record does not remove that step. It is the clearest instance of the general problem, though: a + contract can require a form only the party that understands the software can produce. - **The vault is a ledger.** A module's own secret is "a `secret` provision the controller mints and the vault records" ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). The one module whose job is secrets generates none, and cannot apply a policy (length, form, lifetime) because it never @@ -43,14 +45,21 @@ this one kind of answer on the provider's behalf" is a special case the model wo Rejected. The vault stays a ledger, contracts needing a derived form keep needing bespoke steps, and a data provision stays unable to answer at all. -**2. The provider makes the value and hands it to the consumer itself.** Rejected. This is the fault -0048 fixed. The provider cannot seal to the consumer's node, the two may be on different machines, -and a key both ends hold is the distribution problem one level down. +**2. The provider makes the value and hands it to the consumer itself.** Rejected. The two may be on +different machines with no path between them the mesh has agreed to, and a provider reaching +consumers directly is a second delivery system beside the mesh's. 0048's objection, a symmetric key +both ends hold, is not what rules this out: node keys are asymmetric, and a provider can seal to a +node's public key without sharing anything. -**3. The provider makes the value and gives it to the mesh, and the mesh carries it to the consumer.** -Chosen. The provider answers over its own scoped account. The controller, which already seals to -every node, seals each secret field to the consumer's node and delivers it the way it delivers -everything else. +**3. The provider makes the value and gives it to the controller in plaintext, which seals and +delivers it.** Rejected. It works, and it puts every secret in the controller's memory and on the +broker in the clear, a surface today's design does not have for provider-side values. + +**4. The provider makes the value, seals each secret field to the consumer's node itself, and the +mesh carries the sealed answer.** Chosen. The mesh already tells a provider who each consumer is and +where; it also hands it the consumer node's public key. The provider seals to it, and answers over +its own scoped account. The controller delivers the sealed fields as it delivers everything else, +and can open none of them. ## Decision @@ -58,11 +67,13 @@ everything else. the fields its contract names: a password, an access key, a site id, a registered name, a hashed admin secret. The vault generates the secrets it provides, to their contract, and rotates them. -**The mesh carries the answer back.** The provider hands its answer to the controller over its own -scoped broker account. The controller seals every field the contract marks secret to the consumer's -node, and delivers the answer as the consumer's resolved values. A provider never reaches a consumer -directly. Plaintext exists on the provider's machine, as it does today, and on the consumer's, and -nowhere between. +**The provider seals, and the mesh carries.** With each consumer, the mesh hands the provider that +consumer node's public key. The provider seals every field its contract marks secret to it, and +hands the answer to the controller over its own scoped broker account. The controller delivers the +answer as the consumer's resolved values, and can open none of the sealed fields. A provider never +reaches a consumer directly. A secret's plaintext exists where it is made, on the provider's machine, +and where it is used, on the consumer's, and nowhere between. That is stricter than today, where the +controller holds every minted credential in the clear when it makes it. **Who a consumer is stays the mesh's.** The login a consumer presents is the mesh's derivation, which both ends agree on by construction ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), @@ -72,9 +83,11 @@ issue 023). A provider makes what a consumer is *given*, never what it is *calle and the mesh redelivers it. An operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) is still never replaced by the mesh: its provider is the operator. -**Genesis is the one exception.** The foundation's own credentials and the vault's own access exist -before any provider can answer. Genesis mints those itself, seals them to the operator key as today, -and hands them to their holders. Nothing else is minted by the controller. +**The foundation is the one exception.** The foundation's own credentials, the vault's own access and +the mesh's root secrets exist before any provider can answer. The controller mints those: at genesis, +and when an operator rotates a root secret ([ADR 0085](0085-a-secret-is-a-provision.md)). It seals +them to the operator key as today, including any form the foundation's software needs, such as the +broker admin's hash. Nothing else is minted by the controller. ## What this changes in earlier records @@ -85,7 +98,11 @@ On acceptance, each of these is superseded or amended by this record, not edited option 2 is rejected here. - [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault generates a module's own secret rather than recording one the controller minted. "The vault stores no plaintext, ever" stands. It - makes a value, hands it to the mesh and keeps only what it keeps today. + makes a value, seals it, hands it to the mesh and keeps only what it keeps today. +- [To-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) and + [to-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) are amended: both describe + the controller minting a module's credentials and rotating them. After this, the controller mints + only the foundation's, and a provider rotates by answering again. ## Consequences @@ -96,11 +113,11 @@ On acceptance, each of these is superseded or amended by this record, not edited - The provider harness in the SDK changes: an adapter's create answers with its contract's fields instead of returning nothing, and every provider in the catalogue moves to it. This is one migration per provider, the cost 0048 named for changing the contract, and it is paid once. -- The controller gains the return path: receiving an answer on a provider's account, sealing its - secret fields, and delivering them. Data provisions gain the same path, so the analytics and DNS - providers can finally answer. -- Bespoke derivation steps, like the broker's admin-hash bootstrap, become the provider's own - answer and can be removed. +- The controller gains the return path: receiving a sealed answer on a provider's account and + delivering it. Each contribution gains the consumer node's public key. Data provisions gain the same + path, so the analytics and DNS providers can finally answer. +- A contract needing a derived form is met by its provider. The broker admin's hash stays with the + controller, because that admin is a foundation credential. - **What got harder:** a provider that is down cannot hand out credentials, where today the controller could mint one in its absence. That is honest, because a credential for a resource that does not exist yet was never usable, but it moves a failure from later and silent to earlier and visible. @@ -109,10 +126,11 @@ On acceptance, each of these is superseded or amended by this record, not edited | Rule | Checked by | |---|---| -| A provider's answer reaches the consumer sealed to its node | A controller test delivering a provider's answer: each secret field is sealed to the consumer's node key and to nothing else. | +| A provider's secret fields are sealed before they leave it | An SDK test: an answer whose contract marks a field secret cannot be handed over unsealed. A controller test: an unsealed secret field in an answer is refused, not delivered. | +| A provider's answer reaches the consumer sealed to its node | A controller test delivering a provider's answer: each secret field opens with the consumer node's key and with no other, the controller's included. | | A consumer's identity is still the mesh's | A resolution test: the login a consumer presents is the mesh's derivation, whatever the provider answers. | | A consumer waits for its provider | A resolution test with no answer yet: the requirement shows as waiting on a provider, and nothing is delivered. | -| The controller mints nothing outside genesis | A controller test: outside genesis, no code path mints a credential. The minting function is reachable only from genesis. | +| The controller mints only the foundation's credentials | A controller test: the minting function is reachable only from genesis and from root-secret rotation, and never for a module's provision. | | The vault generates and rotates | A vault test: a requested secret is generated to its contract, and a rotation answers with a new value. | ## References diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 215ae4b..719442f 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -28,7 +28,7 @@ A seat has four properties, fixed by the mesh rather than by any module: | property | is | |---|---| | name | what a manifest claims, and what a person reads in the list | -| scope | node, site or mesh: where there may be only one holder | +| scope | node, site or mesh: where its capacity applies. Every seat in the set has a capacity of one, so one holder per scope. A bench, a seat with several holders, is a word the glossary keeps and no seat uses yet | | delivers | the provision its holder answers for, or nothing | | decision | the record that made it a seat | @@ -63,8 +63,10 @@ argued for is an entry nobody can explain. | `the-showcase` | node | — | the showcase module | The controller holds this set in code, and a test asserts both its size and that every entry names -the record that made it a seat. This document follows the code, not the reverse. If the two disagree, -the test has been changed without this table, and the table is what is wrong. +the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) +govern, and code that disagrees is what is wrong.** The implementation in progress predates two +things here: the `mesh-vault` seat, and the rule that `mesh-store` and `mesh-broker` deliver +nothing. It is brought to this table before it merges. ## A seat that delivers a provision @@ -89,14 +91,22 @@ one is the mesh's, and co-location answering first would let any second provider machine take over for that consumer, silently. So a second provider can run beside the holder and harm nothing. The forge holds `npm-package-registry`. An npm proxy may provide the same provision on another machine, and a module -requiring an npm registry is still served by the forge, without anybody pinning it. Moving the role -to the proxy is moving the seat: unassign the claim from one, assign it to the other, and every -consumer follows. +requiring an npm registry is still served by the forge, without anybody pinning it. + +**Moving the role is changing which module claims the seat, and today that is a definition change.** +A claim is part of a module's definition, so the proxy's definition must claim the seat and the +forge's must stop. The forge cannot simply be unassigned, because it holds `git` as well. Every +consumer follows once the claim moves. Making *which* seats a module holds the assignment's choice, +with the definition saying only which seats it *can* hold, is the consistent answer, and +[27 — A module requires, the mesh resolves](27-a-module-requires-the-mesh-resolves.md) lists it as +not yet settled. **What a consumer receives is a grant**, the same as for any provision: where the provider answers, -what it serves, and a credential where one is minted. A consumer never reads the seat directly. The -one exception is the controller itself, which reaches the store and the broker through a narrow -seat placeholder because it made them before any module existed and cannot be their consumer. +what it serves, and a credential. A consumer never reads the seat directly. The one exception is the +controller itself, which reaches the store and the broker through a narrow seat placeholder, +because it made them before any module existed and cannot be their consumer. One foundation module +also reads it today, to find its own server's port. [27](27-a-module-requires-the-mesh-resolves.md) +moves that to a host port requirement. ## A seat that delivers nothing diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index 606c5ef..42b4b4a 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -23,7 +23,7 @@ beside the model, no literal it carries. This replaces six mechanisms that grew separately: provisions read through bindings, settings, assigned ports, machine facts, minted or accepted secrets, and literals in the definition. Each resolved, validated and failed in its own way ([ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), -[issue 118](../../04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md)). +[issue 119](../../04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md)). ## A requirement @@ -36,9 +36,12 @@ A requirement has three parts: | provider kind | which of the four kinds of provider answers it | **A contract is shared, not per module.** A database's contract is the database's, whoever requires -it. The mesh knows each contract, and a provider is checked against the one it claims to answer. A -module's own specification may narrow a contract (a password of at least this length, a directory -owned by this user) and never widen it. +it. **The controller holds every contract**, one per provision name, declared where the provision is +defined in the catalogue. Today contracts are implicit in each provider's served fields; the first +phase below makes them explicit, because nothing can be checked against a contract that is not +written down. A provider is checked against the contract it claims to answer. A module's own +specification may narrow a contract (a password of at least this length, a directory owned by this +user) and never widen it. ## The four kinds of provider @@ -49,7 +52,7 @@ can come from and a reviewer has to know every one. |---|---|---|---| | **a module** | a database, a bucket, a vhost, a route, a secret | the rule below | provisions and bindings | | **the node's host** | a directory, a port, a fact about the machine | always the module's own node | resource paths, assigned ports, machine placeholders, facts | -| **the mesh** | the module's identity and names | the controller | derived logins, generated names | +| **the mesh** | the module's identity and names, and the delivery of every answer | the controller | derived logins and generated names; the controller's delivery | | **the operator** | a value a person chooses | the assignment, else the requirement's default | settings, carried literals | ### A module provider @@ -61,9 +64,10 @@ Which module answers, in order: 2. **the holder of a seat** that delivers the provision, where one does. Co-location does not apply to these: the seat is the mesh's one answer for everyone ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [26 — The seats](26-the-seats.md)); -3. **the provider on the consumer's own node**, for a provision no seat delivers; -4. **the only provider** in the mesh; -5. otherwise **refused**, naming the candidates, or the unheld seat. +3. for a provision no seat delivers, **the provider on the consumer's own node**; +4. for a provision no seat delivers, **the only provider** in the mesh; +5. otherwise **refused**: naming the unheld seat, for a provision a seat delivers, even when exactly + one provider exists; naming the candidates otherwise. The provider makes what it provides and answers with its contract's fields. The mesh carries the answer back to the consumer, sealing every secret field to the consumer's node @@ -112,9 +116,11 @@ A value a person chooses: a public name for an endpoint, a greeting, how many wo needs no provider module, no grant and no credential. If asking a person for a value took more than that, module authors would route around it, and the literals this replaces would come back. -**A secret operator value**, such as an external API key, is held by the vault as an +**A secret operator value**, such as an external API key, is still an operator requirement: its +provider is the operator. What differs is where it is kept. The vault holds it as an operator-delivered value ([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)), -never stored as a setting. +never as a setting, because anything secret belongs in one place that can seal and audit it. The +vault is its custodian, not its provider. **An endpoint** is an operator value inside a route requirement: the public name is chosen on the assignment, and the route provider answers. A public name already held by another assignment is @@ -128,13 +134,17 @@ name in a configuration file writes the same thing: the requirement's name and t controller fills it at resolution. This one form replaces the placeholders that exist today, one per mechanism: bound values, secrets, -ports, machine facts and seats. The seat placeholder the controller uses to reach its own foundation -stays, because the controller cannot be a consumer of a foundation it made before any module -existed. It is the controller's own and no module uses it. +ports and machine facts. -**A secret field reaches a process as a file**, as today ([ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md)). -Reading one into a plain value, such as an environment variable, is refused when the definition is -parsed, unless the definition declares the exception 0086 allows, with its reason. +**The seat placeholder stays, for the controller alone.** The controller composes its own +declaration and reaches the store and broker it made before any module existed, so it cannot be +their consumer. One module reads the placeholder today: the store module, to find its own server's +port. That is its own port, so it becomes a host port requirement in phase 3, and after that no +module uses the seat placeholder. + +**A secret field reaches a process as a file**, as [ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md) +decided. The one exception 0086 allows is a declared env-file with its reason; a secret field as a +value in a container's environment is refused when the definition is parsed, with no exception. ## An instance @@ -190,12 +200,14 @@ removed from the parser. Each phase ends at a check that holds, so none of them leaves a mechanism half-replaced. -1. **Resolution and the new form.** The controller resolves requirements from the four providers, - refuses as above, and fills the one form. Old mechanisms keep working beside it. *Ends when* a - definition written entirely in the new form installs on a lab machine. -2. **Providers answer.** The SDK harness answers with contract fields, the controller carries answers - back and seals them, and the vault holds its seat and generates. *Ends when* the analytics and DNS - providers answer their consumers and the broker's bootstrap step is removed. +1. **Contracts, resolution and the new form.** Every provision's contract is written down and held by + the controller. The controller resolves requirements from the four providers, refuses as above, + and fills the one form. Old mechanisms keep working beside it. *Ends when* a definition written + entirely in the new form installs on a lab machine. +2. **Providers answer.** The SDK harness answers with contract fields and seals the secret ones, the + controller carries answers back, and the vault holds its seat and generates. *Ends when* the + analytics and DNS providers answer their consumers, and a module's own secret is generated by the + vault and rotated by it. 3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is. *Ends when* the list of definitions using an old form is empty, and the old forms are removed. @@ -208,10 +220,15 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r |---|---| | Every requirement has one of the four provider kinds | The parser refuses any other. | | A definition names no host path, node or mesh | The catalogue tests of [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md). | -| A module provider is chosen by pin, seat, co-location, only one, refusal | Resolution tests for each step, and for a second provider on a consumer's own machine when a seat delivers the provision. | +| A module provider is chosen by pin, seat, co-location, only one, refusal | Resolution tests for each step; for a second provider on a consumer's own machine when a seat delivers the provision; and for an unheld seat with exactly one provider, refused. | +| A provider answers within its contract | A controller test: an answer carrying a field its contract does not name, or missing one it does, is refused and not delivered. | +| A module narrows a contract and never widens it | The parser refuses a module specification that loosens a contract's field. | | A host requirement is answered on its own node | A resolution test placing one elsewhere: refused. | -| An operator value needs no provider | A resolution test: a requirement with a default resolves with no module assigned anywhere. | -| A secret field reaches a process as a file | The parser refuses a secret field read into a plain value, unless the definition declares the 0086 exception with a reason. | +| An operator value needs no provider module | A resolution test: a requirement with a default resolves with no module assigned anywhere. | +| A secret field reaches a process as a file | The parser refuses a secret field as a container environment value, and accepts it in a file or a declared env-file with its reason. | +| A public name already held is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | +| Everything keyed by a module is keyed by its instance | A resolution test: two instances of one module on one node get two directories, two containers, two logins and separate settings. | +| A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | @@ -221,5 +238,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r - The layout a node's default root uses beneath it, beyond one directory per instance. - Whether a module provider's answer can change without the provider being asked, for example a provider moving. The rule so far is that it cannot, and moving is re-resolving. -- A contract registry: where contracts live, and how a new provision gets one. Today contracts are - implicit in each provider's served fields. +- **Which seats a module holds.** Today a claim is part of the definition, so moving a seat is a + definition change ([26 — The seats](26-the-seats.md)). By this document's own logic it belongs to + the assignment: a definition says which seats a module *can* hold, and the assignment says which it + *does*. That changes [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), + and is its own decision. diff --git a/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md similarity index 98% rename from 04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md rename to 04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md index 7a8e416..f5c5c90 100644 --- a/04-ISSUES/118-a-module-definition-decides-where-its-files-live/00-report.md +++ b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md @@ -6,7 +6,7 @@ fixed-by: amended-design: --- -# 118 — A module definition decides where its files live on the machine +# 119 — A module definition decides where its files live on the machine ## What was observed From fa2c09a2c5043ad790afb5a404b76a94fa7bcddf Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 23:10:52 +0200 Subject: [PATCH 17/26] =?UTF-8?q?ADR=200113=20and=20to-be=2027:=20the=20va?= =?UTF-8?q?ult=20makes=20every=20secret=20=E2=80=94=20provisioning=20all?= =?UTF-8?q?=20the=20way=20down?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A secret comes into being seven ways today: provider credentials, own secrets (54 modules), broker accounts through a command that is easy to forget, a vault that only records what the controller mints (6 modules), operator values, licences, and root secrets. The vault was built to end own secrets and did not; the old path was never retired. 0113 is rewritten as a waterfall. The vault makes every secret and nothing else does. A provider that needs a secret for a consumer requires it from the vault, declared once in its provision's contract and expanded per consumer by resolution; the vault delivers it to both holders, each sealed to its own node, so a provider's code is unchanged. Own secrets, broker passwords, operator values and licence credentials take the same path. Genesis is not an exception: it raises the vault first and asks it, so there is one way a secret is made from the first one on. The vault can sit at the bottom because it requires nothing but a broker account. One shared mint function in the SDK was considered and rejected: generation becomes uniform but custody stays spread over every provider's machine, and each SDK language needs its own implementation. Rotation is asked of the vault and is provider-first: the value goes to the holder that accepts it, which confirms, before the holder that presents it gets it, so the lockout window shrinks to the consumer's own restart, and an unconfirmed provider holds the rotation rather than half-doing it. The host derives which processes to restart or recreate from the requirement a definition reads, so no definition declares restart-on for a secret. A rotation shows unconfirmed until each consumer restarted and passed its health check. Issue 103 becomes a prerequisite. The file is renamed to match what it now decides. 0112 follows. --- ...e-definition-names-no-node-mesh-or-path.md | 12 +- ...hat-it-provides-and-the-mesh-carries-it.md | 143 -------------- .../0113-the-vault-makes-every-secret.md | 182 ++++++++++++++++++ 02-DECISIONS/README.md | 2 +- .../27-a-module-requires-the-mesh-resolves.md | 97 +++++++--- 03-DESIGN/01-to-be/README.md | 2 +- 6 files changed, 265 insertions(+), 173 deletions(-) delete mode 100644 02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md create mode 100644 02-DECISIONS/0113-the-vault-makes-every-secret.md diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index f2743c3..fa771d0 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -82,8 +82,9 @@ like an external API key, is still the operator's: the vault is where it is *kep operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), not who provides it. -**What a provider answers with is its contract's fields**, made by the provider and carried back to -the consumer by the mesh ([ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). +**Every secret is made by the vault, and a provider answers with resources and data** +([ADR 0113](0113-the-vault-makes-every-secret.md)). A provider that needs a secret for a consumer +requires it from the vault, like any consumer. The mesh carries every answer back. **A directory is a host provision.** Its contract is the owner and mode the module needs, including the owner its image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). @@ -111,8 +112,9 @@ carries, and a provider's identity. So **one module may be assigned to one node must stay singular stays so by a claim, or by an operator value colliding: a public name already taken is refused like any other singular thing. -**Genesis is the one exception**, as [ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md) -states: the foundation's requirements are answered by genesis itself, before any provider exists. +**Genesis is not an exception.** It raises the vault first and asks it for the foundation's secrets, +so the foundation's requirements are answered the same way as everything else +([ADR 0113](0113-the-vault-makes-every-secret.md)). ## What this changes in earlier records @@ -177,7 +179,7 @@ On acceptance, each of these is amended by a record of its own, not edited: - [ADR 0038](0038-the-mesh-assigns-the-port.md), [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md), [ADR 0084](0084-which-provider-serves-a-consumer.md): the parts already unified - [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): which module provider answers -- [ADR 0113](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md): who makes an answer, and how it travels +- [ADR 0113](0113-the-vault-makes-every-secret.md): the vault makes every secret, and how answers travel - [ADR 0092](0092-an-operator-delivers-a-pair-credential.md): the operator as a provider - [ADR 0051](0051-shared-data-is-the-operators.md), [ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md), [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): what a directory's contract carries, and what it must not diff --git a/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md b/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md deleted file mode 100644 index fe81eb3..0000000 --- a/02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md +++ /dev/null @@ -1,143 +0,0 @@ ---- -topic: what runs on it -status: proposed -date: 2026-09-25 -deciders: jochen -reconstructed: false ---- - -# 113. A provider makes what it provides, and the mesh carries it back to the consumer - -## Context - -[ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) decided that a provider is -handed the credential and makes none: the controller mints one per consumer and provider pair, seals -it to both nodes, and the provider creates the login under it. It fixed a real fault. The provider -harness of the time generated its own password and sealed it with a symmetric key nobody held, so a -consumer could never receive what the provider made. Controller-minting worked because it needed no -way back from provider to consumer. - -**It left that way back undecided, deliberately.** 0048 says so: *"delivering provider-generated data -back to a consumer is a return path the mesh does not have and this decision does not build — a -separate shape, left to a separate decision."* Since then, the missing return path has come up -repeatedly: - -- **Data provisions have nothing to answer with.** The analytics provider assigns a site id the - consumer needs. The DNS provider registers a name the consumer should be told. Both have no path - back, and say so in their code. -- **Some contracts need a value the controller cannot make.** The broker needs its admin password in - a hashed form the mesh's plain secret delivery cannot produce, so a module-specific bootstrap step - was written to derive it. That admin is a foundation credential, which genesis makes, so this - record does not remove that step. It is the clearest instance of the general problem, though: a - contract can require a form only the party that understands the software can produce. -- **The vault is a ledger.** A module's own secret is "a `secret` provision the controller mints and - the vault records" ([ADR 0085](0085-a-secret-is-a-provision.md), as amended). The one module whose - job is secrets generates none, and cannot apply a policy (length, form, lifetime) because it never - makes one. Every other provider creates what it provides. The vault is the exception. - -[ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md) proposes that everything a module -needs is a requirement answered by a provider against a contract. Under that, "the controller mints -this one kind of answer on the provider's behalf" is a special case the model would carry forever. - -## Considered Options - -**1. Keep 0048: the controller mints credentials, and data provisions stay without a way back.** -Rejected. The vault stays a ledger, contracts needing a derived form keep needing bespoke steps, and -a data provision stays unable to answer at all. - -**2. The provider makes the value and hands it to the consumer itself.** Rejected. The two may be on -different machines with no path between them the mesh has agreed to, and a provider reaching -consumers directly is a second delivery system beside the mesh's. 0048's objection, a symmetric key -both ends hold, is not what rules this out: node keys are asymmetric, and a provider can seal to a -node's public key without sharing anything. - -**3. The provider makes the value and gives it to the controller in plaintext, which seals and -delivers it.** Rejected. It works, and it puts every secret in the controller's memory and on the -broker in the clear, a surface today's design does not have for provider-side values. - -**4. The provider makes the value, seals each secret field to the consumer's node itself, and the -mesh carries the sealed answer.** Chosen. The mesh already tells a provider who each consumer is and -where; it also hands it the consumer node's public key. The provider seals to it, and answers over -its own scoped account. The controller delivers the sealed fields as it delivers everything else, -and can open none of them. - -## Decision - -**A provider makes what it provides.** Given a consumer, it creates the resource and answers with -the fields its contract names: a password, an access key, a site id, a registered name, a hashed -admin secret. The vault generates the secrets it provides, to their contract, and rotates them. - -**The provider seals, and the mesh carries.** With each consumer, the mesh hands the provider that -consumer node's public key. The provider seals every field its contract marks secret to it, and -hands the answer to the controller over its own scoped broker account. The controller delivers the -answer as the consumer's resolved values, and can open none of the sealed fields. A provider never -reaches a consumer directly. A secret's plaintext exists where it is made, on the provider's machine, -and where it is used, on the consumer's, and nowhere between. That is stricter than today, where the -controller holds every minted credential in the clear when it makes it. - -**Who a consumer is stays the mesh's.** The login a consumer presents is the mesh's derivation, which -both ends agree on by construction ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), -issue 023). A provider makes what a consumer is *given*, never what it is *called*. - -**Rotation is the provider's act.** Asked to rotate, a provider makes a new value and answers again, -and the mesh redelivers it. An operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) -is still never replaced by the mesh: its provider is the operator. - -**The foundation is the one exception.** The foundation's own credentials, the vault's own access and -the mesh's root secrets exist before any provider can answer. The controller mints those: at genesis, -and when an operator rotates a root secret ([ADR 0085](0085-a-secret-is-a-provision.md)). It seals -them to the operator key as today, including any form the foundation's software needs, such as the -broker admin's hash. Nothing else is minted by the controller. - -## What this changes in earlier records - -On acceptance, each of these is superseded or amended by this record, not edited: - -- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: a provider no - longer receives a minted credential. Its refusal of provider-held symmetric keys stands, and is why - option 2 is rejected here. -- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault generates a module's own secret - rather than recording one the controller minted. "The vault stores no plaintext, ever" stands. It - makes a value, seals it, hands it to the mesh and keeps only what it keeps today. -- [To-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) and - [to-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) are amended: both describe - the controller minting a module's credentials and rotating them. After this, the controller mints - only the foundation's, and a provider rotates by answering again. - -## Consequences - -- **A consumer waits for its provider.** Its requirement is not resolved until the provider has - answered, so resolution gains a state, *waiting on a provider*, that is shown rather than silent. - Today a consumer can receive a credential before the resource behind it exists. After this it - cannot. -- The provider harness in the SDK changes: an adapter's create answers with its contract's fields - instead of returning nothing, and every provider in the catalogue moves to it. This is one - migration per provider, the cost 0048 named for changing the contract, and it is paid once. -- The controller gains the return path: receiving a sealed answer on a provider's account and - delivering it. Each contribution gains the consumer node's public key. Data provisions gain the same - path, so the analytics and DNS providers can finally answer. -- A contract needing a derived form is met by its provider. The broker admin's hash stays with the - controller, because that admin is a foundation credential. -- **What got harder:** a provider that is down cannot hand out credentials, where today the controller - could mint one in its absence. That is honest, because a credential for a resource that does not - exist yet was never usable, but it moves a failure from later and silent to earlier and visible. - -## How it is checked - -| Rule | Checked by | -|---|---| -| A provider's secret fields are sealed before they leave it | An SDK test: an answer whose contract marks a field secret cannot be handed over unsealed. A controller test: an unsealed secret field in an answer is refused, not delivered. | -| A provider's answer reaches the consumer sealed to its node | A controller test delivering a provider's answer: each secret field opens with the consumer node's key and with no other, the controller's included. | -| A consumer's identity is still the mesh's | A resolution test: the login a consumer presents is the mesh's derivation, whatever the provider answers. | -| A consumer waits for its provider | A resolution test with no answer yet: the requirement shows as waiting on a provider, and nothing is delivered. | -| The controller mints only the foundation's credentials | A controller test: the minting function is reachable only from genesis and from root-secret rotation, and never for a module's provision. | -| The vault generates and rotates | A vault test: a requested secret is generated to its contract, and a rotation answers with a new value. | - -## References - -- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, - and the return path it left open -- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): identity stays the mesh's -- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0092](0092-an-operator-delivers-a-pair-credential.md): - the vault, and the operator as a provider -- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md): everything a module needs is a requirement diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md new file mode 100644 index 0000000..7dd624d --- /dev/null +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -0,0 +1,182 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-25 +deciders: jochen +reconstructed: false +--- + +# 113. The vault makes every secret, a provider makes resources and data, and the mesh carries both + +## Context + +**A secret comes into being seven different ways today**, counted across the catalogue and the +controller on 2026-09-25: + +| kind | made by | used by | +|---|---|---| +| a credential between a consumer and a provider | the controller | 19 modules | +| a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules | +| a broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a `secret` from the vault | the controller mints it, and the vault only records it ([ADR 0085](0085-a-secret-is-a-provision.md), as amended) | 6 modules | +| a value an operator accepts | a person ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) | where accepted | +| a licence for model access | a separate controller context with its own store | model consumers | +| the mesh's root secrets | genesis, sealed to the operator key | the foundation | + +**The vault was built to end the second row, and did not.** ADR 0085 says a module's own secret +*"stops being a generated value that nothing owns"*. 54 modules still use one, and 6 use the vault. +The replacement was added and the old path was never retired. The same has happened to the broker +account, a special case of the second row that fails silently when the separate command is forgotten. + +**Rotation has its own gaps.** To-be 13 makes rotation one command, all-or-nothing, and states the +window in which a consumer cannot authenticate: the provider has taken the new password, and the +consumer has not yet restarted with it. A consumer restarts only if its definition remembered to say +so, and a container fed by an env-file is not recreated when that file changes, so it keeps the old +value ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). +Nothing confirms that the new secret works. + +**And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) +left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics +provider's site id and the DNS provider's record have no way back, and say so in their code. + +## Considered Options + +**1. Keep the controller minting, and tidy the seven paths.** Rejected. The paths are the problem: +each is made, kept, rotated and audited differently, and tidying keeps all seven. + +**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. It makes +generation uniform and leaves custody scattered: every provider's machine holds secrets the vault +never sees, so rotation, audit and the operator's break-glass copies cover only some of them. The +function would also need a conforming implementation in every language a provider is written in. + +**3. The vault makes every secret, and a provider that needs one requires it, like any consumer.** +Chosen. A provider serving a consumer requires a secret for that consumer from the vault. Secrets +become provisioning all the way down, with one maker at the bottom. + +## Decision + +**The vault makes every secret in the mesh.** Generation, to a secret's contract, exists in the vault +and nowhere else. The controller mints nothing. + +**A provider that needs a secret for a consumer requires it from the vault.** A provision's contract +declares it: *for each consumer, one secret*. Resolution expands that into one requirement per +consumer, named for the consumer. So gitea requiring a database makes the database's provider require +a secret named for gitea, and the vault answers it. The provider's own code does not change. It is +handed a login and a password, as it is today. + +**A secret has holders, and the vault delivers to each.** The database credential has two: the +provider, which creates the login with it, and the consumer, which presents it. The vault hands it to +the mesh, which delivers it to each holder sealed to that holder's node. Plaintext exists in the +vault while it is made and on each holder's machine, and nowhere else. The controller carries sealed +values it cannot open. + +**Every other secret takes the same path:** + +- a module's **own secret** is a `secret` requirement the vault answers. `own-secrets` is retired; +- a **broker account** is the module's identity on the bus. Its name is the mesh's, its password is a + secret the vault makes, and the controller creates the account with it, as it creates accounts + today. There is no separate command to forget; +- an **operator's value** that is secret is handed to the vault, which provides it like any other + secret. It is never replaced by rotation ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)); +- a **licence's** credential is an operator's value the vault keeps. What the licences context adds, + refreshing a token and a manager holding the refresh credential, is provider behaviour, decided in + its own record. + +**Genesis is the vault's first answer, not an exception.** Genesis raises the vault before anything +else and asks it for the foundation's secrets: the store's superuser, the broker's admin and its hashed +form, and the vault's own broker account. The vault answers with the same code it always uses, before +the bus exists. Genesis mints nothing itself. + +**A provider makes resources and data, and the mesh carries data back.** A provider answers with its +contract's non-secret fields: a site id, a registered name. The mesh delivers them to the consumer as +resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer is *given*, +never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). + +### Rotation + +**It is asked of the vault**, by an operator or by the vault's own policy, such as the maximum age a +secret's contract sets. + +**It is provider-first.** The vault delivers the new value first to the holders that *accept* it, +such as the database, and waits for each to confirm it has applied it. Only then does it release the +value to the holders that *present* it, such as gitea. A consumer is never sent a value its provider +has not accepted, so the window shrinks to the consumer's own restart. A provider that does not +confirm holds the rotation: the consumer keeps the old value, which still works, and `status` shows the +rotation as waiting on that provider. It is never half-done and never silently abandoned. + +**Restarts are derived, not declared.** The mesh knows which process reads which secret, because the +definition reads it through its requirement ([to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md)). +When a secret changes, the host restarts every process that reads it, and recreates a container whose +env-file carries it. No definition has to remember `restart-on` for a secret. + +**It is confirmed.** A rotated secret is shown as unconfirmed until each consumer has restarted with +it and, where its definition declares a health check, passed it. Delivered is not the same as working, +and the mesh says which one it knows. + +| step | who | +|---|---| +| asks | an operator, or the vault's policy | +| makes the value | the vault | +| carries it | the controller, sealed, provider first | +| applies it on the provider | the provider's provisioner, which confirms | +| applies it on the consumer | the host, restarting or recreating what reads it | +| confirms it works | the consumer's restart and health check, shown in `status` | + +## What this changes in earlier records + +On acceptance, each of these is superseded or amended by this record, not edited: + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: the controller + no longer mints a provider's credential; the vault makes it. That a provider is handed its + credential and seals nothing stands. +- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes a module's own secret rather + than recording one the controller minted, own secrets are retired, and genesis asks the vault for + the root secrets. "The vault stores no plaintext, ever" stands. +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret + to the vault. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) and + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is provider-first, + derived and confirmed, and the vault is the only maker. +- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on + its old value. + +## Consequences + +- **The vault is on the path of every new or rotated secret.** Today the controller is, and both run + on the control-node, so no new single point of failure appears. It is stated rather than implied. +- Resolution expands per-consumer requirements from a provision's contract. The contract declares + them, never the provider's code, so what a provider requires stays predictable from the catalogue. +- The SDK's provider loop gains one thing: confirming that a rotation was applied. Adapters are + unchanged. +- The mesh gains the way back from provider to consumer, for confirmations and for data. +- 54 modules move from own secrets to vault requirements, and the broker account stops needing a + separate command. +- **What got harder:** a secret can no longer be made when the vault is down, where today the + controller makes one regardless. And a rotation waits for its provider. Both move failures from + late and silent to early and visible, which is the trade this record makes throughout. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Only the vault makes secrets | A controller test: no code path mints a secret. A vault test: the one generation function is the only one, and genesis reaches it through the vault. | +| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement named for it, answered by the vault. | +| A secret reaches each holder sealed to its node | A controller test: each holder's copy opens with that holder's node key and with no other, the controller's included. | +| Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | +| A broker account needs no separate command | A resolution test: assigning a module that speaks on the bus yields its account. | +| Rotation is provider-first | A rotation test: the consumer is not sent the new value until the provider confirms; a provider that never confirms leaves the consumer on the old value, shown as waiting. | +| Restarts are derived | A host test: changing a secret restarts every process that reads it, and recreates a container whose env-file carries it, with no `restart-on` declared. | +| Rotation is confirmed | A rotation test: the secret shows as unconfirmed until the consumer restarted and passed its health check. | +| A provider answers data back | A resolution test: the analytics provider's site id reaches its consumer as a resolved value. | + +## References + +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, + and the return path it left open +- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), + [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): the vault, the operator, and identity +- [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md): + everything a module needs is a requirement +- [Issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md), + [issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): what fails today diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 4083388..984208e 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -158,7 +158,7 @@ python3 00-META/checks/index.py fail if stale - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) - **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* -- **0113** — [A provider makes what it provides, and the mesh carries it back to the consumer](0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md) *(proposed)* +- **0113** — [The vault makes every secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index 42b4b4a..b9fc2ff 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -5,7 +5,7 @@ code: [] updated: 2026-09-25 decisions: - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - - 02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md + - 02-DECISIONS/0113-the-vault-makes-every-secret.md - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md - 02-DECISIONS/0084-which-provider-serves-a-consumer.md @@ -69,11 +69,29 @@ Which module answers, in order: 5. otherwise **refused**: naming the unheld seat, for a provision a seat delivers, even when exactly one provider exists; naming the candidates otherwise. -The provider makes what it provides and answers with its contract's fields. The mesh carries the -answer back to the consumer, sealing every secret field to the consumer's node -([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). The vault -is a module provider like any other: it holds the `mesh-vault` seat and generates the secrets it -provides. +### Secrets: provisioning all the way down + +**The vault makes every secret, and it is the only thing that does** +([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). It holds the `mesh-vault` seat, +so every `secret` requirement resolves to it. + +**A provider that needs a secret for a consumer requires one, like any consumer.** A provision's +contract declares it: *for each consumer, one secret*. Resolution expands that into one requirement +per consumer, named for that consumer: + +1. gitea requires `postgres-database`; +2. the database provider, to serve gitea, requires a `secret` named for gitea; +3. the vault makes it and hands it to the mesh; +4. the mesh delivers it to both holders, each sealed to its own node: the database's machine, to + create the login, and gitea's, to present it; +5. the database provider creates the login, exactly as it does today, and gitea connects. + +A module's own secret, a broker account's password and a secret operator value take the same path. +Nothing in the mesh makes a secret except the vault. + +**A provider makes resources and data.** Beyond secrets, a provider answers with its contract's +non-secret fields: an analytics site id, a registered public name. The mesh carries them back to the +consumer as resolved values. ### The node's host @@ -116,11 +134,11 @@ A value a person chooses: a public name for an endpoint, a greeting, how many wo needs no provider module, no grant and no credential. If asking a person for a value took more than that, module authors would route around it, and the literals this replaces would come back. -**A secret operator value**, such as an external API key, is still an operator requirement: its -provider is the operator. What differs is where it is kept. The vault holds it as an -operator-delivered value ([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)), -never as a setting, because anything secret belongs in one place that can seal and audit it. The -vault is its custodian, not its provider. +**A secret operator value**, such as an external API key, follows the one rule for secrets: the +vault provides it. The operator hands the value to the vault, once +([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)), and the module requires +a `secret` like any other. The only difference is that rotation never replaces it: the vault cannot +make a new external key, so rotating one means an operator handing over a new value. **An endpoint** is an operator value inside a route requirement: the public name is chosen on the assignment, and the route provider answers. A public name already held by another assignment is @@ -162,10 +180,36 @@ slug, under the same rules as a slug. This has to be settled before a second ins ## Genesis -The one exception. Before any provider exists, genesis answers the foundation's own requirements -itself: the store's and broker's credentials, the vault's own access, and the root secrets. It seals -them to the operator key as it does now ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). -After genesis, nothing is answered except by a provider. +**Genesis is the vault's first answer, not an exception.** It raises the vault before anything else +and asks it for the foundation's secrets: the store's superuser, the broker's admin in the hashed form +the broker needs, and the vault's own broker account. The vault answers with the same code it always +uses, before the bus exists, and seals the root secrets to the operator key as today +([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). Genesis makes no secret itself, so +there is one way a secret is made, from the first one onwards. + +The vault can sit at the bottom because it requires nothing but a broker account: it keeps its data +on its own disk, not in the store. So the waterfall ends at the vault. + +## Rotation + +Rotating a secret is asked of the vault, by an operator or by the vault's own policy, such as a +maximum age in the secret's contract ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). + +1. **The vault makes the new value.** +2. **It is delivered to the holders that accept it first.** For a database credential that is the + database's provider, whose provisioner applies it and confirms it did. +3. **Only then is it released to the holders that present it**, such as gitea. A consumer is never + sent a value its provider has not accepted, so the window in which it cannot log in shrinks to its + own restart. A provider that does not confirm holds the rotation: the consumer keeps the old value, + which still works, and the rotation shows as waiting on that provider. +4. **The host restarts every process that reads the secret**, and recreates a container whose + env-file carries it. It knows which, because a definition reads a secret only through its + requirement. No definition declares a restart for a secret. This needs + [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + fixed, or a container fed by an env-file keeps the old value. +5. **It is confirmed.** The rotation shows as unconfirmed until each consumer has restarted with the + new value and passed its health check, where its definition declares one. Delivered and working + are shown as different things. ## Refusing @@ -175,10 +219,10 @@ requirement it names what is missing and what would answer it: - an unheld seat, and which modules could hold it; - no provider, and which modules could provide it; - an operator value with no default, and that the assignment must give it; -- a provider that has not answered yet, and which one. +- a provider, or the vault, that has not answered yet, and which one. -The last one is a state, not a failure. A consumer waiting for its provider is shown as waiting, and -nothing is delivered until the answer arrives ([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)). +The last one is a state, not a failure. A consumer waiting for its provider or for the vault is shown +as waiting, and nothing is delivered until the answer arrives. ## What this retires @@ -188,7 +232,9 @@ nothing is delivered until the answer arrives ([ADR 0113](../../02-DECISIONS/011 | settings on an assignment ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) | operator requirements, addressed to an instance | | a port the mesh assigns | a host requirement | | machine facts and machine placeholders | host requirements | -| a secret the controller mints | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md)) | +| every secret the controller mints: provider credentials, own secrets, broker passwords, root secrets | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)) | +| a separate command issuing a broker account | a requirement resolved on assignment | +| `restart-on` naming a secret's file | a restart the host derives | | paths in resources, mounts, bindings, secrets and received files | host directory requirements, placed by the assignment | | literals carried in a definition | operator requirements with defaults | @@ -204,10 +250,12 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r the controller. The controller resolves requirements from the four providers, refuses as above, and fills the one form. Old mechanisms keep working beside it. *Ends when* a definition written entirely in the new form installs on a lab machine. -2. **Providers answer.** The SDK harness answers with contract fields and seals the secret ones, the - controller carries answers back, and the vault holds its seat and generates. *Ends when* the - analytics and DNS providers answer their consumers, and a module's own secret is generated by the - vault and rotated by it. +2. **The vault makes every secret, and providers answer.** The vault holds its seat and is the only + maker; resolution expands per-consumer secret requirements; genesis asks the vault first; the mesh + carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + is fixed first. *Ends when* no code path outside the vault makes a secret, the analytics and DNS + providers answer their consumers, and a database credential rotates provider-first, with the + consumer restarted by derivation and the rotation confirmed. 3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is. *Ends when* the list of definitions using an old form is empty, and the old forms are removed. @@ -229,6 +277,9 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | A public name already held is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | | Everything keyed by a module is keyed by its instance | A resolution test: two instances of one module on one node get two directories, two containers, two logins and separate settings. | | A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | +| Only the vault makes secrets | A controller test: no code path mints a secret, genesis included. | +| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both holders. | +| Rotation is provider-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): the consumer waits for the provider's confirmation, is restarted without a declared restart, and shows unconfirmed until healthy. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index fdbfce4..6a913d5 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -35,7 +35,7 @@ document is written and this one's status becomes `implemented`. | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | | [`26-the-seats.md`](26-the-seats.md) | What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | -| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-a-provider-makes-what-it-provides-and-the-mesh-carries-it.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | +| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | ## Not yet written From 1b5f2c2c1a821e806e4eb105119a14e373c4bca6 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 23:27:58 +0200 Subject: [PATCH 18/26] ADR 0113 and to-be 27: address the review of the vault rework MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two decisions taken with the author: - Genesis delivers and the vault adopts. The vault cannot run first — it is built on the runtime base the installation makes after the store, broker and controller, and it learns its work over the bus. Genesis generates the foundation's first shared secrets, seals them to the operator key, and delivers them to the vault through the path an operator's value takes; from then on the vault holds and rotates them. This answers ADR 0085's own reason for rejecting vault-only minting, which 0113 now names instead of stepping around. - Rotation re-confirms on every pass. An applier repeats its confirmation until acknowledged, so a lost message costs one pass; an applier that stops after applying locks readers out until its supervised restart, and that window is stated and shown, not claimed away. Fixes: - Scope: a shared secret is made by the vault; a private key (node sealing keys, the operator's key, the certificate authority) is made where it is used. The inventory adds the makers the first version missed: node and builder broker passwords, and enrolment tokens. - Broker accounts are created by the broker's provisioner, not the controller, so the controller never holds their plaintext; mesh-broker delivers amqp again — one broker per mesh — and only mesh-store delivers nothing. - secret is a reserved provision: only the mesh-vault holder may provide it, and no pin routes around it. - A secret's contract says whether a recipient applies it or reads it at start; appliers are never restarted for it, init-only secrets are applied, and confirmation is to-be 13's standard. - Operator secrets are one rule everywhere: a secret requirement answered by the vault (0112 no longer says otherwise). A data provider's adapter may return fields; the data-return check names a lab consumer. - 'Holder' now means a seat's holder only; a secret has recipients. --- ...s-a-module-assignment-from-a-closed-set.md | 25 +- ...e-definition-names-no-node-mesh-or-path.md | 16 +- .../0113-the-vault-makes-every-secret.md | 233 ++++++++++-------- 02-DECISIONS/README.md | 2 +- 03-DESIGN/01-to-be/23-choosing-a-provider.md | 7 +- 03-DESIGN/01-to-be/26-the-seats.md | 23 +- .../27-a-module-requires-the-mesh-resolves.md | 112 ++++++--- 7 files changed, 253 insertions(+), 165 deletions(-) diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index 6800228..b55badc 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -92,13 +92,21 @@ and which assignment holds it, including seats nobody holds. An unheld seat is a has no X", not an error. **A seat delivers a provision only where the mesh has one answer for everyone.** That is a design -decision about the provision, not about the seat. The artifact store, the npm registry, git and the -vault are each one per mesh by their own records, so their seats deliver them. The store and the -broker are not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its -own stores, with a consumer served by the one on its own machine. So `mesh-store` and `mesh-broker` -keep guarding that the foundation's own server is singular, and deliver nothing. Were they to +decision about the provision, not about the seat. The broker, the artifact store, the npm registry, +git and the vault are each one per mesh by their own records, so their seats deliver them. The store +is not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its own +stores, with a consumer served by the one on its own machine, and the foundation's store is the +controller's own memory, provider to nobody ([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). +So `mesh-store` guards that the foundation's store is singular, and delivers nothing. Were it to deliver, every database consumer on every node would be sent to the control-node's store. +**A seat may reserve its provision.** Where a second provider would break a rule the provision exists +for, only the seat's holder may provide it at all: the parser refuses anyone else, and a pin cannot +choose anyone else. `secret` is the one reserved provision. The vault is one per mesh because a second +one *"would be a second place to lose"* ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and +a second `secret` provider is exactly that, whether a pin chose it or not. Every other delivered +provision may have second providers, which a pin can choose. + **The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and they name twelve seats because two alternative modules claim `the-resolver-configuration`. This record admits every seat the catalogue and the controller claim today, so no module is refused by @@ -108,8 +116,8 @@ it: |---|---|---|---|---| | `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | | `mesh-store` | mesh | — | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-broker` | mesh | — | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-vault` | mesh | `secret` | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | +| `mesh-broker` | mesh | `amqp` | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-vault` | mesh | `secret`, reserved | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | | `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | | `the-catalogue` | mesh | — | `mesh-catalog` | this record | | `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | @@ -166,7 +174,8 @@ question real. | A claim outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat whose claimant does not provide. | | Every module in use claims a seat in the set | A controller test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | | The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own machine, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | -| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` and `mesh-broker` deliver nothing, so a database consumer is still served by co-location. | +| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` delivers nothing, so a database consumer is still served by co-location, and `mesh-broker` delivers `amqp`. | +| A reserved provision has no other provider | The parser refuses a module providing `secret` without claiming `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | ## References diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index fa771d0..56449e6 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -68,7 +68,7 @@ unresolved requirement and what could answer it, all at once. | **another module** | a database, a bucket, a vhost, a secret, a route | provisions and bindings | | **the node's host** | a directory, a port, facts about the machine | resource paths, `${port:}`, `${machine:}`, facts | | **the mesh** | the module's identity and names, and the delivery of every answer | derived logins and generated names; the controller's delivery | -| **the operator, through the assignment** | a value a person chooses: a public name, a greeting, an external key | settings, carried literals | +| **the operator, through the assignment** | a value a person chooses that is not secret: a public name, a greeting, a number of workers | settings, carried literals | A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say: a pin, then the holder of a @@ -77,14 +77,14 @@ A host provider is always the module's own node, because a host path or a port m other. An operator value is the assignment's, or the requirement's default, or unresolved. **A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a -default. It needs no provider module, no grant and no credential. An operator value that is secret, -like an external API key, is still the operator's: the vault is where it is *kept*, as an -operator-delivered value ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), not who -provides it. +default. It needs no provider module, no grant and no credential. -**Every secret is made by the vault, and a provider answers with resources and data** -([ADR 0113](0113-the-vault-makes-every-secret.md)). A provider that needs a secret for a consumer -requires it from the vault, like any consumer. The mesh carries every answer back. +**Every secret is a `secret` requirement, answered by the vault**, with no exception by kind +([ADR 0113](0113-the-vault-makes-every-secret.md)). An external API key an operator chooses is no +different: the operator delivers it to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), +and the module requires a `secret` like any other. A provider that needs a secret for a consumer +requires it from the vault, like any consumer, and answers with resources and data. The mesh carries +every answer back. **A directory is a host provision.** Its contract is the owner and mode the module needs, including the owner its image expects ([ADR 0107](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)). diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index 7dd624d..fe69170 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -6,34 +6,39 @@ deciders: jochen reconstructed: false --- -# 113. The vault makes every secret, a provider makes resources and data, and the mesh carries both +# 113. The vault makes every shared secret, a provider makes resources and data, and the mesh carries both ## Context -**A secret comes into being seven different ways today**, counted across the catalogue and the +**A shared secret comes into being many different ways today**, counted across the catalogue and the controller on 2026-09-25: | kind | made by | used by | |---|---|---| | a credential between a consumer and a provider | the controller | 19 modules | | a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules | -| a broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | +| a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder | +| an enrolment token | the controller | every node joining | | a `secret` from the vault | the controller mints it, and the vault only records it ([ADR 0085](0085-a-secret-is-a-provision.md), as amended) | 6 modules | | a value an operator accepts | a person ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)) | where accepted | | a licence for model access | a separate controller context with its own store | model consumers | -| the mesh's root secrets | genesis, sealed to the operator key | the foundation | +| the foundation's root secrets | genesis, sealed to the operator key | the foundation | **The vault was built to end the second row, and did not.** ADR 0085 says a module's own secret *"stops being a generated value that nothing owns"*. 54 modules still use one, and 6 use the vault. -The replacement was added and the old path was never retired. The same has happened to the broker -account, a special case of the second row that fails silently when the separate command is forgotten. +The replacement was added and the old path was never retired. -**Rotation has its own gaps.** To-be 13 makes rotation one command, all-or-nothing, and states the -window in which a consumer cannot authenticate: the provider has taken the new password, and the -consumer has not yet restarted with it. A consumer restarts only if its definition remembered to say -so, and a container fed by an env-file is not recreated when that file changes, so it keeps the old -value ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). -Nothing confirms that the new secret works. +**ADR 0085 considered and rejected making the vault the only maker**, because *"the controller must +mint in order to deliver any provision — the vault's own credential among them"*: the vault cannot +make the credentials that exist before it does. That objection is real, and this record has to answer +it rather than step around it. + +**Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in +which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so; +a container fed by an env-file is not recreated when that file changes +([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)); +and some secrets are read only when a service first initialises, where a restart changes nothing. **And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics @@ -41,86 +46,112 @@ provider's site id and the DNS provider's record have no way back, and say so in ## Considered Options -**1. Keep the controller minting, and tidy the seven paths.** Rejected. The paths are the problem: -each is made, kept, rotated and audited differently, and tidying keeps all seven. +**1. Keep the controller minting, and tidy the paths.** Rejected. The paths are the problem: each is +made, kept, rotated and audited differently, and tidying keeps them all. -**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. It makes -generation uniform and leaves custody scattered: every provider's machine holds secrets the vault -never sees, so rotation, audit and the operator's break-glass copies cover only some of them. The -function would also need a conforming implementation in every language a provider is written in. +**2. Every provider mints its own secrets, with one shared function in the SDK.** Rejected. Generation +becomes uniform, but custody stays spread over every provider's machine, so rotation, audit and the +operator's break-glass copies cover only some secrets. Each SDK language needs its own implementation. -**3. The vault makes every secret, and a provider that needs one requires it, like any consumer.** -Chosen. A provider serving a consumer requires a secret for that consumer from the vault. Secrets -become provisioning all the way down, with one maker at the bottom. +**3. Raise the vault first at genesis, so it makes even the first secrets.** Rejected. The vault is +built on the shared runtime base, which the installation makes only after the store, the broker and +the controller exist, and the vault learns what to answer from the controller over the bus. Running +it first means reordering the whole installation and giving the vault a second way of being asked. + +**4. The vault makes every shared secret; genesis delivers the first ones to it.** Chosen. It answers +0085's objection with a mechanism the mesh already has: a value delivered to the vault. ## Decision -**The vault makes every secret in the mesh.** Generation, to a secret's contract, exists in the vault -and nowhere else. The controller mints nothing. +**There are two kinds of secret, and each has one rule.** -**A provider that needs a secret for a consumer requires it from the vault.** A provision's contract -declares it: *for each consumer, one secret*. Resolution expands that into one requirement per -consumer, named for the consumer. So gitea requiring a database makes the database's provider require -a secret named for gitea, and the vault answers it. The provider's own code does not change. It is -handed a login and a password, as it is today. +- **A shared secret** is a value more than one party must hold: a password, a token, an API key. **The + vault makes every one.** Nothing else in the mesh generates a shared secret. +- **A private key** is made where it is used and never leaves: a node's sealing key, the operator's + key, the mesh's certificate authority. This is not a second way of making secrets. A private key any + other party ever held would no longer be private. -**A secret has holders, and the vault delivers to each.** The database credential has two: the -provider, which creates the login with it, and the consumer, which presents it. The vault hands it to -the mesh, which delivers it to each holder sealed to that holder's node. Plaintext exists in the -vault while it is made and on each holder's machine, and nowhere else. The controller carries sealed -values it cannot open. +**Every shared secret is a `secret` requirement, answered by the vault:** -**Every other secret takes the same path:** +- a **credential between a consumer and a provider**. A provision's contract declares *for each + consumer, one secret*, and resolution expands it into one requirement per consumer. So gitea + requiring a database makes the database's provider require a secret for gitea, and the vault + answers it. The provider's own code does not change: it is handed a login and a password, as today; +- a module's **own secret**. `own-secrets` is retired; +- every **broker account**: a module's, a node's, the builder's. The broker delivers `amqp` through + its seat ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and the broker's own + provisioner creates each account from the vault's secret, like any provider. The controller no + longer creates accounts, and there is no separate command to forget; +- an **enrolment token**; +- a **secret operator value**, such as an external API key, which the operator delivers to the vault + ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these. + What the licences context adds, refreshing a token, is provider behaviour, decided in its own record. -- a module's **own secret** is a `secret` requirement the vault answers. `own-secrets` is retired; -- a **broker account** is the module's identity on the bus. Its name is the mesh's, its password is a - secret the vault makes, and the controller creates the account with it, as it creates accounts - today. There is no separate command to forget; -- an **operator's value** that is secret is handed to the vault, which provides it like any other - secret. It is never replaced by rotation ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)); -- a **licence's** credential is an operator's value the vault keeps. What the licences context adds, - refreshing a token and a manager holding the refresh credential, is provider behaviour, decided in - its own record. +**Only the vault may provide `secret`.** A module providing it must hold the `mesh-vault` seat, and +the parser refuses one that does not. A pin cannot route a `secret` requirement anywhere else, because +there is nowhere else. -**Genesis is the vault's first answer, not an exception.** Genesis raises the vault before anything -else and asks it for the foundation's secrets: the store's superuser, the broker's admin and its hashed -form, and the vault's own broker account. The vault answers with the same code it always uses, before -the bus exists. Genesis mints nothing itself. +**A secret has recipients, and the vault delivers to each.** The database credential has two: the +provider, which *applies* it by creating the login, and the consumer, which *presents* it. The vault +hands the value to the mesh sealed to each recipient's node. The controller and the broker carry sealed +values they cannot open. -**A provider makes resources and data, and the mesh carries data back.** A provider answers with its -contract's non-secret fields: a site id, a registered name. The mesh delivers them to the consumer as -resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer is *given*, -never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). +**Genesis delivers, and the vault adopts.** The foundation's first shared secrets exist before the +vault can run: the store's superuser, the broker's admin in the hashed form the broker needs, the bus +accounts of the temporary controller and of the vault itself, and the first enrolment token. Genesis +generates these, seals them to the operator key as today, and **delivers them to the vault when the +vault is installed**, through the same path an operator's value takes. From then on the vault holds, +audits and rotates them. Unlike an operator's external key, the vault can make their replacements, so +they are delivered but replaceable. Genesis is the only thing besides the vault that ever generates a +shared secret, once, before the vault exists, and it hands them over. + +**A provider makes resources and data, and the mesh carries data back.** A provider's adapter may +answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to +the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer +is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). ### Rotation -**It is asked of the vault**, by an operator or by the vault's own policy, such as the maximum age a -secret's contract sets. +**It is asked of the vault**, by an operator or by the vault's policy, such as a maximum age in the +secret's contract. A delivered value the vault cannot replace, such as an external API key, is not +rotated by the vault: rotating it means an operator delivering a new one. -**It is provider-first.** The vault delivers the new value first to the holders that *accept* it, -such as the database, and waits for each to confirm it has applied it. Only then does it release the -value to the holders that *present* it, such as gitea. A consumer is never sent a value its provider -has not accepted, so the window shrinks to the consumer's own restart. A provider that does not -confirm holds the rotation: the consumer keeps the old value, which still works, and `status` shows the -rotation as waiting on that provider. It is never half-done and never silently abandoned. +**A secret's contract says how each recipient takes a new value:** -**Restarts are derived, not declared.** The mesh knows which process reads which secret, because the -definition reads it through its requirement ([to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md)). -When a secret changes, the host restarts every process that reads it, and recreates a container whose -env-file carries it. No definition has to remember `restart-on` for a secret. +| recipient takes it by | example | what happens on rotation | +|---|---|---| +| **applying** it | a provider creating the login; the broker's provisioner updating an account; the store's own provisioner changing its superuser | its provisioner applies the new value; it is never restarted for it | +| **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it | -**It is confirmed.** A rotated secret is shown as unconfirmed until each consumer has restarted with -it and, where its definition declares a health check, passed it. Delivered is not the same as working, -and the mesh says which one it knows. +A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks +it applied, and a provisioner makes the change. The host derives which recipients read a secret at +start from the requirement their definition reads it through, so no definition declares a restart for +a secret. + +**It is applier-first.** The vault delivers the new value first to the recipients that apply it. Each +applies it, verifies that the new value authenticates and the old one no longer does, and confirms. +**It repeats that confirmation on every reconcile pass until the vault acknowledges it**, so a lost +message costs one pass. Only after every applier has confirmed does the vault release the value to the +recipients that read it at start. + +**The remaining window is stated.** If an applier applies the new value and its provisioner stops +before confirming, the recipients that present the secret are locked out until the provisioner runs +again, because the old value no longer works and they have not been sent the new one. A provisioner is +supervised and restarted when it exits, so the window is bounded by that restart. The mesh shows the +rotation as waiting on that applier for as long as it lasts, never as done. + +**It is confirmed.** A rotation is shown as unconfirmed until every applier has confirmed, and every +recipient that reads at start has restarted with the new value and passed its health check, where its +definition declares one. | step | who | |---|---| | asks | an operator, or the vault's policy | | makes the value | the vault | -| carries it | the controller, sealed, provider first | -| applies it on the provider | the provider's provisioner, which confirms | -| applies it on the consumer | the host, restarting or recreating what reads it | -| confirms it works | the consumer's restart and health check, shown in `status` | +| carries it | the controller, sealed, appliers first | +| applies it | each applier's provisioner, which verifies and confirms on every pass until acknowledged | +| takes it at start | the host, restarting or recreating what reads it | +| confirms it | applier confirmations, then restarts and health checks, shown in `status` | ## What this changes in earlier records @@ -129,53 +160,61 @@ On acceptance, each of these is superseded or amended by this record, not edited - [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) is superseded: the controller no longer mints a provider's credential; the vault makes it. That a provider is handed its credential and seals nothing stands. -- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes a module's own secret rather - than recording one the controller minted, own secrets are retired, and genesis asks the vault for - the root secrets. "The vault stores no plaintext, ever" stands. +- [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own + secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the + first secrets. "The vault stores no plaintext, ever" stands. - [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret - to the vault. -- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) and - [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is provider-first, - derived and confirmed, and the vault is the only maker. + to the vault, and genesis delivers the foundation's first secrets the same way. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker + account is created by the broker's provisioner, not the controller. Its scoping stands. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), + [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is applier-first, + derived and confirmed; genesis delivers its secrets to the vault; the vault is the only maker. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on its old value. ## Consequences -- **The vault is on the path of every new or rotated secret.** Today the controller is, and both run - on the control-node, so no new single point of failure appears. It is stated rather than implied. +- **The vault is on the path of every new or rotated shared secret.** Today the controller holds that + place, on the same node. A secret can no longer be made while the vault is down. - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. -- The SDK's provider loop gains one thing: confirming that a rotation was applied. Adapters are - unchanged. -- The mesh gains the way back from provider to consumer, for confirmations and for data. -- 54 modules move from own secrets to vault requirements, and the broker account stops needing a - separate command. -- **What got harder:** a secret can no longer be made when the vault is down, where today the - controller makes one regardless. And a rotation waits for its provider. Both move failures from - late and silent to early and visible, which is the trade this record makes throughout. +- The SDK's provider loop gains repeated confirmation of an applied rotation. A credential provider's + adapter is unchanged. A data provider's adapter gains a return value. +- The broker's provisioner gains every bus account, and the controller loses five separate places it + generates a secret today. +- 54 modules move from own secrets to vault requirements. +- **What got harder:** a rotation waits for its appliers, and an applier that stops mid-rotation locks + presenters out until it restarts. Both are shown, not hidden, and the second is bounded by a + supervised restart rather than by someone noticing. ## How it is checked | Rule | Checked by | |---|---| -| Only the vault makes secrets | A controller test: no code path mints a secret. A vault test: the one generation function is the only one, and genesis reaches it through the vault. | -| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement named for it, answered by the vault. | -| A secret reaches each holder sealed to its node | A controller test: each holder's copy opens with that holder's node key and with no other, the controller's included. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | +| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, and the operator's private key never enters the mesh. | +| Only the vault provides `secret` | The parser refuses a module providing `secret` without holding `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. | +| Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. | | Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | -| A broker account needs no separate command | A resolution test: assigning a module that speaks on the bus yields its account. | -| Rotation is provider-first | A rotation test: the consumer is not sent the new value until the provider confirms; a provider that never confirms leaves the consumer on the old value, shown as waiting. | -| Restarts are derived | A host test: changing a secret restarts every process that reads it, and recreates a container whose env-file carries it, with no `restart-on` declared. | -| Rotation is confirmed | A rotation test: the secret shows as unconfirmed until the consumer restarted and passed its health check. | -| A provider answers data back | A resolution test: the analytics provider's site id reaches its consumer as a resolved value. | +| Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | +| An irreplaceable delivered value is not rotated by the vault | A vault test: a rotation request on an operator's external key is refused, naming the operator as its source. | +| Rotation is applier-first and re-confirmed | A rotation test: presenters are not sent the new value until every applier confirms, and a confirmation lost in transit is repeated on the next pass. | +| Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. | +| Rotation is confirmed | A rotation test: an applier confirms only after the new value authenticates and the old one does not, and the rotation shows unconfirmed until every reader restarted and passed its health check. | +| A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | ## References - [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md): the decision this supersedes, and the return path it left open -- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), - [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): the vault, the operator, and identity +- [ADR 0085](0085-a-secret-is-a-provision.md): the vault, and the objection this record answers +- [ADR 0092](0092-an-operator-delivers-a-pair-credential.md), [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), + [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): delivered values, + identity, and broker accounts - [ADR 0112](0112-a-module-definition-names-no-node-mesh-or-path.md), [to-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md): everything a module needs is a requirement - [Issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md), diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 984208e..bd8635a 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -158,7 +158,7 @@ python3 00-META/checks/index.py fail if stale - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) - **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* -- **0113** — [The vault makes every secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* +- **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/23-choosing-a-provider.md b/03-DESIGN/01-to-be/23-choosing-a-provider.md index a16b953..ddb6670 100644 --- a/03-DESIGN/01-to-be/23-choosing-a-provider.md +++ b/03-DESIGN/01-to-be/23-choosing-a-provider.md @@ -57,9 +57,10 @@ consumer's own machine does not take over for that consumer. That is not picking once, mesh-wide, by assigning the holder, rather than once per consumer by naming it ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer -coupled to particular contents has said so. Only provisions the design makes one-per-mesh are -delivered by a seat: the artifact store, a package registry, git and the vault. A database is not. -Node-local stores, served by co-location, are the rule above. +coupled to particular contents has said so, except for `secret`, which only the vault may provide. +Only provisions the design makes one-per-mesh are delivered by a seat: the broker, the artifact store, +a package registry, git and the vault. A database is not. Node-local stores, served by co-location, +are the rule above. **Ambiguity is refused, never resolved by picking.** If several providers of a kind exist, none is named, none is co-located, and no seat delivers it, the requirement is unsatisfiable and is refused diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 719442f..d90dad0 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -48,8 +48,8 @@ argued for is an entry nobody can explain. |---|---|---|---| | `mesh-controller` | mesh | — | the controller | | `mesh-store` | mesh | — | the foundation's store | -| `mesh-broker` | mesh | — | the foundation's broker | -| `mesh-vault` | mesh | `secret` | the vault | +| `mesh-broker` | mesh | `amqp` | the broker | +| `mesh-vault` | mesh | `secret`, reserved | the vault | | `the-artifact-store` | mesh | `artifact-store` | the artifact registry | | `the-catalogue` | mesh | — | the catalogue | | `npm-package-registry` | mesh | `npm-package-registry` | the forge | @@ -64,8 +64,8 @@ argued for is an entry nobody can explain. The controller holds this set in code, and a test asserts both its size and that every entry names the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) -govern, and code that disagrees is what is wrong.** The implementation in progress predates two -things here: the `mesh-vault` seat, and the rule that `mesh-store` and `mesh-broker` deliver +govern, and code that disagrees is what is wrong.** The implementation in progress predates three +things here: the `mesh-vault` seat and its reservation, and the rule that `mesh-store` delivers nothing. It is brought to this table before it merges. ## A seat that delivers a provision @@ -73,11 +73,18 @@ nothing. It is brought to this table before it merges. A seat that delivers a provision may only be held by a module that provides it, at the seat's scope. A mesh seat delivers a mesh-scoped provision. -**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, -the npm registry, git and the vault are each one per mesh by decision. The store and the broker are +**A seat delivers a provision only where the mesh has one answer for everyone.** The broker, the +artifact store, the npm registry, git and the vault are each one per mesh by decision. The store is not: nodes run their own stores and a consumer uses the one on its machine -([23 — Choosing a provider](23-choosing-a-provider.md)). So their seats guard that the foundation's -own server is singular, and route nobody. +([23 — Choosing a provider](23-choosing-a-provider.md)), and the foundation's store is the controller's +own memory, provider to nobody. So `mesh-store` guards that the foundation's store is singular, and +routes nobody. + +**The vault's provision is reserved.** Only the holder of `mesh-vault` may provide `secret` at all: a +module providing it without the seat is refused, and a pin cannot choose another provider, because +there is none. A second provider of secrets would be a second place secrets live, which is what the +vault being one per mesh exists to prevent. Every other delivered provision may have second +providers, which a pin can choose. **Its holder answers for that provision.** A requirement for it resolves, in order, to: diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index b9fc2ff..a76658c 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -71,9 +71,17 @@ Which module answers, in order: ### Secrets: provisioning all the way down -**The vault makes every secret, and it is the only thing that does** -([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). It holds the `mesh-vault` seat, -so every `secret` requirement resolves to it. +**Two kinds of secret, one rule each** ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)): + +- **a shared secret**, a value more than one party must hold (a password, a token, an API key), is + made by the vault, and by nothing else; +- **a private key**, such as a node's sealing key, the operator's key or the mesh's certificate + authority, is made where it is used and never leaves. A private key anyone else held would no + longer be private. + +**Every shared secret is a `secret` requirement, and only the vault provides `secret`.** The vault +holds the `mesh-vault` seat, and that provision is reserved to it: no other module may provide it, and +no pin can choose another provider ([26 — The seats](26-the-seats.md)). **A provider that needs a secret for a consumer requires one, like any consumer.** A provision's contract declares it: *for each consumer, one secret*. Resolution expands that into one requirement @@ -82,16 +90,19 @@ per consumer, named for that consumer: 1. gitea requires `postgres-database`; 2. the database provider, to serve gitea, requires a `secret` named for gitea; 3. the vault makes it and hands it to the mesh; -4. the mesh delivers it to both holders, each sealed to its own node: the database's machine, to - create the login, and gitea's, to present it; +4. the mesh delivers it to both of its **recipients**, each sealed to its own node: the database's + machine, which *applies* it by creating the login, and gitea's, which *presents* it; 5. the database provider creates the login, exactly as it does today, and gitea connects. -A module's own secret, a broker account's password and a secret operator value take the same path. -Nothing in the mesh makes a secret except the vault. +Every other shared secret takes the same path: +- a module's own secret; +- every broker account's password, where the broker's own provisioner creates the account; +- an enrolment token; +- a secret operator value, which the operator delivers to the vault. -**A provider makes resources and data.** Beyond secrets, a provider answers with its contract's -non-secret fields: an analytics site id, a registered public name. The mesh carries them back to the -consumer as resolved values. +**A provider makes resources and data.** Beyond secrets, a provider's adapter may answer with its +contract's non-secret fields: an analytics site id, a registered public name. The mesh carries them +back to the consumer as resolved values. ### The node's host @@ -180,36 +191,54 @@ slug, under the same rules as a slug. This has to be settled before a second ins ## Genesis -**Genesis is the vault's first answer, not an exception.** It raises the vault before anything else -and asks it for the foundation's secrets: the store's superuser, the broker's admin in the hashed form -the broker needs, and the vault's own broker account. The vault answers with the same code it always -uses, before the bus exists, and seals the root secrets to the operator key as today -([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). Genesis makes no secret itself, so -there is one way a secret is made, from the first one onwards. +**Genesis delivers, and the vault adopts.** The vault cannot run first: it is built on the shared +runtime base, which the installation makes only after the store, the broker and the controller exist +([21 — The installation in full](21-the-installation-in-full.md)), and it learns what to answer from +the controller over the bus. So genesis generates the foundation's first shared secrets itself: +- the store's superuser; +- the broker's admin, in the hashed form the broker needs; +- the bus accounts of the temporary controller and of the vault; +- the first enrolment token. -The vault can sit at the bottom because it requires nothing but a broker account: it keeps its data -on its own disk, not in the store. So the waterfall ends at the vault. +It seals them to the operator key as today ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). +When the vault is installed, genesis **delivers them to it**, through the same path an operator's value +takes. From then on the vault holds, audits and rotates them. It can make their replacements, unlike +an operator's external key. + +That is the one time anything but the vault generates a shared secret, and it ends by handing them +over. It is also the answer to the objection ADR 0085 had to the vault being the only maker: the +vault cannot make what exists before it, so what exists before it is delivered to it. ## Rotation Rotating a secret is asked of the vault, by an operator or by the vault's own policy, such as a maximum age in the secret's contract ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). +An operator's external key is not rotated by the vault, which cannot make its replacement: an +operator delivers a new one. + +**A secret's contract says how each recipient takes a new value.** A recipient either *applies* it, +through a provisioner (a provider creating the login, the broker's provisioner updating an account, +the store's own provisioner changing its superuser), or *reads it at start*. A secret a service reads +only when it first initialises is marked applied, because a restart would change nothing. 1. **The vault makes the new value.** -2. **It is delivered to the holders that accept it first.** For a database credential that is the - database's provider, whose provisioner applies it and confirms it did. -3. **Only then is it released to the holders that present it**, such as gitea. A consumer is never - sent a value its provider has not accepted, so the window in which it cannot log in shrinks to its - own restart. A provider that does not confirm holds the rotation: the consumer keeps the old value, - which still works, and the rotation shows as waiting on that provider. -4. **The host restarts every process that reads the secret**, and recreates a container whose - env-file carries it. It knows which, because a definition reads a secret only through its - requirement. No definition declares a restart for a secret. This needs - [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) +2. **It goes first to the recipients that apply it.** Each applies it, verifies that the new value + authenticates and the old one no longer does, and confirms. It repeats that confirmation on every + reconcile pass until the vault acknowledges it, so a lost message costs one pass. +3. **Only then is it released to the recipients that read it at start**, such as gitea. +4. **The host restarts every such recipient**, and recreates a container whose env-file carries the + secret. It knows which, because a definition reads a secret only through its requirement, so no + definition declares a restart for a secret. An applying recipient is never restarted for it. + This needs [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) fixed, or a container fed by an env-file keeps the old value. -5. **It is confirmed.** The rotation shows as unconfirmed until each consumer has restarted with the - new value and passed its health check, where its definition declares one. Delivered and working - are shown as different things. +5. **It is confirmed.** The rotation shows as unconfirmed until every applier has confirmed, and every + recipient that reads at start has restarted and passed its health check, where its definition + declares one. Delivered and working are shown as different things. + +**The remaining window is stated.** An applier whose provisioner stops after applying and before +confirming leaves the recipients that read at start locked out: the old value no longer works, and +they have not been sent the new one. A provisioner is supervised and restarted when it exits, so the +window lasts until that restart. The rotation shows as waiting on that applier throughout, never as done. ## Refusing @@ -250,12 +279,14 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r the controller. The controller resolves requirements from the four providers, refuses as above, and fills the one form. Old mechanisms keep working beside it. *Ends when* a definition written entirely in the new form installs on a lab machine. -2. **The vault makes every secret, and providers answer.** The vault holds its seat and is the only - maker; resolution expands per-consumer secret requirements; genesis asks the vault first; the mesh - carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - is fixed first. *Ends when* no code path outside the vault makes a secret, the analytics and DNS - providers answer their consumers, and a database credential rotates provider-first, with the - consumer restarted by derivation and the rotation confirmed. +2. **The vault makes every shared secret, and providers answer.** The vault holds its seat and its + reserved provision; resolution expands per-consumer secret requirements; genesis delivers the + foundation's first secrets to the vault; the broker's provisioner creates every bus account; the + mesh carries providers' data back. + [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + is fixed first. *Ends when* nothing outside the vault generates a shared secret after genesis, a lab + consumer of analytics receives its site id, and a database credential rotates applier-first, with + the consumer restarted by derivation and the rotation confirmed. 3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is. *Ends when* the list of definitions using an old form is empty, and the old forms are removed. @@ -277,9 +308,10 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | A public name already held is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | | Everything keyed by a module is keyed by its instance | A resolution test: two instances of one module on one node get two directories, two containers, two logins and separate settings. | | A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | -| Only the vault makes secrets | A controller test: no code path mints a secret, genesis included. | -| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both holders. | -| Rotation is provider-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): the consumer waits for the provider's confirmation, is restarted without a declared restart, and shows unconfirmed until healthy. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | +| Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | +| A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | +| Rotation is applier-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): readers wait for every applier's repeated confirmation; an applied secret restarts nothing, and one read at start restarts its reader without a declared restart; the rotation shows unconfirmed until the new value authenticates, the old does not, and readers are healthy. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | From 4a1b2187062ab77d527bf50ba9456e085bcf4fa6 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 23:47:26 +0200 Subject: [PATCH 19/26] Seats held by assignments, one assignment per module per node, and 0113's bottom of the stack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decided with the author: - A seat is held by one assignment, not claimed by a definition. A definition says which seats a module can hold; an assignment says which it does. The store module can run on every node and one assignment holds mesh-store; moving a role changes an assignment, never a definition. The foundation's seats name what the mesh itself uses and route no consumer — database and amqp consumers use co-location, the holder included. This replaces the wrong rationale that the foundation's store is "provider to nobody", which contradicted ADR 0078 and to-be 21. 0079's one-postgres rule becomes one mesh-store holder. - A module is assigned at most once to a node. The instance identity in 0112 and 27 is withdrawn, and the login-length problem with it. Review fixes to 0113: - The bottom of the stack: the vault is installed as soon as the shared runtime base exists, and genesis generates everything needed until then — including the permanent controller's, the control-node agent's, the builder's and the broker provisioner's bus accounts, and the controller's store login. Genesis creates those accounts until the broker's provisioner runs and adopts them. - Genesis's values are delivered recorded as the mesh's own, so 0092's never-replace rule for operator values does not make them unrotatable. - Backend-issued secrets (a forge's once-only API token) enter through the vault. Non-module parties (the controller's logins, node agents' accounts) are answered the same way, the controller asking on their behalf; an enrolment token reaches the controller only as what verifies it. - A secret with no provisioner to apply it is marked not rotatable by the mesh and refused, instead of a restart reported as done. Unused password generators in six provider clients are removed, and a catalogue scan checks no module mints. - Rotation's lock-out cases (offline reader, bus account owner, restarted provisioner) are recorded as open, with overlap and re-confirm-with-safeguards as the two answers, to be chosen before acceptance. 0110, 0111 and 26 are marked proposed: they changed in meaning and are under review, and an accepted record must not rest on proposed ones. To-be 23 and the glossary are restored to main; they change when these records are accepted. --- 00-META/glossary.md | 15 +- ...s-a-module-assignment-from-a-closed-set.md | 209 +++++++++--------- ...d-source-is-on-the-git-seat-or-external.md | 6 +- ...e-definition-names-no-node-mesh-or-path.md | 57 +++-- .../0113-the-vault-makes-every-secret.md | 122 ++++++---- 02-DECISIONS/README.md | 4 +- 03-DESIGN/01-to-be/23-choosing-a-provider.md | 18 +- 03-DESIGN/01-to-be/26-the-seats.md | 110 ++++----- .../27-a-module-requires-the-mesh-resolves.md | 101 +++++---- 9 files changed, 346 insertions(+), 296 deletions(-) diff --git a/00-META/glossary.md b/00-META/glossary.md index 4ffe66e..7d38f6d 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -44,22 +44,15 @@ another — and a mesh you cannot name precisely is a mesh two people describe d ## How modules relate to the mesh -- **seat** — a named role at a scope (node / site / mesh), held by a module assignment, from a - **closed set** the mesh defines: a claim naming a seat outside the set is refused. A seat may - **deliver a provision**, and its holder is then the mesh's answer for it when several modules - provide it ([ADR 0110](../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). - The set, with who holds each seat, is the overview of what a mesh has - ([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)). A seat has a **capacity**: a - capacity-1 seat is exclusive (one holder); a higher-capacity seat is a **bench** (several holders - coexist). +- **seat** — a named position at a scope (node / site / mesh) with a **capacity**. A capacity-1 seat + is exclusive (one holder); a higher-capacity seat is a **bench** (several holders coexist). - **claim** — a module taking a spot on a seat. `claims: [{name, scope}]` in a manifest. A mesh-scoped exclusive claim is how the mesh says "there is one of me". A foundation seat is named after the server it guards: the `mesh-controller`, `postgres` and `lavinmq` modules claim the `mesh-controller`, `mesh-store` and `mesh-broker` seats ([ADR 0079](../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)). - **provision** — a service one module `provides` and others `require`; the mesh resolves a provider - and wires the two with an endpoint and a credential. A provision is a service you offer, a seat - is a role you occupy, and the two meet where a seat delivers a provision: occupying the seat is - what makes a module *the* provider of it. + and wires the two with an endpoint and a credential. This is separate from seats: a provision is + a service you offer, a seat is a slot you occupy. ## How this page is kept diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index b55badc..6bfbbb9 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -1,20 +1,20 @@ --- topic: what runs on it -status: accepted +status: proposed date: 2026-09-25 deciders: jochen reconstructed: false extends: 0009-modules-and-the-graph.md --- -# 110. A seat is a module assignment from a closed set, and it may deliver a provision +# 110. A seat is held by one assignment, from a closed set, and it may deliver a provision ## Context [ADR 0009](0009-modules-and-the-graph.md) introduced claims: a module declares something singular, at a scope, and a second holder is refused. [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) named the foundation's three after their servers. That mechanism is enforced and works. What it -means has drifted, and three things are now true of it that no record says. +means has drifted, and four things are now true of it that no record says. **Any well-formed name becomes a seat by being claimed.** The controller's manifest check refuses a claim only for a malformed name or an unknown scope. Nothing says which seats a mesh has. @@ -24,100 +24,102 @@ The names in use were each invented by the module that claims them: `the-showcas **Nothing can say what a mesh has, or who fills it.** There is no seat table and no command that lists seats. Holdings are assembled while planning, one node at a time, and discarded afterwards. The only way to answer "which seats does this mesh have, and which module holds each" is to read -every manifest in two repositories, because the controller's own manifest lives in its own repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's -code, because one module it ships has its manifest composed there. While this -record was being prepared, that enumeration was done by hand, and it missed both of the last two -sources: eleven claims were reported where there are thirteen. +every manifest in two repositories, because the controller's own manifest lives in its own +repository ([ADR 0069](0069-a-module-is-a-repository-and-a-path.md)), and then the controller's code, +because one module it ships has its manifest composed there. While this record was being prepared, +that enumeration was done by hand, and it missed both of the last two sources: eleven claims were +reported where there are thirteen. -**Some seats are the mesh's one of something that others consume, and nothing uses that fact.** -Of the thirteen claims, four are held by a module that provides something consumers require: -`mesh-store` (`postgres-database`, eleven consumers), `mesh-broker` (`amqp`), `the-artifact-store` -(`artifact-store`) and `the-dns-port`. The rest deliver nothing to anybody, and are still -meaningful: they say which module is this mesh's packet filter, or resolver configuration. -Meanwhile a requirement for a mesh-scoped provision with more than one provider is refused until a -person pins, **per consumer node**, which provider to use. [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) -anticipates exactly that case — gitea and verdaccio both answering npm — and under today's -resolution it would mean a pin on every machine that builds anything. +**A claim in a definition makes a module singular, not a role.** The store module's definition +claims `mesh-store`, so every assignment of it claims the seat, and a second store module on any other +node is refused. What is singular is *the store the mesh itself uses*, not postgres. Any module can +run on any node whose capabilities match, which is a core principle of the module system, and a claim +written into the definition breaks it for every module that claims anything. + +**Some seats are the mesh's one of something everyone consumes, and nothing uses that fact.** A +requirement for a mesh-scoped provision with more than one provider is refused until a person pins, +**per consumer node**, which provider to use. [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) +anticipates exactly that case, gitea and verdaccio both answering npm, and under today's resolution it +would mean a pin on every machine that builds anything. ## Considered Options -**1. Leave seats as free-form exclusion, and keep provisions as the only thing that delivers.** -Rejected. The overview stays unanswerable, and a second provider of anything costs a pin per -consumer node — the decision "gitea is our npm registry" made again on every machine. +**1. Leave seats as free-form exclusion, claimed in definitions.** Rejected. The overview stays +unanswerable, a module that claims a seat can run on only one node, and a second provider of anything +costs a pin per consumer node. **2. Two concepts: seats for exclusion, and a new word for the mesh's one consumable thing.** -Rejected. Both mean "this mesh's one X". Every existing claim would first have to be classified -into one or the other, and the overview a person wants is one list, not two. +Rejected. Both mean "this mesh's one X". Every existing claim would first have to be classified into +one or the other, and the overview a person wants is one list, not two. -**3. A seat is a module assignment from a closed set, and occupying it may deliver a provision.** +**3. A seat is held by one assignment, from a closed set, and holding it may deliver a provision.** Chosen. ## Decision -**The mesh defines a closed set of seats.** Each entry has a name, a scope, what occupying it -delivers (if anything), and the decision that made it a seat. A claim naming a seat outside the set -is refused, and so is a claim at a scope other than the seat's. Adding a seat is a decision, for the -same reason adding a shape to the host's vocabulary is one: the set is what a person reads to learn -what a mesh can have, and a name added without an argument is a name nobody can explain later. +**The mesh defines a closed set of seats.** Each entry has a name, a scope, what holding it delivers +(if anything), and the decision that made it a seat. A seat outside the set is refused wherever it +is named. Adding a seat is a decision, for the same reason adding a shape to the host's vocabulary is +one: the set is what a person reads to learn what a mesh can have, and a name added without an +argument is a name nobody can explain later. -**A seat is held by a module assignment.** What the mesh knows about a seat's holder is what it knows -about that assignment: its node, the node's settings for it, and what it serves. Holdings are not -stored separately. The seat points at an assignment, and a second record of the same fact would be a -second thing to disagree with the first. +**A definition says which seats a module *can* hold. An assignment says which it *does* hold.** The +store module can hold `mesh-store`, and it may be assigned to every node. Exactly one of those +assignments holds the seat, because that assignment said so. A second assignment saying so, at the +seat's scope, is refused. So a seat makes a *role* singular, never a module, and moving the role is +changing which assignment holds it, with no definition changed and nothing unassigned. -**A seat may deliver a provision, and then its holder answers for it.** A seat that delivers a -provision can only be held by a module that provides it, at the seat's scope, and a claim that does -not is refused. A requirement for that provision resolves, in order, to: +**What the mesh knows about a seat's holder is what it knows about that assignment**: its node, the +node's settings for it, and what it serves. Holdings are not stored separately. The seat points at an +assignment, and a second record of the same fact would be a second thing to disagree with the first. -1. a provider the consumer's node was **pinned** to — a consumer coupled to one provider's - contents, as [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) already allows; -2. **the holder of the seat** that delivers it, **even when another provider runs on the consumer's - own node**; +**Holding a seat may deliver a provision.** A seat that delivers a provision may only be held by an +assignment of a module that provides it, at the seat's scope. A requirement for that provision +resolves, in order, to: + +1. a provider the consumer's node was **pinned** to, a consumer coupled to one provider's contents; +2. **the holder of the seat**, **even when another provider runs on the consumer's own node**; 3. otherwise refused, naming the unheld seat. -**Co-location does not apply to a provision a seat delivers.** For every other provision, a provider -on the consumer's own node answers first, then the only provider, then refusal -([to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md)). A seat exists to say *which one is the +**Co-location does not apply to a provision a seat delivers.** A seat exists to say *which one is the mesh's*, and co-location answering first would let any second provider on a consumer's machine take over silently for that consumer. That is the failure [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) -names for the vault. +names for the vault. This keeps [ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with +several answers is never guessed: the seat is the choice made once, mesh-wide, by assigning the holder, +instead of once per consumer by pinning. -This keeps [ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with several answers is -never guessed. The seat is not a guess. It is the choice made once, mesh-wide, by assigning the holder, -instead of once per consumer by pinning. A second provider may run beside the holder, and whatever -requires the provision still resolves to the holder without anybody naming it. +**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, +the npm registry, git and the vault are each one per mesh by their own records, so their seats +deliver them. -**Seats are also informational.** The controller lists every seat in the set, what it delivers, -and which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh -has no X", not an error. +**The foundation's seats deliver nothing.** `mesh-controller`, `mesh-store` and `mesh-broker` name +which assignment the mesh *itself* uses: the controller, the store holding its records, the broker +carrying its bus. The store and broker modules may run on other nodes too, and a database or `amqp` +consumer is served by co-location from whichever runs on its own node, the seat's holder included. +Were `mesh-store` to deliver, every database consumer on every node would be sent to one machine. -**A seat delivers a provision only where the mesh has one answer for everyone.** That is a design -decision about the provision, not about the seat. The broker, the artifact store, the npm registry, -git and the vault are each one per mesh by their own records, so their seats deliver them. The store -is not: [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md) has each node running its own -stores, with a consumer served by the one on its own machine, and the foundation's store is the -controller's own memory, provider to nobody ([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). -So `mesh-store` guards that the foundation's store is singular, and delivers nothing. Were it to -deliver, every database consumer on every node would be sent to the control-node's store. +**A seat may reserve its provision.** Where a second provider would break the reason the provision +exists, only an assignment holding the seat may provide it at all: the parser refuses a definition +that provides it without being able to hold the seat, resolution refuses an assignment providing it +without holding the seat, and a pin cannot choose anyone else. `secret` is the one reserved provision. +The vault is one per mesh because a second *"would be a second place to lose"* +([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and a second `secret` provider is exactly +that, whether a pin chose it or not. -**A seat may reserve its provision.** Where a second provider would break a rule the provision exists -for, only the seat's holder may provide it at all: the parser refuses anyone else, and a pin cannot -choose anyone else. `secret` is the one reserved provision. The vault is one per mesh because a second -one *"would be a second place to lose"* ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and -a second `secret` provider is exactly that, whether a pin chose it or not. Every other delivered -provision may have second providers, which a pin can choose. +**Seats are also informational.** The controller lists every seat in the set, what it delivers, and +which assignment holds it, including seats nobody holds. An unheld seat is an answer, "this mesh has +no X", not an error. -**The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and -they name twelve seats because two alternative modules claim `the-resolver-configuration`. This -record admits every seat the catalogue and the controller claim today, so no module is refused by -it: +**The first set is the twelve seats already claimed, plus two.** Thirteen claims are in use, and they +name twelve seats because two alternative modules claim `the-resolver-configuration`. This record +admits every seat the catalogue and the controller claim today, so no definition is refused by it: -| seat | scope | delivers | held today by | made a seat by | +| seat | scope | delivers | can be held by | made a seat by | |---|---|---|---|---| | `mesh-controller` | mesh | — | `mesh-controller` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | | `mesh-store` | mesh | — | `postgres` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-broker` | mesh | `amqp` | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | -| `mesh-vault` | mesh | `secret`, reserved | nothing yet: `mesh-vault` claims it | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | +| `mesh-broker` | mesh | — | `lavinmq` | [0079](0079-the-foundation-seats-are-named-after-their-servers.md) | +| `mesh-vault` | mesh | `secret`, reserved | `mesh-vault` | this record, for [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) | | `the-artifact-store` | mesh | `artifact-store` | `distribution` | [0075](0075-two-stores-and-which-provides-what.md) | | `the-catalogue` | mesh | — | `mesh-catalog` | this record | | `npm-package-registry` | mesh | `npm-package-registry` | `gitea` | [0109](0109-a-package-registry-seat-is-one-per-ecosystem.md) | @@ -131,57 +133,62 @@ it: There are two additions. `npm-package-registry` is [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md)'s seat, named after the provision it delivers, as 0079 named the foundation's seats after what they are. -gitea claims it. verdaccio provides the same provision and claims nothing, so it is the second -provider this record exists to make harmless. +A gitea assignment holds it. verdaccio provides the same provision and cannot hold the seat, so it is +the second provider this record exists to make harmless. Moving npm to it would take a definition +saying it can hold the seat, and then an assignment saying it does. -`mesh-vault` answers [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md). The vault -is one per mesh ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and until now that was -enforced by nothing. A second vault would have answered requirements silently, and any consumer on -its machine would have been served by it through co-location. The seat is named after its server, by -the 0079 convention, and the vault module claims it. +`mesh-vault` answers [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md). The vault is one +per mesh ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and until now that was enforced by +nothing. The seat is named after its server, by the 0079 convention. `the-dns-port` is listed as delivering nothing, although `dnsmasq` provides `wildcard-resolution`. -That provision is node-scoped and answered on the machine, so no preference between providers -arises. Whether the seat should say it delivers it is left for when a second resolver makes the -question real. +That provision is node-scoped and answered on the machine, so no preference between providers arises. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0009](0009-modules-and-the-graph.md): a claim in a definition says a module *can* hold a seat; + the assignment says it does. +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): "a mesh runs one postgres + and one lavinmq" becomes one holder of `mesh-store` and one of `mesh-broker`. The store and broker + modules may run on other nodes. ## Consequences - The controller carries the set in code. A test asserts its size, and that every entry names the record that made it a seat, so changing the set means finding the argument rather than a number. - This is the pattern the host's vocabulary test already follows. -- Manifest validation refuses an unknown seat, a seat claimed at the wrong scope, and a - provision-delivering seat claimed by a module that does not provide the provision. The three - refusals name the seat and the set. -- Resolution prefers the seat's holder among several providers, after a pin. A provider record - gains the module it came from, because two modules on one node could otherwise not be told apart - as holder and non-holder. -- A `seats` command lists the set with each seat's holders, derived from assignments. -- gitea claims `npm-package-registry`. The catalogue's `package-registry` becomes - `npm-package-registry` throughout, per [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md). +- An assignment gains the seats it holds. Genesis assigns the foundation's store, broker and + controller holding their seats, where today their definitions claim them. +- Manifest validation refuses an unknown seat, a seat named at the wrong scope, and a delivering seat + named by a module that does not provide the provision. Resolution refuses a second holder, and an + assignment holding a seat its module cannot hold. +- Resolution prefers the seat's holder for a provision it delivers, after a pin. A provider record + gains the module it came from. +- A `seats` command lists the set with each seat's holder, derived from assignments. - **What got harder:** a module wanting a new singular role can no longer invent a name. It needs a - record. That is the point, and it costs one record per seat. -- **Not changed:** `${seat::}` stays as it is. It exists so the controller can reach a - foundation it made before any module existed, and it cannot be a consumer. A module that needs - something from a seat's holder requires the provision the seat delivers, and receives it the way - any provision is received: through a grant. + record. And an assignment has one more thing to say. Both are the point. +- **Not changed:** the controller's seat placeholder stays as it is. It exists so the controller can + reach a foundation it made before any module existed. ## How it is checked | Rule | Checked by | |---|---| | The set is closed, and every entry names its decision | A controller unit test asserts the set's size and a non-empty decision for every entry. | -| A claim outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat whose claimant does not provide. | -| Every module in use claims a seat in the set | A controller test parses every catalogue manifest and fails on any refused claim. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | -| The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own machine, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | -| A seat delivers only a one-per-mesh provision | A controller unit test: `mesh-store` delivers nothing, so a database consumer is still served by co-location, and `mesh-broker` delivers `amqp`. | -| A reserved provision has no other provider | The parser refuses a module providing `secret` without claiming `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| A seat outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat named by a module that does not provide it. | +| Every module in use names a seat in the set | A controller test parses every catalogue manifest and fails on any refused seat. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | +| A seat is held by one assignment, not by a module | A resolution test: the store module assigned to two nodes resolves, with one assignment holding `mesh-store`; a second assignment asking to hold it is refused. | +| The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | +| The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`. | +| A reserved provision has no other provider | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`; resolution refuses an assignment providing it without holding the seat, and a pin on a `secret` requirement. | ## References - [ADR 0009](0009-modules-and-the-graph.md): claims, scopes, and "refused, never guessed" - [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): a seat named after what it is - [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md): the per-ecosystem registry seats -- [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): pins, co-location and refusal +- [ADR 0084](0084-which-provider-serves-a-consumer.md), [to-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): + pins, co-location and refusal - `mesh-controller internal/catalogue/resolve.go` (`checkClaims`, the brokered branch of `Resolve`), `internal/catalogue/manifest.go` (claim validation), `cmd/mesh-controller/plan.go` (holdings) diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md index b021ea5..44b7057 100644 --- a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -1,6 +1,6 @@ --- topic: building it -status: accepted +status: proposed date: 2026-09-25 deciders: jochen reconstructed: false @@ -46,7 +46,7 @@ one case it exists for: after the forge moves, old URLs no longer match anything **The mesh has a `git` seat.** It is mesh-scoped and delivers the `git` provision, per [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md). Its holder provides `git`, -serving how a repository on it is cloned: the scheme and the port. gitea claims it. +serving how a repository on it is cloned: the scheme and the port. A gitea assignment holds it. **A source is on the git seat, or it is external, and the mesh records which.** @@ -72,7 +72,7 @@ the controller's job, because only the controller knows where the seat's holder - `build`, `build --behind` and `build --dry-run` resolve a seat source before asking a builder. The recorded source keeps the seat form; the build log keeps the URL that was actually cloned, because that is what happened. -- gitea claims `git` and provides it, serving HTTP clone on its web port. +- gitea can hold `git` and provides it, serving HTTP clone on its web port; the forge's assignment holds the seat. - **Not decided: credentials for private repositories.** The mesh's own repositories are public, and clone without one. A private repository still works only if the build machine's own git configuration authenticates, exactly as before. Delivering a clone credential through the `git` diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index 56449e6..8d68e3e 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -43,7 +43,7 @@ concept that joins them. **1. Keep host paths in definitions, and check that every copy agrees.** Rejected. It checks the agreement of something that should not be there. A definition still could not follow its data to -another disk, be adopted onto a machine whose data is already somewhere, or run twice on one node. +another disk, or be adopted onto a machine whose data is already somewhere. **2. Keep the separate mechanisms, and add directories as a seventh.** Rejected. It fixes paths and keeps the pattern that produced them: each mechanism is resolved, validated and refused differently, @@ -105,39 +105,38 @@ directory. The mesh mounts the assignment's location there. No host path is ever reads, and the mesh's own files (answers, contributions) name nothing by host path, so a provider needs no mount at a machine-identical path. -**An assignment has an identity of its own: an instance name**, defaulting to the module's name. -Everything keyed by the module's name today is keyed by the instance instead: directories, container -names, the login a consumer presents, broker accounts, a claim's holder, the settings an assignment -carries, and a provider's identity. So **one module may be assigned to one node more than once.** What -must stay singular stays so by a claim, or by an operator value colliding: a public name already taken -is refused like any other singular thing. +**A module is assigned at most once to a node.** An assignment is a module on a node, and that pair is +its identity: its directories, containers, login, broker account and settings are keyed by it, as +they are today. A module may run on many nodes, and one of those assignments may hold a seat +([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). Running the same module twice +on one machine is not supported. The cases that seemed to need it, such as two stores of one engine or +two stages of one application, are different modules, or the same module on different machines. The +line is drawn because every identity in the mesh is already a module on a node, and a second +instance would have to rename all of them. -**Genesis is not an exception.** It raises the vault first and asks it for the foundation's secrets, -so the foundation's requirements are answered the same way as everything else -([ADR 0113](0113-the-vault-makes-every-secret.md)). +**What must stay singular stays so** by a seat, or by an operator value colliding: a public name +already held by another assignment is refused like any other singular thing. + +**The foundation's first secrets are delivered, then adopted.** Genesis generates them before the vault +can run and hands them to the vault once it is installed, and from then on they are answered the same +way as every other secret ([ADR 0113](0113-the-vault-makes-every-secret.md)). ## What this changes in earlier records On acceptance, each of these is amended by a record of its own, not edited: - [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings become - operator requirements, addressed to an instance rather than to a module on a node. + operator requirements on an assignment. - [ADR 0038](0038-the-mesh-assigns-the-port.md): a port becomes a host requirement. What 0038 decided is unchanged; it is the first case of this rule. - [ADR 0051](0051-shared-data-is-the-operators.md): an access keeps its shape and its semantics; its path moves from the definition to the assignment. - [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved requirement, checked as resolved rather than as a path the definition declares. -- [ADR 0084](0084-which-provider-serves-a-consumer.md): a provider is a (node, instance) pair, not a - (node, module) pair, so a consumer can name one of two instances on one node. -- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a login is built from the - instance, which gets a short form under the same rules as a slug, so it still fits the tightest - backend. -- [To-be 26](../03-DESIGN/01-to-be/26-the-seats.md): a seat's holder is an instance. - The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to a - requirement answered by any of the four providers, and *instance* is added. Neither lands while this - record is only proposed, because the glossary is the authority on the words in use, not on words - under review. + requirement answered by any of the four providers, and *requirement* and *contract* are added. None + of it lands while this record is only proposed, because the glossary is the authority on the words + in use, not on words under review. ## Consequences @@ -148,13 +147,13 @@ On acceptance, each of these is amended by a record of its own, not edited: - The controller resolves every requirement at assignment and refuses unresolved ones. The host answers directories and ports. The settings, placeholders, facts and bindings that exist today are retired as separate mechanisms, once nothing uses them. -- Identity moves from the module to the instance, which touches logins, broker accounts, settings, - provider selection and every resource name. A login already has a 20-character limit - ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), which a node and an instance - name will strain. The design must answer that before a second instance is possible. -- **What got harder:** a definition no longer says where a module's data is on a machine, or what a - setting's value is. The assignment does, and `plan` shows it. That is the point, and it is also a - real loss of at-a-glance legibility, which the overview has to give back. +- Identity stays a module on a node. Nothing is renamed, and a login still fits the tightest backend + as [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md) arranges. +- **What got harder:** one module cannot run twice on one machine; a second stage or a second store + of one engine is a different module or a different machine. And a definition no longer says where + a module's data is on a machine, or what a setting's value is. The assignment does, and `plan` shows + it. That is the point, and it is also a real loss of at-a-glance legibility, which the overview has + to give back. - **Not decided here:** the syntax a definition reads a requirement's fields with; a node's default layout; the order in which the mechanisms are retired. [To-be 27](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md) proposes all three. @@ -169,8 +168,8 @@ On acceptance, each of these is amended by a record of its own, not edited: | Every requirement has one of the four providers | The parser refuses a requirement whose provider kind is not one of the four. | | Installation resolves every requirement | A resolution test with one requirement unanswered: refused, naming it and what could answer it. | | A host requirement is answered on its module's own node | A resolution test: an assignment placing a directory or a port on another node is refused. | -| A module can run twice on one node | A resolution test assigning one module twice under two instance names: two directories, two logins, two containers, separate settings, no collision. | -| A public name already taken is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | +| A module is assigned at most once to a node | A resolution test: assigning a module to a node that already runs it is refused, naming the existing assignment. | +| A public name already taken is refused | A resolution test: a second assignment asking for a public name another holds is refused, naming the holder. | | An adopted assignment is placed where its data is | An adoption test: the directory resolves to the data's existing location, and nothing is moved. | ## References diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index fe69170..a5b5769 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -78,32 +78,54 @@ it first means reordering the whole installation and giving the vault a second w requiring a database makes the database's provider require a secret for gitea, and the vault answers it. The provider's own code does not change: it is handed a login and a password, as today; - a module's **own secret**. `own-secrets` is retired; -- every **broker account**: a module's, a node's, the builder's. The broker delivers `amqp` through - its seat ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and the broker's own - provisioner creates each account from the vault's secret, like any provider. The controller no - longer creates accounts, and there is no separate command to forget; -- an **enrolment token**; +- every **broker account** on the mesh's bus: a module's, a node agent's, the builder's, the + controller's. The broker holding `mesh-broker` carries the mesh's bus + ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates + each account from the vault's secret, like any provider. The controller no longer creates accounts, + and there is no separate command to forget; +- an **enrolment token**. The vault makes it; the operator receives the token, sealed to the operator + key, to hand to the joining machine; the controller receives only what it needs to verify it, never + the token itself; - a **secret operator value**, such as an external API key, which the operator delivers to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)). A licence's credential is one of these. - What the licences context adds, refreshing a token, is provider behaviour, decided in its own record. + What the licences context adds, refreshing a token, is provider behaviour, decided in its own record; +- a **secret a backend issues itself**, such as an API token a forge hands out exactly once when asked. + The vault cannot make that value. The module that received it delivers it to the vault, which keeps + it and provides it like any other; rotating it means asking the backend again. -**Only the vault may provide `secret`.** A module providing it must hold the `mesh-vault` seat, and -the parser refuses one that does not. A pin cannot route a `secret` requirement anywhere else, because -there is nowhere else. +**Parties that are not modules take the same path.** The controller's own store login and bus account, +and each node agent's bus account, have no definition to require them. The controller asks the vault +on its own behalf, or a node's, and the vault answers the way it answers any requirement: made by the +vault, sealed to the recipient, carried by the mesh. The requirement is not written in a definition, +because the controller and a node agent are the mesh itself, but it is answered no differently. + +**Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat. +The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a +`secret` requirement anywhere else, because there is nowhere else. **A secret has recipients, and the vault delivers to each.** The database credential has two: the -provider, which *applies* it by creating the login, and the consumer, which *presents* it. The vault -hands the value to the mesh sealed to each recipient's node. The controller and the broker carry sealed -values they cannot open. +provider, which *applies* it by creating the login, and the consumer, which *reads* it and presents it +when it connects. The vault hands the value to the mesh sealed to each recipient's node. The controller +and the broker carry sealed values they cannot open. -**Genesis delivers, and the vault adopts.** The foundation's first shared secrets exist before the -vault can run: the store's superuser, the broker's admin in the hashed form the broker needs, the bus -accounts of the temporary controller and of the vault itself, and the first enrolment token. Genesis -generates these, seals them to the operator key as today, and **delivers them to the vault when the -vault is installed**, through the same path an operator's value takes. From then on the vault holds, -audits and rotates them. Unlike an operator's external key, the vault can make their replacements, so -they are delivered but replaceable. Genesis is the only thing besides the vault that ever generates a -shared secret, once, before the vault exists, and it hands them over. +**Genesis delivers, and the vault adopts.** The vault is built on the shared runtime base, which the +installation makes only after the store, the broker and the controller are running +([to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md)). So the vault is installed **as soon +as that base exists**, before any other module built on it, and everything needed before that moment is +generated by genesis: + +- the store's superuser, and the broker's admin in the hashed form the broker needs; +- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder, + the broker's own provisioner and the vault; +- the controller's store login, and the first enrolment token. + +Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's +admin, as the controller does today. Genesis seals all of it to the operator key, and when the vault is +installed it **delivers the values to the vault, recorded as the mesh's own**, not as an operator's. +That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), +and these are, because the vault can make their replacements. The broker's provisioner then adopts the +accounts genesis created. From then on the vault makes every shared secret, and genesis has made its +last one. **A provider makes resources and data, and the mesh carries data back.** A provider's adapter may answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to @@ -124,9 +146,12 @@ rotated by the vault: rotating it means an operator delivering a new one. | **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it | A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks -it applied, and a provisioner makes the change. The host derives which recipients read a secret at -start from the requirement their definition reads it through, so no definition declares a restart for -a secret. +it applied, and a provisioner makes the change. Where no provisioner exists to make it, such as a +module's own bootstrap password, the contract marks the secret **not rotatable by the mesh**, and a +rotation request is refused, saying why, rather than restarting a service that would carry on with +the old value. The host derives which recipients read a secret at start from the requirement their +definition reads it through, so no definition declares a restart for a secret. A provider's +per-consumer secrets are applied, never read at start, so the host never restarts a provider for one. **It is applier-first.** The vault delivers the new value first to the recipients that apply it. Each applies it, verifies that the new value authenticates and the old one no longer does, and confirms. @@ -134,11 +159,20 @@ applies it, verifies that the new value authenticates and the old one no longer message costs one pass. Only after every applier has confirmed does the vault release the value to the recipients that read it at start. -**The remaining window is stated.** If an applier applies the new value and its provisioner stops -before confirming, the recipients that present the secret are locked out until the provisioner runs -again, because the old value no longer works and they have not been sent the new one. A provisioner is -supervised and restarted when it exits, so the window is bounded by that restart. The mesh shows the -rotation as waiting on that applier for as long as it lasts, never as done. +**Open: keeping readers from being locked out.** Review found three cases this rule does not survive: + +- a reader whose machine is offline when an applier has already applied the new value is locked out + until it returns, where to-be 13 would have refused the rotation and kept the old value working; +- a bus account's owner can be locked out permanently, because the confirmation and the new value + travel over the bus it has just lost; +- a provisioner restarted mid-rotation no longer knows the old value, so it cannot verify that the old + value has stopped working. + +Two answers are recorded, and one must be chosen before this record is accepted. **Overlap:** an +applier keeps the old and new credential valid together until every reader has confirmed the new one, +for instance by alternating between two derived logins with the adapter's existing create and remove, +which needs no change on the consumer's side. Or **re-confirm with safeguards:** a pre-check that every +reader is reachable before any applier starts, plus a special path for bus accounts. **It is confirmed.** A rotation is shown as unconfirmed until every applier has confirmed, and every recipient that reads at start has restarted with the new value and passed its health check, where its @@ -164,13 +198,17 @@ On acceptance, each of these is superseded or amended by this record, not edited secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the first secrets. "The vault stores no plaintext, ever" stands. - [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret - to the vault, and genesis delivers the foundation's first secrets the same way. + to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own, + so 0092's rule that an operator's value is never replaced does not apply to them. - [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker account is created by the broker's provisioner, not the controller. Its scoping stands. - [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is applier-first, - derived and confirmed; genesis delivers its secrets to the vault; the vault is the only maker. + derived and confirmed; the vault is installed as soon as the shared runtime base exists, and genesis + delivers its secrets to it; the vault is the only maker. +- The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at + start*, and *reserved provision*, once this record is accepted. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on its old value. @@ -185,24 +223,30 @@ On acceptance, each of these is superseded or amended by this record, not edited adapter is unchanged. A data provider's adapter gains a return value. - The broker's provisioner gains every bus account, and the controller loses five separate places it generates a secret today. -- 54 modules move from own secrets to vault requirements. -- **What got harder:** a rotation waits for its appliers, and an applier that stops mid-rotation locks - presenters out until it restarts. Both are shown, not hidden, and the second is bounded by a - supervised restart rather than by someone noticing. +- 54 modules move from own secrets to vault requirements. Six provider clients export a password + generator nothing uses any more; it is removed, so no module can quietly start minting again. +- The installation changes order: the vault is installed as soon as the shared runtime base exists, + before any other module built on it. +- **What got harder:** a rotation waits for its appliers, and how a reader is kept from being locked + out while it does is still open (above). A secret some services read only at first start can no + longer be "rotated" by a restart that quietly changes nothing; it is refused instead. ## How it is checked | Rule | Checked by | |---|---| -| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | -| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, and the operator's private key never enters the mesh. | -| Only the vault provides `secret` | The parser refuses a module providing `secret` without holding `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls, with none exempt. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. | +| A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. | +| Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | +| Parties that are not modules take the same path | Controller tests: its own store login, its bus account and a node agent's bus account are each made by the vault and delivered sealed; an enrolment token reaches the controller only as what verifies it. | +| A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. | +| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. | | A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. | | Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. | | Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | | Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | -| An irreplaceable delivered value is not rotated by the vault | A vault test: a rotation request on an operator's external key is refused, naming the operator as its source. | -| Rotation is applier-first and re-confirmed | A rotation test: presenters are not sent the new value until every applier confirms, and a confirmation lost in transit is repeated on the next pass. | +| An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. | +| Rotation is applier-first | A rotation test: readers are not sent the new value until every applier confirms. How lock-out is prevented is open, and its check is written when that is decided. | | Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. | | Rotation is confirmed | A rotation test: an applier confirms only after the new value authenticates and the old one does not, and the rotation shows unconfirmed until every reader restarted and passed its health check. | | A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index bd8635a..70b2bcb 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -156,7 +156,7 @@ python3 00-META/checks/index.py fail if stale - **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md) - **0091** — [A mount is declared, and there are three things it can be](0091-a-mount-is-declared-three-ways.md) - **0099** — [A step that runs once names what it reads, and runs again when it changed](0099-a-step-that-runs-once-names-what-it-reads.md) -- **0110** — [A seat is a module assignment from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) +- **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(proposed)* - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* - **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* @@ -178,7 +178,7 @@ python3 00-META/checks/index.py fail if stale - **0096** — [An upstream image is copied between registries, never through a machine's image store](0096-an-upstream-image-is-copied-between-registries.md) - **0097** — [A vendor image is a declared build input, and a recipe fetches nothing undeclared](0097-a-vendor-image-is-a-declared-build-input.md) - **0107** — [Persistent data is a directory bind, never a named volume](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md) -- **0111** — [A build source is on the mesh's git seat, or it is an external repository](0111-a-build-source-is-on-the-git-seat-or-external.md) +- **0111** — [A build source is on the mesh's git seat, or it is an external repository](0111-a-build-source-is-on-the-git-seat-or-external.md) *(proposed)* ### How it is checked diff --git a/03-DESIGN/01-to-be/23-choosing-a-provider.md b/03-DESIGN/01-to-be/23-choosing-a-provider.md index ddb6670..50b3ed3 100644 --- a/03-DESIGN/01-to-be/23-choosing-a-provider.md +++ b/03-DESIGN/01-to-be/23-choosing-a-provider.md @@ -2,11 +2,10 @@ layer: to-be status: designed code: [] -updated: 2026-09-25 +updated: 2026-09-20 decisions: - 02-DECISIONS/0084-which-provider-serves-a-consumer.md - 02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md - - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md --- # 23 — Choosing a provider @@ -51,20 +50,9 @@ provider on a different node. That coupling is exactly what may not be guessed, names the provider. Naming it is also what makes a later move safe — the mesh knows the binding is to that provider and not to whichever one is nearest. -**Some provisions have one provider for the whole mesh, and a seat names it.** Where a seat delivers -the provision, its holder answers for it, **and co-location does not apply**: a second provider on the -consumer's own machine does not take over for that consumer. That is not picking: the choice was made -once, mesh-wide, by assigning the holder, rather than once per consumer by naming it -([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), -[26 — The seats](26-the-seats.md)). A named provider still wins over the seat, because a consumer -coupled to particular contents has said so, except for `secret`, which only the vault may provide. -Only provisions the design makes one-per-mesh are delivered by a seat: the broker, the artifact store, -a package registry, git and the vault. A database is not. Node-local stores, served by co-location, -are the rule above. - **Ambiguity is refused, never resolved by picking.** If several providers of a kind exist, none is -named, none is co-located, and no seat delivers it, the requirement is unsatisfiable and is refused -with the candidates shown — the same stance +named, and none is co-located, the requirement is unsatisfiable and is refused with the candidates +shown — the same stance [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) took against a confidently-wrong match, applied to the instance rather than the dialect. A wrong answer delivered quietly costs more than a refusal. diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index d90dad0..cfb9a9d 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -1,6 +1,6 @@ --- layer: to-be -status: in-progress +status: proposed code: - mesh-controller internal/catalogue/seats.go - mesh-controller internal/catalogue/resolve.go @@ -17,8 +17,8 @@ decisions: # 26 — The seats -**What a mesh can have one of, and who fills each.** A seat is a named role at a scope, taken by a -module assignment. The mesh defines which seats exist. Occupying one may deliver a provision, and the +**What a mesh can have one of, and who fills each.** A seat is a named role at a scope, held by one +module assignment. The mesh defines which seats exist. Holding one may deliver a provision, and the list of seats with their holders is the quickest answer to "what is in this mesh". ## What a seat is @@ -27,28 +27,31 @@ A seat has four properties, fixed by the mesh rather than by any module: | property | is | |---|---| -| name | what a manifest claims, and what a person reads in the list | +| name | what a definition names and an assignment holds, and what a person reads in the list | | scope | node, site or mesh: where its capacity applies. Every seat in the set has a capacity of one, so one holder per scope. A bench, a seat with several holders, is a word the glossary keeps and no seat uses yet | | delivers | the provision its holder answers for, or nothing | | decision | the record that made it a seat | -**A module assignment holds a seat by claiming it.** The claim is the manifest's `claims`, and it is -satisfied by assigning the module somewhere. The seat is not a second record beside the assignment. -It points at the assignment, and everything the mesh knows about the holder is what it knows about -that assignment: the node, the node's settings for the module, and what the module serves. +**A definition says which seats a module can hold. An assignment says which it does hold.** The store +module can hold `mesh-store`, and it may run on every node whose capabilities match. Exactly one of +those assignments holds the seat, because that assignment says so, and a second assignment saying so +is refused. A seat makes a role singular, never a module. -**The set is closed.** A claim naming a seat the mesh does not define is refused, and so is a claim at -the wrong scope. Adding a seat is a decision, recorded, for the reason every addition to the host's -vocabulary is one: the set is what a person reads to learn what a mesh can have, and an entry nobody -argued for is an entry nobody can explain. +**The seat points at the assignment.** Everything the mesh knows about the holder is what it knows +about that assignment: the node, the node's settings for the module, and what the module serves. + +**The set is closed.** A seat the mesh does not define is refused wherever it is named, and so is one +named at the wrong scope. Adding a seat is a decision, recorded, for the reason every addition to the +host's vocabulary is one: the set is what a person reads to learn what a mesh can have, and an entry +nobody argued for is an entry nobody can explain. ## The set | seat | scope | delivers | typically held by | |---|---|---|---| | `mesh-controller` | mesh | — | the controller | -| `mesh-store` | mesh | — | the foundation's store | -| `mesh-broker` | mesh | `amqp` | the broker | +| `mesh-store` | mesh | — | the store the mesh's own records live in | +| `mesh-broker` | mesh | — | the broker carrying the mesh's own bus | | `mesh-vault` | mesh | `secret`, reserved | the vault | | `the-artifact-store` | mesh | `artifact-store` | the artifact registry | | `the-catalogue` | mesh | — | the catalogue | @@ -64,27 +67,24 @@ argued for is an entry nobody can explain. The controller holds this set in code, and a test asserts both its size and that every entry names the record that made it a seat. **This table and [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) -govern, and code that disagrees is what is wrong.** The implementation in progress predates three -things here: the `mesh-vault` seat and its reservation, and the rule that `mesh-store` delivers -nothing. It is brought to this table before it merges. +govern, and code that disagrees is what is wrong.** The implementation in progress predates several +things here: seats held by assignments rather than claimed by definitions, the `mesh-vault` seat and +its reservation, and the foundation's seats delivering nothing. It is brought to this table before it +merges. + +## The foundation's seats + +`mesh-controller`, `mesh-store` and `mesh-broker` name which assignment the mesh *itself* uses: the +controller, the store holding its records, the broker carrying its bus. **They route no consumer.** The +store and broker modules may run on other nodes too. A database or `amqp` consumer is served by +co-location, from whichever runs on its own node, the seat's holder included +([23 — Choosing a provider](23-choosing-a-provider.md)). ## A seat that delivers a provision -A seat that delivers a provision may only be held by a module that provides it, at the seat's scope. -A mesh seat delivers a mesh-scoped provision. - -**A seat delivers a provision only where the mesh has one answer for everyone.** The broker, the -artifact store, the npm registry, git and the vault are each one per mesh by decision. The store is -not: nodes run their own stores and a consumer uses the one on its machine -([23 — Choosing a provider](23-choosing-a-provider.md)), and the foundation's store is the controller's -own memory, provider to nobody. So `mesh-store` guards that the foundation's store is singular, and -routes nobody. - -**The vault's provision is reserved.** Only the holder of `mesh-vault` may provide `secret` at all: a -module providing it without the seat is refused, and a pin cannot choose another provider, because -there is none. A second provider of secrets would be a second place secrets live, which is what the -vault being one per mesh exists to prevent. Every other delivered provision may have second -providers, which a pin can choose. +**A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, +the npm registry, git and the vault are each one per mesh by decision. A seat that delivers a +provision may only be held by an assignment of a module that provides it, at the seat's scope. **Its holder answers for that provision.** A requirement for it resolves, in order, to: @@ -96,21 +96,23 @@ providers, which a pin can choose. Co-location, which answers first for every other provision, does not apply here: a seat says which one is the mesh's, and co-location answering first would let any second provider on a consumer's machine take over for that consumer, silently. So a second provider can run beside the holder and -harm nothing. The forge holds -`npm-package-registry`. An npm proxy may provide the same provision on another machine, and a module -requiring an npm registry is still served by the forge, without anybody pinning it. +harm nothing. A forge assignment holds `npm-package-registry`. An npm proxy may provide the same +provision on another machine, and a module requiring an npm registry is still served by the forge, +without anybody pinning it. -**Moving the role is changing which module claims the seat, and today that is a definition change.** -A claim is part of a module's definition, so the proxy's definition must claim the seat and the -forge's must stop. The forge cannot simply be unassigned, because it holds `git` as well. Every -consumer follows once the claim moves. Making *which* seats a module holds the assignment's choice, -with the definition saying only which seats it *can* hold, is the consistent answer, and -[27 — A module requires, the mesh resolves](27-a-module-requires-the-mesh-resolves.md) lists it as -not yet settled. +**Moving the role is changing which assignment holds the seat.** No definition changes and nothing is +unassigned: the forge keeps running, and keeps holding `git`, when its npm role moves. A module can +take the role only if its definition says it can hold the seat. -**What a consumer receives is a grant**, the same as for any provision: where the provider answers, -what it serves, and a credential. A consumer never reads the seat directly. The one exception is the -controller itself, which reaches the store and the broker through a narrow seat placeholder, +**The vault's provision is reserved.** Only an assignment holding `mesh-vault` may provide `secret` at +all: a definition providing it that cannot hold the seat is refused, an assignment providing it without +holding the seat is refused, and a pin cannot choose another provider, because there is none. A second +provider of secrets would be a second place secrets live, which is what the vault being one per mesh +exists to prevent. + +**What a consumer receives is what it required**, the same as for any provision: where the provider +answers, what it serves, and a credential. A consumer never reads the seat directly. The one exception +is the controller itself, which reaches the store and the broker through a narrow seat placeholder, because it made them before any module existed and cannot be their consumer. One foundation module also reads it today, to find its own server's port. [27](27-a-module-requires-the-mesh-resolves.md) moves that to a host port requirement. @@ -123,9 +125,9 @@ their job, and it is a real one: it is the mesh saying what a machine is, in wor ## The overview -The controller lists every seat in the set with its scope, what it delivers, and each holder as a -node and a module. A seat nobody holds is listed as unheld. That is an answer, "this mesh has no -forge", and not a fault. +The controller lists every seat in the set with its scope, what it delivers, and its holder as a node +and a module. A seat nobody holds is listed as unheld. That is an answer, "this mesh has no forge", +and not a fault. Holdings are derived from assignments whenever they are asked for, never stored. The list is always what the mesh is running, because it is computed from the same thing that decides what the mesh runs. @@ -140,13 +142,13 @@ mesh records which: | on the `git` seat | a repository on the forge that holds the seat | its path on the forge, and the seat | | external | a repository anywhere else, a public forge for instance | its URL, exactly as given | -For a repository on the seat, the controller composes the clone URL at the moment of building, -from where the holder runs and the scheme and port it serves for `git`. The recorded source never -contains an address, so moving the forge changes nothing that was recorded. The build machine is not -told the difference: it receives a URL either way. +For a repository on the seat, the controller composes the clone URL at the moment of building, from +where the holder runs and the scheme and port it serves for `git`. The recorded source never contains +an address, so moving the forge changes nothing that was recorded. The build machine is not told the +difference: it receives a URL either way. With the seat unheld, a build from the seat is refused and says why. External builds carry on. **Not yet designed:** a credential for cloning a private repository. The mesh's own repositories are -public. The natural place for a clone credential is the `git` provision's grant, and that is a -decision still to take. +public. The natural place for a clone credential is a `secret` from the vault, and that is a decision +still to take. diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index a76658c..ca648af 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -96,9 +96,17 @@ per consumer, named for that consumer: Every other shared secret takes the same path: - a module's own secret; -- every broker account's password, where the broker's own provisioner creates the account; -- an enrolment token; -- a secret operator value, which the operator delivers to the vault. +- every broker account's password on the mesh's bus, where the broker's own provisioner creates the + account; +- an enrolment token, which the operator receives and the controller can only verify; +- a secret operator value, which the operator delivers to the vault; +- a secret a backend issues itself, such as a forge's API token, which the module that received it + delivers to the vault. + +**Parties that are not modules take the same path too.** The controller's own store login and bus +account, and each node agent's bus account, have no definition to require them, because the controller +and a node agent are the mesh itself. The controller asks the vault on its own behalf or a node's, and +the answer is made, sealed and carried exactly as for a module. **A provider makes resources and data.** Beyond secrets, a provider's adapter may answer with its contract's non-secret fields: an analytics site id, a registered public name. The mesh carries them @@ -116,7 +124,7 @@ lost is a named volume, not a directory ([ADR 0030](../../02-DECISIONS/0030-data *Where* a directory is on the machine is the assignment's: -- **a node's default layout**, a root per node with one directory per instance beneath it, used when +- **a node's default layout**, a root per node with one directory per assignment beneath it, used when the assignment says nothing; - **a placement**, where the assignment puts one directory elsewhere: on a second disk, or where an adopted machine's data already is ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). @@ -175,35 +183,39 @@ module uses the seat placeholder. decided. The one exception 0086 allows is a declared env-file with its reason; a secret field as a value in a container's environment is refused when the definition is parsed, with no exception. -## An instance +## An assignment -An assignment has an identity: an **instance name**, which defaults to the module's name. Everything -keyed by the module's name today is keyed by the instance: directories, containers, the login it -presents, its broker account, the seats it holds, its settings and its identity as a provider. +**A module is assigned at most once to a node**, and that pair is the assignment's identity +([ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md)). Its directories, +containers, login, broker account and settings are keyed by it, as today, and a login still fits the +tightest backend ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). -So a module may run twice on one node, under two instance names. What must stay singular stays so: -by a seat, or by an operator value colliding, as with a public name. +**A module may run on many nodes, and one assignment may hold a seat** +([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)). The definition +says which seats the module can hold; the assignment says which it does. So the store module can run +on every node, one of those assignments holds `mesh-store`, and moving that role changes an +assignment, not a definition. -**A login still has to fit the tightest backend**, which is twenty characters today -([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). A node name and -an instance name will not fit in full. The instance therefore gets a short form alongside the module's -slug, under the same rules as a slug. This has to be settled before a second instance is allowed. +What must stay singular stays so: by a seat, or by an operator value colliding, as with a public name. ## Genesis **Genesis delivers, and the vault adopts.** The vault cannot run first: it is built on the shared runtime base, which the installation makes only after the store, the broker and the controller exist ([21 — The installation in full](21-the-installation-in-full.md)), and it learns what to answer from -the controller over the bus. So genesis generates the foundation's first shared secrets itself: -- the store's superuser; -- the broker's admin, in the hashed form the broker needs; -- the bus accounts of the temporary controller and of the vault; -- the first enrolment token. +the controller over the bus. So the vault is installed **as soon as that base exists**, before any other +module built on it, and genesis generates what is needed until then: +- the store's superuser, and the broker's admin in the hashed form the broker needs; +- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder, + the broker's own provisioner and the vault; +- the controller's store login, and the first enrolment token. -It seals them to the operator key as today ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)). -When the vault is installed, genesis **delivers them to it**, through the same path an operator's value -takes. From then on the vault holds, audits and rotates them. It can make their replacements, unlike -an operator's external key. +Until the broker's provisioner runs, genesis creates those bus accounts with the broker's admin, as the +controller does today; the provisioner adopts them when it starts. Genesis seals everything to the +operator key as today ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)), and when the +vault is installed it **delivers the values to it, recorded as the mesh's own**. That distinction keeps +them rotatable: an operator's value is never replaced, and these are, because the vault can make their +replacements. That is the one time anything but the vault generates a shared secret, and it ends by handing them over. It is also the answer to the objection ADR 0085 had to the vault being the only maker: the @@ -235,10 +247,16 @@ only when it first initialises is marked applied, because a restart would change recipient that reads at start has restarted and passed its health check, where its definition declares one. Delivered and working are shown as different things. -**The remaining window is stated.** An applier whose provisioner stops after applying and before -confirming leaves the recipients that read at start locked out: the old value no longer works, and -they have not been sent the new one. A provisioner is supervised and restarted when it exits, so the -window lasts until that restart. The rotation shows as waiting on that applier throughout, never as done. +**Open: keeping readers from being locked out.** Steps 2 and 3 leave a reader locked out when its +machine is offline after an applier applied, when the secret is a bus account whose owner loses the bus +it would hear the new value on, or when a restarted provisioner can no longer check the old value. +[ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) records the two answers, overlapping +old and new credentials or re-confirming with safeguards, and one is chosen before it is accepted. +Neither changes a consumer module. + +A secret some service reads only when it first initialises cannot be rotated by restarting it. It is +applied by a provisioner, or, where none exists, marked not rotatable by the mesh, and a rotation is +refused rather than reported done. ## Refusing @@ -258,10 +276,11 @@ as waiting, and nothing is delivered until the answer arrives. | mechanism | becomes | |---|---| | provisions read through bindings | a module requirement; its answer is the contract's fields | -| settings on an assignment ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) | operator requirements, addressed to an instance | +| settings on an assignment ([ADR 0046](../../02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md)) | operator requirements on an assignment | | a port the mesh assigns | a host requirement | | machine facts and machine placeholders | host requirements | -| every secret the controller mints: provider credentials, own secrets, broker passwords, root secrets | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)) | +| every secret the controller mints: provider credentials, own secrets, broker passwords, enrolment tokens | a secret the vault makes ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)) | +| root secrets genesis mints and keeps apart | made by genesis once, then delivered to the vault, which holds and rotates them | | a separate command issuing a broker account | a requirement resolved on assignment | | `restart-on` naming a secret's file | a restart the host derives | | paths in resources, mounts, bindings, secrets and received files | host directory requirements, placed by the assignment | @@ -287,11 +306,11 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r is fixed first. *Ends when* nothing outside the vault generates a shared secret after genesis, a lab consumer of analytics receives its site id, and a database credential rotates applier-first, with the consumer restarted by derivation and the rotation confirmed. -3. **Definitions move.** Every catalogue definition is rewritten, adopted and running assignments - placed where their data already is. *Ends when* the list of definitions using an old form is - empty, and the old forms are removed. -4. **Instances.** The instance name and its short form, and everything keyed by it. *Ends when* one - module runs twice on one lab machine with two public names. +3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted + and running assignments placed where their data already is, and each claim becomes a seat the + module can hold, held by the assignment that holds it today. *Ends when* the list of definitions + using an old form is empty, the old forms are removed, and the store module runs on two lab + machines with one holding `mesh-store`. ## How it is checked @@ -305,8 +324,9 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | A host requirement is answered on its own node | A resolution test placing one elsewhere: refused. | | An operator value needs no provider module | A resolution test: a requirement with a default resolves with no module assigned anywhere. | | A secret field reaches a process as a file | The parser refuses a secret field as a container environment value, and accepts it in a file or a declared env-file with its reason. | -| A public name already held is refused | A resolution test: a second instance asking for a public name the first holds is refused, naming the first. | -| Everything keyed by a module is keyed by its instance | A resolution test: two instances of one module on one node get two directories, two containers, two logins and separate settings. | +| A public name already held is refused | A resolution test: a second assignment asking for a public name another holds is refused, naming the holder. | +| A module is assigned at most once to a node | A resolution test: assigning a module to a node that already runs it is refused. | +| A seat is held by an assignment, not a module | A resolution test: the store module on two nodes, one holding `mesh-store`; a second assignment asking to hold it is refused. | | A consumer waits for its provider | A resolution test with a provider that has not answered: shown as waiting, and nothing delivered. | | Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | | Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | @@ -318,11 +338,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r ## Not settled here - The exact spelling of the one form. It must name a requirement and a field and nothing else. -- The layout a node's default root uses beneath it, beyond one directory per instance. +- The layout a node's default root uses beneath it, beyond one directory per assignment. - Whether a module provider's answer can change without the provider being asked, for example a provider moving. The rule so far is that it cannot, and moving is re-resolving. -- **Which seats a module holds.** Today a claim is part of the definition, so moving a seat is a - definition change ([26 — The seats](26-the-seats.md)). By this document's own logic it belongs to - the assignment: a definition says which seats a module *can* hold, and the assignment says which it - *does*. That changes [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), - and is its own decision. +- **How rotation keeps a recipient from being locked out.** Under review: see + [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), rotation. From 805df3f81ed9ed3ab7464473593269e30c7bb594 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 25 Sep 2026 23:50:20 +0200 Subject: [PATCH 20/26] ADR 0113 and to-be 27: rotation overlaps old and new credentials MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decided with the author. A credential is never changed in place: each consumer has two logins, both derived by the mesh, and uses one at a time. An applier adds the new login beside the old through the adapter's existing create, and confirms both work; only then are readers released to the new one and restarted by derivation; only when every reader has confirmed is the old login retired through the existing remove. It closes the three cases review found in applier-first rotation: an offline reader keeps working on the old login until it returns; a bus account's owner keeps its bus until it has moved; a provisioner restarted mid-rotation is still delivered both values. Nobody is ever without a credential that works, which replaces to-be 13's all-or-nothing rule with a stronger one. No consumer module changes. The alternation is the provider loop's. A provider's adapter gains one duty, giving both logins the same rights over the consumer's data — in postgres, membership of one role that owns it. The mesh derives two logins per consumer, both within ADR 0049's limit, which 0113 now names among what it amends. Every rule has a check: overlap, offline reader, bus account, restarted provisioner, equal rights, login length, and confirmation only once the old login is gone. --- .../0113-the-vault-makes-every-secret.md | 94 ++++++++++++------- .../27-a-module-requires-the-mesh-resolves.md | 52 +++++----- 2 files changed, 88 insertions(+), 58 deletions(-) diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index a5b5769..465f8c6 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -153,39 +153,55 @@ the old value. The host derives which recipients read a secret at start from the definition reads it through, so no definition declares a restart for a secret. A provider's per-consumer secrets are applied, never read at start, so the host never restarts a provider for one. -**It is applier-first.** The vault delivers the new value first to the recipients that apply it. Each -applies it, verifies that the new value authenticates and the old one no longer does, and confirms. -**It repeats that confirmation on every reconcile pass until the vault acknowledges it**, so a lost -message costs one pass. Only after every applier has confirmed does the vault release the value to the -recipients that read it at start. +**Old and new overlap: nobody is ever without a credential that works.** A credential is never +changed in place. The new one is added beside the old, every reader moves to it, and only then is the +old one removed. There is one mechanism, the same for every provider: -**Open: keeping readers from being locked out.** Review found three cases this rule does not survive: +1. **The vault makes the new value.** +2. **Each applier adds it beside the old.** A consumer has two logins, both derived by the mesh, and it + uses one at a time. The provider's loop creates the other with the new value, through the adapter's + existing create, and leaves the one in use untouched. It verifies that the new login works and the + old one still does, and confirms. It repeats that confirmation on every reconcile pass until the + vault acknowledges it, so a lost message costs one pass. +3. **Only then is the new login released to the readers.** A reader receives the new login and its + value together. The host restarts it, or recreates a container whose env-file carries it. +4. **Each reader confirms**, by restarting with the new login and passing its health check, where its + definition declares one. +5. **Only when every reader has confirmed is the old login retired.** Each applier removes it, through + the adapter's existing remove, and verifies that it no longer authenticates. -- a reader whose machine is offline when an applier has already applied the new value is locked out - until it returns, where to-be 13 would have refused the rotation and kept the old value working; -- a bus account's owner can be locked out permanently, because the confirmation and the new value - travel over the bus it has just lost; -- a provisioner restarted mid-rotation no longer knows the old value, so it cannot verify that the old - value has stopped working. +**What overlap closes:** -Two answers are recorded, and one must be chosen before this record is accepted. **Overlap:** an -applier keeps the old and new credential valid together until every reader has confirmed the new one, -for instance by alternating between two derived logins with the adapter's existing create and remove, -which needs no change on the consumer's side. Or **re-confirm with safeguards:** a pre-check that every -reader is reachable before any applier starts, plus a special path for bus accounts. +- a reader whose machine is offline keeps the old login, which still works, until it returns and + moves; the rotation shows as waiting on that reader, and nobody is locked out; +- a bus account's owner keeps its old account until it has confirmed the new one over the bus it still + has, so no party can lose the bus it would hear the new value on; +- a provisioner restarted mid-rotation is still delivered both values until the old is retired, so it + can verify either. -**It is confirmed.** A rotation is shown as unconfirmed until every applier has confirmed, and every -recipient that reads at start has restarted with the new value and passed its health check, where its -definition declares one. +**What overlap costs.** + +- **Consumer modules: nothing.** A consumer reads one login at a time and changes it when it restarts. +- **Providers: one duty.** Both of a consumer's logins must have the same rights over its data, + because the consumer's data was written under one login and is read under the other. In postgres, + both are members of one role that owns the data. That is the adapter's part, and the only place + overlap touches provider code. The alternation itself is the provider loop's, so every provider gets + it by using the harness. +- **The mesh:** it derives two logins per consumer, and both must still fit the tightest backend + ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). +- **Secrets with no applier**, such as a module's own secret read only by itself, have no second party + to overlap with. They are delivered and the reader restarted, where their contract allows rotation at + all. | step | who | |---|---| | asks | an operator, or the vault's policy | | makes the value | the vault | -| carries it | the controller, sealed, appliers first | -| applies it | each applier's provisioner, which verifies and confirms on every pass until acknowledged | -| takes it at start | the host, restarting or recreating what reads it | -| confirms it | applier confirmations, then restarts and health checks, shown in `status` | +| adds the new login beside the old | each applier's provisioner, confirming on every pass until acknowledged | +| moves each reader | the host, restarting or recreating what reads the secret | +| confirms each reader | its restart and health check | +| retires the old login | each applier's provisioner, once every reader has confirmed | +| shows progress | `status`: waiting on which applier or reader, never done until the old is retired | ## What this changes in earlier records @@ -202,11 +218,14 @@ On acceptance, each of these is superseded or amended by this record, not edited so 0092's rule that an operator's value is never replaced does not apply to them. - [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker account is created by the broker's provisioner, not the controller. Its scoping stands. +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md) is amended: the mesh derives two + logins per consumer, and both fit the tightest backend. - [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and - [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation is applier-first, - derived and confirmed; the vault is installed as soon as the shared runtime base exists, and genesis - delivers its secrets to it; the vault is the only maker. + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation overlaps old and new + instead of being all-or-nothing, with restarts derived and each step confirmed; the vault is + installed as soon as the shared runtime base exists, and genesis delivers its secrets to it; the vault + is the only maker. - The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at start*, and *reserved provision*, once this record is accepted. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) @@ -219,17 +238,19 @@ On acceptance, each of these is superseded or amended by this record, not edited place, on the same node. A secret can no longer be made while the vault is down. - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. -- The SDK's provider loop gains repeated confirmation of an applied rotation. A credential provider's - adapter is unchanged. A data provider's adapter gains a return value. +- The SDK's provider loop gains the alternation of two logins per consumer and repeated confirmation + of each step. A credential provider's adapter gains one duty, giving both logins the same rights over + the consumer's data. A data provider's adapter gains a return value. No consumer module changes. - The broker's provisioner gains every bus account, and the controller loses five separate places it generates a secret today. - 54 modules move from own secrets to vault requirements. Six provider clients export a password generator nothing uses any more; it is removed, so no module can quietly start minting again. - The installation changes order: the vault is installed as soon as the shared runtime base exists, before any other module built on it. -- **What got harder:** a rotation waits for its appliers, and how a reader is kept from being locked - out while it does is still open (above). A secret some services read only at first start can no - longer be "rotated" by a restart that quietly changes nothing; it is refused instead. +- **What got harder:** a rotation lasts until its slowest reader has moved, so a reader offline for a + week keeps the old login valid for a week. That is shown, and it is the price of never locking anyone + out. A provider briefly holds two logins per consumer. A secret some services read only at first + start can no longer be "rotated" by a restart that quietly changes nothing; it is refused instead. ## How it is checked @@ -246,9 +267,14 @@ On acceptance, each of these is superseded or amended by this record, not edited | Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | | Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | | An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. | -| Rotation is applier-first | A rotation test: readers are not sent the new value until every applier confirms. How lock-out is prevented is open, and its check is written when that is decided. | +| Old and new overlap | A rotation test: after an applier adds the new login, both authenticate; readers are released only after it confirms; the old login is removed only after every reader confirms, and then no longer authenticates. | +| An offline reader is never locked out | A rotation test with one reader's node offline: it keeps authenticating with the old login throughout, the rotation shows waiting on it, and completes when it returns. | +| A bus account's owner keeps the bus | A rotation test on a node agent's bus account: the agent stays connected on the old account until it has confirmed the new one. | +| A restarted provisioner can still verify | A rotation test restarting the applier's provisioner mid-rotation: it is delivered both values and confirms. | +| Both logins have the same rights | A provider test per credential provider: data written under one of a consumer's logins is read and changed under the other. | +| Both logins fit the tightest backend | A controller test: the two derived logins for the longest node and module names fit the limit ADR 0049 sets. | | Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. | -| Rotation is confirmed | A rotation test: an applier confirms only after the new value authenticates and the old one does not, and the rotation shows unconfirmed until every reader restarted and passed its health check. | +| Rotation is confirmed | A rotation test: the rotation shows unconfirmed until every reader has restarted with the new login and passed its health check, and the old login is retired. | | A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | ## References diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index ca648af..4e7473a 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -233,26 +233,32 @@ through a provisioner (a provider creating the login, the broker's provisioner u the store's own provisioner changing its superuser), or *reads it at start*. A secret a service reads only when it first initialises is marked applied, because a restart would change nothing. -1. **The vault makes the new value.** -2. **It goes first to the recipients that apply it.** Each applies it, verifies that the new value - authenticates and the old one no longer does, and confirms. It repeats that confirmation on every - reconcile pass until the vault acknowledges it, so a lost message costs one pass. -3. **Only then is it released to the recipients that read it at start**, such as gitea. -4. **The host restarts every such recipient**, and recreates a container whose env-file carries the - secret. It knows which, because a definition reads a secret only through its requirement, so no - definition declares a restart for a secret. An applying recipient is never restarted for it. - This needs [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - fixed, or a container fed by an env-file keeps the old value. -5. **It is confirmed.** The rotation shows as unconfirmed until every applier has confirmed, and every - recipient that reads at start has restarted and passed its health check, where its definition - declares one. Delivered and working are shown as different things. +**Old and new overlap, so nobody is ever without a credential that works.** A credential is never +changed in place. Each consumer has two logins, both derived by the mesh, and uses one at a time: -**Open: keeping readers from being locked out.** Steps 2 and 3 leave a reader locked out when its -machine is offline after an applier applied, when the secret is a bus account whose owner loses the bus -it would hear the new value on, or when a restarted provisioner can no longer check the old value. -[ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) records the two answers, overlapping -old and new credentials or re-confirming with safeguards, and one is chosen before it is accepted. -Neither changes a consumer module. +1. **The vault makes the new value.** +2. **Each applier adds it beside the old**, as the consumer's other login, through the adapter's + existing create. It verifies that the new login works and the old one still does, and confirms, + repeating that confirmation on every reconcile pass until the vault acknowledges it. +3. **Only then is the new login released to the readers**, such as gitea, login and value together. +4. **The host restarts every such reader**, and recreates a container whose env-file carries the + secret. It knows which, because a definition reads a secret only through its requirement, so no + definition declares a restart for a secret. An applier is never restarted for it. This needs + [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) + fixed, or a container fed by an env-file keeps the old value. +5. **Each reader confirms** by passing its health check with the new login, where its definition + declares one. +6. **Only when every reader has confirmed is the old login retired**, through the adapter's existing + remove, and verified to no longer authenticate. + +So a reader whose machine is offline keeps working on the old login until it returns, a bus account's +owner keeps its bus until it has moved, and a provisioner restarted mid-rotation is still delivered +both values. The rotation shows as waiting on whichever applier or reader has not moved, and is done +only when the old login is gone. + +**No consumer module changes.** A provider's adapter gains one duty: both of a consumer's logins get the +same rights over its data, which in postgres means both belong to one role that owns it. The +alternation itself is the provider loop's. A secret some service reads only when it first initialises cannot be rotated by restarting it. It is applied by a provisioner, or, where none exists, marked not rotatable by the mesh, and a rotation is @@ -304,8 +310,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r mesh carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) is fixed first. *Ends when* nothing outside the vault generates a shared secret after genesis, a lab - consumer of analytics receives its site id, and a database credential rotates applier-first, with - the consumer restarted by derivation and the rotation confirmed. + consumer of analytics receives its site id, and a database credential rotates with old and new + overlapping, the consumer restarted by derivation and the rotation confirmed. 3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is, and each claim becomes a seat the module can hold, held by the assignment that holds it today. *Ends when* the list of definitions @@ -331,7 +337,7 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | | Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | | A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | -| Rotation is applier-first, derived and confirmed | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): readers wait for every applier's repeated confirmation; an applied secret restarts nothing, and one read at start restarts its reader without a declared restart; the rotation shows unconfirmed until the new value authenticates, the old does not, and readers are healthy. | +| Rotation overlaps old and new | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): both logins authenticate while readers move; an offline reader keeps working on the old login; the old login is retired only after every reader confirms; an applied secret restarts nothing, and one read at start restarts its reader without a declared restart. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | @@ -341,5 +347,3 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r - The layout a node's default root uses beneath it, beyond one directory per assignment. - Whether a module provider's answer can change without the provider being asked, for example a provider moving. The rule so far is that it cannot, and moving is re-resolving. -- **How rotation keeps a recipient from being locked out.** Under review: see - [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), rotation. From 942ebe350f7903512eae63e5c21de45aa7cebfb1 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:14:23 +0200 Subject: [PATCH 21/26] Research 016: survey how each provider can rotate a credential Overlap as drafted in 0113 would have deleted consumer data: seven of eight providers name the resource after the login and five drop it on remove. Rotation is now undecided in 0113 and to-be 27, pending the survey. Also: a requirement naming a seat resolves to its holder, a person chooses among remaining candidates at assignment, the controller's secrets are requirements of its definition, genesis seals to the control-node key, and moving the vault or broker is break-glass. --- .../00-overview.md | 51 ++++++ .../01-the-providers.md | 71 ++++++++ .../02-the-readers.md | 42 +++++ .../03-the-options.md | 75 ++++++++ ...s-a-module-assignment-from-a-closed-set.md | 54 +++--- ...e-definition-names-no-node-mesh-or-path.md | 10 +- .../0113-the-vault-makes-every-secret.md | 168 +++++++----------- 03-DESIGN/01-to-be/26-the-seats.md | 34 ++-- .../27-a-module-requires-the-mesh-resolves.md | 116 ++++++------ 9 files changed, 422 insertions(+), 199 deletions(-) create mode 100644 01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md create mode 100644 01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md create mode 100644 01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md create mode 100644 01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md new file mode 100644 index 0000000..83f184d --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md @@ -0,0 +1,51 @@ +--- +status: active +initiated: 2026-09-26 +touches: + - 02-DECISIONS/0113-the-vault-makes-every-secret.md + - 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md + - 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md + - 03-DESIGN/01-to-be/13-credentials-and-their-rotation.md + - 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md + - 04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md +--- + +# 016 — How a credential can be rotated + +**What.** Which rotation mechanisms the mesh's providers can actually support, measured against +their code rather than assumed. Every provider in the catalogue was read: how it names what it +makes for a consumer, what its remove destroys, whether it re-applies a password, whether its +backend can hold two secrets for one login or two logins on one resource, and how its own +administrative credential is set. The consumer side was read too: when a module reads a secret, and +what makes it read a new one. + +**Why.** [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), as first drafted, +chose *overlap*: add a second login beside the first, move every reader, then remove the old one, +"through the adapter's existing create and remove", with "no consumer changes". A review showed that +claim false. In most providers the consumer's data is named after its login, and remove drops the data +with the login. Overlap as written would have deleted every consumer's database on its first +rotation. The mechanism has to be chosen on what the providers do. + +**What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md), +which both mark it undecided and point here. The identity budget in +[ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer +gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) +describes. + +**Documents.** + +- [01 — The providers](01-the-providers.md): the survey, one row per provider, and what it shows. +- [02 — The readers](02-the-readers.md): how a secret reaches a running process, and what already + recreates it. +- [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a + recommendation. + +**Finding, in one paragraph.** All eight credential providers already re-apply a consumer's password +in place on every create, and the controller's `rotate` command relies on that. It is a working +rotation with a stated window. Seven of the eight name the consumer's resource after its login, +and five drop the resource when they remove the login, so a second login is impossible without +changing the adapter. Only one backend holds two passwords on one login. But every backend can grant +two logins the same rights over one resource. So overlap is possible everywhere, but only after each +adapter separates *the consumer's resource* from *the login that reaches it*. Administrative +credentials are a different case. They have one party, a fixed name, and in three providers they are +taken only at first initialisation. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md new file mode 100644 index 0000000..e537ffa --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md @@ -0,0 +1,71 @@ +# 01 — The providers + +Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`), +the client functions they call, and each definition's own credentials. The provisioner harness in +`mesh-sdk` calls `create` for a consumer when its contribution appears or changes, and after the +provisioner restarts, and `remove` when the contribution goes. Its record of what was applied is +kept in memory. + +## The credential providers + +`login` is the consumer's derived identity, which the adapter receives as `as` +([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)). + +| provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource | +|---|---|---|---|---|---| +| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, both as members of one role that owns the database. Not done today | +| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. Not done today | +| mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today | +| redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login | +| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key and the bucket | yes, by removing the access key and adding it again | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters | +| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host | +| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. An MQTT client identifier still has to be unique per connection | +| gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team | + +## The other providers + +| provider | answers with | credential | +|---|---|---| +| umami | a website, found by its public name | none. The site id it makes has no way back to the consumer today | +| cloudflare-dns | a public name derived from `login` | none handed to the consumer; its own API token is an operator value | +| showcase | a route | none | +| mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today | + +## Each provider's own administrative credential + +| provider | identity | how the backend takes it | +|---|---|---| +| postgres | a fixed superuser | from a file **only at first initialisation**. Afterwards the file is read by the provisioner to connect, and changing it changes nothing in the database | +| mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception | +| mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty | +| redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts | +| minio | a fixed root user | from a file, read when the server starts | +| lavinmq | a fixed admin name | from the module's own secret, which its provisioner connects with | +| mosquitto | a fixed admin client | from the module's own secret, which its provisioner connects with; the broker's dynamic-security file holds it | + +Every provider module also has its own bus account, an own secret, read at start. + +## What the table shows + +1. **Every credential provider already rotates in place.** All eight re-apply the password on the + same login each time `create` runs. The controller's `rotate` command relies on that: it replaces + the credential in the inventory and sends both ends in one push. Its own comments state the window, + between the provider applying and the consumer restarting, in which the consumer cannot + authenticate. +2. **Seven of eight name the consumer's resource after its login.** Only gitea separates them, + because an organisation owns the packages. A second login therefore has no resource of its own to + reach, and cannot share the first one's without the adapter granting it. +3. **Five of eight destroy the resource when they remove the login.** These are postgres, mssql, mongodb, + minio and lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call. + This is the fault the first overlap draft would have triggered. +4. **Only redis holds two passwords on one login**, and gitea through tokens rather than its + password. A rotation built on two secrets per login would work for one provider out of eight. +5. **Every backend can give two logins the same rights over one resource.** Group roles in postgres, + database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared + policy in minio, a shared key prefix in redis, a shared team in gitea. No adapter does it today. +6. **The administrative credentials have one party and a fixed name.** The provider module is both the + one that applies the credential and the only one that reads it. In postgres, mongodb and mssql, a new + value only takes effect through a command run with the old value. A restart changes nothing. +7. **A consumer's identity is already the resource's name.** The login is derived from the + assignment, which is a module on a node, so the current login and "the consumer" are the same + string today. A second login would need a new name. The resource can keep the one it has. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md new file mode 100644 index 0000000..7504391 --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md @@ -0,0 +1,42 @@ +# 02 — The readers + +How a secret reaches a running process, and what makes the process take a new one. + +## No module watches a secret + +A search of every module's code in the catalogue found no file watching of any kind, and no +re-reading of a secret while running. **Every reader reads a secret when it starts.** There is no +consumer that takes a new value live, so every rotation that changes what a consumer presents ends in +the consumer restarting. + +## The host already recreates what read a changed file + +The node host records, for every long-running container, the digest of each file it read when it +was created: its env-files, and every file bind-mounted into it directly. When a digest changes, the +host recreates the container, even though its spec is otherwise unchanged. This is the fix for +[issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md). +It is on the host's main branch, while the issue is still recorded as located, not fixed. + +Two cases are deliberately left out and need `restart-on` in the definition: + +- a file read out of a **mounted directory**, because the host cannot know whether the service reads + it once or watches it (a route proxy re-reads its routes live; a provisioner polls what it receives); +- a **process** rather than a container. + +## What the catalogue does with it + +22 definitions declare a secret they receive. In 16 of them it reaches the service through a +rendered file, usually an env-file. That case the host already covers. 14 declare `restart-on` for +something. Whether each of the 22 is fully covered depends on how its secret travels: through an +env-file or a direct mount, which the host covers, or through a directory or into a process, which +needs `restart-on`. **That was not classified module by module.** It is the check to run before a +rotation mechanism relies on it. + +## What this means for rotation + +- The *read at start* half of 0113's recipient model is already true, and mostly already handled by + the host. The restart is derived from the files a container reads, not declared per secret. +- Any mechanism, in place or overlapping, ends with the reader being recreated. What differs is + whether the credential it held until then still works. +- For a single-party secret, a module's own, the reader is also the only holder. There is nobody to + overlap with, and delivering the new file recreates the reader. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md new file mode 100644 index 0000000..e668691 --- /dev/null +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md @@ -0,0 +1,75 @@ +# 03 — The options + +Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-readers.md). + +## A. In place, as today + +The vault makes a new value. Every applier re-applies it on the same login, which all eight +providers already do. Every reader is recreated by the host. + +- **Works with:** every provider, unchanged. It is what `rotate` does now. +- **Costs:** a window per consumer, from the provider applying to the consumer being recreated. They + are on different machines, and nothing orders them. A reader whose machine is unreachable from the + mesh but still reaches its provider stays locked out until the mesh reaches it again. +- **Admin credentials:** the natural form. The provider module is the only party, and it has to + apply the new value with the old one anyway (finding 6). + +## B. Two secrets on one login + +The applier adds the new password beside the old one, readers move, and the old one is removed. + +- **Works with:** redis natively, and gitea through tokens. **Not** with the other six, whose backends + hold one password per login (finding 4). +- **Verdict:** not a mechanism, a special case. Using it where it exists and something else + elsewhere is the "this way or that way" the design is trying to remove. + +## C. Two logins per consumer, over one resource + +The consumer has two logins derived by the mesh, and uses one at a time. The applier creates the other +with the new value and grants it the same rights over the consumer's resource. Readers move to it, +and then the old login is retired, which removes the login only, never the resource. + +- **Works with:** every backend (finding 5), **after** each adapter changes: + - the resource is named after the consumer, not the login. Today the two are the same string (finding + 7), so existing resources keep their names, the current login stays one of the two, and only the + second is new; + - both logins get the same rights, through a group role or its equivalent; + - *retire a login* and *remove the consumer* become two operations. Today they are one call, and + in five providers that call destroys data (finding 3). This is the whole of the danger, and it has + to be split, whatever else is chosen. +- **Costs:** + - seven adapters change; + - the harness learns the alternation and a confirmation per step; + - the second login's name must fit the tightest backend. That is 20 characters for a minio access + key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and + a suffix spends part of it; + - an MQTT client identifier stays unique per connection, so mosquitto needs the client id kept apart + from the login. +- **Gains:** no window. A reader that cannot be reached keeps a working login until it can. +- **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is + no second party to overlap with. + +## Independent of the choice + +- **Split remove.** Retiring a credential must never be able to destroy a consumer's data. That holds + under A too, because A's remove is the same call. A remove that drops a database should be a + separate, explicit operation, which [ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md) + already implies: data outlives the declaration. +- **Admin credentials are applied by their own provider**, using the old value. That happens in place + whichever mechanism consumers get. Where a backend takes the value only at first initialisation, the + file alone changes nothing, and rotation needs the provider's provisioner to run the change. +- **Classify the 22 readers** ([02](02-the-readers.md)) before relying on derived restarts. + +## Recommendation + +- **Consumer credentials: C**, because it is the only mechanism every backend supports, and it closes + the window instead of shortening it. Its prerequisite, separating the resource from the login and + retiring a login from removing a consumer, is worth doing on its own, because it removes a + data-loss path that exists today. +- **Single-party secrets (admin credentials, a module's own): A.** In place, applied by the provider + that holds them. +- **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is + not replaced by a mechanism the providers cannot yet carry. + +This is two mechanisms, split by a property of the secret rather than by provider: whether it has one +party or two. Every provider is treated the same way for the same kind of secret. diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index 6bfbbb9..fdd08b4 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -74,19 +74,23 @@ node's settings for it, and what it serves. Holdings are not stored separately. assignment, and a second record of the same fact would be a second thing to disagree with the first. **Holding a seat may deliver a provision.** A seat that delivers a provision may only be held by an -assignment of a module that provides it, at the seat's scope. A requirement for that provision -resolves, in order, to: +assignment of a module that provides it, at the seat's scope. -1. a provider the consumer's node was **pinned** to, a consumer coupled to one provider's contents; -2. **the holder of the seat**, **even when another provider runs on the consumer's own node**; -3. otherwise refused, naming the unheld seat. +**A requirement may name a seat, and then the seat's holder answers it.** Naming the seat asks for +*the mesh's* one, not for whichever provider is nearest, so the holder answers **even when another +provider runs on the consumer's own node**, and nothing is asked of anyone. Unheld, the requirement is +refused, naming the seat. A builder asks for the mesh's npm registry this way, and is served by the +holder of `npm-package-registry` wherever it runs, with no pin on any machine. -**Co-location does not apply to a provision a seat delivers.** A seat exists to say *which one is the -mesh's*, and co-location answering first would let any second provider on a consumer's machine take -over silently for that consumer. That is the failure [issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) -names for the vault. This keeps [ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with -several answers is never guessed: the seat is the choice made once, mesh-wide, by assigning the holder, -instead of once per consumer by pinning. +**A requirement that names no seat resolves as [ADR 0084](0084-which-provider-serves-a-consumer.md) +has it**: a pin, then the provider on the consumer's own node, then the only provider. Where several +remain and none is local, **a person chooses when the module is assigned**. Assignment lists the +candidates, with the holder of a seat that delivers the provision suggested first, and records the +answer on the assignment as its pin. Without an answer the module is not assigned. This keeps +[ADR 0009](0009-modules-and-the-graph.md)'s rule that a requirement with several answers is never +guessed. The choice is made either by the requirement naming the seat, or by a person at assignment, +and never silently by what happens to run nearby. That is the failure +[issue 106](../04-ISSUES/106-the-vault-claims-no-seat/00-report.md) names for the vault. **A seat delivers a provision only where the mesh has one answer for everyone.** The artifact store, the npm registry, git and the vault are each one per mesh by their own records, so their seats @@ -95,13 +99,15 @@ deliver them. **The foundation's seats deliver nothing.** `mesh-controller`, `mesh-store` and `mesh-broker` name which assignment the mesh *itself* uses: the controller, the store holding its records, the broker carrying its bus. The store and broker modules may run on other nodes too, and a database or `amqp` -consumer is served by co-location from whichever runs on its own node, the seat's holder included. -Were `mesh-store` to deliver, every database consumer on every node would be sent to one machine. +consumer that names no seat is served by co-location from whichever runs on its own node, the seat's +holder included. Were `mesh-store` to deliver, a consumer could name it and be sent to the store the +mesh keeps its own records in. That is not a store for consumers. **A seat may reserve its provision.** Where a second provider would break the reason the provision exists, only an assignment holding the seat may provide it at all: the parser refuses a definition that provides it without being able to hold the seat, resolution refuses an assignment providing it -without holding the seat, and a pin cannot choose anyone else. `secret` is the one reserved provision. +without holding the seat, and a pin cannot choose anyone else: a requirement for it always names the seat. `secret` is the one +reserved provision. The vault is one per mesh because a second *"would be a second place to lose"* ([ADR 0085](0085-a-secret-is-a-provision.md), as amended), and a second `secret` provider is exactly that, whether a pin chose it or not. @@ -150,9 +156,13 @@ On acceptance, each of these is amended by this record, not edited: - [ADR 0009](0009-modules-and-the-graph.md): a claim in a definition says a module *can* hold a seat; the assignment says it does. -- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md): "a mesh runs one postgres - and one lavinmq" becomes one holder of `mesh-store` and one of `mesh-broker`. The store and broker - modules may run on other nodes. +- [ADR 0079](0079-the-foundation-seats-are-named-after-their-servers.md) and + [ADR 0078](0078-the-store-and-broker-are-modules.md): "a mesh runs one postgres and one + lavinmq" becomes one holder of `mesh-store` and one of `mesh-broker`. The store and broker modules may + run on other nodes. +- [ADR 0084](0084-which-provider-serves-a-consumer.md): a requirement may name a seat, which its holder + answers; and where several providers remain and none is local, the choice is asked when the module is + assigned and recorded as a pin, rather than refused until someone pins it. ## Consequences @@ -163,8 +173,9 @@ On acceptance, each of these is amended by this record, not edited: - Manifest validation refuses an unknown seat, a seat named at the wrong scope, and a delivering seat named by a module that does not provide the provision. Resolution refuses a second holder, and an assignment holding a seat its module cannot hold. -- Resolution prefers the seat's holder for a provision it delivers, after a pin. A provider record - gains the module it came from. +- Resolution answers a requirement naming a seat with its holder. Assignment asks a person where + several providers remain, suggesting the seat's holder first, and records the answer as a pin. A + provider record gains the module it came from. - A `seats` command lists the set with each seat's holder, derived from assignments. - **What got harder:** a module wanting a new singular role can no longer invent a name. It needs a record. And an assignment has one more thing to say. Both are the point. @@ -179,8 +190,9 @@ On acceptance, each of these is amended by this record, not edited: | A seat outside the set is refused | Manifest-validation tests for an unknown seat, the wrong scope, and a delivering seat named by a module that does not provide it. | | Every module in use names a seat in the set | A controller test parses every catalogue manifest and fails on any refused seat. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | | A seat is held by one assignment, not by a module | A resolution test: the store module assigned to two nodes resolves, with one assignment holding `mesh-store`; a second assignment asking to hold it is refused. | -| The holder answers for a provision its seat delivers | Resolution tests: two providers with the seat held; a pin overriding the seat; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | -| The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`. | +| A requirement naming a seat is answered by its holder | Resolution tests: two providers with the seat held; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | +| Several providers and none local is a person's choice | An assignment test: the candidates are listed with the delivering seat's holder first; the answer is recorded as a pin; with no answer the module is not assigned. | +| The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`; a requirement naming `mesh-store` is refused, because it delivers nothing. | | A reserved provision has no other provider | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`; resolution refuses an assignment providing it without holding the seat, and a pin on a `secret` requirement. | ## References diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index 8d68e3e..e247c3e 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -71,8 +71,9 @@ unresolved requirement and what could answer it, all at once. | **the operator, through the assignment** | a value a person chooses that is not secret: a public name, a greeting, a number of workers | settings, carried literals | A module provider is chosen as [ADR 0084](0084-which-provider-serves-a-consumer.md) and -[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say: a pin, then the holder of a -seat that delivers the provision; for a provision no seat delivers, co-location and then the only one. +[ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) say. A requirement naming a seat is +answered by its holder. Otherwise it is a pin, then co-location, then the only provider, and where +several remain, a person chooses at assignment and the choice is recorded as a pin. A host provider is always the module's own node, because a host path or a port means nothing on any other. An operator value is the assignment's, or the requirement's default, or unresolved. @@ -133,6 +134,11 @@ On acceptance, each of these is amended by a record of its own, not edited: path moves from the definition to the assignment. - [ADR 0091](0091-a-mount-is-declared-three-ways.md): a mount's host side is a resolved requirement, checked as resolved rather than as a path the definition declares. +- [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): the step that builds and runs a + store module as a database provider, beside the foundation's store on the same node, would run the + store module twice on one node. The adopted store module ([ADR 0078](0078-the-store-and-broker-are-modules.md)) + holds `mesh-store` and serves that node's database consumers by co-location, so there is no second + one. - The [glossary](../00-META/glossary.md): *provision* widens from "a service one module provides" to a requirement answered by any of the four providers, and *requirement* and *contract* are added. None of it lands while this record is only proposed, because the glossary is the authority on the words diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index 465f8c6..cc70c48 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -36,9 +36,9 @@ it rather than step around it. **Rotation has gaps.** To-be 13 makes rotation one command, all-or-nothing, with a stated window in which a consumer cannot authenticate. A consumer restarts only if its definition remembered to say so; -a container fed by an env-file is not recreated when that file changes -([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)); -and some secrets are read only when a service first initialises, where a restart changes nothing. +a container fed by an env-file was not recreated when that file changed +([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md), +since fixed in the host); and some secrets are read only when a service first initialises, where a restart changes nothing. **And providers cannot answer with data.** [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) left *"delivering provider-generated data back to a consumer"* to a separate decision. The analytics @@ -93,11 +93,12 @@ it first means reordering the whole installation and giving the vault a second w The vault cannot make that value. The module that received it delivers it to the vault, which keeps it and provides it like any other; rotating it means asking the backend again. -**Parties that are not modules take the same path.** The controller's own store login and bus account, -and each node agent's bus account, have no definition to require them. The controller asks the vault -on its own behalf, or a node's, and the vault answers the way it answers any requirement: made by the -vault, sealed to the recipient, carried by the mesh. The requirement is not written in a definition, -because the controller and a node agent are the mesh itself, but it is answered no differently. +**The controller is a module, and takes the same path.** Its store logins (inventory, identity and +licences) and its bus accounts are own secrets of its definition today, and become `secret` requirements +of that definition like any module's. **A node's host is the one party with no definition.** Its bus +account is a requirement the mesh makes for each enrolled node, answered by the vault, sealed to that +node and carried like any other. It is the only requirement not written in a definition, because the +host is what runs definitions. **Only the vault may provide `secret`.** An assignment providing it must hold the `mesh-vault` seat. The parser refuses a definition that provides it and cannot hold the seat, and a pin cannot route a @@ -115,18 +116,25 @@ as that base exists**, before any other module built on it, and everything neede generated by genesis: - the store's superuser, and the broker's admin in the hashed form the broker needs; -- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder, - the broker's own provisioner and the vault; -- the controller's store login, and the first enrolment token. +- the bus accounts of the temporary and permanent controller (its account and the broker-management + login), the control-node's host, the builder, the broker's own provisioner and the vault; +- the controller's three store logins (inventory, identity and licences), and the first enrolment + token. Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's -admin, as the controller does today. Genesis seals all of it to the operator key, and when the vault is -installed it **delivers the values to the vault, recorded as the mesh's own**, not as an operator's. +admin, as the controller does today. Genesis seals all of it to the control-node's key, and when the vault +is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not as an +operator's. Nobody has to be present for it. That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), and these are, because the vault can make their replacements. The broker's provisioner then adopts the accounts genesis created. From then on the vault makes every shared secret, and genesis has made its last one. +**Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker` +seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the +values it needs are delivered, not made by a vault that is not there. That is a break-glass procedure, +stated and checked, never an ordinary assignment. + **A provider makes resources and data, and the mesh carries data back.** A provider's adapter may answer with its contract's non-secret fields: a site id, a registered name. The mesh delivers them to the consumer as resolved values. Who a consumer is stays the mesh's: a provider makes what a consumer @@ -134,74 +142,44 @@ is *given*, never what it is *called* ([ADR 0049](0049-a-consumers-identity-fits ### Rotation -**It is asked of the vault**, by an operator or by the vault's policy, such as a maximum age in the -secret's contract. A delivered value the vault cannot replace, such as an external API key, is not -rotated by the vault: rotating it means an operator delivering a new one. +**Who asks and who makes are decided here; the mechanism is not.** A rotation is asked of the vault, +by an operator or by the vault's policy, such as a maximum age in the requirement's contract, and the +vault makes the new value. A delivered value the vault cannot replace, such as an external API key, is +not rotated by the vault: rotating it means an operator delivering a new one. A secret a backend +issued is rotated by the module that holds the backend asking it again and delivering the new value to +the vault. -**A secret's contract says how each recipient takes a new value:** +**Each recipient takes a new value one of two ways, marked on its requirement:** | recipient takes it by | example | what happens on rotation | |---|---|---| -| **applying** it | a provider creating the login; the broker's provisioner updating an account; the store's own provisioner changing its superuser | its provisioner applies the new value; it is never restarted for it | -| **reading it at start** | a consumer reading its password when it starts | the host restarts it, or recreates a container whose env-file carries it | +| **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it | +| **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed | -A secret read only when a service first initialises cannot be rotated by a restart. Its contract marks -it applied, and a provisioner makes the change. Where no provisioner exists to make it, such as a -module's own bootstrap password, the contract marks the secret **not rotatable by the mesh**, and a -rotation request is refused, saying why, rather than restarting a service that would carry on with -the old value. The host derives which recipients read a secret at start from the requirement their -definition reads it through, so no definition declares a restart for a secret. A provider's -per-consumer secrets are applied, never read at start, so the host never restarts a provider for one. +The marking is on each requirement, not on the secret, because one secret has recipients of both kinds. +Every module in the catalogue reads its secrets at start, and none watches them +([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a +backend takes only when it first initialises is marked applied, and its provider's provisioner makes +the change using the old value. Where no provisioner can make it, the requirement is marked **not +rotatable by the mesh**, and a rotation request is refused, saying why, rather than restarting a service +that would carry on with the old value. -**Old and new overlap: nobody is ever without a credential that works.** A credential is never -changed in place. The new one is added beside the old, every reader moves to it, and only then is the -old one removed. There is one mechanism, the same for every provider: +**How old and new change over is not decided here.** Three mechanisms were measured against every +provider's code in [research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): +in place, as the controller's `rotate` does today; two secrets on one login; and two logins over one +resource. Its findings bound the choice: -1. **The vault makes the new value.** -2. **Each applier adds it beside the old.** A consumer has two logins, both derived by the mesh, and it - uses one at a time. The provider's loop creates the other with the new value, through the adapter's - existing create, and leaves the one in use untouched. It verifies that the new login works and the - old one still does, and confirms. It repeats that confirmation on every reconcile pass until the - vault acknowledges it, so a lost message costs one pass. -3. **Only then is the new login released to the readers.** A reader receives the new login and its - value together. The host restarts it, or recreates a container whose env-file carries it. -4. **Each reader confirms**, by restarting with the new login and passing its health check, where its - definition declares one. -5. **Only when every reader has confirmed is the old login retired.** Each applier removes it, through - the adapter's existing remove, and verifies that it no longer authenticates. +- every credential provider already re-applies a password in place, so today's rotation works, with a + window in which a consumer cannot authenticate; +- seven of eight name the consumer's resource after its login, and five destroy the resource when they + remove the login. **No mechanism may retire a login through today's remove**, because in those five + it deletes the consumer's data; +- only one backend holds two passwords on one login; +- every backend can give two logins the same rights over one resource, once the adapter separates the + resource from the login. -**What overlap closes:** - -- a reader whose machine is offline keeps the old login, which still works, until it returns and - moves; the rotation shows as waiting on that reader, and nobody is locked out; -- a bus account's owner keeps its old account until it has confirmed the new one over the bus it still - has, so no party can lose the bus it would hear the new value on; -- a provisioner restarted mid-rotation is still delivered both values until the old is retired, so it - can verify either. - -**What overlap costs.** - -- **Consumer modules: nothing.** A consumer reads one login at a time and changes it when it restarts. -- **Providers: one duty.** Both of a consumer's logins must have the same rights over its data, - because the consumer's data was written under one login and is read under the other. In postgres, - both are members of one role that owns the data. That is the adapter's part, and the only place - overlap touches provider code. The alternation itself is the provider loop's, so every provider gets - it by using the harness. -- **The mesh:** it derives two logins per consumer, and both must still fit the tightest backend - ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)). -- **Secrets with no applier**, such as a module's own secret read only by itself, have no second party - to overlap with. They are delivered and the reader restarted, where their contract allows rotation at - all. - -| step | who | -|---|---| -| asks | an operator, or the vault's policy | -| makes the value | the vault | -| adds the new login beside the old | each applier's provisioner, confirming on every pass until acknowledged | -| moves each reader | the host, restarting or recreating what reads the secret | -| confirms each reader | its restart and health check | -| retires the old login | each applier's provisioner, once every reader has confirmed | -| shows progress | `status`: waiting on which applier or reader, never done until the old is retired | +The mechanism is decided in its own record, on those facts. Until then rotation stays as the +controller implements it, in place, with its window stated. ## What this changes in earlier records @@ -218,19 +196,17 @@ On acceptance, each of these is superseded or amended by this record, not edited so 0092's rule that an operator's value is never replaced does not apply to them. - [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) is amended: a broker account is created by the broker's provisioner, not the controller. Its scoping stands. -- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md) is amended: the mesh derives two - logins per consumer, and both fit the tightest backend. - [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and - [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: rotation overlaps old and new - instead of being all-or-nothing, with restarts derived and each step confirmed; the vault is - installed as soon as the shared runtime base exists, and genesis delivers its secrets to it; the vault - is the only maker. + [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each + requirement says whether its recipient applies it or reads it at start, while the changeover + mechanism stays as to-be 13 describes until its own record; the vault is installed as soon as the + shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker. - The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at start*, and *reserved provision*, once this record is accepted. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - becomes a prerequisite: rotation cannot be trusted while a changed env-file leaves a container on - its old value. + is a prerequisite, and its fix is in the host: a container is recreated when a file it read at + creation changes. The issue is to be recorded as fixed, and derived restarts rest on it. ## Consequences @@ -238,43 +214,37 @@ On acceptance, each of these is superseded or amended by this record, not edited place, on the same node. A secret can no longer be made while the vault is down. - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. -- The SDK's provider loop gains the alternation of two logins per consumer and repeated confirmation - of each step. A credential provider's adapter gains one duty, giving both logins the same rights over - the consumer's data. A data provider's adapter gains a return value. No consumer module changes. +- A data provider's adapter gains a return value. What a credential provider's adapter must change for + rotation is decided with the mechanism ([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md)). - The broker's provisioner gains every bus account, and the controller loses five separate places it generates a secret today. - 54 modules move from own secrets to vault requirements. Six provider clients export a password generator nothing uses any more; it is removed, so no module can quietly start minting again. - The installation changes order: the vault is installed as soon as the shared runtime base exists, before any other module built on it. -- **What got harder:** a rotation lasts until its slowest reader has moved, so a reader offline for a - week keeps the old login valid for a week. That is shown, and it is the price of never locking anyone - out. A provider briefly holds two logins per consumer. A secret some services read only at first - start can no longer be "rotated" by a restart that quietly changes nothing; it is refused instead. +- **What got harder:** a secret some services read only at first start can no longer be "rotated" by + a restart that quietly changes nothing; it is refused instead, or applied by its provisioner. And + moving the vault or the broker is a procedure, not an assignment. ## How it is checked | Rule | Checked by | |---|---| -| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls, with none exempt. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. | +| Only the vault generates a shared secret after genesis | A controller test: no code path generates a shared secret. A catalogue test: no module's code generates one, found by scanning for generation calls. Exempt are the vault itself, and randomness that is not a secret any other party holds, such as a password hash's salt, each named in a declared list. An installer test: genesis generates exactly the list above and delivers it to the vault, recorded as the mesh's own. | | A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. | | Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | -| Parties that are not modules take the same path | Controller tests: its own store login, its bus account and a node agent's bus account are each made by the vault and delivered sealed; an enrolment token reaches the controller only as what verifies it. | +| The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. | +| Genesis's values reach the vault unattended | An installer test: genesis's values are sealed to the control-node's key and delivered by the controller when the vault is installed, with no operator step. | +| Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. | | A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. | -| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. | +| A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. | +| A rotation never destroys a consumer's data | A provider test per credential provider: rotating a consumer's credential leaves its resource and data intact. It fails today for no provider, because rotation is in place; it guards whichever mechanism replaces it. | | A provider's per-consumer secret comes from the vault | A resolution test: a consumer requiring a database expands to a secret requirement for it, answered by the vault and delivered to both recipients. | | Values are carried sealed | A controller test: each recipient's copy opens with that recipient's node key and no other; neither the controller nor a message on the broker can open one. | | Own secrets are retired | A catalogue test: no definition declares an own secret, with a declared list of exceptions that shrinks to empty. | | Bus accounts come from the broker's provisioner | A resolution test: assigning a module that speaks on the bus yields its account, created by the broker's provisioner with no separate command. | | An operator's value is never rotated by the vault, and genesis's values are | Vault tests: a rotation request on an operator's external key is refused, naming the operator; the same request on a value genesis delivered makes a replacement. | -| Old and new overlap | A rotation test: after an applier adds the new login, both authenticate; readers are released only after it confirms; the old login is removed only after every reader confirms, and then no longer authenticates. | -| An offline reader is never locked out | A rotation test with one reader's node offline: it keeps authenticating with the old login throughout, the rotation shows waiting on it, and completes when it returns. | -| A bus account's owner keeps the bus | A rotation test on a node agent's bus account: the agent stays connected on the old account until it has confirmed the new one. | -| A restarted provisioner can still verify | A rotation test restarting the applier's provisioner mid-rotation: it is delivered both values and confirms. | -| Both logins have the same rights | A provider test per credential provider: data written under one of a consumer's logins is read and changed under the other. | -| Both logins fit the tightest backend | A controller test: the two derived logins for the longest node and module names fit the limit ADR 0049 sets. | -| Restarts are derived from how a secret is read | A host test: a secret read at start restarts its reader, and recreates a container whose env-file carries it; an applied secret restarts nothing. | -| Rotation is confirmed | A rotation test: the rotation shows unconfirmed until every reader has restarted with the new login and passed its health check, and the old login is retired. | +| Restarts are derived from how a secret is read | A host test: a secret read at start recreates the container that read it at creation, through an env-file or a direct mount; an applied secret restarts nothing. A catalogue test: a secret that reaches a process, or a file in a mounted directory, has `restart-on` naming it. | | A provider answers data back | A lab test with a consumer requiring analytics: the provider's site id reaches it as a resolved value. | ## References diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index cfb9a9d..28adb35 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -8,7 +8,7 @@ code: - mesh-controller cmd/mesh-controller/source.go - mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql - mesh-catalog modules/gitea/module.json -updated: 2026-09-25 +updated: 2026-09-26 decisions: - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -78,7 +78,8 @@ merges. controller, the store holding its records, the broker carrying its bus. **They route no consumer.** The store and broker modules may run on other nodes too. A database or `amqp` consumer is served by co-location, from whichever runs on its own node, the seat's holder included -([23 — Choosing a provider](23-choosing-a-provider.md)). +([23 — Choosing a provider](23-choosing-a-provider.md)). A requirement cannot name one of them, +because they deliver nothing. ## A seat that delivers a provision @@ -86,19 +87,18 @@ co-location, from whichever runs on its own node, the seat's holder included the npm registry, git and the vault are each one per mesh by decision. A seat that delivers a provision may only be held by an assignment of a module that provides it, at the seat's scope. -**Its holder answers for that provision.** A requirement for it resolves, in order, to: +**A requirement may name the seat, and then its holder answers.** Naming the seat asks for *the +mesh's* one, so the holder answers **even when another provider runs on the consumer's own machine**, +and nobody is asked anything. With the seat unheld, the requirement is refused, naming the seat. A +second provider can run beside the holder and harm nothing. A forge assignment holds +`npm-package-registry`, and an npm proxy may provide the same provision on another machine. A builder +that names the seat is still served by the forge, without anybody pinning it. -1. the provider the consumer's node was pinned to, because a consumer coupled to one provider's - contents has said so ([23 — Choosing a provider](23-choosing-a-provider.md)); -2. the holder of the seat, **even when another provider runs on the consumer's own machine**; -3. otherwise nothing, and the requirement is refused, naming the unheld seat. - -Co-location, which answers first for every other provision, does not apply here: a seat says which -one is the mesh's, and co-location answering first would let any second provider on a consumer's -machine take over for that consumer, silently. So a second provider can run beside the holder and -harm nothing. A forge assignment holds `npm-package-registry`. An npm proxy may provide the same -provision on another machine, and a module requiring an npm registry is still served by the forge, -without anybody pinning it. +**A requirement that names no seat resolves as any other**: a pin, the provider on the consumer's own +machine, the only provider. If several remain and none is local, a person chooses when the module is +assigned. The candidates are listed with the seat's holder suggested first, and the answer is recorded +as the assignment's pin ([27](27-a-module-requires-the-mesh-resolves.md)). Nothing is guessed, and +nothing changes silently because a second provider happened to appear nearby. **Moving the role is changing which assignment holds the seat.** No definition changes and nothing is unassigned: the forge keeps running, and keeps holding `git`, when its npm role moves. A module can @@ -106,9 +106,9 @@ take the role only if its definition says it can hold the seat. **The vault's provision is reserved.** Only an assignment holding `mesh-vault` may provide `secret` at all: a definition providing it that cannot hold the seat is refused, an assignment providing it without -holding the seat is refused, and a pin cannot choose another provider, because there is none. A second -provider of secrets would be a second place secrets live, which is what the vault being one per mesh -exists to prevent. +holding the seat is refused, and a `secret` requirement always names the seat, because there is no +other provider. A second provider of secrets would be a second place secrets live, which is what the +vault being one per mesh exists to prevent. **What a consumer receives is what it required**, the same as for any provision: where the provider answers, what it serves, and a credential. A consumer never reads the seat directly. The one exception diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index 4e7473a..10d1ac9 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -2,7 +2,7 @@ layer: to-be status: proposed code: [] -updated: 2026-09-25 +updated: 2026-09-26 decisions: - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - 02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -59,15 +59,20 @@ can come from and a reviewer has to know every one. Which module answers, in order: -1. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's +1. **the holder of the seat the requirement names.** A requirement may name a seat instead of leaving + the provider open. It asks for *the mesh's* one, and the mesh answers with whichever assignment + holds that seat, with nothing asked of anyone. Unheld, the requirement is refused, naming the seat + ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), + [26 — The seats](26-the-seats.md)). A `secret` requirement always names `mesh-vault`, because + that provision is reserved; +2. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's contents ([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md)); -2. **the holder of a seat** that delivers the provision, where one does. Co-location does not apply - to these: the seat is the mesh's one answer for everyone ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), - [26 — The seats](26-the-seats.md)); -3. for a provision no seat delivers, **the provider on the consumer's own node**; -4. for a provision no seat delivers, **the only provider** in the mesh; -5. otherwise **refused**: naming the unheld seat, for a provision a seat delivers, even when exactly - one provider exists; naming the candidates otherwise. +3. **the provider on the consumer's own node**; +4. **the only provider** in the mesh; +5. otherwise **a person chooses, at assignment**. Assigning the module lists the candidates, with the + holder of a seat that delivers the provision suggested first, and the answer is recorded on the + assignment as its pin. Without an answer the module is not assigned, and the refusal names the + candidates. Nothing is ever guessed ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)). ### Secrets: provisioning all the way down @@ -103,10 +108,10 @@ Every other shared secret takes the same path: - a secret a backend issues itself, such as a forge's API token, which the module that received it delivers to the vault. -**Parties that are not modules take the same path too.** The controller's own store login and bus -account, and each node agent's bus account, have no definition to require them, because the controller -and a node agent are the mesh itself. The controller asks the vault on its own behalf or a node's, and -the answer is made, sealed and carried exactly as for a module. +**The controller takes the same path, because it is a module.** Its store logins and bus accounts are +own secrets of its definition today, and become requirements of that definition. **A node's host is the +one party with no definition**, because it is what runs definitions. Its bus account is a requirement +the mesh makes for each enrolled node, answered and carried exactly as for a module. **A provider makes resources and data.** Beyond secrets, a provider's adapter may answer with its contract's non-secret fields: an analytics site id, a registered public name. The mesh carries them @@ -206,14 +211,14 @@ runtime base, which the installation makes only after the store, the broker and the controller over the bus. So the vault is installed **as soon as that base exists**, before any other module built on it, and genesis generates what is needed until then: - the store's superuser, and the broker's admin in the hashed form the broker needs; -- the bus accounts of the temporary and permanent controller, the control-node's agent, the builder, - the broker's own provisioner and the vault; -- the controller's store login, and the first enrolment token. +- the bus accounts of the temporary and permanent controller (its account and the broker-management + login), the control-node's host, the builder, the broker's own provisioner and the vault; +- the controller's three store logins, and the first enrolment token. Until the broker's provisioner runs, genesis creates those bus accounts with the broker's admin, as the controller does today; the provisioner adopts them when it starts. Genesis seals everything to the -operator key as today ([ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md)), and when the -vault is installed it **delivers the values to it, recorded as the mesh's own**. That distinction keeps +control-node's key, and when the vault is installed the controller **delivers the values to it, +recorded as the mesh's own**, with nobody present. That distinction keeps them rotatable: an operator's value is never replaced, and these are, because the vault can make their replacements. @@ -221,48 +226,34 @@ That is the one time anything but the vault generates a shared secret, and it en over. It is also the answer to the objection ADR 0085 had to the vault being the only maker: the vault cannot make what exists before it, so what exists before it is delivered to it. +**Raising the vault or the broker again is a genesis act.** Moving either seat to a new assignment, or +recovering either after it is lost, delivers the values it needs the way genesis did. It is a stated +break-glass procedure, and an ordinary assignment attempting it is refused. + ## Rotation Rotating a secret is asked of the vault, by an operator or by the vault's own policy, such as a -maximum age in the secret's contract ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). -An operator's external key is not rotated by the vault, which cannot make its replacement: an -operator delivers a new one. +maximum age in the requirement's contract, and the vault makes the new value +([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)). An operator's external key is +not rotated by the vault, which cannot make its replacement: an operator delivers a new one. -**A secret's contract says how each recipient takes a new value.** A recipient either *applies* it, -through a provisioner (a provider creating the login, the broker's provisioner updating an account, -the store's own provisioner changing its superuser), or *reads it at start*. A secret a service reads -only when it first initialises is marked applied, because a restart would change nothing. +**Each requirement says how its recipient takes a new value.** It either *applies* it, through a +provisioner (a provider setting a login's password, the broker's provisioner updating an account, a +store's provisioner changing its own superuser), or *reads it at start*. Every module in the catalogue +reads its secrets at start, and none watches them. The host already recreates a container when a file +it read at creation changes, its env-files and files mounted into it directly +([issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). +So a reader's restart is derived, and a definition declares `restart-on` only for a secret reaching a +process, or a file in a mounted directory. A secret a service takes only at first initialisation is +applied by its provisioner or marked not rotatable by the mesh, and a rotation of it is refused rather +than reported done. -**Old and new overlap, so nobody is ever without a credential that works.** A credential is never -changed in place. Each consumer has two logins, both derived by the mesh, and uses one at a time: - -1. **The vault makes the new value.** -2. **Each applier adds it beside the old**, as the consumer's other login, through the adapter's - existing create. It verifies that the new login works and the old one still does, and confirms, - repeating that confirmation on every reconcile pass until the vault acknowledges it. -3. **Only then is the new login released to the readers**, such as gitea, login and value together. -4. **The host restarts every such reader**, and recreates a container whose env-file carries the - secret. It knows which, because a definition reads a secret only through its requirement, so no - definition declares a restart for a secret. An applier is never restarted for it. This needs - [issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - fixed, or a container fed by an env-file keeps the old value. -5. **Each reader confirms** by passing its health check with the new login, where its definition - declares one. -6. **Only when every reader has confirmed is the old login retired**, through the adapter's existing - remove, and verified to no longer authenticate. - -So a reader whose machine is offline keeps working on the old login until it returns, a bus account's -owner keeps its bus until it has moved, and a provisioner restarted mid-rotation is still delivered -both values. The rotation shows as waiting on whichever applier or reader has not moved, and is done -only when the old login is gone. - -**No consumer module changes.** A provider's adapter gains one duty: both of a consumer's logins get the -same rights over its data, which in postgres means both belong to one role that owns it. The -alternation itself is the provider loop's. - -A secret some service reads only when it first initialises cannot be rotated by restarting it. It is -applied by a provisioner, or, where none exists, marked not rotatable by the mesh, and a rotation is -refused rather than reported done. +**How old and new change over is not settled.** Until it is, rotation stays as the controller does it +today: in place, both ends sent in one push, with a stated window in which a consumer cannot +authenticate. [Research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) +measured three mechanisms against every provider. One constraint holds whichever is chosen: **retiring a +credential must never remove a consumer's resource.** Today's adapters remove both in one call, and in +five providers that deletes the consumer's data. ## Refusing @@ -271,6 +262,7 @@ requirement it names what is missing and what would answer it: - an unheld seat, and which modules could hold it; - no provider, and which modules could provide it; +- several candidates and no choice made, and which they are; - an operator value with no default, and that the assignment must give it; - a provider, or the vault, that has not answered yet, and which one. @@ -309,9 +301,9 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r foundation's first secrets to the vault; the broker's provisioner creates every bus account; the mesh carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) - is fixed first. *Ends when* nothing outside the vault generates a shared secret after genesis, a lab - consumer of analytics receives its site id, and a database credential rotates with old and new - overlapping, the consumer restarted by derivation and the rotation confirmed. + is fixed in the host already. *Ends when* nothing outside the vault generates a shared secret after + genesis, a lab consumer of analytics receives its site id, and a database credential rotates, the + vault making the value, the consumer recreated by derivation, and its data intact. 3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is, and each claim becomes a seat the module can hold, held by the assignment that holds it today. *Ends when* the list of definitions @@ -324,7 +316,7 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r |---|---| | Every requirement has one of the four provider kinds | The parser refuses any other. | | A definition names no host path, node or mesh | The catalogue tests of [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md). | -| A module provider is chosen by pin, seat, co-location, only one, refusal | Resolution tests for each step; for a second provider on a consumer's own machine when a seat delivers the provision; and for an unheld seat with exactly one provider, refused. | +| A module provider is chosen by named seat, pin, co-location, only one, a person's choice | Resolution tests for each step: a requirement naming a seat served by its holder even with another provider on the consumer's node, and refused when the seat is unheld; several candidates and none local, where assignment lists them with the seat's holder first and records the choice as a pin, and refuses without one. | | A provider answers within its contract | A controller test: an answer carrying a field its contract does not name, or missing one it does, is refused and not delivered. | | A module narrows a contract and never widens it | The parser refuses a module specification that loosens a contract's field. | | A host requirement is answered on its own node | A resolution test placing one elsewhere: refused. | @@ -337,12 +329,16 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | | Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | | A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | -| Rotation overlaps old and new | The rotation tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): both logins authenticate while readers move; an offline reader keeps working on the old login; the old login is retired only after every reader confirms; an applied secret restarts nothing, and one read at start restarts its reader without a declared restart. | +| Restarts are derived, and rotation keeps data | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, one read at start recreates its reader without a declared restart, and rotating a consumer's credential leaves its resource and data intact. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | ## Not settled here +- How old and new credentials change over on rotation: in place, as today, or two logins over one + resource, as [research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md) + recommends for credentials with two parties. It is decided in its own record. + - The exact spelling of the one form. It must name a requirement and a field and nothing else. - The layout a node's default root uses beneath it, beyond one directory per assignment. - Whether a module provider's answer can change without the provider being asked, for example a From 43f63ed41c4f99a88ca874cdbe01abc5db1db8ed Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:22:21 +0200 Subject: [PATCH 22/26] ADR 0114: a two-party credential rotates over two logins Graduates research 016. Retiring a login is separated from removing a consumer, which closes a data-loss path in five providers; single-party secrets rotate in place; the number of parties decides, not the provider. --- .../00-overview.md | 5 +- .../0113-the-vault-makes-every-secret.md | 16 +- ...ared-credential-rotates-over-two-logins.md | 198 ++++++++++++++++++ 02-DECISIONS/README.md | 1 + .../27-a-module-requires-the-mesh-resolves.md | 34 +-- 5 files changed, 234 insertions(+), 20 deletions(-) create mode 100644 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md index 83f184d..4c28849 100644 --- a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md @@ -1,5 +1,8 @@ --- -status: active +status: graduated +became: + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md + - 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md initiated: 2026-09-26 touches: - 02-DECISIONS/0113-the-vault-makes-every-secret.md diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index cc70c48..36b60a6 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -164,10 +164,12 @@ the change using the old value. Where no provisioner can make it, the requiremen rotatable by the mesh**, and a rotation request is refused, saying why, rather than restarting a service that would carry on with the old value. -**How old and new change over is not decided here.** Three mechanisms were measured against every -provider's code in [research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): -in place, as the controller's `rotate` does today; two secrets on one login; and two logins over one -resource. Its findings bound the choice: +**How old and new change over is decided in [ADR +0114](0114-a-shared-credential-rotates-over-two-logins.md).** Three mechanisms were measured against +every provider's code in [research +016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): in place, as the controller's +`rotate` does today; two secrets on one login; and two logins over one resource. Its findings bound the +choice: - every credential provider already re-applies a password in place, so today's rotation works, with a window in which a consumer cannot authenticate; @@ -178,8 +180,8 @@ resource. Its findings bound the choice: - every backend can give two logins the same rights over one resource, once the adapter separates the resource from the login. -The mechanism is decided in its own record, on those facts. Until then rotation stays as the -controller implements it, in place, with its window stated. +On those facts, 0114 rotates a credential two parties hold over two logins, rotates one a single +party holds in place, and separates retiring a login from removing a consumer. ## What this changes in earlier records @@ -215,7 +217,7 @@ On acceptance, each of these is superseded or amended by this record, not edited - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. - A data provider's adapter gains a return value. What a credential provider's adapter must change for - rotation is decided with the mechanism ([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md)). + rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-logins.md)'s. - The broker's provisioner gains every bus account, and the controller loses five separate places it generates a secret today. - 54 modules move from own secrets to vault requirements. Six provider clients export a password diff --git a/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md new file mode 100644 index 0000000..29c1b49 --- /dev/null +++ b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md @@ -0,0 +1,198 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-26 +deciders: jochen +reconstructed: false +extends: 0113-the-vault-makes-every-secret.md +--- + +# 114. A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached + +## Context + +[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the +vault's policy) and who makes the new value (the vault). It leaves open how old and new change over. +[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every provider +in the catalogue against its code: + +- **all eight credential providers re-apply a password in place**, on the same login, every time they + run. The controller's `rotate` command relies on that, and states the window it leaves: between the + provider applying the new value and the consumer restarting with it, the consumer cannot authenticate; +- **seven of eight name the consumer's resource after its login**: a database, a bucket, a virtual host, + a key prefix, a topic prefix. Only the forge's npm registry keeps them apart, because an organisation + owns the packages; +- **five of eight destroy the resource when they remove the login**: postgres, mssql, mongodb, minio and + lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call; +- **only one backend holds two passwords on one login**, redis. A second, the forge, holds several + tokens beside one password; +- **every backend can give two logins the same rights over one resource**: a group role, a database + role, a shared policy, shared permissions, a shared role, a shared key prefix, a shared team. No + adapter does it today; +- **administrative credentials have one party and a fixed name.** The provider module is both the one + that applies the value and the only one that reads it. Three backends take it only at first + initialisation, so it can only be changed through a command run with the old value; +- **no module watches a secret.** Every reader reads at start, and the host already recreates a + container when a file it read at creation changes + ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). + +A first draft of 0113 chose to overlap old and new "through the adapter's existing create and +remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on +what the providers do, and the danger has to be closed whichever mechanism is chosen. + +## Considered Options + +**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials +two parties hold. The window cannot be closed, only shortened, and the two ends are on different +machines with nothing ordering them. A reader the mesh cannot reach, but which still reaches its +provider, is locked out until the mesh reaches it again. + +**2. Two secrets on one login.** The applier adds the new password beside the old one. Rejected. It +works for one provider out of eight. Using it where it exists and something else elsewhere is a +mechanism per provider, which is what the mesh is trying to stop having. + +**3. Two logins over one resource for everything.** Rejected for credentials a single party holds. +The applier and the reader are the same module, so there is no second party to keep working while the +other moves. A fixed administrative name has no second name to alternate with. + +**4. Split by the secret, not by the provider.** A credential two parties hold rotates over two logins; +one a single party holds rotates in place; and retiring a login is separated from removing a consumer +before either is used. Chosen. + +## Decision + +### Retiring a login never removes what it reached + +**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its +database, bucket, virtual host, key or topic prefix) and the *login* that reaches it. They get separate +operations: + +- **ensure the resource**, named after the consumer; +- **ensure a login** with a value, holding the consumer's rights over its resource; +- **retire a login**, which removes the login and nothing else; +- **remove the consumer**, which is what removes the resource. It is run only when the consumer is + unassigned, as today, and never by a rotation. Whether the resource's data is kept beyond that stays + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s. + +**The resource is named after the consumer, not after a login.** A consumer's identity is derived from +its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), and today its login +is that same string. So **no existing resource is renamed**. The login every consumer holds today +becomes the first of its two, under the name it already has. + +### A credential two parties hold rotates over two logins + +This covers a credential between a consumer and a provider, and a bus account, which the broker's +provisioner applies and its module reads. **Each consumer has two logins, derived by the mesh**: its +identity, and its identity with a short fixed suffix. It uses one at a time, and both hold the same +rights over the one resource. + +1. **The vault makes the new value.** +2. **Each applier ensures the unused login with it**, with the consumer's rights, and leaves the login + in use untouched. It verifies that the new login authenticates and the old one still does, and + confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so a + lost message costs one pass. +3. **Only then are the readers given the new login and value, together.** The host recreates each + reader, because a file it read at creation changed. +4. **Each reader confirms** by being recreated with the new login and passing its health check, where + its definition declares one. +5. **Only when every reader has confirmed is the old login retired.** Each applier retires it, the + login and nothing else, and verifies that it no longer authenticates. + +`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done +until the old login is gone. A reader that cannot be reached keeps working on the old login until it +can, and the rotation waits for it. That wait is shown, never hidden. + +**Where a backend identifies a connection separately from a login, the two are kept apart.** An MQTT +client identifier must be unique per connection, so the mosquitto adapter derives the connection's +identifier from the consumer and the login in use, and two logins never collide. + +### A credential a single party holds rotates in place + +This covers a provider's administrative credential and a module's own secret, which only that module +reads. **The vault makes the new value, and the one party takes it:** + +- **applied**: the party's provisioner changes it using the old value, then confirms. This is the only + form for a backend that takes its administrative credential only at first initialisation, where a + restart would change nothing; +- **read at start**: the host recreates the party. + +There is no window between two parties, because there is only one party. Where neither form can change +the value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why +([ADR 0113](0113-the-vault-makes-every-secret.md)). + +### One rule decides which + +**The number of parties that hold the credential decides, never the provider.** A requirement whose +secret has an applier and a reader in different modules rotates over two logins. A secret held by one +module rotates in place. The resolver knows which from the requirement's recipients, so no definition +declares it. + +### Until an adapter can + +**An adapter that cannot yet ensure a second login says so**, and the credentials it applies rotate in +place, as today, with the window stated when the rotation is asked for. That is a migration state, not +a second mechanism. It is listed by a check, and the list shrinks to empty. Separating *retire a +login* from *remove the consumer* comes first in every adapter, because it closes a data-loss path that +exists today, whatever rotation does. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here. +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room + for the second login's suffix within the tightest backend it reaches, and both logins are checked + against it. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker + account is two logins with the same permissions over the same queue, one in use at a time. Its scoping + is unchanged. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party + credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed. + Single-party rotation keeps the form to-be 13 describes. + +## Consequences + +- **Every credential provider's adapter changes**, in two steps. The first separates *retire a login* + from *remove the consumer*, and names the resource after the consumer, which is the name it already + has. The second ensures a second login with the same rights. That is seven adapters for the second + step, since the forge's already holds its packages apart from the user. +- **The SDK's provider harness** carries the alternation, the verification of both logins, and the + repeated confirmation, so no adapter implements them. Its record of what was applied has to survive a + restart of the provisioner mid-rotation. Today it is kept in memory. +- **No consumer module changes.** It reads one login and a value at start, as today, and is recreated + by the host when they change. +- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20 + characters. +- **What got harder:** a provider briefly holds two logins per consumer. A rotation of a two-party + credential lasts until its slowest reader moves, so an unreachable reader keeps the old login valid + until it is reached. And an adapter has four operations where it had two. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Retiring a login never removes a resource | A provider test per credential provider: retiring one of a consumer's logins leaves its resource and data intact, reachable through the other. | +| Removing a consumer is not a rotation | A harness test: no rotation step calls remove; remove runs only when a contribution goes. | +| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, and its existing login becomes the first of its two. | +| Both logins hold the same rights | A provider test per credential provider: data written under one login is read and changed under the other. | +| Readers move only after the applier confirms | A rotation test: readers receive nothing until both logins authenticate at every applier. | +| The old login is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old login, the rotation shows waiting on it, and it completes when the reader returns and confirms. | +| A lost confirmation costs one pass | A harness test dropping the first confirmation: the next pass repeats it. | +| A restarted provisioner resumes a rotation | A harness test restarting the provisioner between steps: it resumes from the step it reached. | +| A single-party secret rotates in place | A vault test: a provider's administrative credential is applied by its own provisioner with the old value; a module's own secret recreates the module; neither has a second login. | +| The number of parties decides | A resolution test: a secret with an applier and a reader in different modules is marked for two logins, and one held by one module for in place, with nothing declared. | +| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. | +| A connection identifier never collides | A mosquitto provider test: two connections under a consumer's two logins are both accepted. | +| Adapters still in place are listed | A catalogue test lists every credential provider that cannot yet ensure a second login. The list shrinks to empty, and a rotation of their credentials states its window. | + +## References + +- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this + rests on, provider by provider +- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving + its declaration +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented +- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): + why a reader's restart can be derived diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 70b2bcb..8bb0aad 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -159,6 +159,7 @@ python3 00-META/checks/index.py fail if stale - **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(proposed)* - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* - **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* +- **0114** — [A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached](0114-a-shared-credential-rotates-over-two-logins.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index 10d1ac9..ed64d18 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -6,6 +6,7 @@ updated: 2026-09-26 decisions: - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - 02-DECISIONS/0113-the-vault-makes-every-secret.md + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md - 02-DECISIONS/0084-which-provider-serves-a-consumer.md @@ -248,12 +249,23 @@ process, or a file in a mounted directory. A secret a service takes only at firs applied by its provisioner or marked not rotatable by the mesh, and a rotation of it is refused rather than reported done. -**How old and new change over is not settled.** Until it is, rotation stays as the controller does it -today: in place, both ends sent in one push, with a stated window in which a consumer cannot -authenticate. [Research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) -measured three mechanisms against every provider. One constraint holds whichever is chosen: **retiring a -credential must never remove a consumer's resource.** Today's adapters remove both in one call, and in -five providers that deletes the consumer's data. +**How old and new change over depends on how many parties hold the credential** +([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md), on +[research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md)): + +- **Two parties**, a consumer and its provider, or a module and the broker: each consumer has two logins + derived by the mesh, both with its rights over one resource, named after the consumer. The vault makes + the new value; each applier ensures the unused login with it and confirms both authenticate; only then + are readers given it and recreated; once every reader has confirmed, the old login is retired. Nobody + is left without a credential that works, and `status` shows who a rotation waits on. +- **One party**, a provider's administrative credential or a module's own secret: in place. Its own + provisioner applies it with the old value, or the host recreates it. + +**Retiring a login never removes what it reached.** An adapter keeps *retire a login* and *remove the +consumer* apart. Only unassigning removes the resource, and never a rotation. Today the two are one +call, and in five providers it deletes the consumer's data, so this separation comes first. An adapter +that cannot yet ensure a second login rotates in place, with its window stated, and is listed until it +can. ## Refusing @@ -302,8 +314,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r mesh carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) is fixed in the host already. *Ends when* nothing outside the vault generates a shared secret after - genesis, a lab consumer of analytics receives its site id, and a database credential rotates, the - vault making the value, the consumer recreated by derivation, and its data intact. + genesis, a lab consumer of analytics receives its site id, and a database credential rotates over its two + logins, the consumer recreated by derivation, never without a working login, and its data intact. 3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is, and each claim becomes a seat the module can hold, held by the assignment that holds it today. *Ends when* the list of definitions @@ -329,15 +341,13 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | Only the vault generates a shared secret after genesis | A controller test: no code path generates one. An installer test: genesis generates exactly the foundation's first secrets and delivers them to the vault. | | Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | | A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | -| Restarts are derived, and rotation keeps data | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, one read at start recreates its reader without a declared restart, and rotating a consumer's credential leaves its resource and data intact. | +| Restarts are derived | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, and one read at start recreates its reader without a declared restart. | +| A two-party credential rotates over two logins | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md): retiring a login leaves the resource intact; readers move only after the applier confirms; an unreachable reader keeps its old login until it returns; a single-party secret rotates in place. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | ## Not settled here -- How old and new credentials change over on rotation: in place, as today, or two logins over one - resource, as [research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md) - recommends for credentials with two parties. It is decided in its own record. - The exact spelling of the one form. It must name a requirement and a field and nothing else. - The layout a node's default root uses beneath it, beyond one directory per assignment. From e387c4bd0e2eb515a005068c138b52e51162a76b Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:38:06 +0200 Subject: [PATCH 23/26] Apply review: two credentials, staged admin rotation, a ninth provider The fact-check found mailu, whose user is its mailbox, so 0114 rotates over two credentials rather than two logins, the adapter choosing what a credential is. Also: minio keeps non-empty buckets; five backends take their admin credential only at first init, so single-party rotation is staged; postgres ownership moves to a non-login role; the harness keys by consumer; rotation state lives with the vault. Consistency fixes across 0110-0113, 26 and 27; issue 103 resolved by mesh-host PR #22. --- .../00-overview.md | 25 +- .../01-the-providers.md | 84 ++++-- .../02-the-readers.md | 4 + .../03-the-options.md | 66 ++--- ...s-a-module-assignment-from-a-closed-set.md | 6 + ...d-source-is-on-the-git-seat-or-external.md | 9 + ...e-definition-names-no-node-mesh-or-path.md | 10 +- .../0113-the-vault-makes-every-secret.md | 58 +++-- ...credential-rotates-over-two-credentials.md | 245 ++++++++++++++++++ ...ared-credential-rotates-over-two-logins.md | 198 -------------- 02-DECISIONS/README.md | 2 +- 03-DESIGN/01-to-be/26-the-seats.md | 16 ++ .../27-a-module-requires-the-mesh-resolves.md | 50 ++-- 03-DESIGN/01-to-be/README.md | 4 +- .../00-report.md | 4 +- .../00-report.md | 9 +- 16 files changed, 472 insertions(+), 318 deletions(-) create mode 100644 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md delete mode 100644 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md index 4c28849..49eabbc 100644 --- a/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md @@ -1,7 +1,7 @@ --- status: graduated became: - - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md - 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md initiated: 2026-09-26 touches: @@ -16,7 +16,7 @@ touches: # 016 — How a credential can be rotated **What.** Which rotation mechanisms the mesh's providers can actually support, measured against -their code rather than assumed. Every provider in the catalogue was read: how it names what it +their code rather than assumed. Every provider in the catalogue was read, found by listing every definition that provides something: how it names what it makes for a consumer, what its remove destroys, whether it re-applies a password, whether its backend can hold two secrets for one login or two logins on one resource, and how its own administrative credential is set. The consumer side was read too: when a module reads a secret, and @@ -30,7 +30,8 @@ with the login. Overlap as written would have deleted every consumer's database rotation. The mechanism has to be chosen on what the providers do. **What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md), -which both mark it undecided and point here. The identity budget in +which [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) decided on +these findings. The identity budget in [ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md) describes. @@ -43,12 +44,14 @@ describes. - [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a recommendation. -**Finding, in one paragraph.** All eight credential providers already re-apply a consumer's password +**Finding, in one paragraph.** All nine credential providers already re-apply a consumer's password in place on every create, and the controller's `rotate` command relies on that. It is a working -rotation with a stated window. Seven of the eight name the consumer's resource after its login, -and five drop the resource when they remove the login, so a second login is impossible without -changing the adapter. Only one backend holds two passwords on one login. But every backend can grant -two logins the same rights over one resource. So overlap is possible everywhere, but only after each -adapter separates *the consumer's resource* from *the login that reaches it*. Administrative -credentials are a different case. They have one party, a fixed name, and in three providers they are -taken only at first initialisation. +rotation with a stated window. Eight of the nine name the consumer's resource after its login, and five +destroy the consumer's data when they remove the login. The harness, keyed by login, would do the same +on any change of login. Only one backend holds two passwords on one login, and two more hold several +tokens. Eight backends can grant two logins the same rights over one resource; the ninth can give one +login a second token. So every provider can hold **two credentials** over one resource, but only after +each adapter separates *the consumer's resource* from *the credential that reaches it*. In postgres +that also means the resource belongs to a role no login owns. Administrative credentials are a +different case. They have one party and a fixed name, and five backends take them only at first +initialisation, so changing one needs the old and the new value at once. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md index e537ffa..8ce34f1 100644 --- a/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/01-the-providers.md @@ -1,10 +1,18 @@ # 01 — The providers Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`), -the client functions they call, and each definition's own credentials. The provisioner harness in -`mesh-sdk` calls `create` for a consumer when its contribution appears or changes, and after the -provisioner restarts, and `remove` when the contribution goes. Its record of what was applied is -kept in memory. +the client functions they call, and each definition's own credentials. The providers were found by +listing every definition that provides something and has a provisioner, not from memory. A first pass +of this survey worked from memory and missed one, mailu. + +**The provisioner harness** in `mesh-sdk` calls `create` for a consumer when its contribution appears +or changes (its login, password or values), and after the provisioner restarts. It calls `remove` for +a login it applied earlier in the same process that is no longer contributed. Its record of what was +applied is kept in memory and keyed by login. Two things follow: + +- a consumer whose derived login changes is removed under the old login and created under the new one, + in one pass; +- a contribution that disappears while the provisioner is down is never removed, and is left behind. ## The credential providers @@ -13,13 +21,14 @@ kept in memory. | provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource | |---|---|---|---|---|---| -| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, both as members of one role that owns the database. Not done today | -| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. Not done today | +| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, but only through a role that cannot log in owning the database, with each login working as it. Otherwise whatever one login creates is its own, and dropping that login means handing its objects over first. Not done today | +| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. A user owning a schema cannot be dropped, and a login with an open session cannot. Not done today | | mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today | | redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login | -| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key and the bucket | yes, by removing the access key and adding it again | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters | -| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host | -| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. An MQTT client identifier still has to be unique per connection | +| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key; the bucket **only if empty**. A bucket holding objects is left, and the failure logged | yes, by removing the access key and adding it again, which leaves a moment with no key | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters | +| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host, with any queued messages, and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host | +| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. The MQTT client identifier is chosen by the consumer, not tied to the login; a duplicate one takes the older session over | +| mailu | a mailbox `login@domain`, unless the consumer contributes its own account name | the mailbox with its mail, for a login-named one; a contributed name is left for an operator | yes, the password is set when the user exists | no for the password; a user can hold several authentication tokens, per the backend's documentation | **no**: a mail user *is* its mailbox | | gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team | ## The other providers @@ -31,41 +40,64 @@ kept in memory. | showcase | a route | none | | mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today | +verdaccio provides the npm registry too, and has no provisioner. + ## Each provider's own administrative credential | provider | identity | how the backend takes it | |---|---|---| -| postgres | a fixed superuser | from a file **only at first initialisation**. Afterwards the file is read by the provisioner to connect, and changing it changes nothing in the database | +| postgres | a fixed superuser | from a file **only at first initialisation** | | mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception | | mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty | +| mosquitto | a fixed admin client | seeded into the broker's dynamic-security file **once**; the seeding step skips when the file exists | +| lavinmq | a fixed admin name | per its own bootstrap code, **only on a first boot** with an empty data directory. No resource in the definition runs that bootstrap; what sets it on a running mesh is outside the catalogue | | redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts | | minio | a fixed root user | from a file, read when the server starts | -| lavinmq | a fixed admin name | from the module's own secret, which its provisioner connects with | -| mosquitto | a fixed admin client | from the module's own secret, which its provisioner connects with; the broker's dynamic-security file holds it | + +**In five of seven, a new administrative value takes effect only through a command run with the old +one.** The credential file is mounted directly into both the server and the provisioner. So replacing +it recreates the provisioner, which then holds only the new value while the backend still expects the +old one, and the provisioner is locked out. That is worse than changing nothing. Every provider module also has its own bus account, an own secret, read at start. -## What the table shows +## What the tables show -1. **Every credential provider already rotates in place.** All eight re-apply the password on the +1. **Every credential provider already rotates in place.** All nine re-apply the password on the same login each time `create` runs. The controller's `rotate` command relies on that: it replaces the credential in the inventory and sends both ends in one push. Its own comments state the window, between the provider applying and the consumer restarting, in which the consumer cannot authenticate. -2. **Seven of eight name the consumer's resource after its login.** Only gitea separates them, +2. **Eight of nine name the consumer's resource after its login.** Only gitea separates them, because an organisation owns the packages. A second login therefore has no resource of its own to reach, and cannot share the first one's without the adapter granting it. -3. **Five of eight destroy the resource when they remove the login.** These are postgres, mssql, mongodb, - minio and lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call. - This is the fault the first overlap draft would have triggered. -4. **Only redis holds two passwords on one login**, and gitea through tokens rather than its - password. A rotation built on two secrets per login would work for one provider out of eight. -5. **Every backend can give two logins the same rights over one resource.** Group roles in postgres, +3. **Five of nine destroy the consumer's data when they remove the login**: postgres, mssql and + mongodb drop the database, lavinmq drops the virtual host with its queued messages, and mailu + deletes the mailbox with its mail. minio drops only an empty bucket, and redis leaves the keys. In + those five, *retire a login* and *delete the consumer's data* are one call. With the harness keyed + by login, a changed login triggers it too. +4. **One backend holds two passwords on one login** (redis). Two hold several tokens beside one + password (gitea and mailu). A rotation built on two secrets per login would work for three + providers out of nine. +5. **Eight of nine can give two logins the same rights over one resource.** Group roles in postgres, database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared - policy in minio, a shared key prefix in redis, a shared team in gitea. No adapter does it today. -6. **The administrative credentials have one party and a fixed name.** The provider module is both the - one that applies the credential and the only one that reads it. In postgres, mongodb and mssql, a new - value only takes effect through a command run with the old value. A restart changes nothing. -7. **A consumer's identity is already the resource's name.** The login is derived from the + policy in minio, a shared key prefix in redis, a shared team in gitea. mailu cannot, because its + user is its mailbox, but it can give one user a second token. So every provider can hold **two + credentials** over one resource, though not every one as two logins. No adapter does either today. +6. **Ownership is a trap in two backends.** In postgres whatever a login creates is that login's, so a + second login cannot alter the first one's tables, and the first cannot be dropped while it owns + them. The one-step way out deletes them. In mssql, a login cannot be dropped with a session open, + nor its user while it owns a schema. +7. **The administrative credentials have one party and a fixed name**, and five backends take them + only at first initialisation. The provisioner needs the old and the new value at once to change + them. Today nothing can give it both. +8. **A consumer's identity is already the resource's name.** The login is derived from the assignment, which is a module on a node, so the current login and "the consumer" are the same string today. A second login would need a new name. The resource can keep the one it has. + +## Seen on the way + +The redis configuration names no ACL file, so a consumer's ACL user exists only in memory. A restart +of the redis server erases every consumer's user. The provisioner does not create them again until it +restarts itself, because its in-memory record says they are done. That is not a rotation finding, but +it is a live fault, and it is recorded here so it is not lost. diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md index 7504391..143f53d 100644 --- a/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md @@ -32,6 +32,10 @@ env-file or a direct mount, which the host covers, or through a directory or int needs `restart-on`. **That was not classified module by module.** It is the check to run before a rotation mechanism relies on it. +The count covers only the `secrets` field. The 49 modules with their own bus account, and 54 with any +own secret, are readers too, and their bus accounts are rotated like any credential two parties hold. +Their files are mounted directly, which the host covers, but the classification has to name them. + ## What this means for rotation - The *read at start* half of 0113's recipient model is already true, and mostly already handled by diff --git a/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md index e668691..5bfb7e5 100644 --- a/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md +++ b/01-RESEARCH/016-how-a-credential-can-be-rotated/03-the-options.md @@ -4,7 +4,7 @@ Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-read ## A. In place, as today -The vault makes a new value. Every applier re-applies it on the same login, which all eight +The vault makes a new value. Every applier re-applies it on the same login, which all nine providers already do. Every reader is recreated by the host. - **Works with:** every provider, unchanged. It is what `rotate` does now. @@ -18,56 +18,60 @@ providers already do. Every reader is recreated by the host. The applier adds the new password beside the old one, readers move, and the old one is removed. -- **Works with:** redis natively, and gitea through tokens. **Not** with the other six, whose backends - hold one password per login (finding 4). +- **Works with:** redis natively, and gitea and mailu through tokens. **Not** with the other six, whose + backends hold one password per login (finding 4). - **Verdict:** not a mechanism, a special case. Using it where it exists and something else elsewhere is the "this way or that way" the design is trying to remove. -## C. Two logins per consumer, over one resource +## C. Two credentials per consumer, over one resource -The consumer has two logins derived by the mesh, and uses one at a time. The applier creates the other -with the new value and grants it the same rights over the consumer's resource. Readers move to it, -and then the old login is retired, which removes the login only, never the resource. +The consumer has two credentials and uses one at a time. The applier ensures the other with the new +value and gives it the same rights over the consumer's resource. Readers move to it, and then the old +credential is retired, which removes the credential only, never the resource. **What a credential is, +is the adapter's**: a second login for eight providers (finding 5), a second token on the same login +for mailu. The mesh sees one mechanism. -- **Works with:** every backend (finding 5), **after** each adapter changes: +- **Works with:** every provider, **after** each adapter changes: - the resource is named after the consumer, not the login. Today the two are the same string (finding - 7), so existing resources keep their names, the current login stays one of the two, and only the - second is new; - - both logins get the same rights, through a group role or its equivalent; - - *retire a login* and *remove the consumer* become two operations. Today they are one call, and - in five providers that call destroys data (finding 3). This is the whole of the danger, and it has - to be split, whatever else is chosen. + 8), so existing resources keep their names, and the current login stays one of the two; + - the resource is owned by the resource, not by a login. In postgres that is a role no one logs in + as, which each login works as, and ownership of an existing database moves to it once (finding 6); + - both credentials get the same rights, over data and structure; + - *retire a credential* and *remove the consumer* become two operations. Today they are one call, and + in five providers that call destroys data (finding 3). The harness must key by consumer, so that a + changed login is not a removal. This is the whole of the danger, and it has to be split, whatever + else is chosen. - **Costs:** - - seven adapters change; - - the harness learns the alternation and a confirmation per step; + - every credential adapter changes; + - the harness learns the alternation and a confirmation per step, and rotation state has to live + somewhere that survives a restart, which the harness's memory does not; - the second login's name must fit the tightest backend. That is 20 characters for a minio access key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and a suffix spends part of it; - - an MQTT client identifier stays unique per connection, so mosquitto needs the client id kept apart - from the login. -- **Gains:** no window. A reader that cannot be reached keeps a working login until it can. + - retiring an mssql login has to end its sessions first. +- **Gains:** no window. A reader that cannot be reached keeps a working credential until it can. - **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is no second party to overlap with. ## Independent of the choice -- **Split remove.** Retiring a credential must never be able to destroy a consumer's data. That holds - under A too, because A's remove is the same call. A remove that drops a database should be a - separate, explicit operation, which [ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md) - already implies: data outlives the declaration. -- **Admin credentials are applied by their own provider**, using the old value. That happens in place - whichever mechanism consumers get. Where a backend takes the value only at first initialisation, the - file alone changes nothing, and rotation needs the provider's provisioner to run the change. -- **Classify the 22 readers** ([02](02-the-readers.md)) before relying on derived restarts. +- **Split remove, and key the harness by consumer.** Retiring a credential, or a login changing, must + never be able to destroy a consumer's data. That holds under A too, because A's remove is the same + call. +- **Admin credentials are applied by their own provider**, using the old value, with the new one staged + beside it. Five backends take the value only at first initialisation. Replacing the file first locks + the provisioner out (finding 7). +- **Classify the readers** ([02](02-the-readers.md)), bus-account readers included, before relying on + derived restarts. ## Recommendation -- **Consumer credentials: C**, because it is the only mechanism every backend supports, and it closes - the window instead of shortening it. Its prerequisite, separating the resource from the login and +- **Two-party credentials, consumer credentials and bus accounts: C**, because it is the only + mechanism every provider supports, and it closes the window instead of shortening it. Its prerequisite, separating the resource from the login and retiring a login from removing a consumer, is worth doing on its own, because it removes a data-loss path that exists today. -- **Single-party secrets (admin credentials, a module's own): A.** In place, applied by the provider - that holds them. +- **Single-party secrets (admin credentials, a module's own): A, staged.** In place, applied by the + provider that holds them, with the new value beside the old until it has taken. - **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is not replaced by a mechanism the providers cannot yet carry. diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index fdd08b4..515b6dd 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -163,6 +163,10 @@ On acceptance, each of these is amended by this record, not edited: - [ADR 0084](0084-which-provider-serves-a-consumer.md): a requirement may name a seat, which its holder answers; and where several providers remain and none is local, the choice is asked when the module is assigned and recorded as a pin, rather than refused until someone pins it. +- [To-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): the same two changes, in the design + that describes choosing a provider. +- [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): genesis assigns the foundation's + store, broker and controller holding their seats, where their definitions claim them today. ## Consequences @@ -191,6 +195,8 @@ On acceptance, each of these is amended by this record, not edited: | Every module in use names a seat in the set | A controller test parses every catalogue manifest and fails on any refused seat. The lab's beds read the same manifests ([ADR 0089](0089-a-bed-reads-the-catalogue-it-proves.md)). | | A seat is held by one assignment, not by a module | A resolution test: the store module assigned to two nodes resolves, with one assignment holding `mesh-store`; a second assignment asking to hold it is refused. | | A requirement naming a seat is answered by its holder | Resolution tests: two providers with the seat held; a second provider on the consumer's own node, where the holder still answers; the seat unheld with two providers, and with **one** provider, both refused naming the seat. | +| An assignment holds only a seat its module can hold | A resolution test: an assignment holding a seat its definition does not name is refused. | +| Every seat is listed with its holder | A `seats` command test: every seat in the set is listed with its scope, what it delivers and its holder, and an unheld seat is listed as unheld. | | Several providers and none local is a person's choice | An assignment test: the candidates are listed with the delivering seat's holder first; the answer is recorded as a pin; with no answer the module is not assigned. | | The foundation's seats route nobody | A resolution test: with the store module on two nodes, a database consumer is served by the one on its own node, whichever holds `mesh-store`; a requirement naming `mesh-store` is refused, because it delivers nothing. | | A reserved provision has no other provider | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`; resolution refuses an assignment providing it without holding the seat, and a pin on a `secret` requirement. | diff --git a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md index 44b7057..5cb4f14 100644 --- a/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md +++ b/02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -65,6 +65,15 @@ failing to clone. **The build machine is not told the difference.** It receives a URL either way. Composing the URL is the controller's job, because only the controller knows where the seat's holder runs. +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0069](0069-a-module-is-a-repository-and-a-path.md): a module's repository is recorded either as + a path on the `git` seat or as an external URL, never as an address of the mesh's own forge. +- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): the closed set gains `git`, + mesh-scoped, delivering `git`, held by a gitea assignment. + ## Consequences - The controller's inventory gains a column saying which seat a source is on. It is empty for diff --git a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md index e247c3e..644268b 100644 --- a/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md +++ b/02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md @@ -22,7 +22,8 @@ copies. The issue records what that has already allowed: - a contributions file that carries host paths into containers, so every provider must mount its grants directory at the identical path; - defaults in code that disagree with their own manifests; -- no way to assign one module to one node twice, because every identity is keyed by the module's name. +- every identity keyed by the module's name, which is why one module cannot be assigned to one node + twice. This record keeps that, and says so below. **Paths are one case of a wider pattern.** A module gets what it needs through at least six separate mechanisms today, each with its own syntax and its own failure modes: @@ -80,8 +81,9 @@ other. An operator value is the assignment's, or the requirement's default, or u **A person's value stays cheap.** An operator requirement's contract is a type and, optionally, a default. It needs no provider module, no grant and no credential. -**Every secret is a `secret` requirement, answered by the vault**, with no exception by kind -([ADR 0113](0113-the-vault-makes-every-secret.md)). An external API key an operator chooses is no +**Every shared secret is a `secret` requirement, answered by the vault**, with no exception by kind +([ADR 0113](0113-the-vault-makes-every-secret.md)). A private key is made where it is used and is not +a requirement. An external API key an operator chooses is no different: the operator delivers it to the vault ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), and the module requires a `secret` like any other. A provider that needs a secret for a consumer requires it from the vault, like any consumer, and answers with resources and data. The mesh carries @@ -124,7 +126,7 @@ way as every other secret ([ADR 0113](0113-the-vault-makes-every-secret.md)). ## What this changes in earlier records -On acceptance, each of these is amended by a record of its own, not edited: +On acceptance, each of these is amended by this record, not edited: - [ADR 0046](0046-a-module-configuration-is-its-assignments-not-its-manifest.md): settings become operator requirements on an assignment. diff --git a/02-DECISIONS/0113-the-vault-makes-every-secret.md b/02-DECISIONS/0113-the-vault-makes-every-secret.md index 36b60a6..a8ae287 100644 --- a/02-DECISIONS/0113-the-vault-makes-every-secret.md +++ b/02-DECISIONS/0113-the-vault-makes-every-secret.md @@ -15,7 +15,7 @@ controller on 2026-09-25: | kind | made by | used by | |---|---|---| -| a credential between a consumer and a provider | the controller | 19 modules | +| a credential between a consumer and a provider | the controller | 16 modules, and 3 more for model access, counted below | | a module's own secret (`own-secrets`) | the controller, as a random value nothing owns | 54 modules | | a module's broker account | the controller, but only when a person runs a separate command; otherwise the random value above, which cannot work ([issue 095](../04-ISSUES/095-a-module-assigned-after-genesis-has-no-broker-account/00-report.md)) | 49 modules | | a node's and the builder's broker accounts | the controller, each in its own code path | every node, the builder | @@ -78,7 +78,7 @@ it first means reordering the whole installation and giving the vault a second w requiring a database makes the database's provider require a secret for gitea, and the vault answers it. The provider's own code does not change: it is handed a login and a password, as today; - a module's **own secret**. `own-secrets` is retired; -- every **broker account** on the mesh's bus: a module's, a node agent's, the builder's, the +- every **broker account** on the mesh's bus: a module's, a node's host's, the builder's, the controller's. The broker holding `mesh-broker` carries the mesh's bus ([ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), and its own provisioner creates each account from the vault's secret, like any provider. The controller no longer creates accounts, @@ -122,9 +122,11 @@ generated by genesis: token. Until the broker's provisioner runs, genesis creates the bus accounts it generated, with the broker's -admin, as the controller does today. Genesis seals all of it to the control-node's key, and when the vault -is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not as an -operator's. Nobody has to be present for it. +admin, as the controller does today. Genesis seals each value twice: to the control-node's key, so that when the +vault is installed the controller **delivers the values to the vault, recorded as the mesh's own**, not +as an operator's, with nobody present; and to the operator key, as the break-glass copy +[ADR 0085](0085-a-secret-is-a-provision.md) keeps of every root secret. The first enrolment token reaches +the operator the same way. That distinction matters: an operator's value is never replaced ([ADR 0092](0092-an-operator-delivers-a-pair-credential.md)), and these are, because the vault can make their replacements. The broker's provisioner then adopts the accounts genesis created. From then on the vault makes every shared secret, and genesis has made its @@ -132,7 +134,9 @@ last one. **Raising the vault or the broker again is a genesis act.** Moving the `mesh-vault` or `mesh-broker` seat to a new assignment, or recovering either after it is lost, is done the way genesis did it: the -values it needs are delivered, not made by a vault that is not there. That is a break-glass procedure, +values it needs are delivered, not made by a vault that is not there. They come from the operator-sealed +copies, which the operator opens. The vault keeps a copy of every secret sealed to the operator key +(0085), so nothing the mesh relies on exists only inside the vault. That is a break-glass procedure, stated and checked, never an ordinary assignment. **A provider makes resources and data, and the mesh carries data back.** A provider's adapter may @@ -149,14 +153,17 @@ not rotated by the vault: rotating it means an operator delivering a new one. A issued is rotated by the module that holds the backend asking it again and delivering the new value to the vault. -**Each recipient takes a new value one of two ways, marked on its requirement:** +**Each recipient takes a new value one of two ways, marked per recipient:** | recipient takes it by | example | what happens on rotation | |---|---|---| | **applying** it | a provider setting a login's password; the broker's provisioner updating an account; a store's provisioner changing its own superuser | its provisioner applies the new value; it is never restarted for it | | **reading it at start** | a consumer reading its password when it starts | the host recreates it, because a file it read at creation changed | -The marking is on each requirement, not on the secret, because one secret has recipients of both kinds. +The marking is per recipient, not per secret, because one secret has recipients of both kinds. A +provision's contract marks its provider's side, which applies. A consumer's side is read at start +unless its requirement says otherwise. The broker's contract marks the host's bus account the same +way: the broker's provisioner applies it, and the host reads it. Every module in the catalogue reads its secrets at start, and none watches them ([research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/02-the-readers.md)). A secret a backend takes only when it first initialises is marked applied, and its provider's provisioner makes @@ -165,7 +172,7 @@ rotatable by the mesh**, and a rotation request is refused, saying why, rather t that would carry on with the old value. **How old and new change over is decided in [ADR -0114](0114-a-shared-credential-rotates-over-two-logins.md).** Three mechanisms were measured against +0114](0114-a-shared-credential-rotates-over-two-credentials.md).** Three mechanisms were measured against every provider's code in [research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): in place, as the controller's `rotate` does today; two secrets on one login; and two logins over one resource. Its findings bound the @@ -173,15 +180,15 @@ choice: - every credential provider already re-applies a password in place, so today's rotation works, with a window in which a consumer cannot authenticate; -- seven of eight name the consumer's resource after its login, and five destroy the resource when they - remove the login. **No mechanism may retire a login through today's remove**, because in those five - it deletes the consumer's data; -- only one backend holds two passwords on one login; -- every backend can give two logins the same rights over one resource, once the adapter separates the - resource from the login. +- eight of nine name the consumer's resource after its login, and five destroy the consumer's data when + they remove the login. **No mechanism may retire a login through today's remove**, because in those + five it deletes the consumer's data; +- one backend holds two passwords on one login, and two more hold several tokens; +- every provider can hold two credentials over one resource, eight as two logins and one as two tokens, + once the adapter separates the resource from the credential. -On those facts, 0114 rotates a credential two parties hold over two logins, rotates one a single -party holds in place, and separates retiring a login from removing a consumer. +On those facts, 0114 rotates a credential two parties hold over two credentials, rotates one a single +party holds in place, staged, and separates retiring a credential from removing a consumer. ## What this changes in earlier records @@ -192,7 +199,9 @@ On acceptance, each of these is superseded or amended by this record, not edited credential and seals nothing stands. - [ADR 0085](0085-a-secret-is-a-provision.md) is amended: the vault makes every shared secret, own secrets are retired, and its rejection of vault-only minting is answered by genesis delivering the - first secrets. "The vault stores no plaintext, ever" stands. + first secrets. Genesis seals its values to the control-node's key as well as to the operator key, so + the controller can deliver them unattended. "The vault stores no plaintext, ever" and the + operator-sealed break-glass copies stand. - [ADR 0092](0092-an-operator-delivers-a-pair-credential.md) is amended: an operator delivers a secret to the vault. Genesis's values reach the vault by delivery too, but are recorded as the mesh's own, so 0092's rule that an operator's value is never replaced does not apply to them. @@ -201,9 +210,14 @@ On acceptance, each of these is superseded or amended by this record, not edited - [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), [to-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md) and [to-be 24](../03-DESIGN/01-to-be/24-the-secrets-vault.md) are amended: the vault makes a rotated value and each - requirement says whether its recipient applies it or reads it at start, while the changeover - mechanism stays as to-be 13 describes until its own record; the vault is installed as soon as the + requirement says whether its recipient applies it or reads it at start, and the changeover is + [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s; the vault is installed as soon as the shared runtime base exists, and genesis delivers its secrets to it; the vault is the only maker. +- [To-be 07](../03-DESIGN/01-to-be/07-the-foundation.md) is amended: genesis seals its values to + the control-node's key as well as the operator key. +- [To-be 12](../03-DESIGN/01-to-be/12-a-module-repository.md), [to-be 16](../03-DESIGN/01-to-be/16-module-coverage.md) + and [to-be 18](../03-DESIGN/01-to-be/18-building-a-module.md) are amended: `own-secrets` is retired from the + manifest they describe. - The [glossary](../00-META/glossary.md) gains *shared secret*, *recipient*, *applies* and *reads at start*, and *reserved provision*, once this record is accepted. - [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) @@ -217,7 +231,7 @@ On acceptance, each of these is superseded or amended by this record, not edited - Resolution expands per-consumer requirements from a provision's contract. The contract declares them, never the provider's code, so what a provider requires stays predictable from the catalogue. - A data provider's adapter gains a return value. What a credential provider's adapter must change for - rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-logins.md)'s. + rotation is [ADR 0114](0114-a-shared-credential-rotates-over-two-credentials.md)'s. - The broker's provisioner gains every bus account, and the controller loses five separate places it generates a secret today. - 54 modules move from own secrets to vault requirements. Six provider clients export a password @@ -236,7 +250,7 @@ On acceptance, each of these is superseded or amended by this record, not edited | A private key is made where it is used | A test per key: a node's sealing key never leaves the node, the operator's private key never enters the mesh, and the certificate authority's private key never leaves the controller's identity store. | | Only the vault provides `secret` | The parser refuses a definition providing `secret` that cannot hold `mesh-vault`, and resolution refuses a pin on a `secret` requirement. | | The controller and each node's host take the same path | A catalogue test: the controller's definition declares no own secret, only requirements. A controller test: a node's bus account is made by the vault and delivered sealed to that node; an enrolment token reaches the controller only as what verifies it. | -| Genesis's values reach the vault unattended | An installer test: genesis's values are sealed to the control-node's key and delivered by the controller when the vault is installed, with no operator step. | +| Genesis's values reach the vault unattended, and the operator keeps a copy | An installer test: each of genesis's values is sealed to the control-node's key and to the operator key; the controller delivers the first to the vault when it is installed, with no operator step; the operator's copy opens only with the operator key. | | Moving the vault or broker is a procedure | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. | | A backend-issued secret enters through the vault | A vault test: a value delivered as issued is provided like any other, and rotating it is refused as the vault's act. | | A secret with no provisioner to apply it is not rotated by restart | A vault test: rotating a secret whose requirement is marked not rotatable by the mesh is refused, naming why. | diff --git a/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md new file mode 100644 index 0000000..6cef363 --- /dev/null +++ b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md @@ -0,0 +1,245 @@ +--- +topic: what runs on it +status: proposed +date: 2026-09-26 +deciders: jochen +reconstructed: false +extends: 0113-the-vault-makes-every-secret.md +--- + +# 114. A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached + +## Context + +[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the +vault's policy) and who makes the new value (the vault). It leaves open how old and new change over. +[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every +credential provider in the catalogue against its code. There are nine: + +- **all nine re-apply a password in place**, on the same login, every time they run. The controller's + `rotate` command relies on that, and states the window it leaves: between the provider applying the + new value and the consumer restarting with it, the consumer cannot authenticate; +- **eight of nine name the consumer's resource after its login**: a database, a bucket, a virtual host, + a key prefix, a topic prefix, a mailbox. Only the forge's npm registry keeps them apart, because an + organisation owns the packages; +- **five of nine destroy the consumer's data when they remove its login**: postgres, mssql and mongodb + drop the database, lavinmq drops the virtual host with its queued messages, and mailu deletes the + mailbox with its mail. In those adapters, *retire a login* and *delete the consumer's data* are one + call. minio drops a bucket only if it is empty. The provisioner harness makes it worse: a consumer + whose derived login changed is removed under the old login and created under the new one, in one pass; +- **one backend holds two passwords on one login** (redis), and two hold several tokens beside one + password (the forge and mailu); +- **eight of nine can give two logins the same rights over one resource**. mailu cannot, because a mail + user *is* its mailbox. It can give one user several tokens. In postgres, a second login is not enough + on its own: objects belong to whichever login created them, so the resource must be owned by a role of + its own; +- **an administrative credential has one party and a fixed name.** The provider module both applies it + and reads it. Five backends take it only at first initialisation: postgres, mssql, mongodb, mosquitto + and lavinmq. Their credential file is mounted directly into both the server and the provisioner, so + replacing the file recreates the provisioner holding only the new value, which the backend does not + know yet. The provisioner is then locked out; +- **no module watches a secret.** Every reader reads at start, and the host recreates a container when + a file it read at creation changes + ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). + +A first draft of 0113 chose to overlap old and new "through the adapter's existing create and +remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on +what the providers do, and the danger has to be closed whichever mechanism is chosen. + +## Considered Options + +**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials +two parties hold. The window cannot be closed, only shortened, and the two ends are on different +machines with nothing ordering them. For an administrative credential, it locks the provisioner out. + +**2. Two secrets on one login.** Rejected as the mechanism. It works for three providers out of nine, +and using it there and something else elsewhere would put the difference in the mesh instead of in the +adapter. + +**3. Two logins over one resource.** Rejected as the mechanism. It works for eight of nine, and not for +mailu. + +**4. Two credentials over one resource, with the adapter choosing what a credential is.** A credential +is what a consumer presents, a login and a secret. The mesh alternates between two of them. Each adapter +makes the second one the way its backend can: a second login for eight providers, a second token on the +same login for mailu. A credential a single party holds is staged in place instead, and retiring a +credential is separated from removing a consumer before either is used. Chosen. + +## Decision + +### Retiring a credential never removes what it reached + +**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its +database, bucket, virtual host, key or topic prefix, mailbox) and a *credential* that reaches it. +They get separate operations: + +- **ensure the resource**, named after the consumer; +- **ensure a credential** with a value, holding the consumer's rights over its resource; +- **retire a credential**. Anything it owns moves first to the resource's owner, and any session it has + open is ended. Then the credential is removed, and nothing else; +- **remove the consumer**, which is what removes the resource, and retires every credential it has. + +**Remove the consumer runs only when the consumer no longer requires the provision from this provider.** +That happens when its assignment goes, when its definition drops the requirement, or when re-resolution +sends it to another provider. It never runs because a login or a value changed. The harness keys what it +applied by the consumer, not by the login, so a changed login is a credential change and never a removal. +What removing a resource does with the data in it stays +[ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s, and re-resolving to another provider +moves no data. + +**The resource is named after the consumer, and owned by the resource, not by a login.** A consumer's +identity is derived from its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), +and today its login is that same string, so **no existing resource is renamed**. Where a backend makes +whatever a login creates the login's own, as postgres does, the resource is owned by a role that cannot +log in, and each credential works as that role. Ownership of an existing resource moves to it once. A +credential is retired by handing what it owns to that role, never by dropping what it owns. + +### A credential two parties hold rotates over two credentials + +**Two parties** means an applier and a reader that are different modules, or a module and a node's +host. The vault's custody copy does not count, because the vault holds every secret. So this covers a +credential between a consumer and a provider, and every bus account: a module's or a host's, applied by +the broker's provisioner and read by its owner. **Each consumer has two credentials, one in use at a +time**, both holding the same rights over the one resource. For eight providers the second is a second +login, derived by the mesh as the consumer's identity with a short fixed suffix. For mailu it is a +second token on the same login. + +**The vault drives each rotation and records every step durably.** A provisioner learns which +credentials to hold from what it receives: both of them, for as long as a rotation is under way. It +never learns them from its own memory, so a provisioner restarted mid-rotation resumes from the step the +vault has recorded. + +1. **The vault makes the new value.** +2. **Each applier ensures the unused credential with it**, with the consumer's rights, and leaves the + one in use untouched. It verifies that the new credential authenticates and the old one still does, + and confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so + a lost message costs one pass. +3. **Only then does the vault release the new credential to the readers.** The mesh delivers it and the + value together, and the host recreates each reader, because a file it read at creation changed. A + node's host is its own reader: it reconnects to the bus with the new login, and confirms over it. +4. **Each reader confirms by authenticating with the new credential.** It shows this through its + health check, where its definition declares one, or the applier sees the new credential in use, + where its backend reports that. A reader for which neither is possible is confirmed by an operator. + It is never assumed from the reader having restarted. +5. **Only when every reader has confirmed is the old credential retired**, as above, and verified to no + longer authenticate. + +**A reader that goes away leaves the rotation.** A reader unassigned, or re-resolved to another +provider, is no longer waited for. A consumer removed mid-rotation has both of its credentials retired +with it. + +**A rotation can be abandoned until the old credential is retired.** An operator abandons it. Readers +that moved are given the old credential back, and recreated. The new credential is retired. Nothing is +lost, because the old one was never removed. + +`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done +until the old credential is gone. A reader that cannot be reached keeps working on the old credential +until it can, and the rotation waits for it. That wait is shown, never hidden. + +**Queues and permissions belong to the consumer, not to a login.** A module's queue on the bus is named +for the module on its node, and both of its logins get the same permissions over it +([ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)). An MQTT client +identifier is chosen by the consumer and is independent of its login. A reader recreated with a new +login keeps it, and the broker hands the session over. + +### A credential a single party holds rotates in place, staged + +This covers a provider's administrative credential and a module's own secret, which only that module +reads. The vault makes the new value, and the one party takes it: + +- **applied**: the vault delivers the new value **staged, beside the current one**, and the current file + is left as it is. The party's provisioner changes the backend using the current value, verifies the + new one, and confirms. Only then does the vault make the new value current. This is the only form for + a backend that takes its administrative credential only at first initialisation. Replacing the file + first would lock the provisioner out; +- **read at start**: the vault delivers the new value as current, and the host recreates the party. + +There is no window between two parties, because there is only one. Where neither form can change the +value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why +([ADR 0113](0113-the-vault-makes-every-secret.md)). + +### One rule decides which + +**The number of parties decides, never the provider.** The resolver knows it from the requirement's +recipients, leaving out the vault's custody copy, so no definition declares it. + +### Until an adapter can + +**An adapter that cannot yet ensure a second credential says so.** The two-party credentials it applies +rotate in place, as today, and the window is stated when the rotation is asked for. So does a +two-party credential whose backend has one fixed name and no second credential for it. These are listed +by a check, and the list is meant to shrink. Separating *retire a credential* from *remove the +consumer*, and keying the harness by consumer, come first. They close a data-loss path that exists +today, whatever rotation does. + +## What this changes in earlier records + +On acceptance, each of these is amended by this record, not edited: + +- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here. +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room + for the second login's suffix within the tightest backend it reaches, and both logins are checked + against it. +- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker + account is two logins with the same permissions over the same queue, one in use at a time. Its scoping + is unchanged. +- [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md), already superseded by 0113: + a provider now ensures and retires credentials over a resource it owns separately. +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party + credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed. + A single-party credential is staged, not replaced. + +## Consequences + +- **Every credential provider's adapter changes**, in two steps. The first separates *retire a + credential* from *remove the consumer*. It names and owns the resource after the consumer, which + keeps the name it has but moves ownership once in postgres and mssql, and the harness is keyed by + consumer. The second ensures a second credential with the same rights. +- **The vault gains rotation state**: each rotation's step, per applier and reader, recorded durably. + Staged delivery is added for single-party secrets. The SDK harness carries the alternation and the + repeated confirmation, so no adapter implements them. +- **No consumer module changes.** It reads one credential at start, as today, and is recreated by the + host when it changes. The exception is a reader that has neither a health check nor a backend that + reports use: its rotations wait for an operator until it declares one. +- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20 + characters. +- **What got harder:** + - a provider briefly holds two credentials per consumer; + - a rotation lasts until its slowest reader moves, so an unreachable reader keeps the old credential + valid until it is reached; + - an adapter has four operations where it had two; + - retiring a login in mssql has to end its sessions first. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Retiring a credential never removes a resource | A provider test per credential provider: retiring one of a consumer's credentials leaves its resource and data intact, reachable through the other. | +| What a retired login owned survives it | A postgres and an mssql test: objects created under login A, tables included, are still there and alterable under login B after A is retired. | +| A changed login is not a removal | A harness test: changing a consumer's derived login ensures a credential and never calls remove. | +| Remove runs only when the requirement goes | Harness tests: unassigning, dropping the requirement and re-resolving each remove the consumer once; a rotation and a login change never do. | +| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, with ownership moved to the resource's own role where the backend needs one. | +| Both credentials hold the same rights | A provider test per credential provider: data and structure created under one credential are read, changed and altered under the other. | +| Readers move only after the applier confirms | A rotation test: readers receive nothing until both credentials authenticate at every applier. | +| A reader confirms by authenticating | A rotation test: a reader recreated but failing to authenticate with the new credential does not confirm, and the old credential is not retired. | +| The old credential is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old credential, the rotation shows waiting on it, and it completes when the reader returns and confirms. | +| A reader that goes away leaves the rotation | A rotation test: unassigning a waiting reader lets the rotation complete; removing the consumer mid-rotation retires both credentials. | +| A rotation can be abandoned | A rotation test: abandoning after readers moved gives them the old credential back and retires the new one. | +| Rotation state survives a restart | A test restarting the applier's provisioner, and then the vault, between steps: the rotation resumes from the recorded step. | +| A single-party applied secret is staged | A rotation test on a first-initialisation administrative credential: the provisioner receives the new value beside the current one, applies it, and only then does the new value become current. At no point does it lose its connection. | +| The number of parties decides | A resolution test: a secret with an applier and a reader in different parties is marked for two credentials, and one held by one module for in place. The vault's copy is not counted. | +| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. | +| A host rotates its bus login | A rotation test on a node's bus account: the host reconnects with the new login and confirms over the bus before the old one is retired. | +| What still rotates in place is listed | A catalogue test lists every adapter that cannot yet ensure a second credential, and every two-party credential with one fixed name. A rotation of these states its window. | + +## References + +- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this + rests on, provider by provider +- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes +- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), + [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving + its declaration +- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented +- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): + why a reader's restart can be derived diff --git a/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md b/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md deleted file mode 100644 index 29c1b49..0000000 --- a/02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md +++ /dev/null @@ -1,198 +0,0 @@ ---- -topic: what runs on it -status: proposed -date: 2026-09-26 -deciders: jochen -reconstructed: false -extends: 0113-the-vault-makes-every-secret.md ---- - -# 114. A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached - -## Context - -[ADR 0113](0113-the-vault-makes-every-secret.md) decides who asks for a rotation (an operator, or the -vault's policy) and who makes the new value (the vault). It leaves open how old and new change over. -[Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md) read every provider -in the catalogue against its code: - -- **all eight credential providers re-apply a password in place**, on the same login, every time they - run. The controller's `rotate` command relies on that, and states the window it leaves: between the - provider applying the new value and the consumer restarting with it, the consumer cannot authenticate; -- **seven of eight name the consumer's resource after its login**: a database, a bucket, a virtual host, - a key prefix, a topic prefix. Only the forge's npm registry keeps them apart, because an organisation - owns the packages; -- **five of eight destroy the resource when they remove the login**: postgres, mssql, mongodb, minio and - lavinmq. In today's adapters, *retire a login* and *delete the consumer's data* are one call; -- **only one backend holds two passwords on one login**, redis. A second, the forge, holds several - tokens beside one password; -- **every backend can give two logins the same rights over one resource**: a group role, a database - role, a shared policy, shared permissions, a shared role, a shared key prefix, a shared team. No - adapter does it today; -- **administrative credentials have one party and a fixed name.** The provider module is both the one - that applies the value and the only one that reads it. Three backends take it only at first - initialisation, so it can only be changed through a command run with the old value; -- **no module watches a secret.** Every reader reads at start, and the host already recreates a - container when a file it read at creation changes - ([issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md)). - -A first draft of 0113 chose to overlap old and new "through the adapter's existing create and -remove". In five providers, that remove deletes the consumer's data. The mechanism has to be chosen on -what the providers do, and the danger has to be closed whichever mechanism is chosen. - -## Considered Options - -**1. In place for everything, as today.** Works with every provider unchanged. Rejected for credentials -two parties hold. The window cannot be closed, only shortened, and the two ends are on different -machines with nothing ordering them. A reader the mesh cannot reach, but which still reaches its -provider, is locked out until the mesh reaches it again. - -**2. Two secrets on one login.** The applier adds the new password beside the old one. Rejected. It -works for one provider out of eight. Using it where it exists and something else elsewhere is a -mechanism per provider, which is what the mesh is trying to stop having. - -**3. Two logins over one resource for everything.** Rejected for credentials a single party holds. -The applier and the reader are the same module, so there is no second party to keep working while the -other moves. A fixed administrative name has no second name to alternate with. - -**4. Split by the secret, not by the provider.** A credential two parties hold rotates over two logins; -one a single party holds rotates in place; and retiring a login is separated from removing a consumer -before either is used. Chosen. - -## Decision - -### Retiring a login never removes what it reached - -**A provider's adapter keeps two things apart that today are one:** the consumer's *resource* (its -database, bucket, virtual host, key or topic prefix) and the *login* that reaches it. They get separate -operations: - -- **ensure the resource**, named after the consumer; -- **ensure a login** with a value, holding the consumer's rights over its resource; -- **retire a login**, which removes the login and nothing else; -- **remove the consumer**, which is what removes the resource. It is run only when the consumer is - unassigned, as today, and never by a rotation. Whether the resource's data is kept beyond that stays - [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)'s. - -**The resource is named after the consumer, not after a login.** A consumer's identity is derived from -its assignment ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)), and today its login -is that same string. So **no existing resource is renamed**. The login every consumer holds today -becomes the first of its two, under the name it already has. - -### A credential two parties hold rotates over two logins - -This covers a credential between a consumer and a provider, and a bus account, which the broker's -provisioner applies and its module reads. **Each consumer has two logins, derived by the mesh**: its -identity, and its identity with a short fixed suffix. It uses one at a time, and both hold the same -rights over the one resource. - -1. **The vault makes the new value.** -2. **Each applier ensures the unused login with it**, with the consumer's rights, and leaves the login - in use untouched. It verifies that the new login authenticates and the old one still does, and - confirms. It repeats the confirmation on every reconcile pass until the vault acknowledges it, so a - lost message costs one pass. -3. **Only then are the readers given the new login and value, together.** The host recreates each - reader, because a file it read at creation changed. -4. **Each reader confirms** by being recreated with the new login and passing its health check, where - its definition declares one. -5. **Only when every reader has confirmed is the old login retired.** Each applier retires it, the - login and nothing else, and verifies that it no longer authenticates. - -`status` shows a rotation as waiting on whichever applier or reader has not moved, and it is not done -until the old login is gone. A reader that cannot be reached keeps working on the old login until it -can, and the rotation waits for it. That wait is shown, never hidden. - -**Where a backend identifies a connection separately from a login, the two are kept apart.** An MQTT -client identifier must be unique per connection, so the mosquitto adapter derives the connection's -identifier from the consumer and the login in use, and two logins never collide. - -### A credential a single party holds rotates in place - -This covers a provider's administrative credential and a module's own secret, which only that module -reads. **The vault makes the new value, and the one party takes it:** - -- **applied**: the party's provisioner changes it using the old value, then confirms. This is the only - form for a backend that takes its administrative credential only at first initialisation, where a - restart would change nothing; -- **read at start**: the host recreates the party. - -There is no window between two parties, because there is only one party. Where neither form can change -the value, the requirement is marked not rotatable by the mesh, and a rotation is refused, saying why -([ADR 0113](0113-the-vault-makes-every-secret.md)). - -### One rule decides which - -**The number of parties that hold the credential decides, never the provider.** A requirement whose -secret has an applier and a reader in different modules rotates over two logins. A secret held by one -module rotates in place. The resolver knows which from the requirement's recipients, so no definition -declares it. - -### Until an adapter can - -**An adapter that cannot yet ensure a second login says so**, and the credentials it applies rotate in -place, as today, with the window stated when the rotation is asked for. That is a migration state, not -a second mechanism. It is listed by a check, and the list shrinks to empty. Separating *retire a -login* from *remove the consumer* comes first in every adapter, because it closes a data-loss path that -exists today, whatever rotation does. - -## What this changes in earlier records - -On acceptance, each of these is amended by this record, not edited: - -- [ADR 0113](0113-the-vault-makes-every-secret.md): the changeover it left open is decided here. -- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md): a consumer's identity leaves room - for the second login's suffix within the tightest backend it reaches, and both logins are checked - against it. -- [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md): a module's broker - account is two logins with the same permissions over the same queue, one in use at a time. Its scoping - is unchanged. -- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation of a two-party - credential is no longer all-or-nothing with a window. It overlaps, with each step confirmed. - Single-party rotation keeps the form to-be 13 describes. - -## Consequences - -- **Every credential provider's adapter changes**, in two steps. The first separates *retire a login* - from *remove the consumer*, and names the resource after the consumer, which is the name it already - has. The second ensures a second login with the same rights. That is seven adapters for the second - step, since the forge's already holds its packages apart from the user. -- **The SDK's provider harness** carries the alternation, the verification of both logins, and the - repeated confirmation, so no adapter implements them. Its record of what was applied has to survive a - restart of the provisioner mid-rotation. Today it is kept in memory. -- **No consumer module changes.** It reads one login and a value at start, as today, and is recreated - by the host when they change. -- **The derived identity is two characters tighter** in the tightest backend, a minio access key of 20 - characters. -- **What got harder:** a provider briefly holds two logins per consumer. A rotation of a two-party - credential lasts until its slowest reader moves, so an unreachable reader keeps the old login valid - until it is reached. And an adapter has four operations where it had two. - -## How it is checked - -| Rule | Checked by | -|---|---| -| Retiring a login never removes a resource | A provider test per credential provider: retiring one of a consumer's logins leaves its resource and data intact, reachable through the other. | -| Removing a consumer is not a rotation | A harness test: no rotation step calls remove; remove runs only when a contribution goes. | -| No existing resource is renamed | A provider test: a consumer created before the change keeps its resource, and its existing login becomes the first of its two. | -| Both logins hold the same rights | A provider test per credential provider: data written under one login is read and changed under the other. | -| Readers move only after the applier confirms | A rotation test: readers receive nothing until both logins authenticate at every applier. | -| The old login is retired only after every reader confirms | A rotation test with one reader's node unreachable: it keeps authenticating with the old login, the rotation shows waiting on it, and it completes when the reader returns and confirms. | -| A lost confirmation costs one pass | A harness test dropping the first confirmation: the next pass repeats it. | -| A restarted provisioner resumes a rotation | A harness test restarting the provisioner between steps: it resumes from the step it reached. | -| A single-party secret rotates in place | A vault test: a provider's administrative credential is applied by its own provisioner with the old value; a module's own secret recreates the module; neither has a second login. | -| The number of parties decides | A resolution test: a secret with an applier and a reader in different modules is marked for two logins, and one held by one module for in place, with nothing declared. | -| Both logins fit the tightest backend | A controller test: both derived logins for the longest node and module names fit the limit ADR 0049 sets. | -| A connection identifier never collides | A mosquitto provider test: two connections under a consumer's two logins are both accepted. | -| Adapters still in place are listed | A catalogue test lists every credential provider that cannot yet ensure a second login. The list shrinks to empty, and a rotation of their credentials states its window. | - -## References - -- [Research 016](../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md): the survey this - rests on, provider by provider -- [ADR 0113](0113-the-vault-makes-every-secret.md): who asks and who makes -- [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md), [ADR 0043](0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md), - [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md): identity, bus accounts, and data outliving - its declaration -- [To-be 13](../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md): rotation as implemented -- [Issue 103](../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md): - why a reader's restart can be derived diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 8bb0aad..dd3835f 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -159,7 +159,7 @@ python3 00-META/checks/index.py fail if stale - **0110** — [A seat is held by one assignment, from a closed set, and it may deliver a provision](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) *(proposed)* - **0112** — [A module definition names no node, no mesh and no path: everything it needs is a requirement the mesh resolves](0112-a-module-definition-names-no-node-mesh-or-path.md) *(proposed)* - **0113** — [The vault makes every shared secret, a provider makes resources and data, and the mesh carries both](0113-the-vault-makes-every-secret.md) *(proposed)* -- **0114** — [A credential two parties hold rotates over two logins; one a single party holds rotates in place; and retiring a login never removes what it reached](0114-a-shared-credential-rotates-over-two-logins.md) *(proposed)* +- **0114** — [A credential two parties hold rotates over two credentials; one a single party holds rotates in place, staged; and retiring a credential never removes what it reached](0114-a-shared-credential-rotates-over-two-credentials.md) *(proposed)* ### How it is built diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 28adb35..3baacb3 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -152,3 +152,19 @@ With the seat unheld, a build from the seat is refused and says why. External bu **Not yet designed:** a credential for cloning a private repository. The mesh's own repositories are public. The natural place for a clone credential is a `secret` from the vault, and that is a decision still to take. + +## How it is checked + +The rules here are [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)'s +and [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md)'s, and each is +checked as their tables say: + +| Rule | Checked by | +|---|---| +| The set is closed, and every entry names its decision | 0110: a unit test on the set's size and decisions; manifest tests refusing an unknown seat or the wrong scope. | +| A seat is held by one assignment, and only by one whose module can hold it | 0110: resolution tests for a second holder and for a seat the definition does not name. | +| A requirement naming a seat is answered by its holder; a foundation seat cannot be named | 0110: resolution tests with a second provider on the consumer's node, with the seat unheld, and naming `mesh-store`. | +| Several providers and none local is a person's choice | 0110: an assignment test listing candidates with the seat's holder first and recording the pin. | +| `secret` is reserved | 0110: the parser and resolution refusals for another provider and a pin. | +| Holdings are derived, and the overview lists every seat | 0110: the `seats` command test, including an unheld seat. | +| A build source on the seat records no address; an unheld seat refuses only self-hosted builds | 0111's tests. | diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index ed64d18..bd0e147 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -6,7 +6,7 @@ updated: 2026-09-26 decisions: - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - 02-DECISIONS/0113-the-vault-makes-every-secret.md - - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md + - 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md - 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md - 02-DECISIONS/0084-which-provider-serves-a-consumer.md @@ -64,8 +64,9 @@ Which module answers, in order: the provider open. It asks for *the mesh's* one, and the mesh answers with whichever assignment holds that seat, with nothing asked of anyone. Unheld, the requirement is refused, naming the seat ([ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), - [26 — The seats](26-the-seats.md)). A `secret` requirement always names `mesh-vault`, because - that provision is reserved; + [26 — The seats](26-the-seats.md)). Only a seat that delivers a provision can be named; naming a + foundation seat is refused, because it delivers nothing. A `secret` requirement always names + `mesh-vault`, because that provision is reserved; 2. **a pin**: the assignment names a provider, because this consumer is coupled to that provider's contents ([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md)); 3. **the provider on the consumer's own node**; @@ -250,22 +251,28 @@ applied by its provisioner or marked not rotatable by the mesh, and a rotation o than reported done. **How old and new change over depends on how many parties hold the credential** -([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md), on +([ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), on [research 016](../../01-RESEARCH/016-how-a-credential-can-be-rotated/00-overview.md)): -- **Two parties**, a consumer and its provider, or a module and the broker: each consumer has two logins - derived by the mesh, both with its rights over one resource, named after the consumer. The vault makes - the new value; each applier ensures the unused login with it and confirms both authenticate; only then - are readers given it and recreated; once every reader has confirmed, the old login is retired. Nobody - is left without a credential that works, and `status` shows who a rotation waits on. -- **One party**, a provider's administrative credential or a module's own secret: in place. Its own - provisioner applies it with the old value, or the host recreates it. +- **Two parties**, a consumer and its provider, or a module or node's host and the broker: each consumer + has two credentials, both with its rights over one resource named after the consumer. For most + providers the second is a second login derived by the mesh; where a backend's user is its resource, it + is a second token. The vault drives the rotation and records each step. It makes the new value; each + applier ensures the unused credential and confirms both authenticate; only then are readers given it + and recreated; each reader confirms by authenticating with it; and only then is the old one retired. + Nobody is left without a credential that works, a rotation can be abandoned until the old one is + retired, and `status` shows who a rotation waits on. +- **One party**, a provider's administrative credential or a module's own secret: in place, staged. + An applied one is delivered beside the current value, the provisioner changes the backend with the + current one, and only then does the new value become current. One read at start is delivered, and + the host recreates the module. -**Retiring a login never removes what it reached.** An adapter keeps *retire a login* and *remove the -consumer* apart. Only unassigning removes the resource, and never a rotation. Today the two are one -call, and in five providers it deletes the consumer's data, so this separation comes first. An adapter -that cannot yet ensure a second login rotates in place, with its window stated, and is listed until it -can. +**Retiring a credential never removes what it reached.** An adapter keeps *retire a credential* and +*remove the consumer* apart, and the harness keys what it applied by consumer, so a changed login is +never a removal. The resource is removed only when the consumer no longer requires it from that +provider: unassigned, the requirement dropped, or re-resolved elsewhere. Today these are one call, and +in five providers it deletes the consumer's data, so this separation comes first. An adapter that cannot +yet ensure a second credential rotates in place, with its window stated, and is listed until it can. ## Refusing @@ -314,8 +321,8 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r mesh carries providers' data back. [Issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md) is fixed in the host already. *Ends when* nothing outside the vault generates a shared secret after - genesis, a lab consumer of analytics receives its site id, and a database credential rotates over its two - logins, the consumer recreated by derivation, never without a working login, and its data intact. + genesis, a lab consumer of analytics receives its site id, and a database credential rotates over + its two credentials, the consumer recreated by derivation, never without a working login, and its data intact. 3. **Definitions move, and seats move to assignments.** Every catalogue definition is rewritten, adopted and running assignments placed where their data already is, and each claim becomes a seat the module can hold, held by the assignment that holds it today. *Ends when* the list of definitions @@ -342,7 +349,12 @@ Each phase ends at a check that holds, so none of them leaves a mechanism half-r | Only the vault provides `secret` | The parser refuses another provider of it, and resolution refuses a pin on a `secret` requirement. | | A provider's per-consumer secret comes from the vault | A resolution test: requiring a database expands to a secret requirement named for the consumer, answered by the vault and delivered to both recipients. | | Restarts are derived | The tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): an applied secret restarts nothing, and one read at start recreates its reader without a declared restart. | -| A two-party credential rotates over two logins | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-logins.md): retiring a login leaves the resource intact; readers move only after the applier confirms; an unreachable reader keeps its old login until it returns; a single-party secret rotates in place. | +| A two-party credential rotates over two credentials | The rotation tests of [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md): retiring a credential leaves the resource intact; a changed login is never a removal; readers move only after the applier confirms and confirm by authenticating; an unreachable reader keeps its old credential until it returns; rotation state survives a restart; a single-party applied secret is staged. | +| A private key is made where it is used | The per-key tests of [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md): a node's sealing key, the operator's key and the certificate authority's key never leave where they were made. | +| The controller and a node's host take the same path | 0113's tests: the controller's definition declares requirements and no own secret; a node's bus account is made by the vault and delivered sealed to that node. | +| Moving the vault or the broker is break-glass | A resolution test: an ordinary assignment moving `mesh-vault` or `mesh-broker` is refused, naming the procedure. | +| A secret that cannot be rotated says so | A vault test: rotating a secret marked not rotatable by the mesh is refused, naming why. | +| Only the controller reads the seat placeholder | A catalogue test, from phase 3: no definition uses the seat placeholder. | | Refusal names everything at once | A resolution test with three unresolved requirements of different kinds: one refusal naming all three. | | The old forms retire | The catalogue test listing definitions still using one. It must be empty before a form is removed. | diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 6a913d5..3c8f3d3 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -34,8 +34,8 @@ document is written and this one's status becomes `implemented`. | [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) | | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | [ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md), [ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md) | | [`24-the-secrets-vault.md`](24-the-secrets-vault.md) | The module that owns a secret — a `secret` provision, and the boundary of what it owns | [ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md), [ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md), [ADR 0048](../../02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md) | -| [`26-the-seats.md`](26-the-seats.md) | What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | -| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | +| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) | +| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md) | ## Not yet written diff --git a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md index f34684d..78eb3dd 100644 --- a/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md +++ b/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md @@ -1,8 +1,8 @@ --- -status: located +status: resolved opened: 2026-09-23 located-in: [mesh-host internal/apply] -fixed-by: +fixed-by: mesh-host PR #22 — a container records the digest of every file it reads at creation, its env-files and files mounted into it directly, and is recreated when one changes; a mounted directory still needs restart-on amended-design: --- diff --git a/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md index f5c5c90..ec7cd08 100644 --- a/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md +++ b/04-ISSUES/119-a-module-definition-decides-where-its-files-live/00-report.md @@ -1,7 +1,7 @@ --- -status: open +status: located opened: 2026-09-25 -located-in: [] +located-in: [mesh-catalog modules, mesh-controller internal/catalogue] fixed-by: amended-design: --- @@ -60,6 +60,11 @@ module to one node would share every one of them. Assigning the same application ordinary need: production beside staging, one site per customer, two instances of one service configured differently, two stores of one engine. +[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), which answers +this report, declines that need rather than meeting it. A module is assigned at most once to a node, +because every identity in the mesh is already a module on a node. The cases above become different +modules, or the same module on different machines. + ## Why it matters beyond this instance A definition that names machine paths is not portable between nodes. It cannot follow data onto a From b3bd50c58825386e1e3047d86e2b4f646f7aecb3 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:44:10 +0200 Subject: [PATCH 24/26] Issue 120: a provisioner remembers what it did, not what is there The harness compares against its own memory, so a backend that loses what was provisioned (the cache's ACL users on a server restart) is never provisioned again, silently. --- .../00-report.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 04-ISSUES/120-a-provisioner-remembers-what-it-did-not-what-is/00-report.md diff --git a/04-ISSUES/120-a-provisioner-remembers-what-it-did-not-what-is/00-report.md b/04-ISSUES/120-a-provisioner-remembers-what-it-did-not-what-is/00-report.md new file mode 100644 index 0000000..9b697f3 --- /dev/null +++ b/04-ISSUES/120-a-provisioner-remembers-what-it-did-not-what-is/00-report.md @@ -0,0 +1,62 @@ +--- +status: located +opened: 2026-09-26 +located-in: [mesh-sdk src/provisioner, mesh-catalog modules/redis] +fixed-by: +amended-design: +--- + +# 120 — A provisioner remembers what it did, not what is there + +## What was observed + +The provisioner harness every provider is built on keeps, in memory, a hash of what it last applied +for each consumer: the login, the password and the values. On each pass it skips a consumer whose +hash has not changed. It never asks the backend whether what it made is still there. + +The cache module shows what that allows. Its server is configured with a password and a data +directory, and **no ACL file**. So the per-consumer ACL users its provisioner creates exist only in +the server's memory. The server and the provisioner run in separate containers: + +1. the provisioner creates an ACL user for each consumer, and records it as applied; +2. the server restarts, for an upgrade or a crash, and comes back with no consumer users; +3. the provisioner, still running, sees nothing changed in what it receives, and does nothing; +4. every consumer of the cache fails to authenticate, and **nothing reports it**. The provisioner's + log is quiet, and the mesh's status is green. + +The consumers recover only when the provisioner itself restarts, because its memory is then empty. +Rotating the cache's administrative password happens to cover it, because that file is mounted into +the provisioner too and recreates it. Nothing else does. + +Evidence, from the catalogue's and the SDK's main branches: the harness's reconcile loop (`applied`, +keyed by login, compared by hash before `create`), and the cache module's rendered configuration, +which names no ACL file. Found during research 016, how a credential can be rotated, proposed +alongside to-be 27. + +## Why it matters beyond this instance + +The cache is the case where the backend forgets on its own. The same gap opens whenever a backend +loses what was provisioned while the provisioner keeps running: a store restored from a backup taken +before a consumer was added, a login removed by hand, a server recreated on an empty data directory. +In each of them the provisioner reports that everything is applied, because it compares against its +own memory and not against the backend. + +The harness's other half has the same shape. A consumer's contribution that disappears while the +provisioner is down is never removed, because only logins the running process applied are candidates +for removal. What the mesh wants and what the backend holds can drift in both directions, and the +harness sees neither. + +This is the design permitting a silent failure. *A provider makes what its consumers require true* +is stated in [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md), and nothing +checks it after the first pass. + +## Open questions + +- Should the harness check each consumer's credential against the backend on every pass, or + periodically, instead of trusting its memory? Most adapters' `create` is already idempotent, so the + cheapest fix may be to drop the hash short-cut and apply every pass. What does that cost for a + provider that recreates an access key on every create, as the object store does? +- Should the cache keep its users in an ACL file, so a restart does not lose them? That fixes this + instance and leaves the gap for the others. +- Where does the record of what was applied live, if not in memory? ADR 0114, still + proposed, puts rotation state with the vault. The same place may answer this. From 0c5eac02184938fbe1be0e77d5749b3322c1e927 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:44:35 +0200 Subject: [PATCH 25/26] 0110 amends 0109: its seats are provisions, and moving npm takes a holdable seat --- ...0110-a-seat-is-a-module-assignment-from-a-closed-set.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md index 515b6dd..3101efb 100644 --- a/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md +++ b/02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md @@ -163,6 +163,13 @@ On acceptance, each of these is amended by this record, not edited: - [ADR 0084](0084-which-provider-serves-a-consumer.md): a requirement may name a seat, which its holder answers; and where several providers remain and none is local, the choice is asked when the module is assigned and recorded as a pin, rather than refused until someone pins it. +- [ADR 0109](0109-a-package-registry-seat-is-one-per-ecosystem.md): one provision per package + ecosystem stands. Where 0109 says *seat*, it means that provision. Only `npm-package-registry` is + also a seat in this set. A cargo or docker registry becomes one by a record, as any seat does. + "Gitea may hold several seats" reads: gitea may provide several ecosystems, and hold the seat of + each one that is a seat. Moving npm to verdaccio is not "assigning `npm-package-registry` to + verdaccio". It takes verdaccio's definition saying it can hold the seat, and then an assignment + holding it. - [To-be 23](../03-DESIGN/01-to-be/23-choosing-a-provider.md): the same two changes, in the design that describes choosing a provider. - [To-be 21](../03-DESIGN/01-to-be/21-the-installation-in-full.md): genesis assigns the foundation's From b5512bed3297ecec358f29f96f8598464efc5e89 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 26 Sep 2026 00:54:27 +0200 Subject: [PATCH 26/26] Research 017: a mesh that heals itself The operator's wish written as intended behaviour for the NATS bus: every loop compares against what is, repairs by the ordinary path, never destroys, and raises a condition for what it cannot fix. What is done before NATS is limited to what survives the move. --- .../00-overview.md | 47 ++++++++++ .../01-the-intended-behaviour.md | 85 +++++++++++++++++++ .../02-now-pragmatically.md | 46 ++++++++++ 3 files changed, 178 insertions(+) create mode 100644 01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md create mode 100644 01-RESEARCH/017-a-mesh-that-heals-itself/01-the-intended-behaviour.md create mode 100644 01-RESEARCH/017-a-mesh-that-heals-itself/02-now-pragmatically.md diff --git a/01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md b/01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md new file mode 100644 index 0000000..243e1ba --- /dev/null +++ b/01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md @@ -0,0 +1,47 @@ +--- +status: active +initiated: 2026-09-26 +touches: + - 00-META/mission.md + - 02-DECISIONS/0106-the-bus-is-nats.md + - 02-DECISIONS/0010-delivery.md + - 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md + - 03-DESIGN/01-to-be/06-the-controller.md + - 03-DESIGN/01-to-be/09-the-node-lifecycle.md + - 03-DESIGN/00-as-is/09-interfaces-and-observability.md +--- + +# 017 — A mesh that heals itself + +**What.** The behaviour the operator wants: a mesh that runs itself. It notices what is wrong, +repairs what it can, and hands what it cannot repair to someone who can, with the reason. This effort +writes that wish down as intended behaviour, designed for the bus the mesh is moving to +([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md): NATS). It also records what can be done +pragmatically before that move. + +**Why.** The mission is *a mesh that controls itself* ([mission](../../00-META/mission.md)). The +mesh can tell whether it is up. It cannot tell whether it is right. The as-is page on observability says so +([as-is 09](../../03-DESIGN/00-as-is/09-interfaces-and-observability.md)). To-be 06 names an +`observability` context in the controller and leaves its store undecided. Nothing routes a condition +the mesh cannot fix to anyone. The cost is measurable: **46 of the 116 issue reports in this +repository describe a failure that was silent.** A mesh that heals itself is, first, a mesh that stops +failing silently. + +**What it touches.** The controller's observability context, the node lifecycle's liveness, delivery +([ADR 0010](../../02-DECISIONS/0010-delivery.md), [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)), +the provisioner harness, and rotation, which is proposed alongside to-be 27 as ADR 0114. + +**Documents.** + +- [01 — The intended behaviour](01-the-intended-behaviour.md): the wish, as principles and as how + the mesh behaves once the bus is NATS. +- [02 — Now, pragmatically](02-now-pragmatically.md): what is done before NATS, why it does not + build anything the move would throw away, and what has been done already. + +**Next.** Two measurements this effort owes before it can graduate: + +1. **Every loop in the mesh**: what it converges, and whether it compares against observed state or + against its own memory. Issue 120 found the provisioner harness trusting memory. The same pattern is + expected elsewhere. +2. **The 46 silent failures, classified**: a missing observation, a loop trusting memory, or a missing + escalation. That shows which mechanism removes the most of them. diff --git a/01-RESEARCH/017-a-mesh-that-heals-itself/01-the-intended-behaviour.md b/01-RESEARCH/017-a-mesh-that-heals-itself/01-the-intended-behaviour.md new file mode 100644 index 0000000..28385d7 --- /dev/null +++ b/01-RESEARCH/017-a-mesh-that-heals-itself/01-the-intended-behaviour.md @@ -0,0 +1,85 @@ +# 01 — The intended behaviour + +The operator's wish, written as behaviour: what a person or an agent sees the mesh doing. This is a +target to design toward, not a design. Every part of it is to be decided through a record before it +is built. + +## Principles + +**1. Every loop compares what should be with what is, never with what it did.** Desired state is the +mesh's: assignments, requirements, seats. Observed state is read from the thing itself: the container, +the backend, the node. A loop that compares against its own memory of what it applied is blind to +anything that changed behind its back. That is issue 120, and it is the pattern this whole effort is +written against. + +**2. Healing is the ordinary path run again, never a second path.** Repairing a lost login is +provisioning it. Repairing a dead container is converging the node. Repairing a stale declaration is +delivering it. A repair that needs its own code is a second way of doing something, which is exactly +what the mesh is removing everywhere else. + +**3. A repair never destroys.** Healing may recreate, re-provision, re-deliver and restart. It may +never delete a consumer's data, retire a credential someone still uses, or pick a winner between two +contradictory states. Where the only repair is destructive, it is escalated. + +**4. Nothing fails silently.** Every condition the mesh cannot repair within its budget becomes +visible. It is named, it says since when, why, and who can resolve it. It is visible until it is +resolved, and resolved by observation, not by someone clicking it away. + +**5. What the mesh cannot fix goes to an agent.** Per the mission, an agent may be human or not. A +condition that needs judgement is handed to one, as work, with what the mesh knows. It is not handed +over as a notification that someone may or may not read. + +**6. Correctness, not only liveness.** A running process that authenticates with a dead credential, +serves an old version, or routes nowhere is not healthy. What a provision's contract promises is what +is checked: the credential authenticates, the route answers, the version is the declared one. + +## The loop, everywhere + +Every part of the mesh that owns something runs the same loop: + +1. **know** what should be true: from assignments, requirements and seats; +2. **observe** what is true: from the thing itself, on its own cadence; +3. **repair** the difference by running the ordinary path again, within a budget of attempts and + time; +4. **raise** a *condition* when the budget is spent or the only repair is destructive; +5. **clear** the condition when observation shows it resolved. + +A **condition** is a durable fact about something the mesh owns, such as a node, an assignment, a +provision, a seat or a rotation: what is wrong, since when, the evidence, what was tried, and who can +resolve it. Conditions are the one thing a person or an agent looks at to know whether the mesh is +right. `status` is the list of open conditions. When it is empty, the mesh is right, not just up. + +## On NATS + +[ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md) moves the bus to NATS, and NATS makes most of +this cheaper, because observation becomes something every component publishes rather than something +a central process polls. + +| the wish needs | on NATS | +|---|---| +| every component says it is alive | a heartbeat on a subject per node and assignment; silence past its interval is a condition, and nobody polls | +| every component says what it observed | observations published on subjects (`mesh.observed..`, for instance), consumed by whoever owns the comparison | +| the last known state survives restarts | a JetStream key-value bucket of observed state per owner; the provisioner's "what I applied" and a rotation's step live there, not in memory | +| conditions are durable and watchable | conditions as entries in a key-value bucket, watched by anyone who cares: a surface, an agent, the controller | +| the bus itself is observed | the server's advisories (a consumer exceeding its deliveries, a slow consumer, a client disconnecting) and its monitoring endpoint become observations like any other | +| a repair is retried, not lost | JetStream redelivery with delay, which is the same mechanism [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)'s guarantee moves to | +| work handed to an agent | a condition that needs judgement published as a task on a subject an agent's queue group consumes | + +**Who compares.** Each owner compares its own: the host for its node's containers and files, a +provisioner for its backend, the vault for rotations, the controller for delivery and seats. The +controller's observability context does not repair anything. It holds conditions, their history, +and the view across the mesh. It notices what no owner can see about itself: an owner gone silent. + +## What stays human + +Some repairs need the operator's key, and the mesh says so rather than pretending otherwise: +re-raising the vault or the broker, and recovering a node's identity. These are conditions too, with +the procedure named, and they are the only ones that can never clear themselves. + +## Open + +- The budgets: how many attempts, over how long, per kind of repair. +- How a condition that needs judgement reaches an agent, and how the agent's action is recorded. +- Where the observability context stores history (to-be 06 left it open; volume argues against the + relational store). +- Which correctness probe each provision's contract offers, and how often it runs. diff --git a/01-RESEARCH/017-a-mesh-that-heals-itself/02-now-pragmatically.md b/01-RESEARCH/017-a-mesh-that-heals-itself/02-now-pragmatically.md new file mode 100644 index 0000000..fe61353 --- /dev/null +++ b/01-RESEARCH/017-a-mesh-that-heals-itself/02-now-pragmatically.md @@ -0,0 +1,46 @@ +# 02 — Now, pragmatically + +The intended behaviour lands on NATS. The bus moves after the migration's core +([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md)). Until then, work toward it is chosen by one +test: + +**Does it survive the move?** A change to what a loop compares against, or to what an adapter can +tell about its backend, survives, because it is independent of the bus. A new AMQP queue for health +reports, a poller written against the broker's management API, or a condition store built on the +current broker does not survive, and is not built. + +## Done + +**The provisioner asks the backend, not memory** (issue 120). The SDK's harness gained an optional +`holds` on the adapter, asked for every applied consumer every minute. A consumer the backend no +longer holds is provisioned again. Being unable to ask is not treated as loss. The cache module +implements it first, because its server keeps its users in memory and forgets them all on a restart. +That was verified against a real server: a restart erases every consumer's user, and `holds` answers +correctly for absent, present, wrong-password, disabled and deleted. +Changes: mesh-sdk PR #7 (0.1.1) and mesh-catalog PR #84. + +This is principle 1 applied to one loop. It survives the move unchanged. On NATS, the harness's +record of what it applied moves from memory into a key-value bucket, and `holds` stays as it is. + +## Next, in order of silent failures removed + +1. **`holds` for the other credential providers.** Each backend can answer whether a login exists + with the mesh's password without changing anything. Where a backend cannot check a password without + logging in, logging in is the check. +2. **The harness's other blind spot.** A consumer that goes away while its provisioner is down is never + removed. The fix is the same principle in reverse: list what the backend holds, and compare it with + what the mesh asks for. Removal stays subject to ADR 0114's rule that it never follows from a login + changing. +3. **`status` reports what owners already know.** The host knows which containers it recreated and + why. A rotation knows who it waits on. Delivery knows what is outstanding. Surfacing those as + conditions in the existing `status` needs no new transport. It is the shape the NATS condition store + will hold. +4. **The loop inventory and the classification** in [00](00-overview.md). They decide what comes after + these three. + +## Not now + +- Heartbeats, observation subjects, key-value state, advisories: all NATS, all after the move. +- Handing conditions to agents: designed with NATS, where a task on a subject is native. +- Choosing the observability store: decided when there is something to store, which is after the + move.