Merge pull request 'Multiple fixes: ADRs 0094–0097; issues 069, 049, 046 resolved; 064's image half decided' (#67) from multiple-fixes into main

2026-09-21 21:05:13 +02:00
parent a259d1292f 43ca9ce0ae
commit f11824d299
15 changed files with 323 additions and 30 deletions
@@ -0,0 +1,63 @@
+---
+topic: the tiers
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0085-a-secret-is-a-provision.md
+---
+
+# 94. A module may hold several secrets from one provider, each a pair of its own
+
+## Context
+
+[ADR 0085](0085-a-secret-is-a-provision.md) makes a module's own secret a provision: the module
+requires `secret` from the vault and reads the pair credential minted for that consumer↔vault
+pair. A pair has one credential, a module requires a provision once, and so a module received
+one value. Read against the catalogue, nine modules hold two or more secrets besides their
+broker account; seven of them hold genuinely independent values with independent lifetimes — a
+root certificate, its key and that key's password; an admin password beside an API token. None
+derives from another, so "one value, derivation the module's business" answers nothing
+([issue 069](../04-ISSUES/069-one-secret-provision-yields-one-value/00-report.md)).
+
+## Considered Options
+
+1. **One value per module; the module derives the rest.** Rejected: the values are independent.
+2. **Require the provision several times.** Rejected: `requires` is a list of names, and a
+   requirement is matched by name everywhere.
+3. **The `secrets` map names several files under local names, and each local name is a pair
+   credential of its own.** Adopted.
+
+## Decision
+
+A module's `secrets` entry for a requirement may be a path, as before, or an object of local
+names to paths. Each local name is its own pair credential, keyed on it beside the provision,
+the consumer node, the consumer module and the provider; its own file on the consumer, referred
+to as `${secret:<local name>}`; its own holder at the provider, named the consumer's identity
+with the local name after it; and rotated apart from the others. A local name may not be one of
+the module's own secrets or something it requires, so what a placeholder means is never
+ambiguous. The plain shape is unchanged, and every credential that exists is the one it was.
+
+The holder's suffix is not a login any backend checks — a secret is not a login — so the
+identity limit that binds a database role or an access key does not apply to it.
+
+## Consequences
+
+The ten modules that could not move onto the vault can. What got harder: `rotate secret` for a
+consumer rotates every local name it holds from that provider; rotating one of several is a
+finer command than the mesh has, and waits for a case that needs it.
+
+## How it is checked
+
+Manifest tests read both shapes, write them back, and refuse a colliding or unusable local
+name. A resolver test asserts two local names are two needs, two files with two credentials,
+and two holders at the provider. An inventory test asserts two local names are two rows, that
+rotating one leaves the other, and that the provider is told both. The vault bed installs a
+consumer that keeps two secrets and asserts two values delivered, two holders in the vault's
+ledger, and both rotated by one command.
+
+## References
+
+- [issue 069](../04-ISSUES/069-one-secret-provision-yields-one-value/00-report.md)
+- [ADR 0085](0085-a-secret-is-a-provision.md), [ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)
+- [`03-DESIGN/01-to-be/24-the-secrets-vault.md`](../03-DESIGN/01-to-be/24-the-secrets-vault.md)
@@ -0,0 +1,60 @@
+---
+topic: the tiers
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md
+---
+
+# 95. The control plane is the way to ask a module
+
+## Context
+
+A module serves tools over the broker under an account scoped to what it emits, consumes and
+serves ([ADR 0047](0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md)). A
+tool call is a request and a reply: the caller creates a reply queue and publishes to the serving
+module's request key, and no module's scope grants either — nor should it, since a module that
+only publishes events has no business declaring queues. So a module could serve tools and nothing
+in the mesh could call them
+([issue 049](../04-ISSUES/049-a-module-can-serve-tools-and-nothing-can-call-them/00-report.md)):
+not an operator at a terminal, not an agent acting for one.
+
+## Considered Options
+
+1. **Calling is a grant**: a module declares it may be asked, and a consumer is issued an
+   account that may create a reply queue and publish to that module's request key. Deferred: a
+   module-to-module call is the only caller that resembles what the mesh mints today, and none
+   asks for one yet.
+2. **The control plane is the way in.** Adopted. It holds a connection that may already, so a
+   person or an agent asks through it, and every question passes one process — which is where
+   an audit of who asked what belongs.
+
+## Decision
+
+`mesh-controller ask <module> <tool> [json]` publishes the request on the tool exchange under
+`<module>.<tool>`, with a private reply queue bound under its own name, waits for the answer
+whose correlation matches, and prints it as the module gave it. A tool that answered with an
+error has answered: the answer is printed and the exit status says so. A module that never
+answers is said to have not answered, with where to look.
+
+A module declares nothing about being asked: serving a tool is being askable through the control
+plane. A module-to-module call, if one is wanted, is a grant like any other and a later decision.
+
+## Consequences
+
+Anything with the control plane in reach can ask any module anything it serves. What got
+harder: nothing outside the control plane can, and the control plane's connection is one more
+thing on the path of every question — a cost accepted for the audit it buys.
+
+## How it is checked
+
+A tools-only bed asks a served tool through the control plane and asserts an answer arrived —
+an error, since the lab has no upstream and no token, which is an answer where a timeout would
+not be.
+
+## References
+
+- [issue 049](../04-ISSUES/049-a-module-can-serve-tools-and-nothing-can-call-them/00-report.md)
+- [ADR 0047](0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md)
+- [`03-DESIGN/01-to-be/19-the-module-protocol.md`](../03-DESIGN/01-to-be/19-the-module-protocol.md)
@@ -0,0 +1,63 @@
+---
+topic: building it
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
+---
+
+# 96. An upstream image is copied between registries, never through a machine's image store
+
+## Context
+
+A module may declare that an artifact is an image published elsewhere, to be copied into the
+mesh's own registry so machines fetch it by a digest this mesh assigned rather than by a name
+somebody else controls. The builder pulled it into the build machine's image store and pushed it
+under the mesh's name, and the push was refused: a published image is an index over several
+architectures, the runtime's store keeps the index, and pushing one platform out of it fails
+however the platform is asked for
+([issue 046](../04-ISSUES/046-an-upstream-image-cannot-be-mirrored-into-the-mesh/00-report.md)).
+Every variant of pull-then-push was tried and failed the same way.
+
+## Considered Options
+
+1. **Resolve the index to one platform and push that.** Tried, reverted: it did not make the
+   push work, and a workaround for a store's behaviour is a thing nobody removes later.
+2. **Tooling that copies between registries**, installed on the build machine. Rejected: one
+   more thing the builder's image carries, for a protocol the builder already speaks for blobs.
+3. **Copy over the registry API**, in the builder. Adopted.
+
+## Decision
+
+The builder copies an upstream image between registries and never through a machine's image
+store: it reads the index and every manifest it names, moves each blob by digest into the mesh's
+registry — skipping what is already there, since blobs are content-named — puts the manifests
+and then the index under the module's repository, and pins the index's digest. Public images are
+read with the anonymous bearer token the registry hands out on challenge, which is how the
+public hub and the others the catalogue names serve them. The mesh mirrors the whole index, so
+what a machine fetches is the image for its own architecture; that every machine on one mesh is
+the same architecture is an assumption this mesh makes and had not written down until now.
+
+Genesis has no registry to copy into and keeps the pull: the image stays in the first machine's
+store, named by its own id, as every artifact does before there is anywhere to publish.
+
+## Consequences
+
+An upstream artifact builds. What got harder: the builder now holds a registry client of its
+own, some two hundred lines, where a runtime command used to do; and a private upstream that
+demands a credential is refused, since the copy is anonymous by design.
+
+## How it is checked
+
+A test raises a fake upstream registry serving an index over two platforms behind a bearer
+challenge, and a fake mesh registry that records what arrives: every blob of both platforms
+arrives once, two manifests and the index are put under their digests, the reference returned
+pins the index under the module's repository, and a second copy uploads nothing. A reference
+test reads names the way a runtime does.
+
+## References
+
+- [issue 046](../04-ISSUES/046-an-upstream-image-cannot-be-mirrored-into-the-mesh/00-report.md)
+- [ADR 0006](0006-the-substrate-and-the-control-plane.md)
+- [`03-DESIGN/01-to-be/18-building-a-module.md`](../03-DESIGN/01-to-be/18-building-a-module.md)
@@ -0,0 +1,64 @@
+---
+topic: building it
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md
+---
+
+# 97. A vendor image is a declared build input, and a recipe fetches nothing undeclared
+
+## Context
+
+Three modules could not be built by the mesh's builder because their recipes reached for what
+no manifest named: a public package, or a binary copied out of a public image
+([issue 064](../04-ISSUES/064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md)).
+A module already names the bases it stands on — another module's artifact, by name — and the
+builder answers with what the mesh holds; a vendor's image had no such declaration, so a recipe
+named it directly and the build worked when the public registry answered, which is sometimes.
+
+## Considered Options
+
+1. **Let the build environment reach public registries.** Rejected: a build that fetches from
+   somebody else's registry on its own is one the mesh cannot rebuild the same way twice.
+2. **A vendor image is a base like any other**, declared under `build.on` with the argument the
+   recipe reads it from, pinned by digest, copied into the mesh's registry before the build
+   ([ADR 0096](0096-an-upstream-image-is-copied-between-registries.md)). Adopted.
+
+## Decision
+
+A build's `on` entry is either a module's artifact or an image published elsewhere, pinned by
+digest, read from one build argument. Before the build the image is copied into the mesh's
+registry under the module's repository and the recipe is handed the copy; genesis, with no
+registry, pulls it into the first machine's store. A recipe whose `COPY --from` names a registry
+image the manifest did not declare is refused before the build, naming the image and the remedy;
+its own stages, declared arguments and `scratch` are not fetches. An unpinned vendor image is
+refused: a tag is what somebody else can move.
+
+A recipe whose `FROM` names an undeclared base is **said, not yet refused**: the mesh's own images
+— the control plane's, the builder's, the tool runtime's — start from a public base and declare
+none, and refusing those refuses genesis. They declare their bases next; until then every build
+names the undeclared base and the remedy.
+
+The package half of the issue is not decided here: the mesh's package registry already proxies
+the public one, and the failure the report saw has to be run again to be placed.
+
+## Consequences
+
+A module's build inputs are all in its manifest, and every one of them is something the mesh
+holds a copy of. What got harder: a recipe that used to name a base image on its first line now
+names an argument, and the manifest names the image.
+
+## How it is checked
+
+A builder test declares a pinned vendor image, asserts it is copied under the module's
+repository and handed to the recipe as the argument, and asserts an unpinned one is refused. A
+recipe test asserts an undeclared `FROM`, an undeclared `COPY --from` and an undeclared argument
+are named, and that stages, declared arguments and `scratch` are not.
+
+## References
+
+- [issue 064](../04-ISSUES/064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md)
+- [ADR 0096](0096-an-upstream-image-is-copied-between-registries.md)
+- [`03-DESIGN/01-to-be/18-building-a-module.md`](../03-DESIGN/01-to-be/18-building-a-module.md)
@@ -113,6 +113,8 @@ python3 00-META/checks/index.py            fail if stale
 - **0078** — [The store and the broker are ordinary modules](0078-the-store-and-broker-are-modules.md)
 - **0079** — [The foundation seats are named after their servers](0079-the-foundation-seats-are-named-after-their-servers.md)
 - **0092** — [An operator delivers a pair credential, and the mesh never replaces it](0092-an-operator-delivers-a-pair-credential.md)
+- **0094** — [A module may hold several secrets from one provider, each a pair of its own](0094-a-module-may-hold-several-secrets-from-one-provider.md)
+- **0095** — [The control plane is the way to ask a module](0095-the-control-plane-is-the-way-to-ask-a-module.md)

 ### What runs on them, and how it gets there

@@ -159,6 +161,8 @@ python3 00-META/checks/index.py            fail if stale
 - **0076** — [The SDK is a published package, and the toolchain resolves it by version](0076-the-sdk-is-a-published-package.md)
 - **0082** — [The registry is reached by name, and the overlay is its security](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)
 - **0086** — [A secret reaches a process as a file, and an exception is declared](0086-a-secret-reaches-a-process-as-a-file.md)
+- **0096** — [An upstream image is copied between registries, never through a machine's image store](0096-an-upstream-image-is-copied-between-registries.md)
+- **0097** — [A vendor image is a declared build input, and a recipe fetches nothing undeclared](0097-a-vendor-image-is-a-declared-build-input.md)

 ### How it is checked

@@ -7,6 +7,8 @@ code:
  - mesh-catalog modules/builder
 updated: 2026-09-21
 decisions:
+  - 02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md
+  - 02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md
  - 02-DECISIONS/0091-a-mount-is-declared-three-ways.md
  - 02-DECISIONS/0087-a-seeded-file-is-created-once.md
  - 02-DECISIONS/0040-what-a-module-is.md
@@ -202,6 +204,28 @@ remedies, and a test parses every manifest in the catalogue beside the checkout.
 | `image` | built from a Dockerfile — for software that needs a particular base |
 | `upstream` | somebody else's image, mirrored and pinned by a digest this mesh assigned |

+**An upstream image is copied between registries, never through a machine's image store**
+([ADR 0096](../../02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md)). A
+published image is an index over several architectures, and a runtime's store refuses to push
+one platform out of an index it pulled. The builder reads the index and every manifest it names
+over the registry API, moves each blob by digest into the mesh's registry, puts the manifests and
+then the index under the module's repository, and pins the index — the whole image, so what a
+machine fetches is the one for its own architecture. Public images are read with the anonymous
+token a registry hands out on challenge; a private upstream is refused. *How it is checked:* a
+test copies an index over two platforms from a fake registry behind a bearer challenge into a fake
+mesh registry and asserts every blob arrived once, the manifests and index under their digests,
+and nothing uploaded on a second copy.
+
+**A vendor image is a declared build input**
+([ADR 0097](../../02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md)). A build's `on`
+entry is a module's artifact or an image published elsewhere, pinned by digest, read from one
+build argument; the image is copied into the mesh's registry before the build and the recipe is
+handed the copy. A recipe whose `COPY --from` names a registry image the manifest did not declare
+is refused before the build, naming it and the remedy; an undeclared `FROM` is said, not yet
+refused, because the mesh's own images start from a public base and declare none. *How it is
+checked:* builder tests on a declared and an unpinned vendor image, and a recipe test on what
+counts as a copy and what as a base.
+
 ### What it puts on a machine

 | resource | is | a module may |
@@ -5,8 +5,9 @@ code:
  - mesh-sdk src
  - mesh-tools src/broker-amqp.ts
  - mesh-controller internal/link
-updated: 2026-09-15
+updated: 2026-09-21
 decisions:
+  - 02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md
  - 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
  - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
  - 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
@@ -137,13 +138,15 @@ A module's tools are its operator-facing surface.
  into any queue on the broker.
 - A caller needs a **reply queue**, and that is what a module's scoped account may not declare
  ([issue 049](../../04-ISSUES/049-a-module-can-serve-tools-and-nothing-can-call-them/00-report.md)).
-  So a module may serve tools and may not call them, and nothing today issues an account to anything
-  that wants to ask.
-
-### Not yet true
-
-The caller's half has no account. Until that is settled, the only thing that can ask a module a
-question is the foundation's bootstrap admin, which is not a protocol so much as a way in.
+  So a module may serve tools and may not call them.
+- **The control plane is the way to ask**
+  ([ADR 0095](../../02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md)):
+  `ask <module> <tool> [json]` publishes on `mesh.rpc` under `<module>.<tool>` with a private reply
+  queue bound under its own name, and prints the answer as the module gave it. A module declares
+  nothing about being asked — serving a tool is being askable through the control plane. A
+  module-to-module call, if one is wanted, is a grant like any other and a later decision.
+  *How it is checked:* a tools-only bed asks a served tool through the control plane and asserts
+  an answer arrived, where a timeout would read differently.

 ---

@@ -4,6 +4,7 @@ status: implemented
 code: [mesh-catalog, mesh-controller, mesh-host]
 updated: 2026-09-21
 decisions:
+  - 02-DECISIONS/0094-a-module-may-hold-several-secrets-from-one-provider.md
  - 02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md
  - 02-DECISIONS/0085-a-secret-is-a-provision.md
  - 02-DECISIONS/0031-the-control-plane-authenticates-nobody.md
@@ -48,8 +49,12 @@ A module that needs a secret for its own use requires a `secret` provision, exac
 a database from the store. The vault generates the value — or takes custody of one an operator
 delivered, through `secret accept … --provider`, which seals it to both ends and records the pair
 as accepted ([ADR 0092](../../02-DECISIONS/0092-an-operator-delivers-a-pair-credential.md)) — and
-the credential belongs to the consumer↔vault pair. An accepted pair is the one exception to what
-follows: the mesh cannot make its replacement, so it is neither remade when a key changes nor
+the credential belongs to the consumer↔vault pair. A module that needs several values names them
+as local names under its `secrets` entry, and each is a pair of its own — keyed on the local
+name, delivered as its own file, held at the vault as its own holder
+([ADR 0094](../../02-DECISIONS/0094-a-module-may-hold-several-secrets-from-one-provider.md)).
+*How it is checked:* manifest, resolver and inventory tests on two local names; the vault bed's
+two-secret consumer. An accepted pair is the one exception to what follows: the mesh cannot make its replacement, so it is neither remade when a key changes nor
 rotated; both are refused aloud, and accepting a new value is the rotation. *How it is checked:*
 an inventory test accepts, reads back unchanged, and asserts the two refusals name the remedy. Because it is an ordinary pair
 credential, **everything already built for pair credentials applies to it unchanged**: it rotates
@@ -1,9 +1,9 @@
 ---
-status: located
+status: resolved
 opened: 2026-09-14
 located-in: [mesh-controller internal/builder (upstream artifacts)]
-fixed-by:
-amended-design:
+fixed-by: ADR 0096; mesh-controller multiple-fixes (the builder copies the index and its manifests between registries over the registry API); proven by a test against a fake upstream serving an index over two platforms
+amended-design: 03-DESIGN/01-to-be/18-building-a-module.md
 ---

 # 046 — An upstream image cannot be mirrored into the mesh's own registry
@@ -12,6 +12,8 @@
   than a limitation; today every machine on one mesh is the same architecture, and that
   assumption is now written down here rather than nowhere.

-**Located in:** the builder's upstream-artifact step. Not fixed here: a registry-to-registry copy
-is a few hundred lines against the registry API and is proven only against a real registry
-serving a real index, which is a lab run of its own.
+**Located in:** the builder's upstream-artifact step. Fixed as
+[ADR 0096](../../02-DECISIONS/0096-an-upstream-image-is-copied-between-registries.md): a copy over
+the registry API, proven against a fake upstream serving an index over two platforms behind a
+bearer challenge. A run against the public hub from a mesh's builder is the remaining proof, and
+the first build of a module with an upstream artifact will be it.
@@ -1,9 +1,9 @@
 ---
-status: located
+status: resolved
 opened: 2026-09-14
-located-in: [mesh-controller cmd/mesh-controller, mesh-tools (the request contract)]
-fixed-by:
-amended-design:
+located-in: [mesh-controller cmd/mesh-controller (ask), mesh-controller internal/link]
+fixed-by: ADR 0095; mesh-controller multiple-fixes (ask: the control plane publishes the request with a private reply queue and prints the answer); proven by the confluence bed
+amended-design: 03-DESIGN/01-to-be/19-the-module-protocol.md
 ---

 # 049 — A module can serve tools, and nothing is allowed to call them
@@ -14,5 +14,6 @@
   like any other, and a later decision.

 **Located in:** the controller (a command speaking the tool request/reply over its own connection)
-and the tool runtime's request contract. Not fixed here: it is a new command against a protocol
-the runtime owns, and needs a lab run against a tools-only bed to be proven.
+and the tool runtime's request contract. Fixed as
+[ADR 0095](../../02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md): `ask`,
+proven by the confluence bed asking a served tool and getting its answer.
@@ -16,5 +16,8 @@
   repositories succeed in the same Dockerfiles.

 **Located in:** the builder (what it tells npm and the runtime) and the catalogue (three modules
-whose Dockerfiles fetch what no manifest names). Still open: a build must be re-run to place the
-404, and a decision is needed on whether a vendor image is declared as a build input.
+whose Dockerfiles fetch what no manifest names). The image half is decided and built:
+[ADR 0097](../../02-DECISIONS/0097-a-vendor-image-is-a-declared-build-input.md) — a vendor image
+is declared under `build.on`, copied in, and a recipe fetching what is undeclared is refused. Still
+open: the package half — a build must be re-run against the mesh's proxying registry to place the
+404 — and the three recipes themselves, which now declare their images or are refused.
@@ -1,9 +1,9 @@
 ---
-status: located
+status: resolved
 opened: 2026-09-20
 located-in: [mesh-controller internal/catalogue (requires/secrets), mesh-controller internal/inventory (secret key)]
-fixed-by:
-amended-design:
+fixed-by: ADR 0094; mesh-controller feat/several-secrets (secrets: under local names, the pair keyed on the local name, migration 0027); proven by the vault bed
+amended-design: 03-DESIGN/01-to-be/24-the-secrets-vault.md
 ---

 # One `secret` provision yields one value, and a module may need several
@@ -18,6 +18,7 @@
   is a manifest-vocabulary decision, and it moves the pair key: the credential must be keyed on
   the local name, not the provision name, or the second pair overwrites the first.

-**Located in:** the manifest's `requires`/`secrets` vocabulary (the catalogue parser) and the pair
-credential's key (the controller's secret store). Not fixed here: the key change touches every
-existing pair and belongs in a feature of its own, with a lab run against the vault bed.
+**Located in:** the manifest's `secrets` vocabulary (the catalogue parser) and the pair
+credential's key (the controller's secret store). Fixed as
+[ADR 0094](../../02-DECISIONS/0094-a-module-may-hold-several-secrets-from-one-provider.md): the
+key gains the local name, empty for every existing pair, so nothing that exists changed.