diff --git a/02-DECISIONS/0189-the-store-keeps-what-the-records-name.md b/02-DECISIONS/0189-the-store-keeps-what-the-records-name.md new file mode 100644 index 0000000..51b7bdf --- /dev/null +++ b/02-DECISIONS/0189-the-store-keeps-what-the-records-name.md @@ -0,0 +1,141 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-02 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md +--- + +# 189. The store keeps what the records name, and a maintenance step holds its writers still + +## Context + +The mesh's artifact store has never collected anything +([issue 108](../04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md)). +Every build pushes another layer set; nothing has ever removed one. The predecessor ran a routine +on a timer — stop the registry, collect, start it — and the conversion carried the settings that +routine depends on without the routine, because the routine was a script beside the module and not +a resource in it. The store now holds fifty-three repositories on the machine that serves +everything else, and the only outcome of leaving it is a full disk reported as somebody else's +failure. + +Three things stood in the way, and the issue names all three. + +**Nothing in the mesh's vocabulary expresses a maintenance window.** The collector requires every +writer stopped while it runs. A `run-once` step runs *beside* containers, not instead of them, and +a scheduled step is the same container on a cadence. There is no way for a module to say *hold this +container of mine still while this runs*. + +**Deletion is not enabled, and the door it would be enabled on has no accounts.** The store is +internal, reached by name over the overlay, trusted because being on that network is the permission +([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)). The predecessor +kept deletion behind an authenticated door, which it could, having one. + +**Nothing says what may be removed.** The registry's own answer — collect everything no tag names — +is wrong here. The mesh pushes each artifact under one moving tag and pins machines by digest, so +every build but the newest is untagged and some machine may still be running it. + +## Decision + +**1. Deletion is enabled on the store's one door, and the overlay stays the permission.** The +objection dissolves on inspection: that door **already accepts a push**, and a writer who can push +can replace any tag in the store with anything it likes. Delete takes nothing a push did not +already have, and the machines that can reach the door are the ones the mesh's own filter admits +([ADR 0168](0168-a-converged-machine-is-filtered-by-the-mesh-alone.md)). Putting an authenticated +door in front of deletion while leaving push open would be a lock on the window beside an open +door, and it would cost the thing ADR 0082 bought: a store every machine can reach without a +credential to distribute first. + +**2. The mesh deletes what it made and no longer keeps; the store reclaims the bytes.** Two halves, +each doing what only it can. + +The **mesh** decides. It does not need to enumerate the store to do it — it has never put anything +there it did not record, so **every digest it could remove is already in its own build records**. +It deletes those manifests through the store's door, by digest, and remembers that it did. + +The **store** reclaims. A deleted manifest frees no bytes until the registry's own collector walks +the storage with nothing writing to it, so the module declares that collector as a scheduled step +with the server held still for its duration. Plain collection, not `--delete-untagged`: what the +mesh keeps is still a manifest in the store, so it is still referenced, so its blobs stay — the +dangerous flag is not needed at all once the mesh is the one deciding. + +**3. What the mesh keeps, stated as three reasons rather than a number.** A digest is kept because: + +- **a definition names it** — every artifact reference in any module's current recorded manifest, + which is what the mesh would hand a machine now. No age limit: this is the floor; +- **the mesh can still go back to it** — every artifact of the **five most recent successful + builds** of each module, so a release that turns out wrong has somewhere to return to; +- **nothing else.** An artifact older than that, which no definition names, is what the store is + carrying for no stated reason. + +A digest the mesh did not record making is never touched. That is not a safety margin, it is the +whole rule restated: the mesh removes what it put there and can account for, and the images genesis +pushed before any record existed are exactly what this must not reach +([04-ISSUES/102](../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md), F4). + +**4. A scheduled step may hold its module's own containers still while it runs** — +`while-stopped`, naming resource ids in the same module. The host stops each, runs the step, and +starts them again **whatever the step did**, including when it failed or the host was interrupted. +Three boundaries: + +- **Its own module's containers only.** A module that could quiesce a neighbour could stop the + mesh; a maintenance window is a statement about one service's own insides. +- **Scheduled steps only, not `run-once`.** At apply time the host already has a window: the + declaration is applied in order and a step gates what follows, so a one-time offline migration + says *before* rather than *instead of*. A recurring window is the case order cannot express. +- **Restoring is not conditional.** A step that fails must leave the service running; the whole + risk of this field is a window that never closes. + +**5. The sweep runs where the records change — after a build the mesh recorded.** That is the +moment new bytes landed and the moment the keep set moved, and it needs no new timer. The +store's collection runs nightly, because reclaiming is slow and the thing it reclaims is already +unreferenced. + +## Consequences + +- Disk stops growing without bound on the machine that serves the mesh. That is the whole point + and it has no other way to be true. +- A machine behind by more than five builds of a module, which recreates a container, cannot pull + what it was running. It is already a machine the mesh reports as behind, and the answer is the + one the mesh already gives it: the current declaration. Stated here rather than discovered. +- The store is a little less of a museum. A digest in an old build record may no longer be + fetchable, and the record still says what that build made — the record is history, not an + index of what is on disk. The collected mark is kept beside it so the two can be told apart. +- `while-stopped` is a second thing the host does to a container it did not start this pass. It is + deliberately the narrowest form: the module's own, by id, restored unconditionally. +- The store is briefly unavailable each night, for as long as collection takes. Everything that + pulls from it retries; nothing in the mesh treats a momentary store as a failure + ([ADR 0185](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md)). +- **An apply arriving during the window reopens it**, because the host's rule for a container it + finds stopped is to replace it, and `while-stopped` is the first thing that makes a stopped + container intentional. Found by reading this before it merged, recorded as + [issue 224](../04-ISSUES/224-an-apply-reopens-a-maintenance-window-by-recreating-what-it-held-still/00-report.md) + rather than fixed here: the two candidate fixes — the window takes the apply lock, or the apply + learns which containers are held — are each a decision with its own cost, and neither belongs + inside this record. Nothing is worse than it was; the store has never collected at all. +- **The sweep is bounded**: at most two hundred artifacts and sixty seconds per build, stopping at + the first refusal, because it runs inside somebody's build. What is left over is offered again + next time. The store stops growing from the first sweep; it does not empty in one. + +## How this is checked + +- The host: a scheduled step with `while-stopped` stops the named containers before the run and + starts them after; it starts them again **when the step fails**; it refuses an id that is not a + container of the same module, its own id, and `while-stopped` on a `run-once` step. Each refusal + is tested for what it says, not only that it says something. +- The controller: given build records and current manifests, the keep set holds every reference a + manifest names and every reference of the five most recent builds per module, and nothing else; + a reference the mesh never recorded is never in the delete set; a delete that answers 404 is + recorded as collected rather than retried forever. +- The sweep is tested against a fake store that records what it was asked to delete, so what is + asserted is the decision and not the registry's behaviour. +- Live: the store's size before and after the first nightly collection, read from the machine. + +## References + +- [issue 108 — the registry has no garbage collection](../04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md) +- [ADR 0082 — the registry is reached by name and trusted by the overlay](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) +- [ADR 0053 — a step that runs on a schedule](0053-a-step-that-runs-on-a-schedule.md) +- [ADR 0156 — an artifact is what a build produces, and the store is named for its scope](0156-an-artifact-is-what-a-build-produces-and-the-store-is-named-for-its-scope.md) +- [design 32 — what a module declares](../03-DESIGN/01-to-be/32-what-a-module-declares.md) diff --git a/02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md b/02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md new file mode 100644 index 0000000..edbb84c --- /dev/null +++ b/02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md @@ -0,0 +1,149 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-02 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md +--- + +# 201. A provider declares what it derives for each consumer, and the mesh tells both ends + +> Written as 0188 on 2026-10-02 and renumbered to 0201 on 2026-10-04: the record of the bundles +> refactor took 0188 on main while this one waited in a pull request, and the mesh's own code now +> cites that one. Only the number moved; the decision is the one taken on the 2nd. + +## Context + +An arrangement between a consumer and a provider is delivered entirely by the mesh. Where the +provider is, which port it answers on, what name the consumer must present, where its password +is — each arrives as a fact the consumer reads from its binding, or as `${bound:…}` filled into a +file before the declaration leaves the control plane. The provider invents none of it and hands +none of it back ([ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md)). + +One kind of value escapes that. Where the **provider names the resource** — a bucket, a database, +a vhost — the name is derived from the consumer, per consumer, and the mesh has no way to carry +it. `serves` is a literal block in the provider's definition: the same values for every consumer. +A provisioner's contract takes a provision and returns nothing. So a value the mesh's own rule +produced reaches neither end as a statement; it is recomputed at one end and transcribed at the +other. + +The object store is the instance ([issue 124](../04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md)). +Its provisioner normalises the login the mesh minted into a bucket name and creates, checks and +removes exactly that; the rule lives in twenty lines of the module's own TypeScript. Its three +consumers each write the answer into their own definition by hand. Two transcribed it correctly; +one named a predecessor's bucket, and would have authenticated successfully and been refused on +every object, which reads like a credential fault and is not one. + +Even corrected, the transcriptions are wrong in a second way. Each is `mesh--`, so +each **names the machine the module happens to run on today** — a definition stating a fact about +one installation, which [ADR 0155](0155-a-definition-names-no-installation-and-how-that-is-checked.md) +forbids and whose check does not catch because the name is not a domain. Move any of the three to +another machine and its configuration points at a bucket its key cannot open. + +The shape is not the object store's. A database provisioner that prefixed names, a queue provider +that scoped vhosts, any provider that derives a resource from who is asking: each forces the +consumer to reproduce somebody else's rule and keep it in agreement by hand. + +## Decision + +**1. A served value may name the consumer the mesh is serving.** A `serves` block, which is +literal today, may interpolate the mesh's own statement of who the consumer is: + +- `${consumer:as}` — the identity the mesh minted for this consumer, exactly as the login it is + told to present ([ADR 0049](0049-a-consumers-identity-fits-the-tightest-backend.md)); +- `${consumer:as:dns}` — the same identity written as a DNS label. + +Nothing else. **The mesh learns no protocol here; it spells its own name in an alphabet it already +knows.** The identity is the mesh's, minted by the mesh, already capped at twenty characters +because of what an S3 access key accepts; `dns` is that same name with its separator written `-` +instead of `_`, which is the whole of the difference between the mesh's identifier alphabet and +the one buckets, vhosts and hostnames use. A provider that needs a prefix or a suffix writes it +around the placeholder, because a served value is a string. + +The rejected alternative is **the provider returning values from provisioning** — the natural +channel, since the provider is what derived them. It is rejected for three reasons, in order of +weight. It inverts the delivery the mesh is built on: a grant would carry data the provider wrote +rather than only data the mesh minted, and [ADR 0048](0048-a-provider-creates-the-credential-the-mesh-minted.md) +removed exactly that second path once already. It makes a consumer's declaration incomplete until +its provider's reconcile loop has run, so a consumer could not be composed before a provider +answered — a bootstrap order the mesh does not have and does not want. And it puts the rule where +nothing can check it: a value that arrives from a running process cannot be refused at resolution, +only discovered wrong later, which is the failure this record exists to end. + +**2. The mesh resolves it once, per consumer, and tells both ends from the one resolution.** At the +moment a consumer's declaration is composed, the mesh knows exactly who the consumer is. There, and +only there, the placeholders are filled. The result reaches: + +- the **consumer**, as the served facts in its binding file and as `${bound::}` in + any file it writes — unchanged mechanisms, carrying one more key; +- the **provider**, as `serves` on that consumer's entry in its contributions file, so the + provisioner is *told* the name rather than recomputing it. + +**The provider stops deriving in code and starts declaring.** One statement, filled once, delivered +to both ends: the two cannot disagree, because there is no second computation to disagree with. + +**3. A served value stays settled before it is per-consumer.** Settings still compose into `serves` +([ADR 0174](0174-a-node-varies-a-module-through-settings-and-kept-regions-never-an-edit.md)), and the consumer +placeholders are filled after that, so an operator may set a prefix and the mesh still derives the +rest. A `${consumer:…}` naming a fact or an alphabet the mesh does not have is refused when the +definition is parsed, with what it may say. + +**4. A consumer may no longer name the resource its provider derives.** With the value delivered, +a literal in a consumer's definition is not merely redundant — it is the one thing that can +disagree with what the provider will actually create. The three object-store consumers lose their +hand-written bucket names in this change. + +**5. A consumer that keeps several holders of one provision may not be served a derived value.** +Each holder gets its own login, `…_` ([ADR 0094](0094-a-module-may-hold-several-secrets-from-one-provider.md)), +and a provider derives from the login — so it would make one resource per holder, while the +consumer's side has one binding and one `${bound::}`, both derived from the +un-suffixed identity. That is this record's own failure one case to the side, and just as quiet: +the consumer would authenticate and be refused on every object. Refused at resolution, naming +both ends. Lifting it means giving the consumer's side a local dimension, which is a decision and +not an omission. + +## Consequences + +- One more thing a definition may say, and one less thing a module may be wrong about. The + vocabulary grows by a placeholder; the catalogue loses three literals that named this + installation's control node. +- A provider's naming rule becomes readable in its definition instead of in its source. `minio`'s + `bucketFor` goes; the manifest says `"bucket": "${consumer:as:dns}"` and the provisioner uses + what it is given. +- A provider that already serves consumers keeps serving them: the derived value equals what the + code derived, so no bucket, database or login changes name. This is a change of **who says it**, + not of **what is said**. +- A refusal here fails **that machine's push**, naming the definition, and nothing else. That is + deliberate and is the opposite of a module quietly left out: a definition that transcribes + somebody else's rule is wrong everywhere, not just here, and the loud failure is in front of + whoever can fix it. +- The mesh now holds a rule in another system's alphabet — one rule, `dns`, stated once. A second + alphabet is a decision, not an addition: the cost of each is that the mesh must be right about + somebody else's naming, and that cost is only worth paying where the mesh already mints the name. + +## How this is checked + +- A served value naming an unknown fact or alphabet is refused at parse, with the list of what it + may say — tested on both halves of the message. +- Resolving a consumer whose provider derives a value puts that value in the consumer's binding + file, in its `${bound:…}` substitutions, and in the provider's contributions entry for that + consumer — one test asserting the three agree, because agreeing is the whole point. +- Two consumers of one provider on one machine get two different derived values, and neither gets + the other's. +- A consumer with several holders of a deriving provider is refused, with both ends named — the + test asserts the refusal, not merely that something failed. +- A catalogue-wide test refuses a consumer definition that writes a literal where its provider + derives: the provider's `serves` names the key, so the catalogue can say which definitions + transcribe one. +- `dns` is checked against the identity the mesh actually mints, not against an invented string: + the test derives an identity with `ConsumerIdentity` and asserts the label it becomes. + +## References + +- [issue 124 — a consumer cannot be told a value its provider derived for it](../04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md) +- [ADR 0048 — a provider creates the credential the mesh minted, and seals nothing](0048-a-provider-creates-the-credential-the-mesh-minted.md) +- [ADR 0049 — a consumer's identity fits the tightest backend](0049-a-consumers-identity-fits-the-tightest-backend.md) +- [ADR 0174 — a node varies a module through settings and kept regions, never through an edit](0174-a-node-varies-a-module-through-settings-and-kept-regions-never-an-edit.md) +- [ADR 0155 — a definition names no installation, and how that is checked](0155-a-definition-names-no-installation-and-how-that-is-checked.md) +- [design 27 — a module requires, the mesh resolves](../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 29a22b0..07994db 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -188,7 +188,9 @@ python3 00-META/checks/index.py fail if stale - **0185** — [A control plane behind its seat's row serves what it can](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md) - **0186** — [A ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang](0186-a-ban-list-never-holds-a-neighbour.md) - **0187** — [A dead tracker is not the machine's failure](0187-a-dead-tracker-is-not-the-machines-failure.md) +- **0189** — [The store keeps what the records name, and a maintenance step holds its writers still](0189-the-store-keeps-what-the-records-name.md) - **0190** — [A seat's work is shared by its holders, and building is the first such role](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) +- **0201** — [A provider declares what it derives for each consumer, and the mesh tells both ends](0201-a-provider-declares-what-it-derives-for-each-consumer.md) ### Its tiers, from the bottom up diff --git a/03-DESIGN/01-to-be/18-building-a-module.md b/03-DESIGN/01-to-be/18-building-a-module.md index 965c5b6..9d196b1 100644 --- a/03-DESIGN/01-to-be/18-building-a-module.md +++ b/03-DESIGN/01-to-be/18-building-a-module.md @@ -5,10 +5,11 @@ code: - mesh-controller cmd/mesh-builder - mesh-controller internal/builder - mesh-catalog modules/build-agent -updated: 2026-10-03 +updated: 2026-10-04 decisions: - 02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md - 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md + - 02-DECISIONS/0189-the-store-keeps-what-the-records-name.md - 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md - 02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md - 02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md @@ -317,6 +318,44 @@ build's lines reach a reader of its subject in order and the stream holds them a against a real server); the seat verb with an id reads the log (controller test); and, live, a build after the roll-out read line by line through the console. +## The store keeps what the records name + +*2026-10-02 — [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md), +[issue 108](../../04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md).* + +Every build pushes another layer set and, until this, nothing ever removed one. The registry's own +answer — collect what no tag names — is wrong for this mesh: each artifact is pushed under one +moving tag and machines are pinned by digest, so every build but the newest is untagged and some +machine may still be running it. + +**The mesh decides and the store reclaims.** Deletion is enabled on the store's one door — that +door already accepts a push, and a writer who can push can replace any tag, so delete takes +nothing a push did not already have, and ADR 0082's bargain (a store every machine reaches with no +credential to distribute first) is kept. The mesh then removes what it put there and no longer +keeps, **naming it from its own build records** rather than enumerating the store: it has never +put anything there it did not record, so a digest it did not record making is never named, which +is what keeps the sweep away from the images genesis pushed before any record existed. + +An artifact stays for one of two reasons and otherwise goes: a definition the mesh holds names it +(no age limit — this is the floor), or it belongs to one of the five most recent successful builds +of its module (somewhere for a wrong release to return to). The sweep runs after a build the mesh +recorded, which is the moment new bytes landed and the moment the keep set moved; it needs no +timer. Deleting a manifest frees no bytes, so the store's own collector runs nightly as a +scheduled step with the server held still — which is what `while-stopped` exists for +([design 20](20-writing-a-module.md)). Plain collection, not `--delete-untagged`: what the mesh +keeps is still a manifest and so still referenced, and the dangerous flag is not needed once the +mesh is the one deciding. + +A machine behind by more than five builds of a module, recreating a container, cannot pull what it +was running. It is already a machine the mesh reports as behind, and the answer is the current +declaration. + +*How it is checked:* the keep set, against records, holds what a manifest names and the five most +recent builds and nothing else; a reference the mesh never recorded is never in the delete set; an +image and an archive are asked for at their own endpoints; a store with deletion off names the +remedy rather than the status code; a store that does not have it is recorded collected rather +than retried for ever. Live: the store's size before and after the first nightly collection. + ## The builder compiles the languages the mesh is written in *2026-09-29 — diff --git a/03-DESIGN/01-to-be/20-writing-a-module.md b/03-DESIGN/01-to-be/20-writing-a-module.md index 6708851..e8e3cd3 100644 --- a/03-DESIGN/01-to-be/20-writing-a-module.md +++ b/03-DESIGN/01-to-be/20-writing-a-module.md @@ -5,11 +5,12 @@ code: - mesh-catalog modules/showcase - mesh-controller internal/builder - mesh-sdk src -updated: 2026-09-30 +updated: 2026-10-02 decisions: - 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md - 02-DECISIONS/0099-a-step-that-runs-once-names-what-it-reads.md - 02-DECISIONS/0053-a-step-that-runs-on-a-schedule.md + - 02-DECISIONS/0189-the-store-keeps-what-the-records-name.md - 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md - 02-DECISIONS/0040-what-a-module-is.md - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md @@ -208,3 +209,27 @@ is recreated with the new fact ([ADR 0099](../../02-DECISIONS/0099-a-step-that-runs-once-names-what-it-reads.md)). *How it is checked:* the host's unit tests run a step again when its named file changed and not otherwise, and recreate a container naming a step after the step ran. + +## A recurring step may hold its own module's containers still + +*Written 2026-10-02, from [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md) +and [issue 108](../../04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md).* + +Some work cannot be done underneath a running service: an artifact store's collector walks the +storage and requires every writer stopped. A `run-once` step runs *beside* containers and a +scheduled one is the same container again, so until this a module had no way to say it — and the +mesh inherited a store that has never collected anything, because the predecessor said it with a +shell script and a script beside a module is not a resource in it. + +A scheduled step may name `while-stopped`: resource ids of **its own module's** containers, which +the host stops before the run and starts again after it, in the reverse order, **whatever the step +did**. Three boundaries, each refused where it can be seen earliest — its own module's containers +only, because a module that could quiesce a neighbour could stop the mesh; scheduled steps only, +because at apply the declaration is applied in order and a step already gates what follows, so a +one-time offline job says *before* rather than *instead of*; and restoring that is not conditional +on anything, because the only real risk of the field is a window that never closes. + +*How it is checked:* the host's unit tests assert stop–run–start in that order, the restart after a +step that **failed**, the reverse order for several containers, and a service left down said +loudly. The controller refuses, from the definition alone, a window with no schedule, one on a +run-once step, one naming a container the module does not declare, and one naming itself. diff --git a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md index c6fada1..f99903d 100644 --- a/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md +++ b/03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md @@ -2,7 +2,7 @@ layer: to-be status: in-progress code: [mesh-controller internal/catalogue] -updated: 2026-09-30 +updated: 2026-10-02 decisions: - 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md - 02-DECISIONS/0155-a-definition-names-no-installation-and-how-that-is-checked.md @@ -14,6 +14,7 @@ decisions: - 02-DECISIONS/0084-which-provider-serves-a-consumer.md - 02-DECISIONS/0046-a-module-configuration-is-its-assignments-not-its-manifest.md - 02-DECISIONS/0038-the-mesh-assigns-the-port.md + - 02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md --- # 27 — A module requires, the mesh resolves @@ -206,6 +207,22 @@ name when nothing sets it. That is the contract half of this design's operator p the placeholder allows: the definition says which values reach which requirement, and nothing else does. *How it is checked:* the unit tests named in issue 173, and the plan comparison that closed it. +*A provider says once what it derives for each consumer (2026-10-02, +[ADR 0201](../../02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md), +[issue 124](../../04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md)):* +where a provider **names the resource** it gives each consumer — a bucket, a database, a vhost — the +name is derived per consumer, and a literal `serves` block could not carry it. A served value may +now name the consumer the mesh is serving: `${consumer:as}`, the identity the mesh minted, and +`${consumer:as:dns}`, that same identity written as a DNS label. Nothing else — **the mesh learns no +protocol here; it spells its own name in an alphabet it already knows.** Settings are laid on first, +so an operator may still set a prefix and the mesh derives the rest. The mesh fills it at the one +moment it knows who the consumer is, and the one filled value reaches both ends: the consumer, as +its binding's served facts and as `${bound::}` in any file it writes; the provider, +as `derived` on that consumer's entry in its contributions file, so its provisioner is told the name +rather than recomputing it. A consumer that writes the derived value into its own definition instead +of asking for it is refused, naming the placeholder to use. *How it is checked:* the unit tests in +ADR 0201's "how this is checked", each run against the unchanged controller first. + ## How a definition reads what was resolved **One form, naming a requirement and a field of its contract.** A definition that needs the database's @@ -214,7 +231,9 @@ name in a configuration file writes the same thing: the requirement's name and t controller fills it at resolution. This one form replaces the placeholders that exist today, one per mechanism: bound values, secrets, -ports and machine facts. +ports and machine facts. It subsumes the consumer placeholder too — a value a provider derives is +read by the consumer exactly as any other field of the contract is, and `${consumer:…}` is only +how the *provider* states the rule. **The seat placeholder stays, for the controller alone.** The controller composes its own declaration and reaches the store and broker it made before any module existed, so it cannot be diff --git a/04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md b/04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md index 74a7709..50a03a6 100644 --- a/04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md +++ b/04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md @@ -1,9 +1,9 @@ --- -status: open +status: resolved opened: 2026-09-23 -located-in: [] -fixed-by: -amended-design: +located-in: [mesh-controller internal/inventory, mesh-controller internal/artifacts, mesh-host internal/apply, mesh-catalog modules/distribution] +fixed-by: 02-DECISIONS/0189-the-store-keeps-what-the-records-name.md +amended-design: 03-DESIGN/01-to-be/18-building-a-module.md --- # 108 — The registry has no garbage collection, and two doors make it harder to add @@ -75,3 +75,32 @@ real thing services need, and the mesh cannot express one. images by digest and moves by version — is retention "the digests no recorded build names"? - Who owns the routine when the store and its public door are two modules — the store, since the volume is its? + +## Answered, 2026-10-02 — [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md) + +The three open questions, answered: + +- **A maintenance step, or a backend that does not need its writers stopped?** The step. A + scheduled container may name `while-stopped` — resource ids of **its own module's** containers, + which the host stops before the run and starts again after it whatever the step did. A storage + backend the mesh does not run would be a bigger thing to own than the mechanism it avoids, and + the mechanism is wanted anyway: a service that cannot have work done underneath it is a real + shape and the mesh could not express it at all. +- **Is retention "the digests no recorded build names"?** Nearly. An artifact stays because a + definition the mesh holds names it (no age limit), or because it belongs to one of the five most + recent successful builds of its module. Last-N-tags was the predecessor's rule for a registry + that knew nothing else; this mesh knows what each digest is for. +- **Who owns the routine now the second door is gone?** Both halves, each where it can be. The + **mesh** decides what may go — only it holds the records — and asks the store to drop it. The + **store** reclaims the bytes, because only it can stop its own server. Neither half can be done + by the other. + +And the sharpened point — enabling deletion on a door with no accounts — dissolved on inspection: +**that door already accepts a push**, so a writer who can reach it can already replace any tag. +Delete takes nothing a push did not have. What it does not do is undo ADR 0082's bargain, which +putting an authenticated door in front of deletion would have. + +The second registry process is not built, as the 2026-09-26 note says, so the shared blob cache +and the deletion-cached-by-the-other-door problem never arise. Plain `garbage-collect` is enough: +what the mesh keeps is still a manifest in the store, so `--delete-untagged` — the flag that would +delete images machines are running — is not needed at all. diff --git a/04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md b/04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md index 2df48fb..8c8b543 100644 --- a/04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md +++ b/04-ISSUES/124-a-consumer-cannot-be-told-what-its-provider-derived/00-report.md @@ -1,9 +1,9 @@ --- -status: located +status: resolved opened: 2026-09-26 -located-in: [mesh-controller internal/catalogue/declaration.go, mesh-sdk src/provisioner, mesh-catalog modules/minio] -fixed-by: -amended-design: +located-in: [mesh-controller internal/catalogue, mesh-sdk src/provisioner, mesh-catalog modules/minio] +fixed-by: 02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md +amended-design: 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md --- # 124 — A consumer cannot be told a value its provider derived for it, so it transcribes one @@ -63,3 +63,27 @@ compares it to what the provider will actually create. The one wrong instance wa - What would have caught the wrong instance? A test that resolves a consumer's grant and compares the bucket in its own configuration against the one the provider would create is a check that could exist today, for any interface, without the mechanism above. + +## Answered, 2026-10-02 — [ADR 0201](../../02-DECISIONS/0201-a-provider-declares-what-it-derives-for-each-consumer.md) + +The channel is the provider's own `serves` block, which may now name the consumer the mesh is +serving: `${consumer:as}` and `${consumer:as:dns}`. The mesh fills it once, where it knows who the +consumer is, and delivers the one filled value to both ends — the consumer's binding and its +`${bound:…}` substitutions, and the provider's contributions entry, so a provisioner is told the +name rather than deriving it. Each open question above, answered: + +- **Should a provider return values from provisioning?** No. It would make a grant carry data the + provider wrote, make a consumer's declaration wait on its provider's reconcile loop, and put the + rule where nothing can refuse it. The reasoning is in the record. +- **Or should `serves` say a value is derived?** Yes, and the mesh performs the derivation — but it + learns no protocol doing it. The only fact is the identity the mesh itself minted, in one of two + alphabets it already knows. +- **Should a consumer that names the resource be refused?** Yes. A consumer's file that already + contains the value the mesh is about to derive for it is refused at resolution, naming the + placeholder to write instead. That is the check this report asked for, and it is exact rather than + heuristic: a derived value carries the identity minted for this consumer on this machine, which + nothing else would spell out. + +minio's `bucketFor` is gone; its manifest serves `"bucket": "${consumer:as:dns}"`. The three +consumers' hand-written bucket names are gone with it — each of them also named the machine the +module happens to run on, which is the second thing wrong with a transcription. diff --git a/04-ISSUES/202-a-module-whose-required-setting-is-unset-is-silently-left-out/00-report.md b/04-ISSUES/202-a-module-whose-required-setting-is-unset-is-silently-left-out/00-report.md new file mode 100644 index 0000000..20c0245 --- /dev/null +++ b/04-ISSUES/202-a-module-whose-required-setting-is-unset-is-silently-left-out/00-report.md @@ -0,0 +1,93 @@ +--- +status: open +opened: 2026-10-02 +located-in: [mesh-catalog modules/dnsmasq, mesh-controller cmd/mesh-controller] +fixed-by: +amended-design: +--- + +# 202 — A module whose required setting nobody set is left out of the machine, and the resolver is the module it happened to + +## What was observed + +Running the controller's own test suite against the catalogue beside it, 2026-10-02. +`TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves` fails with *"the resolver +was not handed the machines"*. Composing the same machine by hand and listing what it receives +shows why: **dnsmasq contributes nothing at all.** Four resources are composed for that node, all +of them the overlay's. The resolver's package, its configuration, its service and the fact that +carries every machine's name are simply not there. + +The cause is one line added to `dnsmasq`'s configuration earlier the same day: the addresses it +listens on beside the machine's own became an operator setting, +`listen-address=${setting:listen-addresses}`, with no default. A `${setting:…}` nothing sets is +refused, a module that cannot be composed is **left out** rather than failing the whole machine +([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md)), and so a node +assigned the resolver is handed a declaration with no resolver in it. + +The failing test is the symptom that surfaced it. The test is not what is wrong. + +**Proven rather than inferred.** Composing the same machine a second time with +`listen-addresses` set to `127.0.0.1` and nothing else changed, every one of dnsmasq's eight +resources appears — `needs-broker`, `mesh-state`, `package`, `config`, `runtime-dns`, `runtime`, +`service` and `fact-node-zones`. The only difference between a machine with a resolver and a +machine without one is whether somebody set a value that did not exist yesterday. + +## Why it matters beyond this instance + +**Leaving a module out is right, and being quiet about it is not.** The rule exists so one +module's broken setting cannot stop a machine converging — a good rule. But the outcome here is a +machine that applies cleanly, reports current, and is missing its DNS resolver. Every name on that +machine then resolves through whatever was there before, or not at all, and nothing in the mesh +says the resolver was dropped. That is the shape +[issue 152](../152-a-nodes-plan-failure-silently-drops-its-routed-names/00-report.md) records for +routed names, here for a whole module. + +**And a setting with no default is a definition that cannot be assigned.** Every other +`${setting:…}` in the catalogue names something that is genuinely particular to one installation — +a public domain, an issuer. "Which addresses besides my own do I answer on" has an obvious correct +default for every machine that is not a LAN gateway: none beside loopback. A definition that +refuses to compose until somebody sets a value most machines do not need is a definition that +breaks the next node to be assigned it, and genesis with it. + +## What this does not claim + +Whether the live machines are affected was not checked — those four have had the setting set, or +their resolvers would already be gone. The claim is about a machine assigned the resolver *from +now on*, and about the silence. + +## Open questions + +- Should the declaration say which modules it left out, where a person or the console can see it? + `left_out` already travels to the host ([ADR 0163](../../02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md)); + what is missing is anything that reads it back and says so. +- ~~Should a `${setting:…}` be allowed a default?~~ **Decided in principle and not built.** + [ADR 0164](../../02-DECISIONS/0164-a-setting-is-declared-with-its-default-its-meaning-and-what-changing-it-costs.md) + (proposed, 2026-10-01) says a setting with a default is a tunable and one without is the + operator's, and narrows 0155's refusal to exactly the second. `listen-addresses` is a tunable by + that rule, and dnsmasq declares no settings block at all. So this issue is, in part, 0164 waiting + to be built — and in part the silence, which 0164 does not address. +- Is leaving a module out ever right for a module a node is **assigned**, as opposed to one it + merely pulls in? An assignment is somebody saying *this machine runs this*; silently not running + it is the one answer nobody asked for. + +## Still true on 2026-10-04, and the evidence had to be re-taken + +Re-checked after the bundles refactor landed (fourteen records, ADRs 0188 and 0190–0200). **The +fault stands and the old evidence no longer reaches it.** + +`TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves` still fails on +mesh-controller main, with the same message — and now for a *different first reason*. `assign` is +refused before composition ever happens: + +> dnsmasq on anchor has no bus credential: nothing was issued for anchor.dnsmasq … (novox/hq issue 203) + +That is [issue 203](../203-a-fresh-assignment-is-pushed-before-its-credential-exists/00-report.md)'s +new guard doing its job on a test harness that mints no credential. Two faults are stacked in one +failing test, and the second was invisible behind the first. + +Proven again, past both: mint `anchor.dnsmasq` so the assignment stands, then compose the machine +twice. **Without `listen-addresses` the node composes four resources, all the overlay's. With it +set to `127.0.0.1`, all eight of dnsmasq's appear** — `needs-broker`, `mesh-state`, `package`, +`config`, `runtime-dns`, `runtime`, `service`, `fact-node-zones`. Nothing else differs. + +The test is now wrong about two things and should be fixed with whichever of these is fixed first. diff --git a/04-ISSUES/224-an-apply-reopens-a-maintenance-window-by-recreating-what-it-held-still/00-report.md b/04-ISSUES/224-an-apply-reopens-a-maintenance-window-by-recreating-what-it-held-still/00-report.md new file mode 100644 index 0000000..b944e6e --- /dev/null +++ b/04-ISSUES/224-an-apply-reopens-a-maintenance-window-by-recreating-what-it-held-still/00-report.md @@ -0,0 +1,64 @@ +--- +status: open +opened: 2026-10-04 +located-in: [mesh-host internal/apply/apply.go, mesh-host internal/apply/schedule.go] +fixed-by: +amended-design: +--- + +# 224 — An apply arriving during a maintenance window reopens it, by recreating the container the window is holding still + +## What was observed + +Reviewing [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md)'s +`while-stopped` before merging it, 2026-10-04. Found by reading, not by running. + +A scheduled step may hold its module's containers still while it runs. The host stops them, runs +the step, starts them again. Nothing tells the **apply** that a window is open, and the apply's +rule for a container it finds stopped is to replace it: + +``` +case existed && (before.Spec == want || legacy) && before.Running && len(reasons) == 0: + out.Action = "unchanged" +case existed: + rm -f +``` + +`before.Running` is false for a container a window is holding, so the second branch takes it: +the container is removed and recreated, **running**, in the middle of the step that required it +to be still. + +## Why it matters + +For the store, which is what the field was built for, the chain is: a push lands at 03:30 → the +apply recreates the registry → the registry accepts an upload from a build running at the same +time → `garbage-collect`, already past its mark phase, sweeps the blob that upload just wrote. +The image is then in the store with a layer missing, and the build that made it reported success. + +Two things have to coincide, so it is not likely. It is also not rare enough to leave unsaid: the +mesh pushes on every merge, at any hour, and a collection over a store this size is minutes rather +than seconds. + +**The general shape is the one that matters.** `while-stopped` is the first thing in the mesh that +makes a container's stopped state *intentional*. Everything else in the host reads "stopped" as +"broken, fix it", which is right everywhere else and wrong here. Any future use of the field +inherits this. + +## What this is not + +Not a regression. The store has never collected anything, so nothing is worse than it was; this +is a hole in something new rather than something that broke. + +## Open questions + +- **Should a window take the apply lock?** The daemon already serialises applies with `applying` + and, across processes, with `store.Lock`. A window that held it would make the race impossible. + The cost is that a push arriving mid-window waits for minutes, and a push that waits is what + [issue 185](../185-a-refused-membership-publish-stops-the-controller/00-report.md)'s + family of outages looked like from outside. +- **Or should the apply learn that a container is held?** Narrower: the scheduler says which + containers a window currently holds, and `applyContainer` reports those unchanged instead of + recreating them. Nothing blocks, and the apply tells the truth for the minutes it matters — + at the cost of a second source for "is this container meant to be running". +- Either way: should the *report* say a window is open, so a machine that looks half-stopped at + 03:31 reads as working rather than broken?