From fa2d199b16c72d9ff68dbb76704b73a8cb8861c5 Mon Sep 17 00:00:00 2001 From: jochen Date: Wed, 4 Nov 2026 00:31:30 +0100 Subject: [PATCH] Design 51: answer the second review (event wakes for every floor source, versions within the state cap, the direct-write and old-form windows shaped as the old name's, release on the caller list, distribution's own family) Also writes the operator's answer of 2026-11-04 on anonymous reads (mesh network only, the public route refusing the forge's package paths), the expected count of three builds per module, and the revert's order by the graph. Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- .../51-the-registries-keep-what-is-named.md | 139 ++++++++++++------ 1 file changed, 93 insertions(+), 46 deletions(-) diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 902fc58f..bc1d097d 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -20,6 +20,10 @@ decisions: - "ADR 0607" - "ADR 0610" - "ADR 0611" + - "ADR 0595" + - "ADR 0596" + - "ADR 0564" + - "ADR 0240" - "ADR 0311" - "ADR 0251" - "ADR 0189" @@ -64,17 +68,20 @@ whose declaration names an artifact has applied a build whose node-engine states (ADR 0614 §4), judged by that stated capability and never by a fingerprint. Its tool `registry_old_name` lists the nodes still holding it open, and drops a retired node from the window by hand, recorded as a hand act; a node holding it open for seven days is a finding naming the node, cleared when the window closes. Closing the old name emits -`old-name-closed` on both image seats' `.events`. A node away when the last `delivered` was said must close the -window when it returns and applies the new build, with no other delivery: the event that says such a node's return, -and the verb that answers each node's stated capabilities, are not yet named by any record or design (the stated -capability comes with the release that adds it, design 47); until they are, the window waits on the next `delivered`, -tracked on issue 1966. `cargo-package-registry` stays +`old-name-closed` (the old seat name, the time, the nodes that held it last, and the nodes dropped by hand) on +`distribution`'s own family `distribution.events`, kept seven days, never on a seat's `.events`: it is one +module's window, not the seats' contract. A node away when the last `delivered` was said closes the window when it +returns and applies the new build, with no other delivery: what wakes `distribution` then is the node-engine seat's +event for a change of a node's stated manifest-reads, and the delivery view's read of them, both decided in ADR 0618 +(proposed after lead-47's final release; the event's family is filled in once it is), never a watch of the +node-engine's have state (ADR 0596). `cargo-package-registry` stays named, without a holder, until a build module publishes cargo packages. **One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `publish-part` (ADR 0611), `versions`, `remove` (a tagged version; the holder itself refuses `remove` of a version with no tag, but for the retention removal below), and -on `container-image-registry` `mirror` and `tags`; the two seats `distribution` holds add `hold` and its release, -with the events `held` and `released`, for the artifact sweep (ADR 0617 §2). The five seats this milestone changes, these four and `mesh-broker`, are defined whole, their contracts moved +on `container-image-registry` `mirror` and `tags`; the two seats `distribution` holds add `hold` and `release`, +with the events `held` and `released`, for the artifact sweep (ADR 0617 §2), both verbs in the seat definition and +in `distribution`'s caller list. The five seats this milestone changes, these four and `mesh-broker`, are defined whole, their contracts moved into `mesh-contracts` (ADR 0613 §1; design 26). Events: `published`, `refused` (added to ADR 0605 §2's two, additively) and `removed`, each carrying the name, version, kind, tag, digest, fingerprint, commit and holder, and `refused` its reason; each seat's events are its own family, `.events`, kept seven days. A holder stores @@ -85,8 +92,9 @@ form ADR 0363 gives a restricted family: `publish` and `publish-part` the build proved caller that opened its transfer; `mirror` mesh-build-image; `remove` the builder, for a tagged version, and the holder of the artifact sweep, for an untagged bundle, archive or copied base with the reason retention (ADR 0610 §3); `hold` (on the two seats `distribution` holds: keep one untagged bundle, archive or copied base that the artifact -sweep's rules keep from removal by retention, and let it go again; never an image of its own) the module that runs -the artifact sweep alone (ADR 0617 §2). Every removal, whoever asked it, emits `removed` with its reason. +sweep's rules keep from removal by retention; never an image of its own) the module that runs the artifact sweep +alone (ADR 0617 §2); `release` (let a hold go) that module, and the operator's session only through +`registry_holds`, each release by hand recorded as a hand act. Every removal, whoever asked it, emits `removed` with its reason. Declaring `invokes` alone grants nothing. The holder checks the caller identity the tool runner proved and stamps on the request (design 25 § 12.4), never a name in the request's body, and answers any other caller with a refusal naming the verb. @@ -139,22 +147,34 @@ the verb. `distribution`'s own program serves the store's port in front of the registry container, which listens on its node's loopback only: reads (the OCI distribution protocol's GET and HEAD) stay open to every node's container runtime, and a write reaches the store only from `distribution`'s own process, after a `publish` it accepted. - **Until every build module publishes through the seats**, the build modules' builds still running push straight to - the store, as they do today: so `distribution`'s program also accepts a write from a build module's own account - until every node's have of every build module, read through `node_haves` at its start and on each `delivered`, runs - a build that publishes through the seats; then it refuses them, naming `publish`. **While the program is down**, - the store answers no read and no write: a node pulling an image fails and retries, and what already runs is not - touched; the holder's health check, which probes the read side every minute, names it, and the program serves - again at once when it restarts, holding nothing but its transfers in `incoming`. Each + **Until every build module publishes through the seats**, a window, the build modules' + builds still running push straight to the store, as they do today, with no credential: today's store takes an + unauthenticated OCI write from the mesh network, and no build module holds an image push credential. So + `distribution`'s program keeps accepting such a write from the mesh network, no wider than today, until every + node's have of every build module, read through `node_haves` at its start and on each `delivered`, runs a build + that publishes through the seats; then it refuses every write but its own process's, naming `publish`, says so in + its log and emits `direct-writes-closed` (the time, the build modules' builds last seen, the nodes dropped by hand) + on `distribution.events`. Since the window has no credential, closing it removes nothing from the store or the + vault: it is the program's refusal alone. Its tool `registry_direct_writes` lists whether the window is open and + the nodes whose have still runs an old build module, and drops a retired node from it by hand, recorded as a hand + act; a node holding it open for seven days is `distribution`'s finding naming the node, cleared when the window + closes. **While the program is down**, the store answers no read and no write: a node pulling an image fails and + retries, a build module's write fails and its build says so, and what already runs is not touched. The holder's + health check is declared in its manifest and run outside the program's process, as every module's (ADR 0240), so + it still answers while the program is down: it probes the read side every minute and names the program. The + program serves again at once when it restarts, holding nothing but its transfers in `incoming`. **On + `distribution`'s own delivery** its new images are pulled from the store before its program stops, since a stopped + program serves no pull, its own images included. Each holder keeps its write side as an own secret beside a read-only tool token (the forge module: `write:package`, its tool token `read:package`). - **The package seats' read sides need no credential** for the mesh's own packages: the forge serves the mesh owner's packages to anonymous reads, so the provisions `npm-package-registry` and `go-package-registry` carry no credential (form none), and mesh-build-node, which held a pair for both writing and installing, requires none (the operator's answer of 2026-11-03, that the mesh's packages may be read without a credential). The scope is the - mesh network only: the store's GET and HEAD and the forge's package reads answer anonymously there, and through no - public route. Whether the forge has a public route that would also serve its package reads anonymously is not - known to this design, to confirm with the operator before the merge. + mesh network only (the operator, 2026-11-04): the forge's package reads and the store's GET and HEAD answer + anonymously on the mesh network only, and the public route refuses the forge's package paths, as it already + refuses the forge's internal API. That refusal is a route-proxy route change for the forge module, delivered in this + milestone. - **The forge module serves both package seats' verbs from its Go bundle**, the one that already serves its package retention, renamed for both packages; its tools bundle in TypeScript is unchanged, so no holder needs generated TypeScript types in this milestone. @@ -186,8 +206,10 @@ the verb. version, and the triggers end. An image, bundle or archive is numbered per fingerprint, as above. - **The builder's state `versions`** holds one key per name, tag and version, with its fingerprint, commit, digest, time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the reason (a - branch's end, retention, the sweep) and the event that removed it. History one, cap 1 GiB; past 80 % the builder - writes a finding, cleared below it. Only the builder reaches it; others call `builder_versions` or hear its events. The build log + branch's end, retention, the sweep) and the event that removed it. History one, cap 64 MiB, the most the template + state allows (ADR 0595 §3). An estimate, not a count: about 30,000 keys a year (some 18,000 trunk versions, about + 50 numbered builds a day, and some 10,000 tagged ones) of about 0.5 KiB each, some 15 MiB a year, so the cap holds + about four years at that rate; past 80 % the builder writes a finding, cleared below it. Only the builder reaches it; others call `builder_versions` or hear its events. The build log carries each change too. - **The floor**: on a name's first claim, or when `versions` holds nothing for it, the count starts above the highest of the seat's `versions`, the package's highest git tag (the forge bridge's tool `bridge_tags`) and the highest @@ -199,9 +221,13 @@ the verb. the builder: waiting would never end. So a trunk build of an image, bundle or archive whose floor is not yet known is claimed with no number, so none travels in its build request, built unnumbered and published by its digest (as a bundle or archive is from every done key, ADR 0605 §3; an image so by ADR 0617 §1). Such fingerprints are numbered - in merge order once the floor answers (ADR 0606 §1). That - publish is what wakes the floor: the builder hears the holder's `published` on its durable consumer of - `.events` and asks the floor again then, never on a timer; once all three answer it numbers the build and emits + in merge order once the floor answers (ADR 0606 §1). What wakes the floor is an event of the source that was + silent, never a timer: for the seat's `versions`, the holder's `published` on the builder's durable consumer of + `.events`, which that by-digest publish itself says; for `bridge_tags`, the forge bridge's next message on + its stream `forge`, which the builder already consumes; for `indexer_requirements`, the indexer's next + `manifest-accepted`; and for each, the condition keeper's event clearing the silent module's health condition, on + the seat `mesh-conditions`' family (ADR 0564 §3), so a source that recovers with nothing to say still wakes it. On + any of these the builder asks the silent sources again; once all three answer it numbers the build and emits `numbered`, which the build module consumes to call `publish` with the version and the held digest, which stores nothing. A package's trunk build waits for its floor, since a package is published only under its version. @@ -246,9 +272,10 @@ With the store's write side closed, the artifact sweep (seat `mesh-artifact-swee longer deletes or holds by writing to the store: it calls `remove` on the seat that holds the thing, with the reason retention, and `hold` to keep an untagged bundle, archive or copied base its rules keep from removal by retention, which then skips it (ADR 0617 §2), both granted to its module alone by `distribution`'s caller list; its rules of what is kept (*The artifact store* below) stand, and it reads both reference forms. -`distribution` keeps the holds in its own state `holds` (bus state, template state), emits `held` and `released` -on the seat's `.events`, and its tool `registry_holds` lists what is held, since when and for whom, and -releases one by hand. A `remove` or `hold` the holder refuses to the sweep writes the sweep's finding, naming the +`distribution` keeps the holds in its own state `holds` (bus state, template state), emits `held` (the thing's +name, kind and digest, the caller that asked, the time) and `released` (the same, and who let it go: the sweep, or +the operator's session by hand) on the seat's `.events`, and its tool `registry_holds` lists what is held, +since when and for whom, and releases one by hand through the seat's `release`, recorded as a hand act. A `remove` or `hold` the holder refuses to the sweep writes the sweep's finding, naming the thing and the reason, cleared by its next run with no refusal. The Go seat keeps what the npm seat keeps: every referenced version and the newest five of each package series. A Go version is referenced when a `go.mod` on a repository's default branch requires it; the forge module reads every @@ -262,7 +289,9 @@ publish refused (the holder when it refuses, the builder when a build module's b is stored, or when a later version of the name is published), a registry seat not answering (each holder's health check, which calls `versions` on each seat it holds and probes the read side every minute, so a dead seat is seen with no publish running), a removal not done (the builder), a floor source silent (the builder), a store past 80 % (the holder, -the builder), a node holding the old seat name open for seven days (`distribution`, cleared when the window closes). A build narrates its publish: asked, waiting on a seat, stored, refused. +the builder), a node holding the old seat name open for seven days (`distribution`, cleared when the window closes), a node +holding open the window of direct writes for seven days (`distribution`, the same), a node holding open the window of +the node-engine's old-form done key for seven days (the builder, the same). A build narrates its publish: asked, waiting on a seat, stored, refused. ### The move, one milestone, ordered by the module graph @@ -278,24 +307,31 @@ its own rule; nothing stores the order (ADR 0612 §2, ADR 0614 §1). The edges t | mesh-build-go's fetch from the Go seat only after the forge module serves its read side | provision: `go-package-registry`, offered only by the forge module's new build (ADR 0603 §5) | the assigner and the provisioner | | No credential goes before its replacement works | the build modules' manifests drop the write credentials in the same change that makes them publish through the seats, so the vault mints none for the build that publishes (ADR 0612 §2) | the served-verb edge above | | A build carrying the new reference form, or a manifest using this milestone's words or seat names, reaches a node only through a node-engine that reads them | a reader's capability the node-engine states, the baseline when it states none (ADR 0614 §4) | the assigner, holding that node's want and naming the capability | -| The holders' `incoming` and `.events` exist before the holders start | bus state: each entry is a requirement on the provision `bus-state` of the bus module (ADR 0614 §5) | the assigner, as for any provision | +| The holders' `incoming`, `.events`, `distribution`'s `holds` and `distribution.events` exist before the holders start | bus state: each entry is a requirement on the provision `bus-state` of the bus module (ADR 0614 §5) | the assigner, as for any provision | | The shared contracts, the forge module that holds their registry and the build module that builds them | package and build module edges are triggers, never waits; the cycle they make holds nothing and is not refused (ADR 0614 §6); inside the group every member takes `mesh-contracts` and mesh-sdk by the group's head (the checker's group pin) | the indexer's graph | -**How many builds** (ADR 0603 §3). The three build modules change in this milestone, so every module they build is -built twice: at once with the recipe that runs, and again once a majority of build nodes runs the new build module. -The contracts move rides that first build and adds none (ADR 0613 §1). Whether the rebuild that a first numbered -publish of mesh-sdk or `mesh-contracts` triggers in their users (ADR 0614 §6) joins the second build or adds a third -is not decided by ADR 0603 or ADR 0614, so this design gives no count beyond two per module; it is put to the -operator. +**How many builds** (ADR 0603 §3). The three build modules change in this milestone, so a module they build can +be built up to three times in one window, and this milestone expects that count: once with the recipe that runs, +once more when the agreement of the build nodes moves to the new build module, and once more when a first numbered +publish of mesh-sdk or `mesh-contracts` triggers their users (ADR 0614 §6). The contracts move rides the first +build and adds none (ADR 0613 §1). ADR 0614 bounds nothing here; a bound, such as a settle before building a +fingerprint whose inputs still move, is a question for a record, put to the operator, and is not designed here. **The node-engine's own build keeps the old reference form** until every node-engine states the capability to read the new one: a node-engine away across the milestone must be able to read the build it wakes to, or it could never update itself (ADR 0603 §7, §9). The builder writes the node-engine's done key in the old form until the stated -capabilities, which the status view reads, show it on every node. +capabilities show it on every node; this is a window like the old seat name's. What wakes the builder is the node-engine seat's event +for a change of a node's stated manifest-reads, and the delivery view's read of them, both decided in ADR 0618 +(proposed after lead-47's final release; the event's family is filled in once it is), never a watch of the +node-engine's have state (ADR 0596). Its tool `builder_old_form` lists whether the window is open and the nodes whose +node-engine does not yet state the capability, and drops a retired node from it by hand, recorded as a hand act; a +node holding it open for seven days is the builder's finding naming the node, cleared when the window closes. **When the group is pushed.** Only once the release that adds the node-engine's stated capability (design 47) runs on every node, read from the status view's haves before the push, and once the progressive insights of ADR 0595 and -ADR 0597 this design is written to are accepted. +ADR 0597 this design is written to are accepted. A node away for weeks does not hold the push for ever: it is left +out of that read only by a recorded hand act naming the node and why, as `registry_old_name` drops one, said in the +group's pull request; the node then takes the new node-engine as any away node does when it returns. **The milestone's own check** runs the changed build modules (ADR 0603 §10) against the running holders, which serve no `publish`: a changed build module whose manifest invokes a verb no running build serves is one the check cannot @@ -313,10 +349,11 @@ guard go in the same change (ADR 0606 §6, read with ADR 0612 §1). **Rolled back by one revert** (ADR 0612 §4): reverting the milestone's merge brings back the build modules' manifests that require the write credentials, so the vault mints them again as own secrets on the revert's delivery, and the builds that publish through the seats go with it. The old seat name still retires on the node-engines' -reports (ADR 0610 §4). Two things of the revert are not decided by any record and are tracked on issue 1966: which -edge delivers `distribution`'s reverted build, which takes direct writes again, before the build modules' reverted -builds that push straight to the store; and whether a done key written in the new form is built again after the -revert or must stay readable by the reverted readers, which the old mesh-sdk refuses. **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images +reports (ADR 0610 §4). The revert's order is the module graph's, as the move's is: `distribution`'s reverted build +is delivered before a build module's reverted build only where that build module's reverted manifest requires or +invokes something only `distribution`'s reverted build offers; the milestone's prepare step checks the build +modules' manifests for such an edge. The done keys written in the new form stay in the builder's state after a +revert and are not built again for their form; whether an older reader can read them is ADR 0618's. **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images with a one-off credential of the bootstrap, discarded once the holders serve `publish`. ### Acceptance tests @@ -354,15 +391,23 @@ answers (a holder down whole is its module's own health check's); `registry_inco refused sweep `remove` writes the sweep's finding and its next clean run clears it; the forge bridge says a branch-deleted kind for the forge module's `forge.delete`, a `forge.delete` from another account is refused by the bus, and an existing consumer of `forge` ignores the new kind; with the program stopped a read fails and the - holder's finding is written, and after its restart the read is answered and the finding cleared; a build module's - old build pushing straight to the store is accepted until every node runs a build module publishing through the - seats, and refused after. + holder's finding is written, a write fails too, and after its restart the read is answered and the finding + cleared; a build module's old build pushing straight to the store is accepted until every node runs a build + module publishing through the seats, and refused after, naming `publish`, with `direct-writes-closed` emitted and + its log line written; `registry_direct_writes` names the node still running an old build module, and drops it by + hand with a hand act; `release` from a caller not on the caller list is refused, and `registry_holds`' release + records its hand act; a waiter on a build's `published` does not wake on `held`, `released`, `old-name-closed` + or `direct-writes-closed`. - **Builder, added for the milestone.** With the seat's holder down at a first claim, a bundle's trunk build is built unnumbered and published by its digest, the holder's `published` makes the builder ask the floor again and number it, and the build module, hearing `numbered`, publishes the version with the held digest; a package's trunk build waits; a second trunk build of a package's revision held `published` keeps its version and publishes nothing, while one numbered but refused is published by its second build under the same version; with `bridge_tags` - silenced (a floor source other than the publish target) the floor finding is written and cleared once it answers; + silenced (a floor source other than the publish target) the floor finding is written and cleared once it answers, + and with `bridge_tags` still silent after the by-digest publish, the build stays unnumbered until the forge + bridge's health clears (no timer runs), and is then numbered and published by version; the node-engine's done key + stays in the old form while one node's node-engine lacks the capability and is written in the new form once every + node states it, `builder_old_form` naming that node meanwhile and dropping it by hand with a hand act; with the image's own seat silent at the claim of `distribution`'s own image, the image is still built, published by its digest, and numbered once its new build answers (ADR 0617 *How it is checked*). - **Checker.** A changed build module invoking a verb no running holder serves is not run, and the verdict names the @@ -374,7 +419,9 @@ answers (a holder down whole is its module's own health check's); `registry_inco `registry_old_name` names the nodes holding the old name open, an away node among them. After the delivery: mesh-sdk's first numbered version is `published` on the Go seat and an image is read back by its digest, both shown by `builder_versions`; a node runs a module from a reference in the new form; a node-engine away across the - milestone wakes and applies the new node-engine. Go's `direct`: its removal is named on the milestone's issue, and + milestone wakes and applies the new node-engine; `distribution`'s own delivery pulls its new images before its + program stops; a package read through the public route is refused, and the same read inside the mesh answers. + Go's `direct`: its removal is named on the milestone's issue, and the change that removes it adds to the Go build module's recipe test a case that refuses `direct` in the fetch. ## Reviewed by architectural domain