From a9745b63cb85508143f7a2cbb01e3f43370931dd Mon Sep 17 00:00:00 2001 From: jochen Date: Wed, 4 Nov 2026 00:42:35 +0100 Subject: [PATCH] Design 51: answer the third review (ADR 0618 only reserved, a module's images pulled before its containers are replaced, public package reads kept, levels on every new case) The windows close on the next delivered until ADR 0618 is accepted; distribution's first delivery needs the node-engine to pull every image first, since its registry serves every pull; the operator kept public reads, so ADR 0346 stands and the route refusal goes. Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- .../51-the-registries-keep-what-is-named.md | 167 ++++++++++++------ 1 file changed, 110 insertions(+), 57 deletions(-) diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index bc1d097d..aa3ea32d 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -9,6 +9,7 @@ code: - mesh-delivery modules/mesh-delivery-builder (versions, tag asks), the forge bridge, the indexer, the checker, the assigner - mesh-sdk (the bus helper for sending in parts) - mesh-catalog modules/docker (image pruning) + - mesh-host internal/apply (every new image of a module pulled before any of its containers is replaced) updated: 2026-11-04 decisions: - "ADR 0605" @@ -22,6 +23,9 @@ decisions: - "ADR 0611" - "ADR 0595" - "ADR 0596" + - "ADR 0615" + - "ADR 0346" + - "ADR 0386" - "ADR 0564" - "ADR 0240" - "ADR 0311" @@ -69,12 +73,14 @@ whose declaration names an artifact has applied a build whose node-engine states nodes still holding it open, and drops a retired node from the window by hand, recorded as a hand act; a node holding it open for seven days is a finding naming the node, cleared when the window closes. Closing the old name emits `old-name-closed` (the old seat name, the time, the nodes that held it last, and the nodes dropped by hand) on -`distribution`'s own family `distribution.events`, kept seven days, never on a seat's `.events`: it is one -module's window, not the seats' contract. A node away when the last `delivered` was said closes the window when it -returns and applies the new build, with no other delivery: what wakes `distribution` then is the node-engine seat's -event for a change of a node's stated manifest-reads, and the delivery view's read of them, both decided in ADR 0618 -(proposed after lead-47's final release; the event's family is filled in once it is), never a watch of the -node-engine's have state (ADR 0596). `cargo-package-registry` stays +`distribution`'s own family `distribution.events`, kept seven days and capped at 16 MiB, dropping its oldest when +full (it carries a handful of events per window), never on a seat's `.events`: it is one module's window, not +the seats' contract. A node away when the last `delivered` was said should close the window when it returns and +applies the new build, with no other delivery: what wakes `distribution` then is to be decided in ADR 0618 +(reserved), the node-engine seat's event for a change of a node's stated manifest-reads and the delivery view's read +of them, never a watch of the node-engine's have state (ADR 0596). **Until ADR 0618 is accepted**, such a node closes +the window on the next `delivered`, whichever delivery says it: `distribution` asks `node_haves` then, finds the +returned node's new build, and closes. `cargo-package-registry` stays named, without a holder, until a build module publishes cargo packages. **One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `publish-part` (ADR 0611), `versions`, `remove` (a @@ -94,7 +100,11 @@ holder of the artifact sweep, for an untagged bundle, archive or copied base wit `hold` (on the two seats `distribution` holds: keep one untagged bundle, archive or copied base that the artifact sweep's rules keep from removal by retention; never an image of its own) the module that runs the artifact sweep alone (ADR 0617 §2); `release` (let a hold go) that module, and the operator's session only through -`registry_holds`, each release by hand recorded as a hand act. Every removal, whoever asked it, emits `removed` with its reason. +`registry_holds`, each release by hand recorded as a hand act. When `registry_holds`, `distribution`'s own tool, +calls its own seat's `release`, the caller the holder proves is the one the tool runner proved and stamped on the +`registry_holds` call, the operator's session (ADR 0386 §7), carried on to `release` by the tool runner's stamp and +never by the tool's own words; `distribution`'s own account is not on `release`'s list, so the tool cannot release in +its own name. Every removal, whoever asked it, emits `removed` with its reason. Declaring `invokes` alone grants nothing. The holder checks the caller identity the tool runner proved and stamps on the request (design 25 § 12.4), never a name in the request's body, and answers any other caller with a refusal naming the verb. @@ -154,7 +164,9 @@ the verb. node's have of every build module, read through `node_haves` at its start and on each `delivered`, runs a build that publishes through the seats; then it refuses every write but its own process's, naming `publish`, says so in its log and emits `direct-writes-closed` (the time, the build modules' builds last seen, the nodes dropped by hand) - on `distribution.events`. Since the window has no credential, closing it removes nothing from the store or the + on `distribution.events`. While the window is open, a direct write to a tag the seat holds `published` is refused, + naming the tag and `publish`, so an old build module cannot overwrite a version published through the seat. Since the + window has no credential, closing it removes nothing from the store or the vault: it is the program's refusal alone. Its tool `registry_direct_writes` lists whether the window is open and the nodes whose have still runs an old build module, and drops a retired node from it by hand, recorded as a hand act; a node holding it open for seven days is `distribution`'s finding naming the node, cleared when the window @@ -162,19 +174,34 @@ the verb. retries, a build module's write fails and its build says so, and what already runs is not touched. The holder's health check is declared in its manifest and run outside the program's process, as every module's (ADR 0240), so it still answers while the program is down: it probes the read side every minute and names the program. The - program serves again at once when it restarts, holding nothing but its transfers in `incoming`. **On - `distribution`'s own delivery** its new images are pulled from the store before its program stops, since a stopped - program serves no pull, its own images included. Each - holder keeps its write side as an own secret beside a read-only tool token (the forge module: `write:package`, its + program serves again at once when it restarts, holding nothing but its transfers in `incoming`. Each holder keeps its write side as an own secret beside a read-only tool token (the forge module: `write:package`, its tool token `read:package`). +- **`distribution`'s first delivery** moves the registry container to its node's loopback and lets the program take + the store's port, and it is the one delivery whose own containers serve every pull. Read on the node-engine's main + (its apply of a container): the node-engine pulls a container's new image just before it replaces that container, + then sets the running one aside, stopped, starts the new one, and starts the old one again if the new one does not + run (issues 1742 and 1945). It does so one container at a time, so a module with two containers pulls its second + image only after its first container was replaced; for `distribution` that second pull would meet the stopped + registry. **So the node-engine pulls every new image of a module before it replaces any of its containers**, and a + pull that fails replaces nothing, the module keeping what it ran: a change in the node-engine, a subtask of this + milestone's node-engine work. The quick fix, a second port for this one delivery, is refused, since every module + with more than one container meets the same order. `distribution`'s declaration lists the registry container before + the program, so the port is free when the program starts. **What a pull meets in the swap**: from the registry + container's stop until the program listens on the port, the store answers nothing. On main, a failed pull fails that + module's apply, and the node agent tries a failed apply again only on the next change of the module's want or + composition, on the node-engine's verb `compose` naming the module (by hand), or when its liveness judge finds a + declared resource missing or down; only a failure of the bus itself is tried again on a clock (a bounded backoff, + at most four times before the have says failed). So another module's apply whose pull falls in that window stays + failed, its running containers kept, until one of those: its have says failed naming the image, and `compose` + naming the module is its recovery. - **The package seats' read sides need no credential** for the mesh's own packages: the forge serves the mesh owner's packages to anonymous reads, so the provisions `npm-package-registry` and `go-package-registry` carry no credential (form none), and mesh-build-node, which held a pair for both writing and installing, requires none (the - operator's answer of 2026-11-03, that the mesh's packages may be read without a credential). The scope is the - mesh network only (the operator, 2026-11-04): the forge's package reads and the store's GET and HEAD answer - anonymously on the mesh network only, and the public route refuses the forge's package paths, as it already - refuses the forge's internal API. That refusal is a route-proxy route change for the forge module, delivered in this - milestone. + operator's answer of 2026-11-03, that the mesh's packages may be read without a credential). Public reads stay (the + operator, 2026-11-04): the forge's package reads and the store's GET and HEAD answer anonymously on the mesh network + and, where a public route serves them today, through it, the forge's public route included. The forge keeps serving + the registry's public root (ADR 0346 §6, unchanged), so a lock file naming that root installs as today, and the lab + trusts it as before (ADR 0346 §7). - **The forge module serves both package seats' verbs from its Go bundle**, the one that already serves its package retention, renamed for both packages; its tools bundle in TypeScript is unchanged, so no holder needs generated TypeScript types in this milestone. @@ -227,7 +254,8 @@ the verb. its stream `forge`, which the builder already consumes; for `indexer_requirements`, the indexer's next `manifest-accepted`; and for each, the condition keeper's event clearing the silent module's health condition, on the seat `mesh-conditions`' family (ADR 0564 §3), so a source that recovers with nothing to say still wakes it. On - any of these the builder asks the silent sources again; once all three answer it numbers the build and emits + any of these the builder asks the silent sources again; beside them, its tool `builder_floor_again` asks the three + sources of one name again at once by hand, recorded as a hand act. Once all three answer it numbers the build and emits `numbered`, which the build module consumes to call `publish` with the version and the held digest, which stores nothing. A package's trunk build waits for its floor, since a package is published only under its version. @@ -237,7 +265,8 @@ the verb. after a holder failed; when the build's file is gone it builds the fingerprint again under the same number, and the new digest replaces the recorded one only if no `published` was ever emitted for that number, else it takes a new number; it refuses a version refused for other bytes), `builder_tag` (ask a tagged version), `builder_tag_asks` (open asks and - their state) and `builder_untag` (remove a branch's tagged versions). + their state), `builder_untag` (remove a branch's tagged versions), `builder_floor_again` (ask a name's floor + again by hand, a hand act) and `builder_old_form` (below). ### Tagged versions @@ -320,16 +349,23 @@ fingerprint whose inputs still move, is a question for a record, put to the oper **The node-engine's own build keeps the old reference form** until every node-engine states the capability to read the new one: a node-engine away across the milestone must be able to read the build it wakes to, or it could never update itself (ADR 0603 §7, §9). The builder writes the node-engine's done key in the old form until the stated -capabilities show it on every node; this is a window like the old seat name's. What wakes the builder is the node-engine seat's event -for a change of a node's stated manifest-reads, and the delivery view's read of them, both decided in ADR 0618 -(proposed after lead-47's final release; the event's family is filled in once it is), never a watch of the -node-engine's have state (ADR 0596). Its tool `builder_old_form` lists whether the window is open and the nodes whose -node-engine does not yet state the capability, and drops a retired node from it by hand, recorded as a hand act; a -node holding it open for seven days is the builder's finding naming the node, cleared when the window closes. +capabilities show it on every node; this is a window like the old seat name's. What should wake the builder for a +node that returns is to be decided in ADR 0618 (reserved): the node-engine seat's event for a change of a node's stated +manifest-reads and the delivery view's read of them, never a watch of the node-engine's have state (ADR 0596). **Until +ADR 0618 is accepted**, the builder hears the status view's `delivered` events, asks `node_haves` at its start and on +each, and a node that was away closes the window on the next `delivered`. Its tool `builder_old_form` lists whether the +window is open and the nodes whose node-engine does not yet state the capability, and drops a retired node from it by +hand, recorded as a hand act; a node holding it open for seven days is the builder's finding naming the node, cleared +when the window closes. Closing it says so in the build log and emits `old-form-closed` (the time, the nodes that held +it last, the nodes dropped by hand) on the builder's own family beside `numbered`; a waiter on a build's end does not +wake on it. **When the group is pushed.** Only once the release that adds the node-engine's stated capability (design 47) runs on every node, read from the status view's haves before the push, and once the progressive insights of ADR 0595 and -ADR 0597 this design is written to are accepted. A node away for weeks does not hold the push for ever: it is left +ADR 0597 this design is written to are accepted, and once the running judge accepts the process-subject form, the +manifest form by which the indexer and the assigner serve their verbs on their own process subjects (design 25 +§ 12.2): the judge that runs accepts that form before the indexer and the assigner use it, decided in a record of +this milestone extending ADR 0615 §3 (to be proposed). A node away for weeks does not hold the push for ever: it is left out of that read only by a recorded hand act naming the node and why, as `registry_old_name` drops one, said in the group's pull request; the node then takes the new node-engine as any away node does when it returns. @@ -353,7 +389,8 @@ reports (ADR 0610 §4). The revert's order is the module graph's, as the move's is delivered before a build module's reverted build only where that build module's reverted manifest requires or invokes something only `distribution`'s reverted build offers; the milestone's prepare step checks the build modules' manifests for such an edge. The done keys written in the new form stay in the builder's state after a -revert and are not built again for their form; whether an older reader can read them is ADR 0618's. **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images +revert and are not built again for their form; whether an older reader can read them is to be decided in ADR 0618 +(reserved). **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images with a one-off credential of the bootstrap, discarded once the holders serve `publish`. ### Acceptance tests @@ -372,9 +409,10 @@ test). after the build's file is gone builds again under the same number; with the seat's holder stopped at a first claim nothing is numbered and the finding names it; two merges claimed in reverse order are numbered in merge order; a retention removal turns the entry `removed`. -- **Builder, added in round 2.** Two merges close together, the first replaced before its claim: the second is - numbered at once; a build module's bound running out writes the builder's finding, and a later version of the name - clears it; `builder_publish_again` after a `published` takes a new number. +- **Builder, added in round 2** (integration, cases of the builder's whole-module test). Two merges close together, + the first replaced before its claim: the second is numbered at once; a build module's bound running out writes the + builder's finding, and a later version of the name clears it; `builder_publish_again` after a `published` takes a + new number. - **Holder (gitea, distribution).** A caller not on the caller list is refused for each verb, and one module naming another as the caller in the request's body is refused; a part sent by a module other than its transfer's opener is refused; a transfer cut in the middle resumes from the parts held, @@ -382,34 +420,48 @@ test). survives retention, and an unreadable `go.mod` stops the run; a part whose bytes do not match its digest is refused; only the missing layers are sent; `remove` of a version with no tag is refused; with no publish running, a Go seat that stops answering while its holder runs writes its finding and clears it when it -answers (a holder down whole is its module's own health check's); `registry_incoming` + answers (a holder down whole is its module's own health check's); `registry_incoming` lists and drops. -- **Holder, added for the milestone.** A write to the store's port from any account but `distribution`'s process is - refused, and a read from a node's container runtime is answered; the artifact sweep's `remove` with the reason - retention removes an untagged bundle and emits `removed`, and the same call from another caller is refused; `hold` - keeps an archive from retention's removal, a `hold` of an image or from another caller is refused, emits `held` and is listed by `registry_holds`, which releases it with `released`; a - refused sweep `remove` writes the sweep's finding and its next clean run clears it; the forge bridge says a - branch-deleted kind for the forge module's `forge.delete`, a `forge.delete` from another account is refused by the - bus, and an existing consumer of `forge` ignores the new kind; with the program stopped a read fails and the +- **Holder, added for the milestone, unit** (table cases of `distribution`'s caller list and store front): a write to + the store's port from any account but `distribution`'s process is refused, and a read is answered; the artifact + sweep's `remove` with the reason retention is accepted and the same call from another caller is refused; a `hold` + of an image or from another caller is refused; `release` from a caller not on the caller list is refused, a + `release` carrying the operator's session as the proved caller is accepted and `distribution`'s own account as + the caller is refused; while the direct-write window is open, a direct write to a tag the seat holds `published` is + refused naming the tag and `publish`. +- **Holder, added for the milestone, integration** (cases of `distribution`'s and the forge bridge's whole-module + tests): the sweep's `remove` removes an untagged bundle and emits `removed`; `hold` keeps an archive from + retention's removal, emits `held` and is listed by `registry_holds`, which releases it with `released` and records + its hand act; a refused sweep `remove` writes the sweep's finding and its next clean run clears it; the forge bridge + says a branch-deleted kind for the forge module's `forge.delete`, a `forge.delete` from another account is refused + by the bus, and an existing consumer of `forge` ignores the new kind; with the program stopped a read fails and the holder's finding is written, a write fails too, and after its restart the read is answered and the finding cleared; a build module's old build pushing straight to the store is accepted until every node runs a build module publishing through the seats, and refused after, naming `publish`, with `direct-writes-closed` emitted and its log line written; `registry_direct_writes` names the node still running an old build module, and drops it by - hand with a hand act; `release` from a caller not on the caller list is refused, and `registry_holds`' release - records its hand act; a waiter on a build's `published` does not wake on `held`, `released`, `old-name-closed` - or `direct-writes-closed`. -- **Builder, added for the milestone.** With the seat's holder down at a first claim, a bundle's trunk build is built - unnumbered and published by its digest, the holder's `published` makes the builder ask the floor again and number - it, and the build module, hearing `numbered`, publishes the version with the held digest; a package's trunk build - waits; a second trunk build of a package's revision held `published` keeps its version and publishes nothing, - while one numbered but refused is published by its second build under the same version; with `bridge_tags` - silenced (a floor source other than the publish target) the floor finding is written and cleared once it answers, - and with `bridge_tags` still silent after the by-digest publish, the build stays unnumbered until the forge - bridge's health clears (no timer runs), and is then numbered and published by version; the node-engine's done key - stays in the old form while one node's node-engine lacks the capability and is written in the new form once every - node states it, `builder_old_form` naming that node meanwhile and dropping it by hand with a hand act; - with the image's own seat silent at the claim of `distribution`'s own image, the image is still built, published by - its digest, and numbered once its new build answers (ADR 0617 *How it is checked*). + hand with a hand act; a node that was away, returning with the new build, closes the old seat name's window on + the next `delivered` (until ADR 0618 is accepted); a waiter on a build's `published` does not wake on `held`, + `released`, `old-name-closed` or `direct-writes-closed`. +- **Builder, added for the milestone, integration** (cases of the builder's whole-module test): with the seat's + holder down at a first claim, a bundle's trunk build is built unnumbered and published by its digest, the holder's + `published` makes the builder ask the floor again and number it, and the build module, hearing `numbered`, + publishes the version with the held digest; a package's trunk build waits; with `bridge_tags` silenced (a floor + source other than the publish target) the floor finding is written and cleared once it answers, and with + `bridge_tags` still silent after the by-digest publish, the build stays unnumbered until the forge bridge's health + clears (no timer runs), and is then numbered and published by version; `builder_floor_again` asks the three + sources at once and records the hand act; the node-engine's done key stays in the old form while one node's + node-engine lacks the capability and is written in the new form once every node states it, `builder_old_form` + naming that node meanwhile and dropping it by hand with a hand act; a node that was away, returning with the + capability, closes that window on the next `delivered` (until ADR 0618 is accepted), with `old-form-closed` emitted + and its build-log line written, and a waiter on a build's end does not wake on it; with the image's own seat silent + at the claim of `distribution`'s own image, the image is still built, published by its digest, and numbered once + its new build answers (ADR 0617 *How it is checked*). +- **Builder, added for the milestone, unit** (cases of the builder's numbering): a second trunk build of a package's + revision held `published` keeps its version and publishes nothing, while one numbered but refused is published by + its second build under the same version. +- **Node-engine, unit** (a case of its apply test, with the runner it already fakes): a module of two containers + whose second new image cannot be pulled replaces neither container, says the image, and keeps both running; with + both images pullable, both are pulled before the first container is set aside. - **Checker.** A changed build module invoking a verb no running holder serves is not run, and the verdict names the modules composed from their manifests instead, and the verb. - **Vault.** Reverting the milestone's merge mints the build modules' credentials again; the bootstrap's one-off @@ -419,10 +471,11 @@ answers (a holder down whole is its module's own health check's); `registry_inco `registry_old_name` names the nodes holding the old name open, an away node among them. After the delivery: mesh-sdk's first numbered version is `published` on the Go seat and an image is read back by its digest, both shown by `builder_versions`; a node runs a module from a reference in the new form; a node-engine away across the - milestone wakes and applies the new node-engine; `distribution`'s own delivery pulls its new images before its - program stops; a package read through the public route is refused, and the same read inside the mesh answers. - Go's `direct`: its removal is named on the milestone's issue, and - the change that removes it adds to the Go build module's recipe test a case that refuses `direct` in the fetch. + milestone wakes and applies the new node-engine; the node-engine's apply journal for `distribution` on each node + shows every new image pulled before its first container was replaced, and no node's pull of any module failed + during the delivery (each node's haves, none failed naming an image); mesh-build-node installs from a lock file + naming the registry's public root with no credential. Go's `direct`: its removal is named on the milestone's issue, + and the change that removes it adds to the Go build module's recipe test a case that refuses `direct` in the fetch. ## Reviewed by architectural domain