From 27063a492be2f76891bf52b673523b0ddf322495 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 3 Nov 2026 17:10:36 +0100 Subject: [PATCH 1/4] Graduate research 081 into designs 51, 26 and 32: registries are seats published into over the bus, the builder numbers every build, seat definitions on the bus, contracts modules (ADR 0605-0609) --- 00-META/glossary.md | 39 +++++- 03-DESIGN/01-to-be/26-the-seats.md | 46 ++++++- .../01-to-be/32-what-a-module-declares.md | 46 +++++++ .../51-the-registries-keep-what-is-named.md | 119 +++++++++++++++++- 4 files changed, 241 insertions(+), 9 deletions(-) diff --git a/00-META/glossary.md b/00-META/glossary.md index 110f3a44..e04f3b60 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -58,6 +58,9 @@ is listed under [Homonyms](#homonyms), with the qualified form each domain uses. promised as optional, then served, then required (ADR 0246). A node seat may also name **the paths its holder owns**, such as `/etc/hosts`: a claim whose module does not declare them is refused, and only a claimant declares them (ADR 0287). +- **seat definition** — the one language-neutral file (a schema with conformance fixtures) that names a seat's + verbs and their messages, its events, its state and the forms of its provisions; its Go and TypeScript types + are generated from it. The indexer keeps every trunk's and open pull request's as bus state (ADR 0608 §1, §2). - **operator** — the person who runs the mesh, and the one the mesh talks to. *Not:* ~~master~~ (hq) - **person** — any human, as against the mesh acting unattended: *a person's word* releases a held @@ -95,8 +98,17 @@ and repair). - **tool** — one capability a module serves through the tool runner, addressed `.`. A seat's **verb** is the same thing carried by a seat rather than a module. - **mesh-sdk** — the library a module's own code is written against, including the SDK harness that serves a - tools bundle to the tool runner. The SDK harness is part of it, not a library of its own. + tools bundle to the tool runner. The SDK harness is part of it, not a library of its own. It keeps the shared + runtime, and its seat contracts move into `mesh-contracts` (ADR 0608 §5). *Not:* ~~tools-sdk~~ +- **contracts module** — a module whose work is to publish the contract packages of seats (their verbs, events, + the layouts of their states, their messages) in every language the seats' users are written in, and never logic: + `mesh-contracts` for the seat contracts mesh-sdk held, `mesh--contracts` for an area's. A seat's holder + implements the contract; the contracts module holds no seat (ADR 0608 §5). + *Not:* ~~SDK module~~, ~~domain package~~ +- **package series** — the first two numbers of a package's versions (`0.1`), declared in its manifest and raised + only by a reviewed change, for a change that breaks its users; a user's manifest names the package series it + accepts (ADR 0606 §4, ADR 0609 §1). Bare "series" only where the package is plain (see [Homonyms](#homonyms)). - **invokes** — the manifest word for the tools a module calls, `.` each or `*` for every one. A grant on the publish side and nothing else; a module that declares none calls nothing. - **run for** — an assignment, or a start of a module, that serves one agent identity with that identity's machine @@ -513,12 +525,29 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel and at the end the outcome and each artifact's digest). A build's duration is read from its two entries, and no other part records durations; the builder's module tool `builder_builds_since` answers the builds a reader missed from it (ADR 0571 §1, §12). -- **package** — what code resolves when it is **compiled**: an npm/cargo/pypi dependency, by **version**. - Served by the **package-registry** (gitea). Only a build module talks to it. +- **package** — what code resolves when it is **compiled**: a Go module, or an npm, cargo or pypi dependency, by + **version**. Published by the registry seat of its language, on the request of its language's build module, from + a trunk build under the **next version** or for a **tagged version** (ADR 0605 §3, ADR 0607). Only a build module + asks a registry seat to store one. +- **registry seat** — a seat held once for the mesh whose holder stores what build modules publish, under the + version the builder gave, and serves its toolchain's read side: `npm-package-registry` and + `go-package-registry` (the **package-registry**, held today by the forge module), `container-image-registry` + and `oci-artifact-registry` (held today by the `distribution` module). One contract: `publish`, `versions`, a + verb removing a tagged version, the events `published` and `removed` (ADR 0605 §1, §2). Not the forge. + *Not:* ~~docker-package-registry~~ +- **next version** — the version the builder gives a trunk build of a published thing: its package series and one + above the highest count it holds for that name, kept with its fingerprint, commit, digest and time in the + builder's versions on the bus (ADR 0606 §1 to §3). Counted, never typed. +- **tagged version** — a published version with a tag (a branch's name, or a word such as `beta`), numbered in + Semantic Versioning's pre-release form below the next version, never a package's latest, taken only by a build + that asks its tag, and removed when its branch is merged or deleted (ADR 0607). +- **mesh delivery case** — a kind of change a developer makes (a package's change, a renamed config value, a new + seat), and how the mesh carries it from the pull request to the nodes; research 081 lists them. - **artifact** — anything a build produces and the mesh delivers to a node by **digest**: an `image`, a mirrored `upstream` image, a `bundle` of the module's own code, an `archive`. Served by the **artifact store**, an OCI registry that holds every kind as content-addressed blobs - (ADR 0156). + (ADR 0156), reached through two registry seats: `container-image-registry` for images and + `oci-artifact-registry` for bundles and archives, which replace the seat `mesh-artifact-store` (ADR 0605 §1). An image is one kind of artifact, and a module is not an image. A package and an artifact are two protocols, not one store being weak (ADR 0075). - **artifact sweep** — letting go of the artifacts no one needs any more, the seat `mesh-artifact-sweep`, held @@ -1125,6 +1154,8 @@ renaming one sense, the old sense moves to a *Not:* line and becomes mechanical. | harness | the part of `mesh-sdk` that serves a tools bundle | **SDK harness** | Module | | | the program that runs an agent's sessions | **agent harness** | Operator and conversation | | | the TypeScript `provisioner` helper for database providers | **`provisioner`**, renamed by design 57 | Provisioning | +| series | the first two numbers of a package's versions | **package series** | Module | +| | a kind of content told in episodes | the kind **series** | Media | | role | what a seat names, at a scope | **seat**; the bare word in older designs is tracked by issue 872 | Module | | | what an identity may call | **role** | Identity and access | | ask | a request for the operator's input | **ask** | Operator and conversation | diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index bb0b2073..4ea5fb1a 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -15,8 +15,10 @@ code: - mesh-catalog modules/gitea/module.json - mesh-controller internal/inventory/migrations/0063-a-machines-names-are-one-seats.sql - mesh-controller internal/inventory/migrations/0064-the-resolver-file-is-the-uplinks.sql -updated: 2026-10-10 +updated: 2026-11-03 decisions: + - "ADR 0608" + - "ADR 0605" - "ADR 0287" - "ADR 0223" - "ADR 0220" @@ -141,9 +143,11 @@ convention, which later seats departed from. | `mesh-store` | — | mesh | — | the store the mesh's own records live in | | `mesh-broker` | — | mesh | `mesh-bus` | the broker carrying the mesh's own bus | | `mesh-vault` | — | mesh | `secret` | the vault (in the seed since ADR 0161; the earlier *reserved* named an effect no rule produced) | -| `mesh-artifact-store` | `the-artifact-store` (renamed 2026-09-30, ADR 0156) | mesh | `artifact-store` | the artifact registry | +| `container-image-registry` | `mesh-artifact-store` for images (ADR 0605; the old name readable until every node-engine reads the new one) | mesh | `artifact-store` | the artifact registry (`distribution`) | +| `oci-artifact-registry` | `mesh-artifact-store` for bundles and archives (ADR 0605) | mesh | `artifact-store` | the artifact registry (`distribution`) | | `mesh-catalog` | `the-catalogue` | mesh | — | the catalog | -| `mesh-npm-package-registry` | `npm-package-registry` | mesh | `npm-package-registry` | the forge | +| `npm-package-registry` | — (its rename to `mesh-*` stays deferred, ADR 0156 §8) | mesh | `npm-package-registry` | the forge | +| `go-package-registry` | — (ADR 0605) | mesh | `go-package-registry` | the forge | | `mesh-git` | `git` | mesh | `git` | the forge | | `mesh-build-machine` | `the-build-machine` | node | — | a builder | | `mesh-resolver` | — | mesh | — | the mesh's resolvers, each holding every node's internal domain (ADR 0194); replicated, held on the anchor and the home server (ADR 0223) | @@ -165,6 +169,42 @@ things here: seats held by assignments rather than claimed by definitions, the ` its reservation, and the foundation's seats delivering nothing. It is brought to this table before it merges. +## A seat's definition (from 2026-11-03) + +**A seat is defined once, in one language-neutral file, and every part learns it from the bus** (ADR 0608, from +research 081). The registry seats (design 51) are the first defined this way, then `mesh-broker`'s verbs. + +- **The file** is a schema with conformance fixtures: the seat's verbs and their messages, its events and their + bodies, its state and the forms of its provisions. It lives in the contracts module of its area. Its Go and + TypeScript types are generated from it and committed beside it; nobody writes them by hand. +- **The generator** is each language's build module's: the Go build module's package for Go, the Node and TypeScript + build module's for TypeScript (the logic belongs to the module whose work it is). A check regenerates and fails, + naming the file, when a committed type differs. +- **On the bus**: the indexer reads every seat definition from every trunk and open pull request and keeps them in its + state `seat-definitions`, keyed by seat and source (the trunk, or a pull request's head), as their only writer, and + emits `seat-definition-changed`. A definition that does not parse fails its pull request's check; the trunk's + stays. The merge is the only approval. The foundation's own definitions are bootstrapped before the bus exists, as + the bootstrap of a fresh mesh has it (research 075). +- **Conditions**, each raised and cleared by the indexer: a seat definition that does not parse; generated types out + of step with their definition. **Tools** of the indexer: `indexer_seat_definition` (read one), + `indexer_seat_definitions_again` (read a trunk's or a pull request's again now) and `indexer_out_of_step` (the + generated types out of step). +- **Contracts modules.** A seat's contract is published by a contracts module, which holds contracts only (every file + generated from a seat definition) and no seat; the seat's holder implements it. The seat contracts mesh-sdk holds + move into `mesh-contracts` in one change, delivered with the first numbered publish (design 51, *The move*); an + area's contracts move into `mesh--contracts` when the area's contract next changes, and that change creates + the module. mesh-sdk keeps its name and the shared runtime. A catalog check refuses a contracts module holding a file + its generator did not write, and a seat-specific type in mesh-sdk once `mesh-contracts` publishes. +- **Older readers keep working**: a moved contract keeps its subjects, keys and message shapes, and a reader ignores + a field it does not know. +- **A vendor-specific** provision or contribution lives in its module's own extension package, published by version + like any package; a later holder of the seat may publish its own the same way. A seat's definition stays + vendor-neutral, which the seams review checks. + +Acceptance tests: research 081's 14, 21, 25 and 26, and an integration test of the indexer: a definition that does not +parse raises its condition and a fixed one clears it; a generated type changed by hand raises the out-of-step condition +and regenerating clears it; each of the three tools answers. + ## The foundation's seats `mesh-controller`, `mesh-store` and `mesh-broker` name which assignment the mesh *itself* uses: the diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index 63e6cd85..e749369f 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -15,6 +15,8 @@ code: - mesh-sdk (the typed bus-state entry in `requires` and its binding, ADR 0595) updated: 2026-11-03 decisions: + - "ADR 0609" + - "ADR 0606" - "ADR 0595" - "ADR 0596" - "ADR 0307" @@ -470,6 +472,50 @@ meaning into shape wherever it can go — a required `channel` field is caught, interpretation of an existing one is not. Saying so is better than implying the fingerprint is complete, because a team that believes it is complete stops reviewing for the case it misses. +### A package, its series, and the series its users accept (from 2026-11-03) + +(ADR 0606 §4, ADR 0609, from research 081.) This is the manifest's half of design 51's versions. + +- **A package is declared as an artifact**: its name, the kind `package`, its language (`go` or `typescript`), its + paths and its **package series** (`0.1`). An image, a bundle or an archive may declare a package series the same + way; one that declares none is numbered in `0.1`. Nobody types a version: the builder counts it (design 51). +- **Raising a package series** is a reviewed change of the package's manifest, for a change that breaks its users. + The checker's verdict names every change of a package series it sees, so the review of the change sees it. +- **A user names, in each `uses` of a mesh package, the package series it accepts.** It builds, and is checked, + against the newest version of that series; a publish in another series does not move it, and moving is a reviewed + change of the user's manifest. A user that names none accepts the series of the package's first numbered version, + so nothing is edited in today's users. +- **Inside one repository** a module uses its sibling package by path, from the same commit, and waits for no publish. +- **A user outside the package's delivery group is not built before the merge**: it is built after it, when the new + version is in its series. A user that breaks fails its own build, is not wanted, keeps its previous build, and a + condition tells the operator. A group's members are built in the group's check, against the group's tagged version. + +### A config value renamed (from 2026-11-03) + +(ADR 0609 §4.) A config value's declaration may name its old name, `renamed-from`. On the merge, mesh-config moves the +value it holds under the old name to the new name in its own state, without the operator giving it again, and records +the move. This design decides what ADR 0609 leaves open: + +- **Which values**: a module's own config values only, never a secret, and never another module's value. +- **How long the old name is read**: until every node's want of the module names a build that reads the new name, as + the delivery view's wants show; until then mesh-config answers both names. A later change may drop `renamed-from` + only after that, and the checker refuses it before. +- **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and `config_rename_again` (move one + value by hand when a move was refused). + +### The mesh delivery cases this design carries + +Research 081 listed the mesh delivery cases; these are the ones it handed to a design, each with its acceptance test. + +| case | what holds | acceptance test | +|---|---|---| +| C6, a package series raised | users on `0.1` keep the newest `0.1`; a reviewed change moves a user | integration, builder: a `0.2.0` publish moves no `0.1` user, its check takes the newest `0.1` too, a reviewed change naming `0.2` moves it, a user naming no series builds against the newest `0.1` | +| C14, a provision's form gains a field | the provider serves it in a new version; the consumer reads it once rebuilt; an older consumer ignores it | integration, mesh-sdk: a consumer built against the older form reads a provision with the new field | +| C18, a new required config value with no default | the check names the nodes that lack it; the want waits; the operator is asked once | live: a module gaining such a value waits on each node lacking it, the operator is asked once naming the nodes, and once given the module is applied | +| C20, a config value renamed | the value moves; both names read until every node runs the new build | integration, mesh-config: a renamed value is present under the new name with no operator input, the old name answers until the wants show the new build everywhere, and dropping `renamed-from` early is refused | +| C22, a secret only a person can give | the want waits; the operator is asked once what to give and where | live: a module requiring such a secret waits, one ask names it and where, and once given the module starts | +| C24, a config value or secret no longer read | it is retiring, removed only on the operator's Approve | integration, mesh-config: a value no manifest reads any more is marked retiring and stays until an Approve removes it | + ## 9. Provisioning over the bus Provisioning rides the bus, and the provider stops having an address. diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 06dad5ed..2a3bdd57 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -6,15 +6,18 @@ code: - mesh-catalog modules/artifact-store-tools (the store's tools), modules/distribution (its unbuilt code removed) - mesh-catalog modules/gitea (package retention) - mesh-catalog modules/docker (image pruning) -updated: 2026-10-12 +updated: 2026-11-03 decisions: + - "ADR 0605" + - "ADR 0606" + - "ADR 0607" - "ADR 0311" - "ADR 0251" - "ADR 0189" - "ADR 0257" --- -# 51 — The registries keep what is named +# 51 — The registries: what is published, under which version, and what is kept **Three places hold built things: the artifact store, the package registry and each machine's container runtime. Each gets tools that say what it holds, what names each thing, and what could go. @@ -24,6 +27,118 @@ registry, and the machine's declaration for its images.** (ADR 0251, extending ADR 0189.) +## What is published, and how (from 2026-11-03) + +**Every registry is a seat; a build module publishes into it over the bus; the builder gives every version.** +(ADR 0605, ADR 0606, ADR 0607, from research 081.) What follows is the to-be shape; the sections from *The parts* +on keep what the registries hold, and stand. + +### The four registry seats + +| registry seat | holds | read side it serves | holder today | +|---|---|---|---| +| `npm-package-registry` | npm packages | the npm registry protocol | the forge module | +| `go-package-registry` | Go modules | Go's module proxy protocol | the forge module | +| `container-image-registry` | container images | OCI distribution, with an image's own verbs (its tags, mirroring an outside image) | `distribution` | +| `oci-artifact-registry` | bundles and archives, by digest | OCI distribution | `distribution` | + +Each is held once for the mesh. The last two replace the seat `mesh-artifact-store` and front the one artifact store; +the provision `artifact-store` keeps its name. `mesh-artifact-store` stays readable until every node's want of the +node-engine names a build that reads the new names, which the delivery view's wants show. `cargo-package-registry` +stays named, without a holder, until a build module publishes cargo packages. + +**One contract, a seat definition** (design 26, *A seat's definition*): the verbs `publish`, `versions` and +`remove` (a tagged version only), and the events `published` and `removed`, each carrying the name, version, kind, +tag, digest, fingerprint, commit and holder. A holder stores under the version it is given and counts nothing. + +### Publishing + +``` + builder ── build request (name, version or tag) ──▶ build module + │ hands the file: object store `publish-handoff` + │ calls .publish + ▼ + registry seat's holder ── reads the builder's `versions` (announced state) + │ stores, deletes the handed object + └── emits `published` ──▶ builder (entry → published), indexer, status view +``` + +- **Only build modules may call `publish`**, and only from a trunk build or for a tagged version; the bus grants + nothing else. No build module holds a registry's write credential: the npm write token and the image push + credential go, and each holder keeps its write credential as an own secret beside a read-only tool token (the forge + module: `write:package`, its tool token `read:package`). +- **The handed object** lives in the object store `publish-handoff`, which the builder owns: build modules write, + holders read and delete. An image is handed as an OCI layout holding only the layers the holder lacks: `publish` + names every layer's digest, the holder answers the digests it is missing, and only those are handed. The store is + capped at 20 GiB, an object lives at most 24 hours, and a publish not stored within 10 minutes is refused by its + build module, naming the seat, which leaves the entry `refused` (the condition below). +- **A holder refuses loudly**: a version or digest the builder's `versions` do not record, and a held version or tag + offered with other bytes. A rebuild of a held fingerprint answers the held version and stores nothing. +- **The done key** names the reference the seat answered: `//@sha256:` for an + image, bundle or archive, `@` for a package. +- **Go packages** are fetched through the Go seat's provision first and Go's public proxy second, with no + `GOPRIVATE` and the checksum database off for the forge's paths; `go.sum` and the recorded digest refuse a mesh + package with other content. Outside Go modules keep coming from the public proxy; no mirror is built, and one could + come later as a holder of a seat of its own without any build module changing. + +### The versions + +- **The builder numbers every trunk build** of a package, image, bundle or archive at its claim: the package series + its manifest declares (design 32 § 6; `0.1` when it declares none) and the next count for that name, under + compare-and-set. Numbers follow merge order. +- **The builder's state `versions`** (announced, the builder its only writer, readable by the indexer, the registry + seats' holders and the status view) holds per name, and per tag, every version with its fingerprint, commit, + digest, time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the branch + and the forge bridge's event that removed it. The build log carries each change too. +- **The floor**: on a name's first claim, or when `versions` holds nothing for it, the count starts above the highest + of what the seat's `versions` answers, the package's highest git tag (the forge bridge's state) and the highest + pseudo-version a consumer requires (the indexer's state); a name with none starts at `0.1.0`. Today that gives Go + `v0.1.14` and npm `0.1.18`. +- **A trunk build that declares a package is done** only once its done key names a published version for each. +- **Events of the builder**: `numbered`. **Tools of the builder**: `builder_versions` (a name's versions in order, + one call), `builder_publish_again` (one fingerprint under its number, after a holder failed; it refuses a version + refused for other bytes), `builder_tag` (ask a tagged version) and `builder_untag` (remove a branch's tagged versions). + +### Tagged versions + +- **A tagged version is asked by a key, never by a push.** The checker, for each head of a delivery group member that + declares a package, and a person, through `builder_tag`, write an ask in the builder's state `tag-asks`, keyed by + repository, branch and head; the builder watches it. The tag is cleaned to Semantic Versioning's characters. +- **It is numbered below the next version** (`0.1.15-feat-x.2`), never the latest; npm keeps it under a dist-tag of its + own and Go's `@latest` skips it. A package new in a group is numbered from its floor (`0.1.0-feat-x.1`). +- **A group's check takes it from the registry**: Go members by the tagged version, TypeScript members with their lock + naming it and the registry's integrity. Nothing is packed for a check, and a lone pull request's check publishes + nothing. +- **It is removed** when its branch is merged or deleted: the builder hears the forge bridge's event on a durable + consumer, calls each seat's `remove`, the seat emits `removed`, and the entry turns `removed`. A version without a + tag is never removed. + +### Health and conditions + +Every holder declares a health check. Conditions, each raised and cleared: a publish refused (cleared when it is +stored), a registry seat not answering (cleared when it answers). A build narrates its publish: asked, stored, refused. + +### The move, in four steps + +Each step is its own delivery. 1: the holders serve the verbs and `versions` exists. 2: mesh-sdk's first publish +through the seats, read back, delivered together with the move of mesh-sdk's seat contracts into `mesh-contracts` +(design 26), so every module is rebuilt once for both. 3: the Go fetch switches to the seat. 4: the old credentials +and `package-version.sum` go. Steps 1 to 3 are rolled back by reverting their pull request; step 4 by reverting it, +which mints the build modules' credentials again. On a fresh mesh the builder publishes into a holder the +foundation's bootstrap raised. + +### Acceptance tests + +Research 081's tests 1-3, 5-7, 9, 10, 12, 13, 15-20, 22 and 23 (its Direction) are this design's, with their labels +(live or integration, and the module). Added here: + +- **Integration, builder.** A branch pushed with no ask publishes nothing; a lone pull request's check publishes + nothing; a group introducing a new package takes `0.1.0-.1`; `builder_untag` removes a branch's tagged + versions with `removed`, and the entry shows the branch and the event. +- **Integration, builder.** An image whose layers the holder holds hands only the missing ones; a publish not stored + within 10 minutes leaves the entry `refused` and raises the condition. +- **Live.** A revert of step 4 on one node mints the build modules' credentials again before it reaches the others. + ## The parts ``` From 20157fcafe38a63803ebef32c81cf8e9022cb676 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 3 Nov 2026 18:02:09 +0100 Subject: [PATCH 2/4] Designs 51, 26, 32: answer the first review by architectural domain (ADR 0596 paths, holder-side caller lists, the file in the publish verb, findings for the condition keeper, windows on node-engine reports) Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- 03-DESIGN/01-to-be/26-the-seats.md | 36 ++-- .../01-to-be/32-what-a-module-declares.md | 49 +++-- .../51-the-registries-keep-what-is-named.md | 196 +++++++++++------- 3 files changed, 180 insertions(+), 101 deletions(-) diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 4ea5fb1a..688ba837 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -178,21 +178,27 @@ research 081). The registry seats (design 51) are the first defined this way, th bodies, its state and the forms of its provisions. It lives in the contracts module of its area. Its Go and TypeScript types are generated from it and committed beside it; nobody writes them by hand. - **The generator** is each language's build module's: the Go build module's package for Go, the Node and TypeScript - build module's for TypeScript (the logic belongs to the module whose work it is). A check regenerates and fails, - naming the file, when a committed type differs. -- **On the bus**: the indexer reads every seat definition from every trunk and open pull request and keeps them in its - state `seat-definitions`, keyed by seat and source (the trunk, or a pull request's head), as their only writer, and - emits `seat-definition-changed`. A definition that does not parse fails its pull request's check; the trunk's - stays. The merge is the only approval. The foundation's own definitions are bootstrapped before the bus exists, as - the bootstrap of a fresh mesh has it (research 075). -- **Conditions**, each raised and cleared by the indexer: a seat definition that does not parse; generated types out - of step with their definition. **Tools** of the indexer: `indexer_seat_definition` (read one), + build module's for TypeScript (the logic is the module's whose work it is). The check of a change regenerates and + fails, naming the file, when a committed type differs; its verdict event names the out-of-step files, and that is + how the indexer learns of them. The indexer runs no generator. +- **On the bus**: the indexer reads every seat definition from every trunk and open pull request, from the forge + bridge's messages, and keeps them in its own state `seat-definitions`, keyed by seat and source (the trunk, or a + pull request's head), as their only writer; a definition is at most 256 KiB. It emits `seat-definition-changed` + (seat, source, digest), kept seven days. **Only the trunk's entry is the seat's contract**: the readers (the contracts + modules' builds, the tool runner's descriptions, the review) ask `indexer_seat_definition`, which answers the trunk's + unless a pull request's head is named, and says which it answered. A pull request's entries are removed on the forge + bridge's event that it closed or merged. A definition that does not parse fails its pull request's check; the + trunk's stays. The merge is the only approval. The foundation's own definitions are bootstrapped before the bus + exists, as the bootstrap of a fresh mesh has it (research 075). +- **Findings**, written by the indexer, from which the condition keeper raises and clears the conditions (ADR 0564): a + trunk's seat definition that does not parse (a pull request's is its check's verdict, not a condition); generated + types on a trunk out of step with their definition. **Tools** of the indexer: `indexer_seat_definition` (read one), `indexer_seat_definitions_again` (read a trunk's or a pull request's again now) and `indexer_out_of_step` (the generated types out of step). - **Contracts modules.** A seat's contract is published by a contracts module, which holds contracts only (every file generated from a seat definition) and no seat; the seat's holder implements it. The seat contracts mesh-sdk holds - move into `mesh-contracts` in one change, delivered with the first numbered publish (design 51, *The move*); an - area's contracts move into `mesh--contracts` when the area's contract next changes, and that change creates + move into `mesh-contracts` in one change, delivered with the first numbered publish (design 51, *The move*, step 2); + an area's contracts move into `mesh--contracts` when the area's contract next changes, and that change creates the module. mesh-sdk keeps its name and the shared runtime. A catalog check refuses a contracts module holding a file its generator did not write, and a seat-specific type in mesh-sdk once `mesh-contracts` publishes. - **Older readers keep working**: a moved contract keeps its subjects, keys and message shapes, and a reader ignores @@ -201,9 +207,11 @@ research 081). The registry seats (design 51) are the first defined this way, th like any package; a later holder of the seat may publish its own the same way. A seat's definition stays vendor-neutral, which the seams review checks. -Acceptance tests: research 081's 14, 21, 25 and 26, and an integration test of the indexer: a definition that does not -parse raises its condition and a fixed one clears it; a generated type changed by hand raises the out-of-step condition -and regenerating clears it; each of the three tools answers. +Acceptance tests: research 081's 14, 21, 25 and 26, and cases of the indexer's whole-module integration test: a trunk +definition that does not parse writes its finding and a fixed one clears it; a check verdict naming out-of-step types +writes the finding and a clean one clears it; `indexer_seat_definition` answers the trunk's entry while a pull request +holds another; a pull request's entries go when it closes; a definition over 256 KiB is refused; each of the three +tools answers. ## The foundation's seats diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index e749369f..37054c73 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -481,40 +481,55 @@ complete, because a team that believes it is complete stops reviewing for the ca way; one that declares none is numbered in `0.1`. Nobody types a version: the builder counts it (design 51). - **Raising a package series** is a reviewed change of the package's manifest, for a change that breaks its users. The checker's verdict names every change of a package series it sees, so the review of the change sees it. -- **A user names, in each `uses` of a mesh package, the package series it accepts.** It builds, and is checked, - against the newest version of that series; a publish in another series does not move it, and moving is a reviewed - change of the user's manifest. A user that names none accepts the series of the package's first numbered version, - so nothing is edited in today's users. +- **A user names the mesh packages it takes in a section of its own, `packages`**, each with the package series it + accepts. It is not `uses`, which grants publishing on a seat's `accepts` subjects (§ 3); a package is not a seat. A + user builds, and is checked, against the newest version of that series; a publish in another series does not move + it, and moving is a reviewed change of the user's manifest. A user that names none accepts the series of the + package's first numbered version, so nothing is edited in today's users. The indexer's tool `indexer_package_users` + lists, per package, its users and the series each accepts, so a user left on an older series is seen. - **Inside one repository** a module uses its sibling package by path, from the same commit, and waits for no publish. - **A user outside the package's delivery group is not built before the merge**: it is built after it, when the new version is in its series. A user that breaks fails its own build, is not wanted, keeps its previous build, and a condition tells the operator. A group's members are built in the group's check, against the group's tagged version. +- **Cost**: the new manifest fields are read through mesh-sdk's manifest types, so their change rebuilds every module + once; it is delivered with design 51's step 2, the one rebuild that step already takes. ### A config value renamed (from 2026-11-03) (ADR 0609 §4.) A config value's declaration may name its old name, `renamed-from`. On the merge, mesh-config moves the -value it holds under the old name to the new name in its own state, without the operator giving it again, and records -the move. This design decides what ADR 0609 leaves open: +value it holds under the old name to the new name in its own state, without the operator giving it again, emits +`renamed` (module, old name, new name, when) and records it. This design decides what ADR 0609 leaves open: -- **Which values**: a module's own config values only, never a secret, and never another module's value. -- **How long the old name is read**: until every node's want of the module names a build that reads the new name, as - the delivery view's wants show; until then mesh-config answers both names. A later change may drop `renamed-from` - only after that, and the checker refuses it before. -- **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and `config_rename_again` (move one - value by hand when a move was refused). +- **Which values**: a module's own config values only, never a secret, never another module's value. A rename keeps + the value's trust: a value set without the operator's Approve stays untrusted under its new name (ADR 0348 §8), and + the checker refuses a rename that would change a value's trust. +- **How long the old name is read**: until every node-engine reports applied a build of the module that reads the new + name, as the node-engine seat's reports answer; until then mesh-config answers both names. A later change may drop + `renamed-from` only after that, and the checker refuses it before. The checker also refuses `renamed-from` in any + manifest until a build of mesh-config that understands it is delivered. +- **A refused move** (the old name holds nothing, or a secret) is mesh-config's finding, from which the condition keeper + raises the condition. **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and + `config_rename_again` (move one value by hand when a move was refused). + +### A value no longer read (from 2026-11-03) + +A config value or secret no manifest reads any more is marked retiring by mesh-config, which emits `retiring` and asks +the operator once, through the operator channel, whether to remove it; it is removed only on the operator's Approve. +`config_retiring` lists what is retiring and since when. ### The mesh delivery cases this design carries -Research 081 listed the mesh delivery cases; these are the ones it handed to a design, each with its acceptance test. +Research 081 listed the mesh delivery cases; these are the ones it handed to a design, each with its acceptance test, +each a case of its module's one whole-module integration test unless it says live. | case | what holds | acceptance test | |---|---|---| -| C6, a package series raised | users on `0.1` keep the newest `0.1`; a reviewed change moves a user | integration, builder: a `0.2.0` publish moves no `0.1` user, its check takes the newest `0.1` too, a reviewed change naming `0.2` moves it, a user naming no series builds against the newest `0.1` | -| C14, a provision's form gains a field | the provider serves it in a new version; the consumer reads it once rebuilt; an older consumer ignores it | integration, mesh-sdk: a consumer built against the older form reads a provision with the new field | +| C6, a package series raised | users on `0.1` keep the newest `0.1`; a reviewed change moves a user | builder: a `0.2.0` publish moves no `0.1` user, its check takes the newest `0.1` too, a reviewed change naming `0.2` moves it, a user naming no series builds against the newest `0.1` | +| C14, a provision's form gains a field | the provider serves it in a new version; the consumer reads it once rebuilt; an older consumer ignores it | mesh-sdk: a consumer built against the older form reads a provision with the new field | | C18, a new required config value with no default | the check names the nodes that lack it; the want waits; the operator is asked once | live: a module gaining such a value waits on each node lacking it, the operator is asked once naming the nodes, and once given the module is applied | -| C20, a config value renamed | the value moves; both names read until every node runs the new build | integration, mesh-config: a renamed value is present under the new name with no operator input, the old name answers until the wants show the new build everywhere, and dropping `renamed-from` early is refused | +| C20, a config value renamed | the value moves, keeps its trust, both names read until every node runs the new build | mesh-config: a renamed value is present under the new name with no operator input and its trust; the old name answers until the node-engines report the new build; dropping `renamed-from` early, or changing trust, is refused; a refused move writes its finding | | C22, a secret only a person can give | the want waits; the operator is asked once what to give and where | live: a module requiring such a secret waits, one ask names it and where, and once given the module starts | -| C24, a config value or secret no longer read | it is retiring, removed only on the operator's Approve | integration, mesh-config: a value no manifest reads any more is marked retiring and stays until an Approve removes it | +| C24, a config value or secret no longer read | it is retiring, the operator is asked, removed only on the Approve | mesh-config: a value no manifest reads is marked retiring, `retiring` is emitted, one ask reaches the operator, and it stays until an Approve removes it | ## 9. Provisioning over the bus diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 2a3bdd57..d3a61983 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -11,6 +11,7 @@ decisions: - "ADR 0605" - "ADR 0606" - "ADR 0607" + - "ADR 0610" - "ADR 0311" - "ADR 0251" - "ADR 0189" @@ -30,8 +31,9 @@ extending ADR 0189.) ## What is published, and how (from 2026-11-03) **Every registry is a seat; a build module publishes into it over the bus; the builder gives every version.** -(ADR 0605, ADR 0606, ADR 0607, from research 081.) What follows is the to-be shape; the sections from *The parts* -on keep what the registries hold, and stand. +(ADR 0605, ADR 0606, ADR 0607, ADR 0610, from research 081.) What follows is the to-be shape; the sections from +*The parts* on keep what the registries hold, and stand. Every interaction below is a seat verb, a module's tool or an +event: no module reads or writes another's bus state (ADR 0596). ### The four registry seats @@ -43,101 +45,155 @@ on keep what the registries hold, and stand. | `oci-artifact-registry` | bundles and archives, by digest | OCI distribution | `distribution` | Each is held once for the mesh. The last two replace the seat `mesh-artifact-store` and front the one artifact store; -the provision `artifact-store` keeps its name. `mesh-artifact-store` stays readable until every node's want of the -node-engine names a build that reads the new names, which the delivery view's wants show. `cargo-package-registry` -stays named, without a holder, until a build module publishes cargo packages. +while one module holds both, both deliver the provision `artifact-store`, and splitting them between two holders takes a +decision of its own. `mesh-artifact-store` stays served until every node-engine reports applied a build that reads the +new names (ADR 0610 §4); a sleeping node holds it open. `cargo-package-registry` stays named, without a holder, until a +build module publishes cargo packages. -**One contract, a seat definition** (design 26, *A seat's definition*): the verbs `publish`, `versions` and -`remove` (a tagged version only), and the events `published` and `removed`, each carrying the name, version, kind, -tag, digest, fingerprint, commit and holder. A holder stores under the version it is given and counts nothing. +**One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `versions`, `remove` (a +tagged version only; the holder itself refuses `remove` of a version with no tag), and on `container-image-registry` +`mirror` and `tags`. Events: `published`, `refused` (added to ADR 0605 §2's two, additively) and `removed`, each +carrying the name, version, kind, tag, digest, fingerprint, commit and holder, and `refused` its reason. A holder stores +under the version it is given and counts nothing. + +**Who may call which verb is the holder's to say**, in a trusted config value of the holder listing its callers, the +form ADR 0363 gives a restricted family: `publish` the build modules; `mirror` mesh-build-image; `remove` the builder. +Declaring `invokes` alone grants nothing. The holder answers any other caller with a refusal naming the verb. ### Publishing ``` builder ── build request (name, version or tag) ──▶ build module - │ hands the file: object store `publish-handoff` - │ calls .publish + │ publish(name, version, digests of every part) ▼ - registry seat's holder ── reads the builder's `versions` (announced state) - │ stores, deletes the handed object - └── emits `published` ──▶ builder (entry → published), indexer, status view + registry seat's holder ── answers the parts it lacks + │ the build module sends those parts, in chunks, by `publish` + holder: hashes what it received, asks `builder_versions`, + stores in its own store, emits `published` or `refused` + ▼ + builder (durable consumer: entry, done key), indexer, status view, checker ``` -- **Only build modules may call `publish`**, and only from a trunk build or for a tagged version; the bus grants - nothing else. No build module holds a registry's write credential: the npm write token and the image push - credential go, and each holder keeps its write credential as an own secret beside a read-only tool token (the forge - module: `write:package`, its tool token `read:package`). -- **The handed object** lives in the object store `publish-handoff`, which the builder owns: build modules write, - holders read and delete. An image is handed as an OCI layout holding only the layers the holder lacks: `publish` - names every layer's digest, the holder answers the digests it is missing, and only those are handed. The store is - capped at 20 GiB, an object lives at most 24 hours, and a publish not stored within 10 minutes is refused by its - build module, naming the seat, which leaves the entry `refused` (the condition below). -- **A holder refuses loudly**: a version or digest the builder's `versions` do not record, and a held version or tag - offered with other bytes. A rebuild of a held fingerprint answers the held version and stores nothing. -- **The done key** names the reference the seat answered: `//@sha256:` for an - image, bundle or archive, `@` for a package. -- **Go packages** are fetched through the Go seat's provision first and Go's public proxy second, with no - `GOPRIVATE` and the checksum database off for the forge's paths; `go.sum` and the recorded digest refuse a mesh - package with other content. Outside Go modules keep coming from the public proxy; no mirror is built, and one could - come later as a holder of a seat of its own without any build module changing. +- **The file travels in the verb.** `publish` opens with every part's digest (an image's layers, a package's file, + a bundle's blob); the holder answers the digests it lacks, and the build module sends only those, in chunks no + larger than the bus's message limit, into the holder's own store. This is how ADR 0605 §3's hand-off holds under ADR + 0596: no store is reached by two modules. The holder hashes every part it received and refuses one whose bytes do + not match its digest; a part it never received completed is dropped after 24 hours. The holder's tool + `registry_incoming` lists what is in transit, its fill against a 20 GiB cap, and drops one by hand; past 80 % of the + cap the holder writes a finding. +- **The holder asks the builder**, through `builder_versions`, whether the name, version and digest are recorded, and + refuses loudly otherwise (ADR 0605 §4). A rebuild of a held fingerprint answers the held version and stores nothing. +- **How a publish ends**: the holder emits `published` or `refused`. The build module waits for one of them, at most + 10 minutes, narrating "waiting on " at least every five minutes; past the bound it writes the key refused, + naming the seat. A `published` that arrives after that is still heard by the builder on its durable consumer: the + entry turns `published` and the builder writes the done key then, so a late store is never lost. +- **What is published when**: a package or an image from a trunk build under its version, or as a tagged version; a + bundle or an archive from every done key by its digest alone, a candidate's included, and at the trunk's claim the + build module calls `publish` with the version and the held digest, which stores nothing. An upstream image is copied + by `mirror` with its pinned digest. The done key names the reference the seat answered; a key in the old + `artifact-store://` form keeps it, every reader reads both forms, and none is built again for its form. +- **No build module holds a registry's write credential**: the npm write token, the image push credential and the + artifact store's write binding go in step 4. Each holder keeps its write side as an own secret beside a read-only + tool token (the forge module: `write:package`, its tool token `read:package`). +- **Go packages** are fetched through the Go seat's provision first and Go's public proxy second, with no `GOPRIVATE` + and the checksum database off for the forge's paths; `go.sum` and the digest the builder records (Go's `h1:` hash, + computed by the Go build module at the publish) refuse a mesh package with other content. The builder also tags + each Go version in git (`go/v0.1.`), so a user that names a version can fetch it from the forge's git as well. + No mirror of outside modules is built; one could come later as a holder of a seat of its own. ### The versions - **The builder numbers every trunk build** of a package, image, bundle or archive at its claim: the package series - its manifest declares (design 32 § 6; `0.1` when it declares none) and the next count for that name, under - compare-and-set. Numbers follow merge order. -- **The builder's state `versions`** (announced, the builder its only writer, readable by the indexer, the registry - seats' holders and the status view) holds per name, and per tag, every version with its fingerprint, commit, - digest, time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the branch - and the forge bridge's event that removed it. The build log carries each change too. + its manifest declares (design 32; `0.1` when it declares none) and the next count for that name, under + compare-and-set. Numbers follow the order of the forge bridge's merge events: a claim of a later merge waits until + every earlier merge of the same name is numbered. A fingerprint already numbered keeps its number. +- **The builder's state `versions`** holds one key per name, tag and version, with its fingerprint, commit, digest, + time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the reason (a + branch's end, retention, the sweep) and the event that removed it. History one, cap 1 GiB; past 80 % the builder + writes a finding. Only the builder reaches it; others call `builder_versions` or hear its events. The build log + carries each change too. - **The floor**: on a name's first claim, or when `versions` holds nothing for it, the count starts above the highest - of what the seat's `versions` answers, the package's highest git tag (the forge bridge's state) and the highest - pseudo-version a consumer requires (the indexer's state); a name with none starts at `0.1.0`. Today that gives Go - `v0.1.14` and npm `0.1.18`. -- **A trunk build that declares a package is done** only once its done key names a published version for each. -- **Events of the builder**: `numbered`. **Tools of the builder**: `builder_versions` (a name's versions in order, - one call), `builder_publish_again` (one fingerprint under its number, after a holder failed; it refuses a version - refused for other bytes), `builder_tag` (ask a tagged version) and `builder_untag` (remove a branch's tagged versions). + of the seat's `versions`, the package's highest git tag (the forge bridge's tool `bridge_tags`) and the highest + pseudo-version a consumer requires (the indexer's tool `indexer_requirements`). The builder numbers nothing for that + name until all three answered, and writes a finding naming the silent one meanwhile. Today that gives Go `v0.1.14` + and npm `0.1.18`; a name with none starts at `0.1.0`. +- **Events of the builder**: `numbered` (name, version, tag, fingerprint, commit, head when it answers an ask), on its + own family, kept seven days; a waiter on a build's end does not wake on it. **Tools of the builder**: + `builder_versions` (a name's versions in order, one call), `builder_publish_again` (one fingerprint under its number, + after a holder failed; when the build's file is gone it builds the fingerprint again under the same number; it + refuses a version refused for other bytes), `builder_tag` (ask a tagged version), `builder_tag_asks` (open asks and + their state) and `builder_untag` (remove a branch's tagged versions). ### Tagged versions -- **A tagged version is asked by a key, never by a push.** The checker, for each head of a delivery group member that - declares a package, and a person, through `builder_tag`, write an ask in the builder's state `tag-asks`, keyed by - repository, branch and head; the builder watches it. The tag is cleaned to Semantic Versioning's characters. -- **It is numbered below the next version** (`0.1.15-feat-x.2`), never the latest; npm keeps it under a dist-tag of its - own and Go's `@latest` skips it. A package new in a group is numbered from its floor (`0.1.0-feat-x.1`). -- **A group's check takes it from the registry**: Go members by the tagged version, TypeScript members with their lock - naming it and the registry's integrity. Nothing is packed for a check, and a lone pull request's check publishes - nothing. -- **It is removed** when its branch is merged or deleted: the builder hears the forge bridge's event on a durable - consumer, calls each seat's `remove`, the seat emits `removed`, and the entry turns `removed`. A version without a - tag is never removed. +- **Asked by an event, never by a push** (ADR 0610 §1). The checker emits `tag-wanted` (repository, branch, head, + tag) for each head of a delivery group member that declares a package; a person asks through `builder_tag`. The + builder consumes the event on a durable consumer and keeps the ask in its own state `tag-asks`, removed when its + branch is merged or deleted. An ask the builder has not answered within five minutes is narrated in the build log + with what it waits on. +- **Numbered below the next version** (`0.1.15-feat-x.2`), never the latest; npm keeps it under a dist-tag of its own + and Go's `@latest` skips it. A package new in a group is numbered from its floor (`0.1.0-feat-x.1`). +- **A group's check builds after `published`**: Go members by the tagged version, TypeScript members with their lock + naming it and the registry's integrity; nothing is packed for a check, and a lone pull request's check publishes + nothing. No `published` within the publish's bound (10 minutes) fails the check naming the package. +- **Removed with its branch**: on the forge bridge's merged or deleted event, heard on a durable consumer, the builder + calls each seat's `remove`. It acknowledges the event only once every seat emitted `removed`; until then it retries + every minute and, past 15 minutes, writes a finding naming the seat. A version with no tag is not removed by a + branch's end; retention may remove an old unreferenced one (below). + +### Retention + +The retention of *The package registry* below, and the artifact sweep, keep working, now through the seats: each +removal emits `removed` and the builder marks its entry, so `versions` never says `published` for bytes that are gone. +The Go seat keeps what the npm seat keeps: every referenced version and the newest five of each package series. ### Health and conditions -Every holder declares a health check. Conditions, each raised and cleared: a publish refused (cleared when it is -stored), a registry seat not answering (cleared when it answers). A build narrates its publish: asked, stored, refused. +Observers write findings; the condition keeper raises and clears the conditions from them (ADR 0564). Findings: a +publish refused (the holder; cleared when it is stored), a registry seat not answering (each holder's health check, +which calls `versions` on each seat it holds and probes the read side every minute, so a dead seat is seen with no +publish running), a removal not done (the builder), a floor source silent (the builder), a store past 80 % (the holder, +the builder). A build narrates its publish: asked, waiting on a seat, stored, refused. ### The move, in four steps -Each step is its own delivery. 1: the holders serve the verbs and `versions` exists. 2: mesh-sdk's first publish -through the seats, read back, delivered together with the move of mesh-sdk's seat contracts into `mesh-contracts` -(design 26), so every module is rebuilt once for both. 3: the Go fetch switches to the seat. 4: the old credentials -and `package-version.sum` go. Steps 1 to 3 are rolled back by reverting their pull request; step 4 by reverting it, -which mints the build modules' credentials again. On a fresh mesh the builder publishes into a holder the -foundation's bootstrap raised. +Each step is its own delivery, and no credential goes before its replacement works (ADR 0605 §7). + +1. The holders serve every verb of the four seats with their caller lists; the builder's `versions`, `tag-asks`, its + events and tools; the forge bridge's `bridge_tags` and the indexer's `indexer_requirements`. +2. Every build module publishes through the seats: mesh-sdk's first numbered publish, images, bundles, archives and + upstream copies, each read back; delivered with the move of mesh-sdk's seat contracts into `mesh-contracts` (design + 26), so every module is rebuilt once for both. The checker emits `tag-wanted`. Until step 3, Go users fetch from the + forge's git, where the builder's tags let them take the numbered version. +3. The Go fetch switches to the Go seat. Its rollback stays safe after a user moved, since the git tags exist. +4. The three credentials go, and `package-version.sum`. The old seat name retires after it, on the node-engines' + reports. + +Steps 1 to 3 are rolled back by reverting their pull request; step 4 by reverting it, which mints the build modules' +credentials again. **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images with a +one-off credential of the bootstrap, discarded once the holders serve `publish`. ### Acceptance tests -Research 081's tests 1-3, 5-7, 9, 10, 12, 13, 15-20, 22 and 23 (its Direction) are this design's, with their labels -(live or integration, and the module). Added here: +Research 081's tests 1-3, 5-7, 9, 10, 12, 13, 15-20, 22 and 23 (its Direction) are this design's, with their labels. +Each integration test below is a case of its module's one whole-module integration test, which starts the module +against the real bus (test budget: mesh-delivery holds 76 integration tests today, over its 50; these add cases, not +tests, and each change that touches the builder's or the indexer's tests merges two old ones into its whole-module +test). -- **Integration, builder.** A branch pushed with no ask publishes nothing; a lone pull request's check publishes - nothing; a group introducing a new package takes `0.1.0-.1`; `builder_untag` removes a branch's tagged - versions with `removed`, and the entry shows the branch and the event. -- **Integration, builder.** An image whose layers the holder holds hands only the missing ones; a publish not stored - within 10 minutes leaves the entry `refused` and raises the condition. -- **Live.** A revert of step 4 on one node mints the build modules' credentials again before it reaches the others. +- **Builder.** A branch pushed with no ask publishes nothing; a lone pull request's check publishes nothing; a group + introducing a new package takes `0.1.0-.1`; the checker's `tag-wanted` yields `numbered` and `published`, and + members are built only after `published`; no `published` in 10 minutes fails the check naming the package; + `builder_untag` removes a branch's tagged versions with `removed`; a `remove` while the holder is down is retried and + past 15 minutes writes its finding; a late `published` after the bound writes the done key; `builder_publish_again` + after the build's file is gone builds again under the same number; with the seat's holder stopped at a first claim + nothing is numbered and the finding names it; two merges claimed in reverse order are numbered in merge order; a + retention removal turns the entry `removed`. +- **Holder (gitea, distribution).** A caller not on the caller list is refused for each verb; a part whose bytes do + not match its digest is refused; only the missing layers are sent; `remove` of a version with no tag is refused; + with no publish running, a stopped Go seat writes its finding and clears it when it answers; `registry_incoming` + lists and drops. +- **Vault.** Reverting step 4 mints the build modules' credentials again. ## The parts From 9845b78e5b55cc862f01f0c0c3f92290a5edf77e Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 3 Nov 2026 18:08:00 +0100 Subject: [PATCH 3/4] Designs 51, 26, 32: answer the second review by architectural domain (ADR 0611, caller identity, node-engine reports, families and retention, Go references); add each design's review record Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- 03-DESIGN/01-to-be/26-the-seats.md | 8 ++ .../01-to-be/32-what-a-module-declares.md | 27 +++-- .../51-the-registries-keep-what-is-named.md | 103 +++++++++++++----- 3 files changed, 104 insertions(+), 34 deletions(-) diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 688ba837..58cb32b4 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -395,3 +395,11 @@ checked as their tables say: | The uplink's step-back comes from the seat | 0287: the composition tests for the uplink and the resolver on one machine pass with the case in the collision check removed. | | Holdings are derived, and the overview lists every seat | 0118: the `seats` command test, including an unheld seat. | | A build source on the seat records no address; an unheld seat refuses only self-hosted builds | 0111's tests. | + +## Reviewed by architectural domain + +The amendment of 2026-11-03 for research 081 (ADR 0605 to 0611; *A seat's definition*) was reviewed with designs 51 +and 32 by the four reviewers by architectural domain; the findings and what answers them are listed in design 51's +*Reviewed by architectural domain*. For this design: only the trunk's definition is the contract (security), the +indexer learns out-of-step types from the check's verdict event and runs no generator (seams), findings go to the +condition keeper (health), a pull request's entries are removed when it closes (running qualities). diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index 37054c73..e8d68d5e 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -498,24 +498,30 @@ complete, because a team that believes it is complete stops reviewing for the ca (ADR 0609 §4.) A config value's declaration may name its old name, `renamed-from`. On the merge, mesh-config moves the value it holds under the old name to the new name in its own state, without the operator giving it again, emits -`renamed` (module, old name, new name, when) and records it. This design decides what ADR 0609 leaves open: +`renamed` (module, old name, new name, when; its family `config.events`, kept seven days) and records it. This design decides what ADR 0609 leaves open: - **Which values**: a module's own config values only, never a secret, never another module's value. A rename keeps the value's trust: a value set without the operator's Approve stays untrusted under its new name (ADR 0348 §8), and the checker refuses a rename that would change a value's trust. -- **How long the old name is read**: until every node-engine reports applied a build of the module that reads the new - name, as the node-engine seat's reports answer; until then mesh-config answers both names. A later change may drop +- **How long the old name is read**: until every node-engine whose node has a want of the module reports applied a + build of it that reads the new name: mesh-config hears the node-engine seat's `applied` events, and asks each node's + `node-engine.haves` at its start; until then mesh-config answers both names. A later change may drop `renamed-from` only after that, and the checker refuses it before. The checker also refuses `renamed-from` in any - manifest until a build of mesh-config that understands it is delivered. + manifest until a build of mesh-config that understands it is delivered, which it learns from the node-engine seat's + `applied` events for mesh-config. `config_renames` names, per rename, the nodes still holding the old name open. - **A refused move** (the old name holds nothing, or a secret) is mesh-config's finding, from which the condition keeper raises the condition. **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and `config_rename_again` (move one value by hand when a move was refused). ### A value no longer read (from 2026-11-03) -A config value or secret no manifest reads any more is marked retiring by mesh-config, which emits `retiring` and asks -the operator once, through the operator channel, whether to remove it; it is removed only on the operator's Approve. -`config_retiring` lists what is retiring and since when. +A config value or secret no manifest on any trunk reads any more (mesh-config learns it from the indexer's +`indexer_config_readers`, on each of the indexer's events that a manifest was taken in) is marked retiring only once no +node-engine reports applied a build that reads it, as the rename's window is (§ above). mesh-config then emits +`retiring` (family `config.events`, kept seven days) and asks the operator once, through the operator channel, whether +to remove it, naming any node that still read it until lately; it is removed only on the operator's Approve. An ask +unanswered for seven days is asked once more, and then written as a finding. `config_retiring` lists what is retiring, +since when, and its ask. ### The mesh delivery cases this design carries @@ -727,3 +733,10 @@ ruled on every finding. What was found, and what answers it: condition `full` (ADR 0595 §6 raises and clears it at 75 percent, so an entry sitting there flaps), and an event of its own for `drop` and `recreate`. **A correction of fact** for ADR 0595 §4: the bus module deletes through `recreate` as well as `drop`. Each is noted on issue 1966. + +The amendment of 2026-11-03 for research 081 (ADR 0606, ADR 0609; the package, rename, retiring and delivery-case +sections of § 8) was reviewed with designs 51 and 26 by the four reviewers by architectural domain; the findings and +what answers them are listed in design 51's *Reviewed by architectural domain*. For this design: a rename keeps a +value's trust (security), `packages` is a section of its own, not `uses`, and mesh-config hears the node-engines' +`applied` events (seams), a value retires only once no node-engine runs a build reading it (security), and `renamed` +and `retiring` have a family and an ask that is asked again (events). diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index d3a61983..1211d32c 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -31,9 +31,10 @@ extending ADR 0189.) ## What is published, and how (from 2026-11-03) **Every registry is a seat; a build module publishes into it over the bus; the builder gives every version.** -(ADR 0605, ADR 0606, ADR 0607, ADR 0610, from research 081.) What follows is the to-be shape; the sections from +(ADR 0605, ADR 0606, ADR 0607, ADR 0610, ADR 0611, from research 081.) What follows is the to-be shape; the sections from *The parts* on keep what the registries hold, and stand. Every interaction below is a seat verb, a module's tool or an -event: no module reads or writes another's bus state (ADR 0596). +event: no module reads or writes another's bus state (ADR 0596). A module asks another's data through its tools, and +asks work of another by an event of its own (ADR 0610 §1). ### The four registry seats @@ -47,18 +48,24 @@ event: no module reads or writes another's bus state (ADR 0596). Each is held once for the mesh. The last two replace the seat `mesh-artifact-store` and front the one artifact store; while one module holds both, both deliver the provision `artifact-store`, and splitting them between two holders takes a decision of its own. `mesh-artifact-store` stays served until every node-engine reports applied a build that reads the -new names (ADR 0610 §4); a sleeping node holds it open. `cargo-package-registry` stays named, without a holder, until a +new names (ADR 0610 §4); a sleeping node holds it open. `distribution` closes it itself: it hears the node-engine +seat's `applied` events (and asks each node's `node-engine.haves` at its start), stops serving the old name once every +node whose declaration names an artifact reports the new build, and its tool `registry_old_name` lists the nodes still +holding it open. `cargo-package-registry` stays named, without a holder, until a build module publishes cargo packages. **One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `versions`, `remove` (a tagged version only; the holder itself refuses `remove` of a version with no tag), and on `container-image-registry` `mirror` and `tags`. Events: `published`, `refused` (added to ADR 0605 §2's two, additively) and `removed`, each -carrying the name, version, kind, tag, digest, fingerprint, commit and holder, and `refused` its reason. A holder stores +carrying the name, version, kind, tag, digest, fingerprint, commit and holder, and `refused` its reason; each seat's +events are its own family, `.events`, kept seven days. A holder stores under the version it is given and counts nothing. **Who may call which verb is the holder's to say**, in a trusted config value of the holder listing its callers, the form ADR 0363 gives a restricted family: `publish` the build modules; `mirror` mesh-build-image; `remove` the builder. -Declaring `invokes` alone grants nothing. The holder answers any other caller with a refusal naming the verb. +Declaring `invokes` alone grants nothing. The holder checks the caller identity the tool runner proved and stamps on +the request (design 25 § 12.4), never a name in the request's body, and answers any other caller with a refusal naming +the verb. ### Publishing @@ -74,18 +81,20 @@ Declaring `invokes` alone grants nothing. The holder answers any other caller wi builder (durable consumer: entry, done key), indexer, status view, checker ``` -- **The file travels in the verb.** `publish` opens with every part's digest (an image's layers, a package's file, - a bundle's blob); the holder answers the digests it lacks, and the build module sends only those, in chunks no - larger than the bus's message limit, into the holder's own store. This is how ADR 0605 §3's hand-off holds under ADR - 0596: no store is reached by two modules. The holder hashes every part it received and refuses one whose bytes do - not match its digest; a part it never received completed is dropped after 24 hours. The holder's tool - `registry_incoming` lists what is in transit, its fill against a 20 GiB cap, and drops one by hand; past 80 % of the - cap the holder writes a finding. +- **The file travels in parts through the seat** (ADR 0611). `publish` opens a transfer, naming it and the digest of + every part (an image's layers, a package's file, a bundle's blob); the holder answers the digests it lacks. The build + module sends only those through `publish-part` (transfer, part digest, sequence, last), each at most 1 MiB, with + mesh-sdk's bus helper for sending in parts, added to mesh-sdk's runtime in step 2's rebuild. A `publish` opening a + transfer already open answers the parts held, so a cut transfer goes on from there. The parts land in the holder's + own object store `incoming`, declared in its manifest with a cap of 20 GiB and a lifetime of 24 hours; a transfer with + no part for 24 hours is dropped and its parts deleted. The holder hashes every part and refuses one whose bytes do not + match its digest. At 80 % of the cap it writes a finding (cleared below 80 %); at the cap it refuses a new transfer + with `refused` naming the cap. Its tool `registry_incoming` lists open transfers and the fill, and drops one by hand. - **The holder asks the builder**, through `builder_versions`, whether the name, version and digest are recorded, and refuses loudly otherwise (ADR 0605 §4). A rebuild of a held fingerprint answers the held version and stores nothing. - **How a publish ends**: the holder emits `published` or `refused`. The build module waits for one of them, at most 10 minutes, narrating "waiting on " at least every five minutes; past the bound it writes the key refused, - naming the seat. A `published` that arrives after that is still heard by the builder on its durable consumer: the + naming the seat, and the builder writes the finding for a refused publish. A `published` that arrives after that is still heard by the builder on its durable consumer: the entry turns `published` and the builder writes the done key then, so a late store is never lost. - **What is published when**: a package or an image from a trunk build under its version, or as a tagged version; a bundle or an archive from every done key by its digest alone, a candidate's included, and at the trunk's claim the @@ -106,28 +115,32 @@ Declaring `invokes` alone grants nothing. The holder answers any other caller wi - **The builder numbers every trunk build** of a package, image, bundle or archive at its claim: the package series its manifest declares (design 32; `0.1` when it declares none) and the next count for that name, under compare-and-set. Numbers follow the order of the forge bridge's merge events: a claim of a later merge waits until - every earlier merge of the same name is numbered. A fingerprint already numbered keeps its number. + every earlier merge of the same name whose fingerprint the indexer still holds is numbered (a fingerprint replaced + by a later merge before its claim is skipped, design 47 *After the merge*); a wait past 30 minutes writes a finding + naming the merge it waits on. A fingerprint already numbered keeps its number. - **The builder's state `versions`** holds one key per name, tag and version, with its fingerprint, commit, digest, time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the reason (a branch's end, retention, the sweep) and the event that removed it. History one, cap 1 GiB; past 80 % the builder - writes a finding. Only the builder reaches it; others call `builder_versions` or hear its events. The build log + writes a finding, cleared below it. Only the builder reaches it; others call `builder_versions` or hear its events. The build log carries each change too. - **The floor**: on a name's first claim, or when `versions` holds nothing for it, the count starts above the highest of the seat's `versions`, the package's highest git tag (the forge bridge's tool `bridge_tags`) and the highest pseudo-version a consumer requires (the indexer's tool `indexer_requirements`). The builder numbers nothing for that - name until all three answered, and writes a finding naming the silent one meanwhile. Today that gives Go `v0.1.14` + name until all three answered, and writes a finding naming the silent one meanwhile, cleared once all three answer. Today that gives Go `v0.1.14` and npm `0.1.18`; a name with none starts at `0.1.0`. - **Events of the builder**: `numbered` (name, version, tag, fingerprint, commit, head when it answers an ask), on its own family, kept seven days; a waiter on a build's end does not wake on it. **Tools of the builder**: `builder_versions` (a name's versions in order, one call), `builder_publish_again` (one fingerprint under its number, - after a holder failed; when the build's file is gone it builds the fingerprint again under the same number; it - refuses a version refused for other bytes), `builder_tag` (ask a tagged version), `builder_tag_asks` (open asks and + after a holder failed; when the build's file is gone it builds the fingerprint again under the same number, and the + new digest replaces the recorded one only if no `published` was ever emitted for that number, else it takes a new + number; it refuses a version refused for other bytes), `builder_tag` (ask a tagged version), `builder_tag_asks` (open asks and their state) and `builder_untag` (remove a branch's tagged versions). ### Tagged versions - **Asked by an event, never by a push** (ADR 0610 §1). The checker emits `tag-wanted` (repository, branch, head, - tag) for each head of a delivery group member that declares a package; a person asks through `builder_tag`. The + tag), on a family of its own, `checker.asks`, kept seven days, apart from its verdicts, so a waiter on a verdict does + not wake on it, for each head of a delivery group member that declares a package; a person asks through `builder_tag`. The builder consumes the event on a durable consumer and keeps the ask in its own state `tag-asks`, removed when its branch is merged or deleted. An ask the builder has not answered within five minutes is narrated in the build log with what it waits on. @@ -145,12 +158,16 @@ Declaring `invokes` alone grants nothing. The holder answers any other caller wi The retention of *The package registry* below, and the artifact sweep, keep working, now through the seats: each removal emits `removed` and the builder marks its entry, so `versions` never says `published` for bytes that are gone. -The Go seat keeps what the npm seat keeps: every referenced version and the newest five of each package series. +The Go seat keeps what the npm seat keeps: every referenced version and the newest five of each package series. A +Go version is referenced when a `go.mod` on a repository's default branch requires it; the forge module reads every +`go.mod` as it reads lockfiles, and a `go.mod` it could not read stops a real run before anything is deleted, as an +unread lockfile does. ### Health and conditions Observers write findings; the condition keeper raises and clears the conditions from them (ADR 0564). Findings: a -publish refused (the holder; cleared when it is stored), a registry seat not answering (each holder's health check, +publish refused (the holder when it refuses, the builder when a build module's bound ran out; cleared when that version +is stored, or when a later version of the name is published), a registry seat not answering (each holder's health check, which calls `versions` on each seat it holds and probes the read side every minute, so a dead seat is seen with no publish running), a removal not done (the builder), a floor source silent (the builder), a store past 80 % (the holder, the builder). A build narrates its publish: asked, waiting on a seat, stored, refused. @@ -159,9 +176,11 @@ the builder). A build narrates its publish: asked, waiting on a seat, stored, re Each step is its own delivery, and no credential goes before its replacement works (ADR 0605 §7). -1. The holders serve every verb of the four seats with their caller lists; the builder's `versions`, `tag-asks`, its +1. A node-engine that reads both reference forms and both seat names; the holders serve every verb of the four seats + with their caller lists, and `incoming`; the builder's `versions`, `tag-asks`, its events and tools; the forge bridge's `bridge_tags` and the indexer's `indexer_requirements`. -2. Every build module publishes through the seats: mesh-sdk's first numbered publish, images, bundles, archives and +2. Starts only once every node-engine reports step 1's build applied (a sleeping node holds it). Every build module + publishes through the seats: mesh-sdk's first numbered publish, images, bundles, archives and upstream copies, each read back; delivered with the move of mesh-sdk's seat contracts into `mesh-contracts` (design 26), so every module is rebuilt once for both. The checker emits `tag-wanted`. Until step 3, Go users fetch from the forge's git, where the builder's tags let them take the numbered version. @@ -169,7 +188,8 @@ Each step is its own delivery, and no credential goes before its replacement wor 4. The three credentials go, and `package-version.sum`. The old seat name retires after it, on the node-engines' reports. -Steps 1 to 3 are rolled back by reverting their pull request; step 4 by reverting it, which mints the build modules' +Steps are rolled back in reverse order. Steps 1 to 3 are rolled back by reverting their pull request; step 4 by +reverting it, which mints the build modules' credentials again. **On a fresh mesh** the foundation's bootstrap (research 075) pushes the holders' own images with a one-off credential of the bootstrap, discarded once the holders serve `publish`. @@ -189,11 +209,40 @@ test). after the build's file is gone builds again under the same number; with the seat's holder stopped at a first claim nothing is numbered and the finding names it; two merges claimed in reverse order are numbered in merge order; a retention removal turns the entry `removed`. -- **Holder (gitea, distribution).** A caller not on the caller list is refused for each verb; a part whose bytes do +- **Builder, added in round 2.** Two merges close together, the first replaced before its claim: the second is + numbered at once; a build module's bound running out writes the builder's finding, and a later version of the name + clears it; `builder_publish_again` after a `published` takes a new number. +- **Holder (gitea, distribution).** A caller not on the caller list is refused for each verb, and one module naming + another as the caller in the request's body is refused; a transfer cut in the middle resumes from the parts held, + and one left 24 hours is dropped; at the cap a new transfer is refused naming the cap; a version a `go.mod` requires + survives retention, and an unreadable `go.mod` stops the run; a part whose bytes do not match its digest is refused; only the missing layers are sent; `remove` of a version with no tag is refused; - with no publish running, a stopped Go seat writes its finding and clears it when it answers; `registry_incoming` + with no publish running, a Go seat that stops answering while its holder runs writes its finding and clears it when it +answers (a holder down whole is its module's own health check's); `registry_incoming` lists and drops. -- **Vault.** Reverting step 4 mints the build modules' credentials again. +- **Vault.** Reverting step 4 mints the build modules' credentials again; the bootstrap's one-off credential is gone + once the holders serve `publish`. +- **Live.** Step 2 waits while one node-engine has not reported step 1's build; `registry_old_name` names the nodes + holding the old name open. + +## Reviewed by architectural domain + +The amendment of 2026-11-03 (ADR 0605 to 0611, research 081's graduation, designs 51, 26 and 32 together) was reviewed +by the four reviewers by architectural domain side by side (playbook 01), and the lead ruled on every finding. Two +findings reached a decision and became records: ADR 0610 (a module's bus state is its own, ADR 0596, so the checker +asks by an event and holders ask the builder's tool; retention; the old seat name closes on what node-engines applied) +and ADR 0611 (the file travels in parts through the seat). What was found, and what answers it: + +| Domain | Found | Answered by | +|---|---|---| +| security | caller lists named only on the caller's side; the handed object's bytes unchecked; no caller identity; pull request seat definitions served as contracts; the bootstrap's credential | *Who may call which verb* (holder-side lists, the identity the tool runner proved); *Publishing* (parts hashed); design 26 (only the trunk's entry is the contract); *The move* (a one-off bootstrap credential) | +| upgrade | the steps moved no image, bundle or archive before their credentials went; Go users between steps 2 and 3; no node-engine reading the new forms before step 2 | *The move* (every move in step 2, a node-engine reading both forms in step 1, step 2 waiting on the node-engines' reports, git tags for Go, reverse-order rollback) | +| seams | five reaches into another module's bus state; a shared hand-off store; how a publish ends; how the indexer learns out-of-step types; no named path for the node-engines' reports | ADR 0610, ADR 0611; *Publishing* (`published` or `refused`); design 26 (the check's verdict event); `applied` and `node-engine.haves` | +| tools | no read or drop tool for asks, transfers, the old name's window | `builder_tag_asks`, `registry_incoming`, `registry_old_name`, design 32's `config_renames` and `config_retiring` | +| running qualities | the versions value growing; the floor with a part down; merge order and a replaced merge; windows closing on wants; retention against "never removed"; a Go reference undefined; the in-transit cap | *The versions* (one key per version, a cap, the floor's wait, merge order skipping a replaced fingerprint); *Retention* (a Go reference); *Publishing* (the cap refused) | +| testing | failure paths untested; the test budget | cases of each module's one whole-module integration test, listed under *Acceptance tests* | +| health | conditions raised by their observers; no health check of a seat between publishes; a refused publish with no clearing | findings for the condition keeper (ADR 0564); the holder's health check; *Health and conditions* (each finding with what clears it) | +| events and logs | removal and `published` failing silently; a publish silent for 10 minutes; events with no family or retention | the builder's durable consumers and acknowledgement rule; narration every five minutes; each event's family and seven days | ## The parts From 5b2cead5e3bc5e1ce97428e83b37e6a156d8e394 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 3 Nov 2026 18:10:19 +0100 Subject: [PATCH 4/4] Designs 51 and 32: answer the third review (publish-part listed with its callers, the node-engines' reports through the status view, C24 tested) Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- .../01-to-be/32-what-a-module-declares.md | 14 +++++------ .../51-the-registries-keep-what-is-named.md | 24 ++++++++++++------- 2 files changed, 22 insertions(+), 16 deletions(-) diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index e8d68d5e..3ca7ae5d 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -504,11 +504,11 @@ value it holds under the old name to the new name in its own state, without the the value's trust: a value set without the operator's Approve stays untrusted under its new name (ADR 0348 §8), and the checker refuses a rename that would change a value's trust. - **How long the old name is read**: until every node-engine whose node has a want of the module reports applied a - build of it that reads the new name: mesh-config hears the node-engine seat's `applied` events, and asks each node's - `node-engine.haves` at its start; until then mesh-config answers both names. A later change may drop + build of it that reads the new name: mesh-config hears the status view's `delivered` events, and asks its + `node_haves` at its start (design 47); until then mesh-config answers both names. A later change may drop `renamed-from` only after that, and the checker refuses it before. The checker also refuses `renamed-from` in any - manifest until a build of mesh-config that understands it is delivered, which it learns from the node-engine seat's - `applied` events for mesh-config. `config_renames` names, per rename, the nodes still holding the old name open. + manifest until a build of mesh-config that understands it is delivered, which it learns from the status view's + `delivered` events for mesh-config. `config_renames` names, per rename, the nodes still holding the old name open. - **A refused move** (the old name holds nothing, or a secret) is mesh-config's finding, from which the condition keeper raises the condition. **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and `config_rename_again` (move one value by hand when a move was refused). @@ -535,7 +535,7 @@ each a case of its module's one whole-module integration test unless it says liv | C18, a new required config value with no default | the check names the nodes that lack it; the want waits; the operator is asked once | live: a module gaining such a value waits on each node lacking it, the operator is asked once naming the nodes, and once given the module is applied | | C20, a config value renamed | the value moves, keeps its trust, both names read until every node runs the new build | mesh-config: a renamed value is present under the new name with no operator input and its trust; the old name answers until the node-engines report the new build; dropping `renamed-from` early, or changing trust, is refused; a refused move writes its finding | | C22, a secret only a person can give | the want waits; the operator is asked once what to give and where | live: a module requiring such a secret waits, one ask names it and where, and once given the module starts | -| C24, a config value or secret no longer read | it is retiring, the operator is asked, removed only on the Approve | mesh-config: a value no manifest reads is marked retiring, `retiring` is emitted, one ask reaches the operator, and it stays until an Approve removes it | +| C24, a config value or secret no longer read | it is retiring, the operator is asked, removed only on the Approve | mesh-config: with one node-engine still reporting a build that reads the value it is not marked retiring and no ask is sent; once that node reports the new build it is marked retiring, `retiring` is emitted and one ask reaches the operator; an ask unanswered seven days is asked once more, then written as a finding; the value stays until an Approve removes it | ## 9. Provisioning over the bus @@ -737,6 +737,6 @@ as well as `drop`. Each is noted on issue 1966. The amendment of 2026-11-03 for research 081 (ADR 0606, ADR 0609; the package, rename, retiring and delivery-case sections of § 8) was reviewed with designs 51 and 26 by the four reviewers by architectural domain; the findings and what answers them are listed in design 51's *Reviewed by architectural domain*. For this design: a rename keeps a -value's trust (security), `packages` is a section of its own, not `uses`, and mesh-config hears the node-engines' -`applied` events (seams), a value retires only once no node-engine runs a build reading it (security), and `renamed` +value's trust (security), `packages` is a section of its own, not `uses`, and mesh-config hears the status view's +`delivered` events (seams), a value retires only once no node-engine runs a build reading it (security), and `renamed` and `retiring` have a family and an ask that is asked again (events). diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 1211d32c..d4bcc5c6 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -12,6 +12,7 @@ decisions: - "ADR 0606" - "ADR 0607" - "ADR 0610" + - "ADR 0611" - "ADR 0311" - "ADR 0251" - "ADR 0189" @@ -48,13 +49,13 @@ asks work of another by an event of its own (ADR 0610 §1). Each is held once for the mesh. The last two replace the seat `mesh-artifact-store` and front the one artifact store; while one module holds both, both deliver the provision `artifact-store`, and splitting them between two holders takes a decision of its own. `mesh-artifact-store` stays served until every node-engine reports applied a build that reads the -new names (ADR 0610 §4); a sleeping node holds it open. `distribution` closes it itself: it hears the node-engine -seat's `applied` events (and asks each node's `node-engine.haves` at its start), stops serving the old name once every +new names (ADR 0610 §4); a sleeping node holds it open. `distribution` closes it itself: it hears the status view's +`delivered` events (and asks the status view's `node_haves` at its start, design 47), stops serving the old name once every node whose declaration names an artifact reports the new build, and its tool `registry_old_name` lists the nodes still holding it open. `cargo-package-registry` stays named, without a holder, until a build module publishes cargo packages. -**One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `versions`, `remove` (a +**One contract, a seat definition** (design 26, *A seat's definition*). Verbs: `publish`, `publish-part` (ADR 0611), `versions`, `remove` (a tagged version only; the holder itself refuses `remove` of a version with no tag), and on `container-image-registry` `mirror` and `tags`. Events: `published`, `refused` (added to ADR 0605 §2's two, additively) and `removed`, each carrying the name, version, kind, tag, digest, fingerprint, commit and holder, and `refused` its reason; each seat's @@ -62,7 +63,8 @@ events are its own family, `.events`, kept seven days. A holder stores under the version it is given and counts nothing. **Who may call which verb is the holder's to say**, in a trusted config value of the holder listing its callers, the -form ADR 0363 gives a restricted family: `publish` the build modules; `mirror` mesh-build-image; `remove` the builder. +form ADR 0363 gives a restricted family: `publish` and `publish-part` the build modules, and a part only from the +proved caller that opened its transfer; `mirror` mesh-build-image; `remove` the builder. Declaring `invokes` alone grants nothing. The holder checks the caller identity the tool runner proved and stamps on the request (design 25 § 12.4), never a name in the request's body, and answers any other caller with a refusal naming the verb. @@ -92,9 +94,11 @@ the verb. with `refused` naming the cap. Its tool `registry_incoming` lists open transfers and the fill, and drops one by hand. - **The holder asks the builder**, through `builder_versions`, whether the name, version and digest are recorded, and refuses loudly otherwise (ADR 0605 §4). A rebuild of a held fingerprint answers the held version and stores nothing. -- **How a publish ends**: the holder emits `published` or `refused`. The build module waits for one of them, at most +- **How a publish ends**: the holder emits `published` or `refused`. The build module waits for one of them, matching + on its transfer, name and version, at most 10 minutes, narrating "waiting on " at least every five minutes; past the bound it writes the key refused, - naming the seat, and the builder writes the finding for a refused publish. A `published` that arrives after that is still heard by the builder on its durable consumer: the + naming the seat, and the builder, hearing that build's end event as it does every build's, writes the finding for a + refused publish. A `published` that arrives after that is still heard by the builder on its durable consumer: the entry turns `published` and the builder writes the done key then, so a late store is never lost. - **What is published when**: a package or an image from a trunk build under its version, or as a tagged version; a bundle or an archive from every done key by its digest alone, a candidate's included, and at the trunk's claim the @@ -213,7 +217,8 @@ test). numbered at once; a build module's bound running out writes the builder's finding, and a later version of the name clears it; `builder_publish_again` after a `published` takes a new number. - **Holder (gitea, distribution).** A caller not on the caller list is refused for each verb, and one module naming - another as the caller in the request's body is refused; a transfer cut in the middle resumes from the parts held, + another as the caller in the request's body is refused; a part sent by a module other than its transfer's opener is + refused; a transfer cut in the middle resumes from the parts held, and one left 24 hours is dropped; at the cap a new transfer is refused naming the cap; a version a `go.mod` requires survives retention, and an unreadable `go.mod` stops the run; a part whose bytes do not match its digest is refused; only the missing layers are sent; `remove` of a version with no tag is refused; @@ -222,7 +227,8 @@ answers (a holder down whole is its module's own health check's); `registry_inco lists and drops. - **Vault.** Reverting step 4 mints the build modules' credentials again; the bootstrap's one-off credential is gone once the holders serve `publish`. -- **Live.** Step 2 waits while one node-engine has not reported step 1's build; `registry_old_name` names the nodes +- **Live.** Step 2 waits while one node has not got step 1's build, as `node_haves` shows (the lead's gate before + step 2's merge); `registry_old_name` names the nodes holding the old name open. ## Reviewed by architectural domain @@ -237,7 +243,7 @@ and ADR 0611 (the file travels in parts through the seat). What was found, and w |---|---|---| | security | caller lists named only on the caller's side; the handed object's bytes unchecked; no caller identity; pull request seat definitions served as contracts; the bootstrap's credential | *Who may call which verb* (holder-side lists, the identity the tool runner proved); *Publishing* (parts hashed); design 26 (only the trunk's entry is the contract); *The move* (a one-off bootstrap credential) | | upgrade | the steps moved no image, bundle or archive before their credentials went; Go users between steps 2 and 3; no node-engine reading the new forms before step 2 | *The move* (every move in step 2, a node-engine reading both forms in step 1, step 2 waiting on the node-engines' reports, git tags for Go, reverse-order rollback) | -| seams | five reaches into another module's bus state; a shared hand-off store; how a publish ends; how the indexer learns out-of-step types; no named path for the node-engines' reports | ADR 0610, ADR 0611; *Publishing* (`published` or `refused`); design 26 (the check's verdict event); `applied` and `node-engine.haves` | +| seams | five reaches into another module's bus state; a shared hand-off store; how a publish ends; how the indexer learns out-of-step types; no named path for the node-engines' reports | ADR 0610, ADR 0611; *Publishing* (`published` or `refused`); design 26 (the check's verdict event); the status view's `delivered` events and `node_haves` | | tools | no read or drop tool for asks, transfers, the old name's window | `builder_tag_asks`, `registry_incoming`, `registry_old_name`, design 32's `config_renames` and `config_retiring` | | running qualities | the versions value growing; the floor with a part down; merge order and a replaced merge; windows closing on wants; retention against "never removed"; a Go reference undefined; the in-transit cap | *The versions* (one key per version, a cap, the floor's wait, merge order skipping a replaced fingerprint); *Retention* (a Go reference); *Publishing* (the cap refused) | | testing | failure paths untested; the test budget | cases of each module's one whole-module integration test, listed under *Acceptance tests* |