From 27063a492be2f76891bf52b673523b0ddf322495 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 3 Nov 2026 17:10:36 +0100 Subject: [PATCH] Graduate research 081 into designs 51, 26 and 32: registries are seats published into over the bus, the builder numbers every build, seat definitions on the bus, contracts modules (ADR 0605-0609) --- 00-META/glossary.md | 39 +++++- 03-DESIGN/01-to-be/26-the-seats.md | 46 ++++++- .../01-to-be/32-what-a-module-declares.md | 46 +++++++ .../51-the-registries-keep-what-is-named.md | 119 +++++++++++++++++- 4 files changed, 241 insertions(+), 9 deletions(-) diff --git a/00-META/glossary.md b/00-META/glossary.md index 110f3a44..e04f3b60 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -58,6 +58,9 @@ is listed under [Homonyms](#homonyms), with the qualified form each domain uses. promised as optional, then served, then required (ADR 0246). A node seat may also name **the paths its holder owns**, such as `/etc/hosts`: a claim whose module does not declare them is refused, and only a claimant declares them (ADR 0287). +- **seat definition** — the one language-neutral file (a schema with conformance fixtures) that names a seat's + verbs and their messages, its events, its state and the forms of its provisions; its Go and TypeScript types + are generated from it. The indexer keeps every trunk's and open pull request's as bus state (ADR 0608 §1, §2). - **operator** — the person who runs the mesh, and the one the mesh talks to. *Not:* ~~master~~ (hq) - **person** — any human, as against the mesh acting unattended: *a person's word* releases a held @@ -95,8 +98,17 @@ and repair). - **tool** — one capability a module serves through the tool runner, addressed `.`. A seat's **verb** is the same thing carried by a seat rather than a module. - **mesh-sdk** — the library a module's own code is written against, including the SDK harness that serves a - tools bundle to the tool runner. The SDK harness is part of it, not a library of its own. + tools bundle to the tool runner. The SDK harness is part of it, not a library of its own. It keeps the shared + runtime, and its seat contracts move into `mesh-contracts` (ADR 0608 §5). *Not:* ~~tools-sdk~~ +- **contracts module** — a module whose work is to publish the contract packages of seats (their verbs, events, + the layouts of their states, their messages) in every language the seats' users are written in, and never logic: + `mesh-contracts` for the seat contracts mesh-sdk held, `mesh--contracts` for an area's. A seat's holder + implements the contract; the contracts module holds no seat (ADR 0608 §5). + *Not:* ~~SDK module~~, ~~domain package~~ +- **package series** — the first two numbers of a package's versions (`0.1`), declared in its manifest and raised + only by a reviewed change, for a change that breaks its users; a user's manifest names the package series it + accepts (ADR 0606 §4, ADR 0609 §1). Bare "series" only where the package is plain (see [Homonyms](#homonyms)). - **invokes** — the manifest word for the tools a module calls, `.` each or `*` for every one. A grant on the publish side and nothing else; a module that declares none calls nothing. - **run for** — an assignment, or a start of a module, that serves one agent identity with that identity's machine @@ -513,12 +525,29 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel and at the end the outcome and each artifact's digest). A build's duration is read from its two entries, and no other part records durations; the builder's module tool `builder_builds_since` answers the builds a reader missed from it (ADR 0571 §1, §12). -- **package** — what code resolves when it is **compiled**: an npm/cargo/pypi dependency, by **version**. - Served by the **package-registry** (gitea). Only a build module talks to it. +- **package** — what code resolves when it is **compiled**: a Go module, or an npm, cargo or pypi dependency, by + **version**. Published by the registry seat of its language, on the request of its language's build module, from + a trunk build under the **next version** or for a **tagged version** (ADR 0605 §3, ADR 0607). Only a build module + asks a registry seat to store one. +- **registry seat** — a seat held once for the mesh whose holder stores what build modules publish, under the + version the builder gave, and serves its toolchain's read side: `npm-package-registry` and + `go-package-registry` (the **package-registry**, held today by the forge module), `container-image-registry` + and `oci-artifact-registry` (held today by the `distribution` module). One contract: `publish`, `versions`, a + verb removing a tagged version, the events `published` and `removed` (ADR 0605 §1, §2). Not the forge. + *Not:* ~~docker-package-registry~~ +- **next version** — the version the builder gives a trunk build of a published thing: its package series and one + above the highest count it holds for that name, kept with its fingerprint, commit, digest and time in the + builder's versions on the bus (ADR 0606 §1 to §3). Counted, never typed. +- **tagged version** — a published version with a tag (a branch's name, or a word such as `beta`), numbered in + Semantic Versioning's pre-release form below the next version, never a package's latest, taken only by a build + that asks its tag, and removed when its branch is merged or deleted (ADR 0607). +- **mesh delivery case** — a kind of change a developer makes (a package's change, a renamed config value, a new + seat), and how the mesh carries it from the pull request to the nodes; research 081 lists them. - **artifact** — anything a build produces and the mesh delivers to a node by **digest**: an `image`, a mirrored `upstream` image, a `bundle` of the module's own code, an `archive`. Served by the **artifact store**, an OCI registry that holds every kind as content-addressed blobs - (ADR 0156). + (ADR 0156), reached through two registry seats: `container-image-registry` for images and + `oci-artifact-registry` for bundles and archives, which replace the seat `mesh-artifact-store` (ADR 0605 §1). An image is one kind of artifact, and a module is not an image. A package and an artifact are two protocols, not one store being weak (ADR 0075). - **artifact sweep** — letting go of the artifacts no one needs any more, the seat `mesh-artifact-sweep`, held @@ -1125,6 +1154,8 @@ renaming one sense, the old sense moves to a *Not:* line and becomes mechanical. | harness | the part of `mesh-sdk` that serves a tools bundle | **SDK harness** | Module | | | the program that runs an agent's sessions | **agent harness** | Operator and conversation | | | the TypeScript `provisioner` helper for database providers | **`provisioner`**, renamed by design 57 | Provisioning | +| series | the first two numbers of a package's versions | **package series** | Module | +| | a kind of content told in episodes | the kind **series** | Media | | role | what a seat names, at a scope | **seat**; the bare word in older designs is tracked by issue 872 | Module | | | what an identity may call | **role** | Identity and access | | ask | a request for the operator's input | **ask** | Operator and conversation | diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index bb0b2073..4ea5fb1a 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -15,8 +15,10 @@ code: - mesh-catalog modules/gitea/module.json - mesh-controller internal/inventory/migrations/0063-a-machines-names-are-one-seats.sql - mesh-controller internal/inventory/migrations/0064-the-resolver-file-is-the-uplinks.sql -updated: 2026-10-10 +updated: 2026-11-03 decisions: + - "ADR 0608" + - "ADR 0605" - "ADR 0287" - "ADR 0223" - "ADR 0220" @@ -141,9 +143,11 @@ convention, which later seats departed from. | `mesh-store` | — | mesh | — | the store the mesh's own records live in | | `mesh-broker` | — | mesh | `mesh-bus` | the broker carrying the mesh's own bus | | `mesh-vault` | — | mesh | `secret` | the vault (in the seed since ADR 0161; the earlier *reserved* named an effect no rule produced) | -| `mesh-artifact-store` | `the-artifact-store` (renamed 2026-09-30, ADR 0156) | mesh | `artifact-store` | the artifact registry | +| `container-image-registry` | `mesh-artifact-store` for images (ADR 0605; the old name readable until every node-engine reads the new one) | mesh | `artifact-store` | the artifact registry (`distribution`) | +| `oci-artifact-registry` | `mesh-artifact-store` for bundles and archives (ADR 0605) | mesh | `artifact-store` | the artifact registry (`distribution`) | | `mesh-catalog` | `the-catalogue` | mesh | — | the catalog | -| `mesh-npm-package-registry` | `npm-package-registry` | mesh | `npm-package-registry` | the forge | +| `npm-package-registry` | — (its rename to `mesh-*` stays deferred, ADR 0156 §8) | mesh | `npm-package-registry` | the forge | +| `go-package-registry` | — (ADR 0605) | mesh | `go-package-registry` | the forge | | `mesh-git` | `git` | mesh | `git` | the forge | | `mesh-build-machine` | `the-build-machine` | node | — | a builder | | `mesh-resolver` | — | mesh | — | the mesh's resolvers, each holding every node's internal domain (ADR 0194); replicated, held on the anchor and the home server (ADR 0223) | @@ -165,6 +169,42 @@ things here: seats held by assignments rather than claimed by definitions, the ` its reservation, and the foundation's seats delivering nothing. It is brought to this table before it merges. +## A seat's definition (from 2026-11-03) + +**A seat is defined once, in one language-neutral file, and every part learns it from the bus** (ADR 0608, from +research 081). The registry seats (design 51) are the first defined this way, then `mesh-broker`'s verbs. + +- **The file** is a schema with conformance fixtures: the seat's verbs and their messages, its events and their + bodies, its state and the forms of its provisions. It lives in the contracts module of its area. Its Go and + TypeScript types are generated from it and committed beside it; nobody writes them by hand. +- **The generator** is each language's build module's: the Go build module's package for Go, the Node and TypeScript + build module's for TypeScript (the logic belongs to the module whose work it is). A check regenerates and fails, + naming the file, when a committed type differs. +- **On the bus**: the indexer reads every seat definition from every trunk and open pull request and keeps them in its + state `seat-definitions`, keyed by seat and source (the trunk, or a pull request's head), as their only writer, and + emits `seat-definition-changed`. A definition that does not parse fails its pull request's check; the trunk's + stays. The merge is the only approval. The foundation's own definitions are bootstrapped before the bus exists, as + the bootstrap of a fresh mesh has it (research 075). +- **Conditions**, each raised and cleared by the indexer: a seat definition that does not parse; generated types out + of step with their definition. **Tools** of the indexer: `indexer_seat_definition` (read one), + `indexer_seat_definitions_again` (read a trunk's or a pull request's again now) and `indexer_out_of_step` (the + generated types out of step). +- **Contracts modules.** A seat's contract is published by a contracts module, which holds contracts only (every file + generated from a seat definition) and no seat; the seat's holder implements it. The seat contracts mesh-sdk holds + move into `mesh-contracts` in one change, delivered with the first numbered publish (design 51, *The move*); an + area's contracts move into `mesh--contracts` when the area's contract next changes, and that change creates + the module. mesh-sdk keeps its name and the shared runtime. A catalog check refuses a contracts module holding a file + its generator did not write, and a seat-specific type in mesh-sdk once `mesh-contracts` publishes. +- **Older readers keep working**: a moved contract keeps its subjects, keys and message shapes, and a reader ignores + a field it does not know. +- **A vendor-specific** provision or contribution lives in its module's own extension package, published by version + like any package; a later holder of the seat may publish its own the same way. A seat's definition stays + vendor-neutral, which the seams review checks. + +Acceptance tests: research 081's 14, 21, 25 and 26, and an integration test of the indexer: a definition that does not +parse raises its condition and a fixed one clears it; a generated type changed by hand raises the out-of-step condition +and regenerating clears it; each of the three tools answers. + ## The foundation's seats `mesh-controller`, `mesh-store` and `mesh-broker` name which assignment the mesh *itself* uses: the diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index 63e6cd85..e749369f 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -15,6 +15,8 @@ code: - mesh-sdk (the typed bus-state entry in `requires` and its binding, ADR 0595) updated: 2026-11-03 decisions: + - "ADR 0609" + - "ADR 0606" - "ADR 0595" - "ADR 0596" - "ADR 0307" @@ -470,6 +472,50 @@ meaning into shape wherever it can go — a required `channel` field is caught, interpretation of an existing one is not. Saying so is better than implying the fingerprint is complete, because a team that believes it is complete stops reviewing for the case it misses. +### A package, its series, and the series its users accept (from 2026-11-03) + +(ADR 0606 §4, ADR 0609, from research 081.) This is the manifest's half of design 51's versions. + +- **A package is declared as an artifact**: its name, the kind `package`, its language (`go` or `typescript`), its + paths and its **package series** (`0.1`). An image, a bundle or an archive may declare a package series the same + way; one that declares none is numbered in `0.1`. Nobody types a version: the builder counts it (design 51). +- **Raising a package series** is a reviewed change of the package's manifest, for a change that breaks its users. + The checker's verdict names every change of a package series it sees, so the review of the change sees it. +- **A user names, in each `uses` of a mesh package, the package series it accepts.** It builds, and is checked, + against the newest version of that series; a publish in another series does not move it, and moving is a reviewed + change of the user's manifest. A user that names none accepts the series of the package's first numbered version, + so nothing is edited in today's users. +- **Inside one repository** a module uses its sibling package by path, from the same commit, and waits for no publish. +- **A user outside the package's delivery group is not built before the merge**: it is built after it, when the new + version is in its series. A user that breaks fails its own build, is not wanted, keeps its previous build, and a + condition tells the operator. A group's members are built in the group's check, against the group's tagged version. + +### A config value renamed (from 2026-11-03) + +(ADR 0609 §4.) A config value's declaration may name its old name, `renamed-from`. On the merge, mesh-config moves the +value it holds under the old name to the new name in its own state, without the operator giving it again, and records +the move. This design decides what ADR 0609 leaves open: + +- **Which values**: a module's own config values only, never a secret, and never another module's value. +- **How long the old name is read**: until every node's want of the module names a build that reads the new name, as + the delivery view's wants show; until then mesh-config answers both names. A later change may drop `renamed-from` + only after that, and the checker refuses it before. +- **Tools of mesh-config**: `config_renames` (what moved, from which name, when) and `config_rename_again` (move one + value by hand when a move was refused). + +### The mesh delivery cases this design carries + +Research 081 listed the mesh delivery cases; these are the ones it handed to a design, each with its acceptance test. + +| case | what holds | acceptance test | +|---|---|---| +| C6, a package series raised | users on `0.1` keep the newest `0.1`; a reviewed change moves a user | integration, builder: a `0.2.0` publish moves no `0.1` user, its check takes the newest `0.1` too, a reviewed change naming `0.2` moves it, a user naming no series builds against the newest `0.1` | +| C14, a provision's form gains a field | the provider serves it in a new version; the consumer reads it once rebuilt; an older consumer ignores it | integration, mesh-sdk: a consumer built against the older form reads a provision with the new field | +| C18, a new required config value with no default | the check names the nodes that lack it; the want waits; the operator is asked once | live: a module gaining such a value waits on each node lacking it, the operator is asked once naming the nodes, and once given the module is applied | +| C20, a config value renamed | the value moves; both names read until every node runs the new build | integration, mesh-config: a renamed value is present under the new name with no operator input, the old name answers until the wants show the new build everywhere, and dropping `renamed-from` early is refused | +| C22, a secret only a person can give | the want waits; the operator is asked once what to give and where | live: a module requiring such a secret waits, one ask names it and where, and once given the module starts | +| C24, a config value or secret no longer read | it is retiring, removed only on the operator's Approve | integration, mesh-config: a value no manifest reads any more is marked retiring and stays until an Approve removes it | + ## 9. Provisioning over the bus Provisioning rides the bus, and the provider stops having an address. diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 06dad5ed..2a3bdd57 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -6,15 +6,18 @@ code: - mesh-catalog modules/artifact-store-tools (the store's tools), modules/distribution (its unbuilt code removed) - mesh-catalog modules/gitea (package retention) - mesh-catalog modules/docker (image pruning) -updated: 2026-10-12 +updated: 2026-11-03 decisions: + - "ADR 0605" + - "ADR 0606" + - "ADR 0607" - "ADR 0311" - "ADR 0251" - "ADR 0189" - "ADR 0257" --- -# 51 — The registries keep what is named +# 51 — The registries: what is published, under which version, and what is kept **Three places hold built things: the artifact store, the package registry and each machine's container runtime. Each gets tools that say what it holds, what names each thing, and what could go. @@ -24,6 +27,118 @@ registry, and the machine's declaration for its images.** (ADR 0251, extending ADR 0189.) +## What is published, and how (from 2026-11-03) + +**Every registry is a seat; a build module publishes into it over the bus; the builder gives every version.** +(ADR 0605, ADR 0606, ADR 0607, from research 081.) What follows is the to-be shape; the sections from *The parts* +on keep what the registries hold, and stand. + +### The four registry seats + +| registry seat | holds | read side it serves | holder today | +|---|---|---|---| +| `npm-package-registry` | npm packages | the npm registry protocol | the forge module | +| `go-package-registry` | Go modules | Go's module proxy protocol | the forge module | +| `container-image-registry` | container images | OCI distribution, with an image's own verbs (its tags, mirroring an outside image) | `distribution` | +| `oci-artifact-registry` | bundles and archives, by digest | OCI distribution | `distribution` | + +Each is held once for the mesh. The last two replace the seat `mesh-artifact-store` and front the one artifact store; +the provision `artifact-store` keeps its name. `mesh-artifact-store` stays readable until every node's want of the +node-engine names a build that reads the new names, which the delivery view's wants show. `cargo-package-registry` +stays named, without a holder, until a build module publishes cargo packages. + +**One contract, a seat definition** (design 26, *A seat's definition*): the verbs `publish`, `versions` and +`remove` (a tagged version only), and the events `published` and `removed`, each carrying the name, version, kind, +tag, digest, fingerprint, commit and holder. A holder stores under the version it is given and counts nothing. + +### Publishing + +``` + builder ── build request (name, version or tag) ──▶ build module + │ hands the file: object store `publish-handoff` + │ calls .publish + ▼ + registry seat's holder ── reads the builder's `versions` (announced state) + │ stores, deletes the handed object + └── emits `published` ──▶ builder (entry → published), indexer, status view +``` + +- **Only build modules may call `publish`**, and only from a trunk build or for a tagged version; the bus grants + nothing else. No build module holds a registry's write credential: the npm write token and the image push + credential go, and each holder keeps its write credential as an own secret beside a read-only tool token (the forge + module: `write:package`, its tool token `read:package`). +- **The handed object** lives in the object store `publish-handoff`, which the builder owns: build modules write, + holders read and delete. An image is handed as an OCI layout holding only the layers the holder lacks: `publish` + names every layer's digest, the holder answers the digests it is missing, and only those are handed. The store is + capped at 20 GiB, an object lives at most 24 hours, and a publish not stored within 10 minutes is refused by its + build module, naming the seat, which leaves the entry `refused` (the condition below). +- **A holder refuses loudly**: a version or digest the builder's `versions` do not record, and a held version or tag + offered with other bytes. A rebuild of a held fingerprint answers the held version and stores nothing. +- **The done key** names the reference the seat answered: `//@sha256:` for an + image, bundle or archive, `@` for a package. +- **Go packages** are fetched through the Go seat's provision first and Go's public proxy second, with no + `GOPRIVATE` and the checksum database off for the forge's paths; `go.sum` and the recorded digest refuse a mesh + package with other content. Outside Go modules keep coming from the public proxy; no mirror is built, and one could + come later as a holder of a seat of its own without any build module changing. + +### The versions + +- **The builder numbers every trunk build** of a package, image, bundle or archive at its claim: the package series + its manifest declares (design 32 § 6; `0.1` when it declares none) and the next count for that name, under + compare-and-set. Numbers follow merge order. +- **The builder's state `versions`** (announced, the builder its only writer, readable by the indexer, the registry + seats' holders and the status view) holds per name, and per tag, every version with its fingerprint, commit, + digest, time and state: `numbered`, `published`, `refused` with its reason, or `removed` with the time, the branch + and the forge bridge's event that removed it. The build log carries each change too. +- **The floor**: on a name's first claim, or when `versions` holds nothing for it, the count starts above the highest + of what the seat's `versions` answers, the package's highest git tag (the forge bridge's state) and the highest + pseudo-version a consumer requires (the indexer's state); a name with none starts at `0.1.0`. Today that gives Go + `v0.1.14` and npm `0.1.18`. +- **A trunk build that declares a package is done** only once its done key names a published version for each. +- **Events of the builder**: `numbered`. **Tools of the builder**: `builder_versions` (a name's versions in order, + one call), `builder_publish_again` (one fingerprint under its number, after a holder failed; it refuses a version + refused for other bytes), `builder_tag` (ask a tagged version) and `builder_untag` (remove a branch's tagged versions). + +### Tagged versions + +- **A tagged version is asked by a key, never by a push.** The checker, for each head of a delivery group member that + declares a package, and a person, through `builder_tag`, write an ask in the builder's state `tag-asks`, keyed by + repository, branch and head; the builder watches it. The tag is cleaned to Semantic Versioning's characters. +- **It is numbered below the next version** (`0.1.15-feat-x.2`), never the latest; npm keeps it under a dist-tag of its + own and Go's `@latest` skips it. A package new in a group is numbered from its floor (`0.1.0-feat-x.1`). +- **A group's check takes it from the registry**: Go members by the tagged version, TypeScript members with their lock + naming it and the registry's integrity. Nothing is packed for a check, and a lone pull request's check publishes + nothing. +- **It is removed** when its branch is merged or deleted: the builder hears the forge bridge's event on a durable + consumer, calls each seat's `remove`, the seat emits `removed`, and the entry turns `removed`. A version without a + tag is never removed. + +### Health and conditions + +Every holder declares a health check. Conditions, each raised and cleared: a publish refused (cleared when it is +stored), a registry seat not answering (cleared when it answers). A build narrates its publish: asked, stored, refused. + +### The move, in four steps + +Each step is its own delivery. 1: the holders serve the verbs and `versions` exists. 2: mesh-sdk's first publish +through the seats, read back, delivered together with the move of mesh-sdk's seat contracts into `mesh-contracts` +(design 26), so every module is rebuilt once for both. 3: the Go fetch switches to the seat. 4: the old credentials +and `package-version.sum` go. Steps 1 to 3 are rolled back by reverting their pull request; step 4 by reverting it, +which mints the build modules' credentials again. On a fresh mesh the builder publishes into a holder the +foundation's bootstrap raised. + +### Acceptance tests + +Research 081's tests 1-3, 5-7, 9, 10, 12, 13, 15-20, 22 and 23 (its Direction) are this design's, with their labels +(live or integration, and the module). Added here: + +- **Integration, builder.** A branch pushed with no ask publishes nothing; a lone pull request's check publishes + nothing; a group introducing a new package takes `0.1.0-.1`; `builder_untag` removes a branch's tagged + versions with `removed`, and the entry shows the branch and the event. +- **Integration, builder.** An image whose layers the holder holds hands only the missing ones; a publish not stored + within 10 minutes leaves the entry `refused` and raises the condition. +- **Live.** A revert of step 4 on one node mints the build modules' credentials again before it reaches the others. + ## The parts ```