diff --git a/03-DESIGN/00-as-is/04-delivery.md b/03-DESIGN/00-as-is/04-delivery.md index 307bfafb..bce6942d 100644 --- a/03-DESIGN/00-as-is/04-delivery.md +++ b/03-DESIGN/00-as-is/04-delivery.md @@ -1,8 +1,8 @@ --- layer: as-is status: implemented -code: [mesh-controller, mesh-catalog modules/mesh-delivery, mesh-catalog modules/gitea, mesh-catalog modules/build-agent] -updated: 2026-10-17 +code: [mesh-controller, mesh-delivery, mesh-catalog modules/gitea, mesh-catalog modules/mesh-build-go] +updated: 2026-10-24 decisions: - "ADR 0239" - "ADR 0267" @@ -21,70 +21,62 @@ decisions: # Delivery — from a merge to the nodes, as it runs -What runs on the mesh's main branches on 2026-10-16, read from the code and the live record by research 055 -(findings 1 to 12). It is what [to-be 47](../01-to-be/47-delivery-from-commit-to-delivered.md) replaces, milestone -by milestone. Until 2026-10-16 this document described the predecessor's pipeline, which no longer runs. +What runs on 2026-10-24, after the delivery of release 4 and release 5's first half (milestone R4g, design 47), +read from the live controller (`mesh-controller.modules`, `.seats`, `.plans`), the delivery checker's `checker_status` +and the machines' module lists. It is what [to-be 47](../01-to-be/47-delivery-from-commit-to-delivered.md) has not yet +finished: the node agent applying wants and the deletion of plans are release 6, and the controller still plans, +orders and sends a delivery's builds to the machines. Until 2026-10-23 this document described the pipeline with the +build agent, the build queue and the two statuses `mesh/merge-gate` and `mesh/repo-check`, which no longer run. ## Who does what -- **mesh-delivery** owns deliveries and delivery groups (ADR 0239): one delivery per commit, a group per branch - name shared across repositories. It records each transition in its state on the bus (key-value buckets - `deliveries` and `groups`), marks pull requests, asks the controller to deliver, and merges a pull request marked - ready at its head once both required checks pass against the trunk's head (ADR 0364) and the operator approved - the merge into main on a channel that proves who answered (ADR 0398 §3). -- **The controller** plans, orders, sends and judges. It computes a delivery plan for a repository and the files a - merge changes, moves only modules whose build source changed with their build dependents (ADR 0267), and - reuses a merge check's build of the same reuse key (ADR 0350). -- **The builders** (holders of the build seat) take build requests and checks from a work-queue stream on the bus - and build inside a toolchain image another module provides, with the network off; on 2026-10-13 no node had Go - or npm installed (issue 626). -- **The forge module** publishes every forge event on the bus through the forge's system webhook since 2026-10-14 - (issue 729), and still polls the forge every 30 s, or every 300 s once it hears events. It keeps files of what it - announced so it does not announce twice. +- **The parts of `mesh-delivery`** are modules of one repository, each a process with its own bus credential that + watches keys and writes keys (design 47, ADR 0429). Running now: the forge bridge (`mesh-delivery-forge-bridge`, + novox), the indexer (novox), the builder (`mesh-delivery-builder`, shanks), the checker (novox), the gate + (`mesh-delivery-gate`, which holds the seat `mesh-delivery`, novox), the teller (novox), the watchman (novox), the + status view (shanks) and the assigner (`mesh-delivery-assigner`, ace). +- **The forge bridge** turns the forge's events into stream messages and is the only part that calls the forge. The + forge module (`gitea`, novox) still announces them through the forge's system webhook. +- **The indexer** computes each module's fingerprint at the merge preview; **the builder** claims a build once per + fingerprint and orders it as an ask on the stream of the artifact's language. +- **The build modules** (`mesh-build-go` and `mesh-build-node` on three machines, `mesh-build-image` on one) hold the + node seats `node-build-go`, `node-build-node` and `node-build-image`. Each pulls asks from the bus's work-queue + streams (deliveries first, then checks, then re-checks) while its node has a free slot of build capacity; no part + chooses the node. The seat `node-build-agent` and the module `build-agent` have no holder any more, and the seat + `mesh-build-machine` has none either. +- **The checker** (`mesh-delivery-checker`) runs the merge check: a change's tests through the build module of each + language, running only packages whose inputs changed, and writes the one status `mesh/check`. It also serves a + **try** (`checker_try`, `checker_try_read`): the same check on a pushed branch, before a pull request. `checker_status` + shows the steps in flight by node and the candidates still without a verdict. +- **The gate** merges a delivery group's pull requests once the check, the review and the operator's Approve agree on + the same heads. The forge requires no status on any repository (ADR 0534). +- **The assigner** composes the machines a change touches (the composition check the checker reads) and writes which + fingerprint each node should want, by its six rules; it only compares wants with what the controller sends. +- **The controller** (`mesh-controller`, novox) plans, orders, sends and judges. It computes a delivery plan for the + files a merge changes, moves only modules whose build source changed with their build dependents, and writes the + chain's links it owns. On 2026-10-24 the seat still lists 64 verbs, `queue` and `replay` among them; whether + those still do work was not read here, and the build agent they served has no holder. -## From a merge to a rollout +## From a merge to a delivery plan -- **The merge window.** The controller collects merges into an open batch; each merge restarts the window, which - closes after 90 seconds without a merge or 10 minutes after the batch's first merge (ADR 0276). A lone merge is - cut at once (ADR 0282 §2, as ADR 0304 §1 changed it). The batch and its merges are rows in the controller's - store, cut by a timer every 5 s, with a catch-up timer every 5 minutes for merges no batch holds. -- **The rollout.** A closed batch becomes one rollout, at most one commit per repository, tiered along the module - graph. Rollouts run one at a time: a batch cut while a rollout is open is queued behind it, whatever it shares - with it. Running unrelated rollouts side by side (ADR 0282 §3, ADR 0304 §2) is decided and not built (issue 525). A rollout's builds go to every machine at once, with no first-machine gate, and the rollout ends on - the machines' reports or after fifteen minutes of silence (ADR 0360). A controller timer advances rollouts every - 30 s. -- **A merge during a rollout.** Its place in the next batch is a store row the open rollout's holder writes. On - 2026-10-16, when another controller worked the plans, a merge was not recorded and waited until a person released - it (issue 921). -- **"Release branches".** A pull request into a branch other than a trunk is judged by the controller built from that - branch (ADR 0358); feature 57 shipped about 37 packages that way over about 2.5 days and held the controller for - 55.7 h. - -## Checks - -- Every pull request's head gets `mesh/merge-gate` (the composed machines) and `mesh/repo-check` (the repository's - `merge-check.sh`), each judged against one trunk commit (ADR 0323). A trunk move asks the check of every active, - cleanly merging pull request again (ADR 0395). -- A delivery group's members are each checked beside their siblings' heads, and the group also gets a composed - check, `mesh/delivery-group`; each member keeps its own gate beside it (ADR 0382). -- The queue's order (delivery builds first, then ready checks, then the rest) is kept by pausing the worker and - deleting and republishing a request (issue 558); an old head's check is ended after asking the forge for the - current head (issue 726). -- mesh-delivery ticks every 10 s, reconciles with the controller every 30 s and polls the forge for ready marks - every minute, although the forge module already publishes labeled, commented and synchronized events. +- A pull request's head is built and checked as it comes in. One status, `mesh/check`, carries the verdict; a delivery + group has one verdict for every member. A move of the trunk asks the checks again, and a running check stops itself + when its head or trunk commit moves. +- A merge is made by the gate on the operator's Approve; the controller then makes a delivery plan, tiered along the + module graph, and sends it. `mesh-controller.plans` lists the plans and the delivery plans' tiers; a plan that + fails on a node says so and is retried by hand. +- **hq** builds no module: the checker runs hq's Go tests (`go test ./...`) as its check. ## Where the record of what runs lives -Builds and build requests, rollouts with their tiers, batches and merges, and assignments are rows in the -controller's store; artifacts are blobs in the artifact store by digest (ADR 0251); packages are versions in the -package registry. The controller writes three record mirrors to the bus: assignments, declared config and -predefined keys; of the five ADR 0289 §7 decided, the nodes, rollouts, builds and settings mirrors are not in the -code. An assignment names its module and node but not the build, artifact or rollout that put it there: which commit -runs where, and which delivery brought it, takes three verbs and a join by hand. - -No backup is taken before a delivery moves a module that holds data; backups run nightly and on demand (ADR 0214). +Fingerprints, builds, verdicts and wants are keys in the bus's key-value buckets of the parts; builds' files are in +the artifact store by digest; packages are versions in the package registry. Plans, rollouts with their tiers, +assignments and settings are still rows in the controller's store. An assignment names its module and node but not +the build, artifact or plan that put it there. ## What is not covered -The integration tests run only in each repository's merge check; there are no end-to-end tests of delivery yet, -and the lab is not used (the operator, 2026-10-16). +The node agent does not yet apply wants: a node runs what the controller sent. The checker's left-running keys of +build modules that stopped (`checker_status`'s `left_running`) are listed and hold no place. The integration tests run +in each repository's check; there are no end-to-end tests of delivery yet, and the lab is not used (the operator, +2026-10-16). diff --git a/03-DESIGN/00-as-is/13-the-console.md b/03-DESIGN/00-as-is/13-the-console.md index d9ee8a98..2099495e 100644 --- a/03-DESIGN/00-as-is/13-the-console.md +++ b/03-DESIGN/00-as-is/13-the-console.md @@ -2,7 +2,7 @@ layer: as-is status: implemented code: [mesh-tools, mesh-host, mesh-controller] -updated: 2026-10-17 +updated: 2026-10-24 decisions: - "ADR 0152" - "ADR 0175" @@ -92,12 +92,12 @@ command, not inline, because one call blocks until it answers or times out. **These already answer directly, not as a stated future.** Beside every assigned module's tools, the tool runner asks the `mesh-controller` seat for its own verbs the same way (ADR 0154): measured -live, the mesh-controller runtime is one instance, mesh-wide, and it answers 63 verbs for that one +live, the mesh-controller runtime is one instance, mesh-wide, and it answers 64 verbs for that one seat (`status`, `push`, `assign`, `build`, `artifacts` and the rest), and the `mesh-controller` *module* answers none of its own — it serves no module tools, only its seat's, exactly as design 33 described while it was still only designed. The 56-to-93 seat-verb counts each node's tool runner answers (below) are a different thing: the tool runner serves the verbs of whichever node-scoped -seats are held on that node (`node-backup`, `node-build-agent`, `node-service-manager` among them), +seats are held on that node (`node-backup`, `node-build-go`, `node-service-manager` among them), on behalf of the modules holding them, not the mesh-controller seat's — that one is served by the mesh-controller runtime itself, a separate process, and the tool runner only asks it and relays what it answers.