Files
hq/03-DESIGN/00-as-is/04-delivery.md
T
jschoubben 9eb963d5f1 Give each restated acceptance test of design 47 its own proof, and add puts, the hand path and the builds bucket to as-is 04 (issue 1130, review round 1)
Tests 18 and 19 do not cover tests 3, 7, 9, 10 and 11 as restated, so each now names the test or live read that proves it, or says it is not yet proved.
2026-10-26 01:27:21 +01:00

9.6 KiB

layer, status, code, updated, decisions
layer status code updated decisions
as-is implemented
mesh-delivery
mesh-catalog modules/mesh-delivery-gate
mesh-catalog modules/gitea
mesh-catalog modules/mesh-build-go
mesh-catalog modules/messenger
mesh-catalog modules/mesh-issues
mesh-controller
mesh-host
mesh-sdk
2026-10-26
ADR 0239
ADR 0267
ADR 0282
ADR 0289
ADR 0304
ADR 0350
ADR 0358
ADR 0360
ADR 0364
ADR 0382
ADR 0395
ADR 0398
ADR 0429
ADR 0524
ADR 0525
ADR 0527
ADR 0528
ADR 0529
ADR 0539
ADR 0546
ADR 0547

Delivery — from a merge to the nodes, as it runs

What runs on 2026-10-26, after the delivery of release 6 (milestone R6 of to-be 47, ADR 0539), read from the code on each repository's main (mesh-delivery, the gate in mesh-catalog, the node agent in mesh-host, mesh-controller) and from the live mesh: the controller's seat verbs, mesh-controller.modules, the gate's table, and the view's assigner_wants and node_haves. Until 2026-10-26 this document described release 5, in which the controller still made a delivery plan for each merge, ordered it in tiers and sent whole declarations to the machines; none of that runs any more.

Who does what

  • The parts of mesh-delivery are modules of one repository, each a process with its own bus credential that watches keys and writes keys (ADR 0429). Running now: the forge bridge, the indexer, the checker, the teller and the watchman on the control-node; the builder and the status view (mesh-delivery-view) on the workstation; and the assigner on the home server. The gate (mesh-delivery-gate, holding the seat mesh-delivery) is a module of mesh-catalog and runs on the control-node.
  • The forge bridge turns the forge's events into stream messages and is the only part that calls the forge; the forge module (gitea) still announces them through the forge's system webhook.
  • The indexer writes each module's fingerprint at the merge preview and at a trunk merge; the builder claims a build once per fingerprint and orders it as an ask on the stream of the artifact's language.
  • The build modules (mesh-build-go on all four machines, mesh-build-node on three, mesh-build-image on one; ADR 0525) hold the node seats node-build-go, node-build-node and node-build-image and pull asks from the bus's work-queue streams (deliveries first, then checks, then re-checks) while their node has a free slot of build capacity.
  • The checker runs the merge check and writes the one status mesh/check, and serves a try on a pushed branch. The forge requires no status on any repository (ADR 0534), and no repository holds a merge-check.sh (ADR 0529).
  • The gate keeps one key per pull request and the merge's states only: proposed, ready, rejected, held, merged and closed, read from its table verb. On the operator's Approve it merges a delivery group's pull requests in one pass and writes their merged heads as one revision of source.main. It sends nothing to a machine and asks the controller nothing.
  • The controller (mesh-controller, on the control-node) keeps registration, placement in its assignments bucket and every node's inputs, and composes: on events it writes each module's signed composition for each node at each fingerprint into the node's bucket composed-<node>, with its digest and its judgement of the move in mesh-controller_composed, and it issues a module's credential and grant when its build is done. It plans, orders and sends nothing: of its seat's 65 verbs, 20 are refused by name with their replacement, among them plans, plan, push, deliver, delivery-stop, delivery-order and delivery-walks (read 2026-10-26 01:20).
  • The assigner writes each node's want of each module, want.<node>.<module>, naming a fingerprint and the digest of the one composition it allows, when its seven rules hold: the build is done, the fingerprint is not held, the node is the module's canary or the canary runs it healthy, at most step others ahead, every provider runs healthy, the grant exists and the composition exists. In the code the canary is the first of the module's nodes by name that is online (one ring), and rule 4 always holds, because the mesh keeps no step. It also runs the composition check the checker reads. Its differences bucket, from release 5's comparison with what the controller sent, is no longer written.
  • Two ways around the rules, both by hand. A put (ADR 0546), written by the assigner's tool assigner_put, puts one version of a module on one node: while a put owns a node's want of a module, from its ask until it ends, the rules pass over that want. And the operator's credential may write a want itself, with a composition the operator signed and wrote through mesh-sdk's mesh-hand-compose (ADR 0539 §10).
  • The node agent, a process of the node-engine (mesh-host) on every node, watches its node's wants and its bucket of compositions, and applies a module only when the want's fingerprint or digest differs from what runs, and only the composition whose digest the want names, after checking its signature. It asks its node's node-backup seat to back up a data module first and waits for the backup's end. It writes the have, have.<node>.<module> in node-engine_have, with the fingerprint, the composition's digest, the phase and the health. A failed apply is retried by the next change of the want or the composition; an apply that failed only because the bus did not answer for a moment is retried with a bounded backoff, and at once when the bus is back.
  • The watchman mirrors every want as a pulse owed by that node's agent until its have names the want's fingerprint, so a node that does not move shows as a stall.
  • The status view derives from source.main, the wants and the haves whether the source revision is delivered, and says the event delivered once per revision, and moving, settled and stuck once for a module on a node at one composition. The messenger holds the operator's notices on those events, and the issue tracker hears delivered.
  • The grants part (mesh-grants, on the control-node; ADR 0547) publishes each assignment's membership on the assignments stream when it differs from the one last published. The bus's user list, assertions and grants it composes only in its own state, compared with what the bus holds, and they take no effect; the controller still composes the user list the bus loads.

From a merge to the nodes

  • A pull request's head is built and checked as it comes in; one status, mesh/check, carries the verdict, and a delivery group has one verdict for every member. A move of the trunk asks the checks again, and a running check stops itself when its head or trunk commit moves.
  • The gate merges on the operator's Approve and moves the source revision in one write. Nothing collects merges or waits for others: the indexer writes the new fingerprints, the builder claims each new one once, the controller composes and grants on the done build, and the assigner writes each want once its rules hold, the canary first.
  • Each node applies its own wants one module at a time and writes its haves; the source revision is delivered when every want equals its module's fingerprint and every have equals its want, running and healthy. Read on 2026-10-26 at 01:20: 303 wants, every one equal to its node's have in fingerprint and composition digest, and 12 fingerprints held.
  • hq builds no module: the checker runs hq's Go tests (go test ./...) as its check (ADR 0527).

Where the record of what runs lives

Fingerprints, builds, verdicts, wants and haves are keys in the parts' buckets on the bus; compositions are in each node's composed-<node> bucket, readable only by that node, with a file over 64 KiB in the object store; builds' files are in the artifact store by digest; packages are versions in the package registry. The controller still records every build it takes in, in its builds bucket (RecordBuild in cmd/mesh-controller/key_takein.go); its assignments bucket keeps placement alone, and its rollouts bucket keeps the record of releases 1 to 5 read-only, and nothing writes it now.

What is not covered

Nothing checks the delivery budget: the over-budget condition and the gate's times went with release 6, and no reader of the view's settled and delivered events measures it yet (issue 1945, note 10). The bus's own move stays a person's step through the controller's bus verb (ADR 0539 §15). The checker's left-running keys of build modules that stopped (checker_status's left_running) are listed and hold no place. There are no end-to-end tests of delivery, and the lab is not used (the operator, 2026-10-16).

How it is checked

  • mesh-delivery-view.assigner_wants and mesh-delivery-view.node_haves: each want beside its have; indexer_modules and builder_builds for the fingerprints and builds they name.
  • mesh-delivery.table: the gate's states and transitions; mesh-delivery.deliveries and show for one delivery.
  • mesh-controller.seats and mesh_overview: the controller's verbs, and which are refused by name.
  • The code: mesh-delivery (modules/mesh-delivery-assigner/internal/assign, modules/mesh-delivery-view/internal/view, modules/mesh-delivery-watchman/internal/watch, modules/mesh-grants), mesh-host (internal/agent, cmd/mesh-host/nodeagent.go), and the gate in mesh-catalog (modules/mesh-delivery-gate).