diff --git a/00-META/glossary.md b/00-META/glossary.md index ba87277c..035e8b89 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -127,6 +127,29 @@ another — and a mesh you cannot name precisely is a mesh two people describe d - **invokes** — the manifest word for the tools a module calls, `.` each or `*` for every one. A grant on the publish side and nothing else; a module that declares none calls nothing. +## How a change reaches the machines + +- **delivery** — one commit in one repository on its way to the machines, from its pull request's head being + announced to delivered, failed, superseded or stopped: one pull request, one status, one note + ([ADR 0239](../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)). + Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`, `held`, + and the final four). Replaces **release plan** as a concept: the controller's plan is now the *walk* of one + delivery's trunk commit across machines, its record and nothing more. Not "change" (a diff), not + "pipeline" (to-be 10 retires it), not "deployment" (one stage of it). +- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's + repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One level: + a group holds deliveries, never groups. Its state is derived from its members, never set. A lone delivery + needs none. +- **delivery plan** — what a delivery does to the mesh: its build plan (the modules it moves and their + dependents, in tiers), its deploy plan (per machine, what it receives and what waits for a person) and its + verdict (the composed machines, the replays). Computed by the controller's planner from a diffset. ADR 0238 + called it the **change plan**; that word is retired. +- **mesh-delivery** — the module that owns deliveries and delivery groups, holding the mesh-scoped seat of the + same name. It records and decides; the controller sends, judges and rolls back when it is asked. +- **walk** — the controller's sending of one trunk commit's builds across machines, tier by tier, one machine + first and judged at the gate (ADR 0236). A primitive the delivering stage asks for, not an object anyone + manages. + ## How this page is kept A new name for an existing thing lands here first, in the same change that introduces it in code. A diff --git a/02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md b/02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md index 2573feea..932a9a5a 100644 --- a/02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md +++ b/02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md @@ -9,6 +9,13 @@ extends: 02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-t # 236. A build is judged on its first machine and put back by something other than itself, and so it rolls out unattended +> **The mechanism changed — 2026-10-06, by [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).** The gate, the rollback, the +> witness, the policy and the bus step stand as decided here. What moved: a **release plan** is no longer an +> object of its own. A plan is the *walk* of one delivery's trunk commit, and the backlog walk of §4a credits +> each build it carries to the delivery that published it. While `mesh-delivery` is held, a walk that moves +> no core module starts on that module's word, and *held* is a delivery's state, released by a person through +> `mesh-delivery`'s `release`. These are the gate rules ADR 0239 applies when a member of a delivery group fails. + ## Context **The operator said "start phase 4"** of [to-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) diff --git a/02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md b/02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md index 310ac007..9ab33b5d 100644 --- a/02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md +++ b/02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md @@ -11,6 +11,13 @@ extends: 02-DECISIONS/0237-a-change-is-judged-against-the-mesh-that-runs-before- # 238. A commit is the build at hand: one commit, one change plan, checked off the trunk and published only on it +> **Superseded in part — 2026-10-06, by [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).** Decision 6's state machine is +> built as a **delivery**'s, owned by the module `mesh-delivery` rather than by the controller. Its states are +> renamed (`releasing` → `delivering`, `done` → `delivered`) and a delivery group sits above it. Decision 7's +> note and status link are written by the forge's holder when it hears a delivery's transition, and the link +> leads to the delivery's view on the pull request. The *change plan* is called the **delivery plan**. Decisions +> 1 to 5 stand as decided here. + ## Context [ADR 0237](0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md) put a diff --git a/02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md b/02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md new file mode 100644 index 00000000..3497e7e4 --- /dev/null +++ b/02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md @@ -0,0 +1,314 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-06 +deciders: jochen +reconstructed: false +supersedes-in-part: + - 0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md +extends: 02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md +--- + +# 239. A delivery is owned by the mesh-delivery module and runs from commit to delivered + +## Context + +[ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md) +made the commit the build at hand and gave it one change plan, and decided (its decision 6) that the plan is a +state machine in one table. It did not say who owns that machine, and nothing built it. On 2026-10-06 the +operator decided that a new module owns it, named it `mesh-delivery`, and anchored it in +[to-be 10](../03-DESIGN/01-to-be/10-delivery.md), whose title is *Modules and delivery*. + +What the day showed: + +- **"Did my change go out?" had no owner.** The forge's holder knew the pull request, the controller knew the + merge's plan, the build seat knew the builds, the gate knew the first machine, and the hand-act log knew + the 47 pushes made by hand. A person joined those five records by hand to answer for one commit. +- **The release plan's lifecycle was implicit.** `building`, `rolling`, `done`, `failed`, `superseded`, and + the release backlog's `held` were strings in `inventory.Plan.State` and in the note beside it. No table said + which state may follow which. A plan could be closed by hand, by healer H2, by a newer merge, or by a gate. + Each did it in its own code, and none said so on the forge. +- **Nearly every change crossed repositories, in an order that was not written down.** A catalogue manifest + used a field only a newer controller parses, so the controller had to go first. The node-engine's genesis + lock had to mirror the controller's grants. A toolchain image had to be built before the builds that use it. + Each order was known to the person merging and to nothing in the mesh. A wrong order failed at + registration (version skew) or on a machine. +- **The controller is the one component that cannot be judged by itself.** Every orchestration put in it + adds to what has to be right before anything can be repaired. The witness ([ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) + §5) exists because of that. + +To-be 10 says *"There is no pipeline as a state machine. No stage list something can be omitted from, and +no run to lose."* That was written against a pipeline that decides what to build. It still holds: what is +built is decided by comparing source with artifacts, and nothing here changes that. What it lacked is a +record of one commit's journey through the mesh that has an owner. That record must stay durable when the +controller is replaced, and it must refuse an impossible step instead of drifting into one. + +**Checked against GENESIS.** *Anything requiring a human to notice it will be noticed late*: a person joining +five records is that sentence. *Failure must be loud*: a delivery stopped half-way, with nothing saying where, +is the silent failure the core exists to end. Nothing here conflicts with GENESIS. + +## Considered Options + +**Who owns a delivery.** + +1. *The controller, as ADR 0238 decision 6 left it implicitly.* Rejected. The controller would own the + delivery of the controller. Its states would sit in the store the controller migrates. Every new rule about + order, holds and groups would be one more thing the component that cannot be judged by itself must get + right first. +2. **A module, `mesh-delivery`, holding a mesh-scoped seat of the same name.** Chosen. One owner per thing. + The controller stays the owner of what it already owns: machine declarations, bus objects, grants, + registration and the gate's primitives. The delivery is the new module's. + +**What it is called.** *Change* was rejected: it is the forge's and git's word for a diff, and it names the +input, not the journey. *Upgrade-planner* was rejected: `upgrade` already names a module's policy (ADR 0236 +§4), and the thing is more than a planner. *Release* was rejected: a release plan is one stage of the journey, +the stage this decision folds in, and keeping the word would keep two concepts. *Pipeline* was rejected +because to-be 10 retires it. **Delivery** is to-be 10's word for the whole of it. + +**What a delivery is.** + +1. *A merge.* Rejected: a pull request is checked long before it merges, and a merge's commit is not the one + the forge shows the verdict on. +2. *A group of commits across repositories, as the unit.* Considered on the day and replaced. A group as the + only object has no one commit status, no one note and no one pull request. Everything the forge shows is + per commit. +3. **One commit in one repository, and a delivery group of deliveries one level above it (the composite).** + Chosen. A delivery is 1:1 with git and the forge: one pull request, one head commit, one status, one note. + A group holds deliveries and never groups. A lone delivery needs no group. + +**How a delivery joins a group.** + +1. *By timing* (merged close together). Rejected: guessing is the fault being removed. +2. *By a trailer, a field in the pull request, or a verb.* Rejected as the mechanism. Each is a second thing a + person must keep in step with the branch they already named, and a verb is a hand act on every + cross-repository change. +3. **By the pull request's head branch name, across the mesh's repositories.** Chosen. + [Playbook 07](../00-META/process/07-feature-branches.md) already requires one feature to be one branch + name in every repository it touches. The mesh reads the name the person already gave. Two or more open + pull requests with the same head branch form one group. A pull request whose branch name is used in no + other repository is a lone delivery. To keep a change out of a group, rename its branch. + +**Dependencies between separately planned deliveries (`needs:`).** Dropped. A delivery that depends on another +is in the same feature and belongs in its group. A dependency on a delivery already delivered needs nothing. +A second linking mechanism would be a second graph to keep acyclic and to show. + +**Where the walk across machines runs.** + +1. *Ported into mesh-delivery*: the module asks for each tier's build, picks the first machine, asks for one + gated send, polls the judgement and asks for the rest. Rejected for now. The walk holds the fixes of issues + 214, 219, 249, 254, 256, 280 and 281. The bootstrap rule below requires the controller to keep a walk for + its own updates and for mesh-delivery's. Porting it means two walks, and one of them would be less tested. +2. **The walk of one trunk commit across machines stays the controller's primitive, and mesh-delivery decides + when it may start, records each step, and stops it.** Chosen. Sending, the gate and the rollback are the + primitives the controller keeps. The walk is their sequence for one commit, as `push ` is a + person's sequence of one. Everything above that sequence moves to mesh-delivery: whether and when, in what + order, waiting for whom, superseded by what, stopped by whom, and the record. + +**Where a commit's check is triggered.** + +1. *mesh-delivery hears every pull request and asks the controller to check it.* Rejected. A mesh-delivery + that is down would check nothing, including the pull request that fixes mesh-delivery, and that is a mesh + that cannot be fixed. +2. **The controller keeps answering every pull request's head itself (ADR 0238 decision 4), and mesh-delivery + records the verdict as the delivery's `checked` and asks only for what the controller cannot know: the + group's composed check.** Chosen. + +**Where a delivery is kept.** + +1. *A database from the store's provider.* Rejected. It would make mesh-delivery depend on a provider whose + policy is `record` (ADR 0236 §4) and whose restart drops every consumer, adding a second dependency to the + bus mesh-delivery cannot work without anyway. Its queries are a few dozen keys a day. +2. **Key-value state the module declares, on the bus ([ADR 0201](0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md)).** + Chosen. Current state per delivery and per group. History is the events, the git notes and a bounded list + of transitions in each delivery. The seat has one holder, so the state has one writer. + +## Decision + +**1. Three things, one owner.** + +- **A delivery** is one commit in one repository: a pull request's head, or a commit that reached the trunk + without one. Its id is `/@`. It carries its **delivery plan**, + the ADR 0238 change plan under its new name: the **build plan** (the modules it moves, their dependents in + tiers, a move or not by source fingerprint), the **deploy plan** (per machine, what it receives in that + order, what waits there for a person, the steps that are not an ordinary send), and the **verdict** (the + composed machines and the replays). It also carries its transitions, its per-machine steps and, once + merged, the commit it landed on the trunk as. +- **A delivery group** is two or more deliveries with the same head branch name in the mesh's repositories. + It has one level, an order among its members, and a state derived from theirs, never set on its own. +- **mesh-delivery** is the module that owns both. It holds the mesh-scoped seat `mesh-delivery`, one holder, + in the controller's compiled seat set, and serves the seat's verbs: `deliveries`, `show`, `groups`, + `what-if`, `stop`, `release`, `recheck`, `stalled`, `close`, `table`. + +**2. A delivery is a state machine, one compiled table.** Each row of the table is a transition: a from-state, +an event, a to-state and a guard. Any transition not in the table is refused and the refusal is said. A test +walks the table. Each state carries its bound and what healer H2 may do once the bound has passed. + +| From | Event | To | Guard | +|---|---|---|---| +| proposed | checked | checked | the verdict names this commit | +| checked | accepted | ready | the gate passed or warned; the repository's own check did not fail | +| checked | refused | rejected | the gate failed or could not run | +| rejected | recheck | proposed | a person asked, or the head was announced again | +| ready | recheck | proposed | a person asked, or its group changed | +| proposed, checked, ready, rejected | new head | superseded | a newer head of the same pull request | +| proposed, checked, ready, rejected | closed | stopped | the pull request closed unmerged | +| ready | merged | published | the commit is on the trunk its modules follow; its first tier's builds are registered | +| proposed, checked, rejected | merged | held | merged without a passing check: waits for a person | +| published | go | delivering | the order of its group allows it; nothing holds it | +| published | hold | held | the release backlog, a bus step, a module whose policy records, or a group member before it that did not deliver | +| held | release | delivering | a person's decision, with why | +| delivering | done | delivered | every machine of its deploy plan passed or was left as its policy says | +| delivering | gate failed | failed | a first machine's gate failed; what it carried was put back (ADR 0236 §3) | +| published, delivering | superseded | superseded | a newer delivery to the same trunk took over its walk (ADR 0218) | +| any state not final | stop | stopped | a person, with why | + +`delivered`, `failed`, `superseded` and `stopped` are final. A delivery never goes back to a state before +`published` once it has left one. Inside `delivering`, each machine has its own row in a second table: +`sent` → `judging` → `passed` | `failed`, `failed` → `rolled-back`. These are read from the walk's own +record (rule 7), never guessed. + +**3. The trunk rule ([ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md) +decision 1) is the guard on `published`.** A commit off the trunk can reach `ready` and no further. Anything +built to check it is built in a scratch namespace of the artifact store that registration refuses. Only a +commit on the trunk is published, and only a published delivery is delivered. + +**4. A delivery group.** + +- **Membership** is the head branch name (above). A group that has started delivering is closed: a new pull + request with the same branch name is a lone delivery, and the reason is given. +- **Order** is declared and inferred. It is declared by a line `after: ` in a member's pull + request description. It is inferred by the controller's planner, which holds the graph, through the verb + `delivery-order`, from four rules: + - a member that moves a module goes before a member whose modules are built by it or stand on it + (toolchains and the build agent before their dependents); + - a member that moves the controller goes before a member that changes any module's manifest (version skew: + the newer controller parses what the newer manifest says); + - a member that moves the node-engine goes before the controller's member, because the witness reads + nothing the controller does not yet grant and a controller sends nothing an older engine refuses (ADR 0236 + *Rollout order*); + - otherwise, the order of the repositories' names, so the order is the same on every reading. + A declared order that contradicts an inferred one is a cycle. The group is rejected, naming the cycle. +- **Checked** means all members' heads are composed together as one future state of the mesh: one gate over + every machine with every member's definitions, judged by the group's own controller when a member moves it. + The verdict is posted on every member's head as `mesh/delivery-group`. The group is `ready` only when that + verdict passed and every member is `ready`. +- **Delivering** follows the order, member by member. The next member starts only when the one before is + `delivered`. +- **A failed member stops the group cleanly.** The members after it are `stopped`, and the reason names the + member and why. **What was already delivered stays.** Each earlier member passed its own gate on its first + machine and its own judgement on every machine. A later member's failure says nothing about it, and putting + back a build that passed would be a rollback with no verdict behind it. **The failed member's own builds are + put back at its gate, once (ADR 0236 §3)**, by the controller as today. A person releases what was stopped + once the fix is merged, which makes a new delivery. +- The group's state is derived: `checking` while any member is `proposed` or the composed check is + outstanding; `rejected` when any member or the composed check is; `ready`; `delivering`; `delivered` when + all are; `failed` or `stopped` when a member is. Nothing writes a group's state except this derivation. + +**5. Every transition is said four ways, and the owner of each says it.** mesh-delivery keeps the transition +in its state before anything else. It emits it as the event `mesh-delivery.transition`, with no secret and no +address. It asks the forge's holder to append it to the commit's note under `refs/notes/mesh-plan`: predicted +on the head, executed on the commit it landed as. It also asks the forge's holder to reflect it in the +delivery's view, one comment on the pull request kept current, which every status of the commit links to as +its `target_url`, `mesh/merge-gate` among them. The forge's holder writes notes as the mesh's own forge +account, appending: running twice adds nothing. + +**6. The boundary.** mesh-delivery never sends to a machine, never writes a declaration, a bus object or a +grant, and never registers a build. It asks: + +- **the controller**, through the seat's verbs: `delivery-plan` (the planner's one answer for a diffset), + `delivery-order`, `delivery-check` (compose a group's heads), `deliver` (let one published trunk commit's + walk start), `delivery-stop` (end a walk, with why), `delivery-walks` (the walks and their steps); +- **the build seat**, through the controller's `delivery-check`. A group's check is a check ask like any + other; +- **the forge's holder**, through events it consumes: the notes and the delivery's view. + +The controller keeps composition, registration, the trunk rule, sending, the gate, the rollback and the walk +of one commit's tiers across machines, and says each step of a walk as the event `plan-moved`. + +**7. The walk's record is the controller's, and the delivery's state is mesh-delivery's.** A plan the +controller keeps is from now on the walk of one delivery, read by the repository and commit it walks. Its +`building` and `rolling` are the walk's progress, and its end is reported to the delivery, which decides the +delivery's state from it. **A release plan stops being a concept of its own.** The backlog walk of ADR 0236 +§4a is the walk of every build that waits on its machines, and each build it carries is credited to the +delivery that published it. `plans` stays as the walk's record. `deliveries` is what a person reads. + +**8. The bootstrap rule.** mesh-delivery cannot gate or deliver itself, and the mesh must never depend on it to +be repaired: + +- A walk that moves **the controller, the node-engine, the node tools, the bus or mesh-delivery** runs on the + controller's built-in path: started by the merge, judged at the gate, witnessed on the machine (ADR 0236 + §5), the previous build kept and switched back to. mesh-delivery records it and does not start it. +- Any other walk, **while the seat `mesh-delivery` has a holder on record**, is built and published by the + merge and then waits for mesh-delivery's `deliver` before its first send. With no holder on record, the + controller starts it itself, exactly as before this record. +- **If mesh-delivery is down**, a walk waiting for it waits. Past its bound, the stall is said as the condition + `stalled` with the remedy. A person delivers by hand: `push `, which carries what waits there + (ADR 0236 §4a), or `plans go --why`, which gives the walk a person's word in place of + mesh-delivery's. Nothing in the controller waits for mesh-delivery to repair the controller or + mesh-delivery. +- A pull request's own check is triggered by the controller, not by mesh-delivery (above). The fix for a + broken mesh-delivery is checked, merged and delivered without it. + +**9. Durable across restarts.** The state is mesh-delivery's key-value state. A restarted holder reads it back +before it takes an event. It then reconciles from the controller's `delivery-walks` and the forge's open pull +requests, so an event missed while it was down is found by comparison, never lost. A state held past its +bound is listed by `stalled`. The controller's self-check reads that list and raises +`delivery..stalled`. **Healer H2 works from the table**: it closes a stalled delivery only by a +transition the table allows for that state (`superseded` when a newer delivery took over, `delivered` when +the walk's record says done), through mesh-delivery's `close`. + +**10. The switch, without a gap.** Before mesh-delivery is assigned, the controller walks every merge as it +does today. Once the seat has a holder on record, a walk that starts after that waits for its word. Walks +already open finish where they are: the holder adopts each open plan as a delivery in `delivering`, and an open +release plan's carried builds are credited to their deliveries, or listed as carried with no delivery. Only a +walk's start reads whether a holder is on record, so no walk is ever both started by the controller and +waiting for mesh-delivery. Unassigning mesh-delivery returns the mesh to the controller's own path. + +## Consequences + +- **"Did my change go out?" is one verb.** `deliveries` and `show ` answer it from the pull request + to the last machine. The pull request's statuses link to the same answer, and the commit's note keeps it + after the state is pruned. +- **A cross-repository change merges in an order the mesh enforces.** A member that would cause version skew + waits for the member it depends on. A group that cannot be ordered is rejected before anything merges. +- **A merge no longer sends anything until mesh-delivery says so** while it is held, except the core and + mesh-delivery itself. A down mesh-delivery means walks wait, said as a condition. A person can still deliver + by hand. +- **The controller gains six verbs and one event and loses no primitive.** Its release planner's lifecycle, + supersession and hold decisions remain its code until mesh-delivery is held. After that they run only on the + built-in path. Removing them is a later decision, once mesh-delivery has delivered for a while. +- **The forge's holder gains the notes and the view.** It needs git on its machine for notes, declared as a + package of its module. +- **What is not built by this record**: porting the walk itself into mesh-delivery (option 1 of *where the walk + runs*); a web view of deliveries beyond the pull request's comment; requiring `mesh/delivery-group` on the + trunks, which is the operator's setting through the forge module's protection tool. + +## How it is checked + +| Rule | Checked by | +|---|---| +| a transition not in the table is refused; the table is walked | mesh-delivery's test that walks every row and every pair not in the table; the machine-step table walked the same way | +| a pull request's head goes proposed → checked → ready, or rejected | mesh-delivery's test from a `pull.updated` and a `checked` | +| a merge publishes, delivers and is delivered; a failed gate fails it; a newer commit supersedes it | mesh-delivery's tests driven by the controller's `plan-moved` snapshots: done, gate failed with rollback, superseded | +| the trunk rule | mesh-delivery's test that an off-trunk commit never reaches published; the controller's publish-rule test (ADR 0238) | +| a group's membership, order, composed check and clean stop | mesh-delivery's group tests: by branch name, declared and inferred order, a cycle rejected, ready only when all are, a failed member stopping those after it and leaving those before it delivered | +| a restart resumes | mesh-delivery's test that a holder restarted over the same state resumes each delivery and reconciles a walk it missed | +| the bootstrap rule | the controller's tests: a walk moving a core module or mesh-delivery never waits; another waits only while the seat has a holder on record; `plans go` and `push` deliver with mesh-delivery down | +| the inferred order | the controller's `delivery-order` test over the real dependency relation: toolchain first, controller before a manifest change, node-engine before controller | +| every transition is persisted, said, noted and reflected | mesh-delivery's test that each transition is put before it is emitted; the forge module's note test (append-only, idempotent) and view test (statuses carry its link) | +| the seat is in the set and its holder is one | the controller's seat test; the catalogue's manifest test of mesh-delivery | +| live | the next merge with mesh-delivery held waits for `deliver`, and `deliveries` shows it from head to delivered; `git log --notes=mesh-plan` shows the note on its merge commit | + +## References + +- [ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md) + decision 6, which this record gives an owner and builds; + [ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) + §3, §4a and §5, whose release plan becomes the delivering stage and whose gate rules decide what stays; + [ADR 0218](0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md), + [ADR 0201](0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md), + [ADR 0231](0231-a-healer-acts-on-what-observation-raised-and-only-observation-says-it-worked.md). +- [To-be 47](../03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md), the design; + [to-be 10](../03-DESIGN/01-to-be/10-delivery.md), the delivery it carries out; + [playbook 07](../00-META/process/07-feature-branches.md), whose branch name is a group's membership. +- mesh-controller, mesh-catalog and mesh-host: the branches `feat/mesh-delivery`. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index e74b7092..d7045644 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -207,6 +207,7 @@ python3 00-META/checks/index.py fail if stale - **0236** — [A build is judged on its first machine and put back by something other than itself, and so it rolls out unattended](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) - **0237** — [A change is judged against the mesh that runs, before it merges, on the build seat](0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md) - **0238** — [A commit is the build at hand: one commit, one change plan, checked off the trunk and published only on it](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md) +- **0239** — [A delivery is owned by the mesh-delivery module and runs from commit to delivered](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md) ### Its tiers, from the bottom up diff --git a/03-DESIGN/01-to-be/10-delivery.md b/03-DESIGN/01-to-be/10-delivery.md index 94f98fcd..03020489 100644 --- a/03-DESIGN/01-to-be/10-delivery.md +++ b/03-DESIGN/01-to-be/10-delivery.md @@ -5,8 +5,9 @@ code: - mesh-controller internal/builder - mesh-controller cmd/mesh-controller (build, build --behind, push, status) - mesh-controller internal/inventory/builds.go -updated: 2026-09-29 +updated: 2026-10-06 decisions: + - 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md - 02-DECISIONS/0090-a-failure-that-repeats-is-said-to-be-stuck.md - 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md - 02-DECISIONS/0010-delivery.md @@ -100,7 +101,9 @@ That is the same shape the host uses on a machine, one layer up: | the host | machine state | declarations | **There is no pipeline as a state machine.** No stage list something can be omitted from, and no -run to lose. +run to lose. *What* is built stays decided by this comparison. *One commit's journey* through the mesh is a +delivery with a state table of its own: it records and orders that journey and never decides what to build +(below, *A delivery has an owner*). ### An artifact is current, or it is not @@ -257,3 +260,26 @@ A verification mechanism was drafted for this and withdrawn. It would have repor would not have prevented it, and the part of it that was hard — deciding which network position to check from — existed only because the rule was wrong. Whether the mesh should check that a grant works is still open, in issue 145; it is not the remedy for a configuration error. + +## A delivery has an owner + +*2026-10-06, by [ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md); +designed in full in [to-be 47](47-delivery-from-commit-to-delivered.md).* + +The comparison above decides what is built, and that does not change. What it never had is an owner for +**did my change go out?** for one commit: from its pull request's head through its check, the merge, its +builds and every machine's gate. Five records answered it in parts, and a person joined them. + +**A delivery is one commit in one repository, and the module `mesh-delivery` owns it.** Its states are one +compiled table: proposed, checked, ready or rejected, published, delivering, held, and four final states. +A transition the table does not hold is refused. Two or more deliveries that share a head branch name are a +**delivery group**, delivered in an order the planner infers and the pull requests may declare, and checked +together as one future state of the mesh. + +**This answers the first open question above.** A fit artifact does not declare itself. Off the trunk it is +only checked. On the trunk it is published, and its walk across the machines starts when its delivery says +so: by itself for the core, which cannot wait for anything to be repaired, and by `mesh-delivery` for +everything else while that module is held. A person can always deliver by hand. + +The controller keeps the comparison, the planner, the gate, sending and rollback. `mesh-delivery` keeps the +record and the order, and asks. diff --git a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md index 6daf0bfb..e88afbad 100644 --- a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md +++ b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md @@ -737,6 +737,10 @@ the status's link to the kept plan; the notes under `refs/notes/mesh-plan`; chec namespace (the check builds no module today, so there is none to keep apart yet); the statuses made required — prepared as calls of the forge module's tool, applied by the operator. +**Given an owner 2026-10-06** ([ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)): the state machine, the +notes and the status's link are built as a delivery's, owned by the module `mesh-delivery`. They are designed +in [to-be 47](47-delivery-from-commit-to-delivered.md), and their progress is tracked there. + ## What is not decided here - The bus as a cluster of three, to upgrade it live. diff --git a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md new file mode 100644 index 00000000..223ee2a4 --- /dev/null +++ b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md @@ -0,0 +1,177 @@ +--- +layer: to-be +status: designed +code: [] +updated: 2026-10-06 +decisions: + - 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md + - 02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md + - 02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md + - 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md +--- + +# 47 — Delivery: from a commit to delivered + +**A delivery is one commit in one repository, from the moment its pull request's head is announced to the +moment every machine of its deploy plan runs it, or to the moment it fails, is superseded or is stopped. A +delivery group is two or more deliveries that share a head branch name, delivered as one unit in order. Both +are owned by one module, `mesh-delivery`, which records every step and asks the controller for each act.** +([ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).) + +This is the delivery of [to-be 10](10-delivery.md) carried out. To-be 10 says what is built: the comparison +of source and artifacts decides, and nothing here changes that. This design says who answers for one +commit's journey and in what order the steps are allowed. + +## The parts + +``` + forge's holder ──pull.updated / pull.merged──► mesh-delivery ◄──plan-moved / checked── controller + (statuses, the view, ▲ (deliveries, groups, (planner, gate, + the notes) │ the table, its state) registration, + ▲ │ │ sending, walk) + └──── transition ────────┴───────────────────────┘── verbs: delivery-plan, -order, -check, + deliver, delivery-stop, delivery-walks +``` + +- **The controller** stays the only writer of what it owns: machine declarations, bus objects, grants, + registration. It keeps the primitives: the planner (one function, ADR 0238 decision 3), the gate, the trunk + rule, a gated send, the rollback and the walk of one trunk commit's tiers across machines. It still answers + every pull request's head with its check itself, so a check never depends on mesh-delivery. +- **mesh-delivery** is a Go bundle holding the mesh-scoped seat `mesh-delivery`. It owns the delivery, the + group, the state table and their state, and nothing else. +- **The forge's holder** sets a commit's statuses, keeps the delivery's view as one comment on the pull + request, and appends to the commit's note under `refs/notes/mesh-plan`. +- **The build seat** builds and checks, asked by the controller as before. + +## A delivery + +Its id is `/@`. It holds: + +- the pull request (number, base, head branch, description), or none for a commit that reached the trunk + without one; +- its **delivery plan**: build plan, deploy plan and verdict, as the controller's planner computed them for + its diffset (the change plan of ADR 0238 under the glossary's name); +- its **state**, its **transitions** (the last hundred, each with when, the event, from, to and why), and its + **machine steps** while delivering; +- once merged, **the commit it landed on the trunk as**, and the walk that delivers it. + +## The state table + +One table, compiled into mesh-delivery, read by its code, its test, its `table` verb, the controller's +self-check and healer H2. Each state has a **bound**: how long it may be held before `stalled` lists it. +Each state also says **what H2 may do** after the bound, which is only ever a transition the table already +allows. + +| State | Means | Bound | H2 after the bound | +|---|---|---|---| +| proposed | a head was announced and is being checked | 1 hour | nothing: the check's own watchdog (S6) speaks | +| checked | a verdict arrived | 1 minute | nothing: it is decided at once | +| ready | it passed; it waits to be merged | none | — | +| rejected | it failed its check | none | — | +| published | merged on the trunk, its first tier registered; it waits for its turn or its word | 30 minutes | `superseded` when a newer delivery to the same trunk took over its walk | +| held | it waits for a person | 1 day | nothing: it is the operator's | +| delivering | its walk runs | 2 hours | `delivered` when the walk's record says done; `superseded` when a newer walk closed it | +| delivered, failed, superseded, stopped | final | — | — | + +The transitions are those of ADR 0239 decision 2. A delivery that is merged while not `ready` goes to +`held`. Its check did not pass, and only a person decides that it goes on. + +**The machine steps.** While a delivery is `delivering`, each machine its walk reaches has a row with the +module and the build: `sent` (the first machine's send, or the rest's), `judging` (the gate's healthy +readings so far), `passed`, `failed`, `rolled-back`. They are read from the walk's own record in the +controller's `plan-moved` event, never inferred from time. + +## A delivery group + +- **Membership**: the pull request's head branch name, across the mesh's repositories, among open pull + requests. Two or more make a group. A group that has started delivering takes no new member. +- **Order**: `after: ` lines in a member's description, and the controller's `delivery-order` + over the graph (ADR 0239 decision 4). The plan shows the order and why each pair is ordered: *declared*, + *built by*, *version skew*, *engine before controller*, *by name*. +- **Check**: the members' heads composed together, one gate over every machine with all their definitions. It + is asked by mesh-delivery through `delivery-check` when the group forms or a member's head moves, and + posted on every member's head as `mesh/delivery-group`. +- **Delivering**: in order. A member waits in `published` until the one before is `delivered`. +- **A failed member**: the members after it are `stopped`, naming it. The members before it stay delivered. + Its own builds were put back at its gate. + +A group's state is derived from its members and its composed check, and is shown, never stored apart from +them. + +## What every transition does + +In this order, each by its owner: + +1. **kept**: mesh-delivery puts the delivery in its state (`deliveries`, one key per delivery; `groups`, one + key per group). Nothing is said before it is kept; +2. **said**: the event `mesh-delivery.transition`, with the delivery's id, from, to, event, why and plan + summary, and no secret or address; +3. **noted**: the forge's holder, hearing it, appends a line to the commit's note: the head while the + delivery is off the trunk, and the commit it landed as once it is on it. It appends as the mesh's own forge + account, and a line already there is not appended again; +4. **reflected**: the forge's holder edits the delivery's view, one comment on the pull request with the + state, the plan, the machine steps and the group, and sets every status of the commit with the view as its + `target_url`. + +## The verbs + +**mesh-delivery's seat** (served by its holder; the first five read): + +| Verb | Answers or does | +|---|---| +| `deliveries` | every delivery not final, and the final ones of the last day, one line each; filtered by state, repository or group | +| `show` | one delivery or group whole: plan, transitions, machine steps, order, the walk | +| `groups` | every group with its members in order and its derived state | +| `what-if` | the delivery plan a diffset would have, asked of the controller's planner, kept nowhere | +| `table` | the state table and the machine-step table, with bounds and H2's repairs | +| `stalled` | every delivery held past its bound, with the transition H2 may take | +| `recheck` | a rejected or ready delivery back to proposed, with why | +| `release` | a held delivery to delivering, with why: a person's word | +| `stop` | any delivery not final to stopped, with why; its walk is ended through the controller | +| `close` | H2's only verb: the transition the table names for a stalled delivery, refused otherwise | + +**The controller's verbs for it** (on the mesh-controller seat): `delivery-plan`, `delivery-order`, +`delivery-check`, `deliver`, `delivery-stop`, `delivery-walks`. A person has `plans go --why`, which +does what `deliver` does under their name. + +## The store + +Key-value state on the bus, declared by the module (ADR 0201): `deliveries` and `groups`. One writer, the +seat's holder, whose code changes state from a single goroutine. A final delivery is kept for thirty days and +then removed. Its note on the commit keeps its record after that. The bus's own snapshot (ADR 0235) backs it +up. + +## The bootstrap + +- A walk that moves the controller, the node-engine, the node tools, the bus or mesh-delivery is the + controller's own, started by the merge and witnessed on the machine. mesh-delivery records it. +- Any other walk waits for `deliver` only while the seat has a holder on record. With none on record, the + controller starts it as it always did. +- With the holder down, waiting walks are `stalled` past their bound, said with the remedy: `push ` + or `plans go`. + +## The switch + +1. The controller's change (the seat, its verbs, `plan-moved`, the wait for a holder's word, `plans go`) is + merged and rolls out as the core does. With no holder on record it behaves exactly as before. +2. The catalogue's change (mesh-delivery, the forge holder's notes and view) is merged. mesh-delivery is built + and published. +3. A person assigns mesh-delivery to the control node. The seat now has a holder on record. Walks that start + from then on wait for its word. Walks already open finish on their own and are adopted as deliveries in + `delivering`. +4. To step back, unassign it. Waiting walks are then started by the controller at its next pass. + +## Phases + +| Phase | Delivers | Done when | +|---|---|---| +| A — the owner | mesh-delivery with its table, state, verbs, events and adoption; the controller's seat, verbs, `plan-moved` and the wait; the forge holder's view and notes; the group's order and composed check | the tests of ADR 0239's *How it is checked* pass; a merge on the live mesh with mesh-delivery held is delivered by it and shown by `deliveries` | +| B — the conditions | the self-check reads `stalled`; `delivery..stalled`; H2 closing by the table | a delivery stalled on purpose is raised and closed by H2 through `close` | +| C — the walk itself | the per-tier walk moved into mesh-delivery behind single-send verbs, the controller's planner kept only for the built-in path | decided by its own record once Phase A has delivered for a while | + +## What is not decided here + +- A web view of deliveries beyond the pull request's comment. +- Whether `mesh/delivery-group` is required on the trunks. That is the operator's setting, through the forge + module's protection tool. +- Moving the per-tier walk out of the controller (Phase C). diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 95fcbbfc..d23c596e 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -47,6 +47,7 @@ document is written and this one's status becomes `implemented`. | [`42-the-machines-modules-in-order.md`](42-the-machines-modules-in-order.md) | **In progress.** The order the machines' modules of research 026 and 027 are built and rolled out: every machine's first (sudo, localization, time sync, pacman, logrotate, avahi, systemd, docker, `~/.ssh`, scripts, kernel), then both workstations', then one machine model's; each proven on one workstation before the rest | [ADR 0173](../../02-DECISIONS/0173-the-operators-machine-is-the-meshs-and-a-module-is-what-it-declares.md), [ADR 0182](../../02-DECISIONS/0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md), [ADR 0205](../../02-DECISIONS/0205-software-the-distribution-does-not-package-ships-as-a-pinned-archive-of-the-module.md) | | [`45-a-core-that-cannot-fail-silently.md`](45-a-core-that-cannot-fail-silently.md) | **Designed.** The core says when it is wrong, refuses what is stale or unreadable, heals what it knows, upgrades one machine at a time with a witness that rolls it back, and is checked against the real mesh before merge: the writers and signals tables, the condition store, `doctor`, the minimal output channel, healers, the lease and report order, staged upgrades, the facts snapshot and replays, in six phases | [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md), [ADR 0224](../../02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md), [ADR 0218](../../02-DECISIONS/0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md) | | [`46-the-conversation-with-the-operator.md`](46-the-conversation-with-the-operator.md) | **Designed.** The mesh tells and asks its operator over channels that are holders of two kinded benches, `channel` and `intake`, declaring capabilities from a fixed vocabulary; the router orders them by work context and never lowers the bar; an answer that performs an action is checked and performed by the controller, on a TOTP code or a verified Telegram sender, never on a desk click alone; operator messages addressed or answered by the mesh's responder, untrusted input kept as data, references for what words may not carry, and a recoverable factor; Telegram first, in six phases | [ADR 0234](../../02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md), [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md) | +| [`47-delivery-from-commit-to-delivered.md`](47-delivery-from-commit-to-delivered.md) | **Designed.** A delivery is one commit in one repository, from its pull request's head to every machine; a delivery group is deliveries sharing a branch name, ordered and checked as one future state; both owned by the `mesh-delivery` module with one state table, its state on the bus, every transition said, noted on the commit and shown on the pull request; the controller keeps the planner, the gate, sending and the walk, and the core's own updates never wait for the module | [ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md), [ADR 0238](../../02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md), [ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) | ## Not yet written