novox/hq
6
0
Fork 0
Code Issues Pull Requests 7 Releases Wiki Activity

Merge pull request 'ADR 0239: a delivery is owned by mesh-delivery and runs from commit to delivered' (#156) from design/0239-a-delivery-is-owned-by-mesh-delivery into main

Browse Source
This commit was merged in pull request #156.
This commit is contained in:
mesh-admin
2026-10-06 22:07:18 +00:00
parent 7216ffc825 e1b95d09cd
commit c51a3da5d5
9 changed files with 621 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
00-META/glossary.md
+23
View File
@@ -127,6 +127,29 @@ another — and a mesh you cannot name precisely is a mesh two people describe d
- **invokes** — the manifest word for the tools a module calls, `<module>.<tool>` each or `*` for
every one. A grant on the publish side and nothing else; a module that declares none calls nothing.
## How a change reaches the machines
- **delivery** — one commit in one repository on its way to the machines, from its pull request's head being
announced to delivered, failed, superseded or stopped: one pull request, one status, one note
([ADR 0239](../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)).
Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`, `held`,
and the final four). Replaces **release plan** as a concept: the controller's plan is now the *walk* of one
delivery's trunk commit across machines, its record and nothing more. Not "change" (a diff), not
"pipeline" (to-be 10 retires it), not "deployment" (one stage of it).
- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's
repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One level:
a group holds deliveries, never groups. Its state is derived from its members, never set. A lone delivery
needs none.
- **delivery plan** — what a delivery does to the mesh: its build plan (the modules it moves and their
dependents, in tiers), its deploy plan (per machine, what it receives and what waits for a person) and its
verdict (the composed machines, the replays). Computed by the controller's planner from a diffset. ADR 0238
called it the **change plan**; that word is retired.
- **mesh-delivery** — the module that owns deliveries and delivery groups, holding the mesh-scoped seat of the
same name. It records and decides; the controller sends, judges and rolls back when it is asked.
- **walk** — the controller's sending of one trunk commit's builds across machines, tier by tier, one machine
first and judged at the gate (ADR 0236). A primitive the delivering stage asks for, not an object anyone
manages.
## How this page is kept
A new name for an existing thing lands here first, in the same change that introduces it in code. A
02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md
+7
View File
@@ -9,6 +9,13 @@ extends: 02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-t
# 236. A build is judged on its first machine and put back by something other than itself, and so it rolls out unattended
> **The mechanism changed — 2026-10-06, by [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).** The gate, the rollback, the
> witness, the policy and the bus step stand as decided here. What moved: a **release plan** is no longer an
> object of its own. A plan is the *walk* of one delivery's trunk commit, and the backlog walk of §4a credits
> each build it carries to the delivery that published it. While `mesh-delivery` is held, a walk that moves
> no core module starts on that module's word, and *held* is a delivery's state, released by a person through
> `mesh-delivery`'s `release`. These are the gate rules ADR 0239 applies when a member of a delivery group fails.
## Context
**The operator said "start phase 4"** of [to-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md)
02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md
+7
View File
@@ -11,6 +11,13 @@ extends: 02-DECISIONS/0237-a-change-is-judged-against-the-mesh-that-runs-before-
# 238. A commit is the build at hand: one commit, one change plan, checked off the trunk and published only on it
> **Superseded in part — 2026-10-06, by [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).** Decision 6's state machine is
> built as a **delivery**'s, owned by the module `mesh-delivery` rather than by the controller. Its states are
> renamed (`releasing` → `delivering`, `done` → `delivered`) and a delivery group sits above it. Decision 7's
> note and status link are written by the forge's holder when it hears a delivery's transition, and the link
> leads to the delivery's view on the pull request. The *change plan* is called the **delivery plan**. Decisions
> 1 to 5 stand as decided here.
## Context
[ADR 0237](0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md) put a
02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
+360
View File
@@ -0,0 +1,360 @@
---
topic: the mesh
status: accepted
date: 2026-10-06
deciders: jochen
reconstructed: false
supersedes-in-part:
- 0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md
extends: 02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md
---
# 239. A delivery is owned by the mesh-delivery module and runs from commit to delivered
## Context
[ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md)
made the commit the build at hand and gave it one change plan, and decided (its decision 6) that the plan is a
state machine in one table. It did not say who owns that machine, and nothing built it. On 2026-10-06 the
operator decided that a new module owns it, named it `mesh-delivery`, and anchored it in
[to-be 10](../03-DESIGN/01-to-be/10-delivery.md), whose title is *Modules and delivery*.
What the day showed:
- **"Did my change go out?" had no owner.** The forge's holder knew the pull request, the controller knew the
merge's plan, the build seat knew the builds, the gate knew the first machine, and the hand-act log knew
the 47 pushes made by hand. A person joined those five records by hand to answer for one commit.
- **The release plan's lifecycle was implicit.** `building`, `rolling`, `done`, `failed`, `superseded`, and
the release backlog's `held` were strings in `inventory.Plan.State` and in the note beside it. No table said
which state may follow which. A plan could be closed by hand, by healer H2, by a newer merge, or by a gate.
Each did it in its own code, and none said so on the forge.
- **Nearly every change crossed repositories, in an order that was not written down.** A catalogue manifest
used a field only a newer controller parses, so the controller had to go first. The node-engine's genesis
lock had to mirror the controller's grants. A toolchain image had to be built before the builds that use it.
Each order was known to the person merging and to nothing in the mesh. A wrong order failed at
registration (version skew) or on a machine.
- **The controller is the one component that cannot be judged by itself.** Every orchestration put in it
adds to what has to be right before anything can be repaired. The witness ([ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)
§5) exists because of that.
To-be 10 says *"There is no pipeline as a state machine. No stage list something can be omitted from, and
no run to lose."* That was written against a pipeline that decides what to build. It still holds: what is
built is decided by comparing source with artifacts, and nothing here changes that. What it lacked is a
record of one commit's journey through the mesh that has an owner. That record must stay durable when the
controller is replaced, and it must refuse an impossible step instead of drifting into one.
**Checked against GENESIS.** *Anything requiring a human to notice it will be noticed late*: a person joining
five records is that sentence. *Failure must be loud*: a delivery stopped half-way, with nothing saying where,
is the silent failure the core exists to end. Nothing here conflicts with GENESIS.
## Considered Options
**Who owns a delivery.**
1. *The controller, as ADR 0238 decision 6 left it implicitly.* Rejected. The controller would own the
delivery of the controller. Its states would sit in the store the controller migrates. Every new rule about
order, holds and groups would be one more thing the component that cannot be judged by itself must get
right first.
2. **A module, `mesh-delivery`, holding a mesh-scoped seat of the same name.** Chosen. One owner per thing.
The controller stays the owner of what it already owns: machine declarations, bus objects, grants,
registration and the gate's primitives. The delivery is the new module's.
**What it is called.** *Change* was rejected: it is the forge's and git's word for a diff, and it names the
input, not the journey. *Upgrade-planner* was rejected: `upgrade` already names a module's policy (ADR 0236
§4), and the thing is more than a planner. *Release* was rejected: a release plan is one stage of the journey,
the stage this decision folds in, and keeping the word would keep two concepts. *Pipeline* was rejected
because to-be 10 retires it. **Delivery** is to-be 10's word for the whole of it.
**What a delivery is.**
1. *A merge.* Rejected: a pull request is checked long before it merges, and a merge's commit is not the one
the forge shows the verdict on.
2. *A group of commits across repositories, as the unit.* Considered on the day and replaced. A group as the
only object has no one commit status, no one note and no one pull request. Everything the forge shows is
per commit.
3. **One commit in one repository, and a delivery group of deliveries one level above it (the composite).**
Chosen. A delivery is 1:1 with git and the forge: one pull request, one head commit, one status, one note.
A group holds deliveries and never groups. A lone delivery needs no group.
**How a delivery joins a group.**
1. *By timing* (merged close together). Rejected: guessing is the fault being removed.
2. *By a trailer, a field in the pull request, or a verb.* Rejected as the mechanism. Each is a second thing a
person must keep in step with the branch they already named, and a verb is a hand act on every
cross-repository change.
3. **By the pull request's head branch name, across the mesh's repositories.** Chosen.
[Playbook 07](../00-META/process/07-feature-branches.md) already requires one feature to be one branch
name in every repository it touches. The mesh reads the name the person already gave. Two or more open
pull requests with the same head branch form one group. A pull request whose branch name is used in no
other repository is a lone delivery. To keep a change out of a group, rename its branch.
**Dependencies between separately planned deliveries (`needs:`).** Dropped. A delivery that depends on another
is in the same feature and belongs in its group. A dependency on a delivery already delivered needs nothing.
A second linking mechanism would be a second graph to keep acyclic and to show.
**Where the walk across machines runs.**
1. *Ported into mesh-delivery*: the module asks for each tier's build, picks the first machine, asks for one
gated send, polls the judgement and asks for the rest. Rejected for now. The walk holds the fixes of issues
214, 219, 249, 254, 256, 280 and 281. The bootstrap rule below requires the controller to keep a walk for
its own updates and for mesh-delivery's. Porting it means two walks, and one of them would be less tested.
2. **The walk of one trunk commit across machines stays the controller's primitive, and mesh-delivery decides
when it may start, records each step, and stops it.** Chosen. Sending, the gate and the rollback are the
primitives the controller keeps. The walk is their sequence for one commit, as `push <machine>` is a
person's sequence of one. Everything above that sequence moves to mesh-delivery: whether and when, in what
order, waiting for whom, superseded by what, stopped by whom, and the record.
**Where a commit's check is triggered.**
1. *mesh-delivery hears every pull request and asks the controller to check it.* Rejected. A mesh-delivery
that is down would check nothing, including the pull request that fixes mesh-delivery, and that is a mesh
that cannot be fixed.
2. **The controller keeps answering every pull request's head itself (ADR 0238 decision 4), and mesh-delivery
records the verdict as the delivery's `checked` and asks only for what the controller cannot know: the
group's composed check.** Chosen.
**When a published delivery's builds are registered.**
1. *At the merge, as before, and the walk held back until its turn.* Rejected. A send carries a machine's
whole declaration, and a gated send carries every registered build that waits on its machine
([ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)
§4a). A build registered before its turn would reach the machines with the next send of anything else
there. The group's order would be broken by a send nobody made for it, and a catalogue member registered
ahead of its controller is the version skew the order exists to prevent.
2. **When its walk starts.** Chosen. The merge opens the walk and asks nothing. Its first tier is asked when
the delivery's word comes, and a later tier is asked once the earlier one runs, as before. `published` is
the trunk commit accepted with its walk opened and waiting. Its builds are registered in `delivering`.
**Where a delivery is kept.**
1. *A database from the store's provider.* Rejected. It would make mesh-delivery depend on a provider whose
policy is `record` (ADR 0236 §4) and whose restart drops every consumer, adding a second dependency to the
bus mesh-delivery cannot work without anyway. Its queries are a few dozen keys a day.
2. **Key-value state the module declares, on the bus ([ADR 0201](0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md)).**
Chosen. Current state per delivery and per group. History is the events, the git notes and a bounded list
of transitions in each delivery. The seat has one holder, so the state has one writer.
## Decision
**1. Three things, one owner.**
- **A delivery** is one commit in one repository: a pull request's head, or a commit that reached the trunk
without one. Its id is `<owner>/<repository>@<commit, twelve characters>`. It carries its **delivery plan**,
the ADR 0238 change plan under its new name: the **build plan** (the modules it moves, their dependents in
tiers, a move or not by source fingerprint), the **deploy plan** (per machine, what it receives in that
order, what waits there for a person, the steps that are not an ordinary send), and the **verdict** (the
composed machines and the replays). It also carries its transitions, its per-machine steps and, once
merged, the commit it landed on the trunk as.
- **A delivery group** is two or more deliveries with the same head branch name in the mesh's repositories.
It has one level, an order among its members, and a state derived from theirs, never set on its own.
- **mesh-delivery** is the module that owns both. It holds the mesh-scoped seat `mesh-delivery`, one holder,
in the controller's compiled seat set, and serves the seat's verbs: `deliveries`, `show`, `groups`,
`what-if`, `stop`, `release`, `recheck`, `stalled`, `close`, `table`.
**2. A delivery is a state machine, one compiled table.** Each row of the table is a transition: a from-state,
an event, a to-state and a guard. Any transition not in the table is refused and the refusal is said. A test
walks the table. Each state carries its bound and what healer H2 may do once the bound has passed.
| From | Event | To | Guard |
|---|---|---|---|
| (none) | announced | proposed | a pull request's head the forge announced |
| (none) | appeared | held | a trunk commit whose walk waits, with no pull request known: merged before the seat was held, or pushed to the trunk |
| (none) | adopted | delivering | a walk already running: at the switch, or on the controller's own path |
| proposed | checked | checked | the verdict names this commit |
| checked | accepted | ready | the gate passed or warned; the repository's own check did not fail |
| checked | refused | rejected | the gate failed or could not run |
| rejected | recheck | proposed | a person asked, or the head was announced again |
| ready | recheck | proposed | a person asked, or its group changed |
| proposed, checked, ready, rejected | new head | superseded | a newer head of the same pull request |
| proposed, checked, ready, rejected | closed | stopped | the pull request closed unmerged |
| ready | merged | published | merged on the trunk its modules follow, and its walk opened; or nothing for a walk to move |
| proposed, checked, rejected | merged unchecked | held | merged without a passing check; a verdict known with it is taken first |
| published | done | delivered | nothing for a walk to move |
| published, ready | hold | held | merged, and its group's composed check did not pass for the heads that merged, or no walk was opened for it within ten minutes |
| published | go | delivering | its walk started: let go by this owner in its group's order, by a person, or on the controller's own path |
| held | go | delivering | its walk started on a word that was not this owner's: the controller's own path, or a person's `plans go` |
| held | release | delivering | a person's decision, with why |
| delivering | done | delivered | every machine of its deploy plan passed or was left as its policy says |
| delivering | failed | failed | its walk failed: a first machine's gate (what it carried put back, ADR 0236 §3), a build, a machine |
| delivering | stop | stopped | its walk was stopped through this owner |
| published, delivering | superseded | superseded | a newer delivery to the same trunk took over its walk (ADR 0218) |
| any state not final | stop | stopped | a person, with why, or its group stopped by a member before it |
`delivered`, `failed`, `superseded` and `stopped` are final. A delivery never goes back to a state before
`published` once it has left one. A row is either *observed*, taken whenever its guard holds, or an *act*,
taken only when a person or healer H2 asks (recheck, release, stop). The observed rows are taken in the
table's order until none holds. A delivery therefore stands where its facts put it, whatever order the facts
arrived in. Inside `delivering`, each machine has its own row in a second table: `sent` → `judging` →
`passed` | `failed`, `failed` → `rolled-back`. A step may pass through states between two readings of the
walk. A move the table does not reach, such as `passed` → `failed`, is refused. Steps are read from the
walk's own record (rule 7), never guessed.
**3. The trunk rule ([ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md)
decision 1) is the guard on `published`.** A commit off the trunk can reach `ready` and no further, and the
walk that publishes it exists only for a merge into a branch a module follows. Only a commit on the trunk is
published, and only a published delivery is delivered. A check builds nothing today. When one does, what it
builds goes under a scratch namespace of the artifact store that registration refuses.
**4. A delivery group.**
- **Membership** is the head branch name (above). A group that has started delivering is closed: a new pull
request with the same branch name is a lone delivery, and the reason is given.
- **Order** is declared and inferred. It is declared by a line `after: <repository>` in a member's pull
request description. It is inferred by the controller's planner, which holds the graph, through the verb
`delivery-order`, from four rules:
- a member that moves a module goes before a member whose modules are built by it or stand on it
(toolchains and the build agent before their dependents);
- a member that moves the controller goes before a member that changes any module's manifest (version skew:
the newer controller parses what the newer manifest says);
- a member that moves the node-engine goes before the controller's member, because the witness reads
nothing the controller does not yet grant and a controller sends nothing an older engine refuses (ADR 0236
*Rollout order*);
- otherwise, the order of the repositories' names, so the order is the same on every reading.
A declared order that contradicts an inferred one is a cycle. The group is rejected, naming the cycle.
- **Checked** means all members' heads are composed together as one future state of the mesh: one gate over
every machine with every member's definitions, judged by the group's own controller when a member moves it.
The verdict is posted on every member's head as `mesh/delivery-group`. The group is `ready` only when that
verdict passed and every member is `ready`.
- **Delivering** follows the order, member by member. The next member starts only when the one before is
`delivered`.
- **A failed member stops the group cleanly.** The members after it are `stopped`, and the reason names the
member and why. **What was already delivered stays.** Each earlier member passed its own gate on its first
machine and its own judgement on every machine. A later member's failure says nothing about it, and putting
back a build that passed would be a rollback with no verdict behind it. **The failed member's own builds are
put back at its gate, once (ADR 0236 §3)**, by the controller as today. A person releases what was stopped
once the fix is merged, which makes a new delivery.
- The group's state is derived: `checking` while any member is `proposed` or the composed check is
outstanding; `rejected` when any member or the composed check is; `ready`; `delivering`; `delivered` when
all are; `failed` or `stopped` when a member is. Nothing writes a group's state except this derivation.
**5. Every transition is said four ways, and the owner of each says it.** mesh-delivery keeps the transition
in its state before anything else, and does nothing else for it until it is kept. What it owes outside is
kept with the delivery and retried until done:
- the event `mesh-delivery.transition` (a group's change of state: `mesh-delivery.group`), with no secret and
no address;
- a line on the commit's note under `refs/notes/mesh-plan`, asked of the forge's holder (`gitea_note_append`).
The line goes on the head while the delivery is off the trunk, on the commit it landed as once it is on it,
and at its end with what was executed. The forge's holder appends it in its own repository as the forge's
own account. A line the note already holds is not appended again;
- the delivery's view, one comment on the pull request kept current (`gitea_delivery_view`);
- the status `mesh/delivery` on the head, and `mesh/delivery-group` on each member's head
(`gitea_commit_status`).
Every status links to the pull request where the view is, `mesh/merge-gate` among them. The forge's holder
also says when a pull request closes unmerged (`pull.closed`), and says the head and its statuses with a
merge.
**6. The boundary.** mesh-delivery never sends to a machine, never writes a declaration, a bus object or a
grant, and never registers a build. It asks:
- **the controller**, through the seat's verbs: `delivery-plan` (the planner's one answer for a diffset),
`delivery-order`, `delivery-check` (compose a group's heads, or check one head again), `deliver` (let one
published trunk commit's walk start), `delivery-stop` (end a walk, with why), `delivery-walks` (the walks
and their steps);
- **the build seat**, through the controller's `delivery-check`. A group's check is a check ask like any
other;
- **the forge's holder**, through its tools: the notes, the view and the statuses.
The controller keeps composition, registration, the trunk rule, sending, the gate, the rollback and the walk
of one commit's tiers across machines, and says each step of a walk as the event `plan-moved`.
**7. The walk's record is the controller's, and the delivery's state is mesh-delivery's.** A plan the
controller keeps is from now on the walk of one delivery, read by the repository and commit it walks. Its
`building` and `rolling` are the walk's progress, and its end is reported to the delivery, which decides the
delivery's state from it. **A release plan stops being a concept of its own.** The backlog walk of ADR 0236
§4a is the walk of every build that waits on its machines, and each build it carries is credited to the
delivery that published it. `plans` stays as the walk's record. `deliveries` is what a person reads.
**8. The bootstrap rule.** mesh-delivery cannot gate or deliver itself, and the mesh must never depend on it to
be repaired:
- A walk that moves **the controller, the node-engine, the node tools, the bus or mesh-delivery** runs on the
controller's built-in path: started by the merge, judged at the gate, witnessed on the machine (ADR 0236
§5), the previous build kept and switched back to. mesh-delivery records it and does not start it.
- Any other walk, **while the seat `mesh-delivery` has a holder on record**, is built and published by the
merge and then waits for mesh-delivery's `deliver` before its first send. With no holder on record, the
controller starts it itself, exactly as before this record.
- **If mesh-delivery is down**, a walk waiting for it waits. Past its bound, the stall is said as the condition
`stalled` with the remedy. A person delivers by hand: `push <machine>`, which carries what waits there
(ADR 0236 §4a), or `plans go <plan> --why`, which gives the walk a person's word in place of
mesh-delivery's. Nothing in the controller waits for mesh-delivery to repair the controller or
mesh-delivery.
- A pull request's own check is triggered by the controller, not by mesh-delivery (above). The fix for a
broken mesh-delivery is checked, merged and delivered without it.
**9. Durable across restarts.** The state is mesh-delivery's key-value state. A restarted holder reads it back
before it takes an event. It then reads the controller's `delivery-walks`, and again every half minute, so a
walk's move it missed while it was down, or one a command made that says nothing, is found by comparison and
never lost. A pull request merged while it was down is made a delivery from the forge's word: its head and
the statuses the forge holds on it. A state held past its
bound is listed by `stalled`. The controller's self-check reads that list and raises
`delivery.<delivery>.stalled`. **Healer H2 works from the table**: it closes a stalled delivery only by a
transition the table allows for that state (`superseded` when a newer delivery took over, `delivered` when
the walk's record says done), through mesh-delivery's `close`.
**10. The switch, without a gap.** Before mesh-delivery is assigned, the controller walks every merge as it
does today. Once the seat has a holder on record, a walk that starts after that waits for its word. Walks
already open finish where they are: the holder adopts each open plan as a delivery in `delivering`, and an open
release plan's carried builds are credited to their deliveries, or listed as carried with no delivery. Only a
walk's start reads whether a holder is on record, so no walk is ever both started by the controller and
waiting for mesh-delivery. Unassigning mesh-delivery returns the mesh to the controller's own path.
## Consequences
- **"Did my change go out?" is one verb.** `deliveries` and `show <delivery>` answer it from the pull request
to the last machine. The pull request's statuses link to the same answer, and the commit's note keeps it
after the state is pruned.
- **A cross-repository change merges in an order the mesh enforces.** A member that would cause version skew
waits for the member it depends on. A group that cannot be ordered is rejected before anything merges.
- **A merge no longer sends anything until mesh-delivery says so** while it is held, except the core and
mesh-delivery itself. A down mesh-delivery means walks wait, said as a condition. A person can still deliver
by hand.
- **The controller gains six verbs, one event (`plan-moved`, in the installer's first user list too) and a
`go` for `plans`, and loses no primitive.** Its release planner's lifecycle,
supersession and hold decisions remain its code until mesh-delivery is held. After that they run only on the
built-in path. Removing them is a later decision, once mesh-delivery has delivered for a while.
- **The forge's holder gains the notes, the view and two statuses.** A note is written inside the forge's own
container, in the bare repository, as the forge's user. The forge's API reads notes and writes none, and a
push from anywhere else would need a credential to the forge that nothing else should hold.
- **A module named as its seat says its events as itself.** A consumed `<seat>.<event>` names the seat's event
only when the seat emits it. Otherwise it names the event of the module of that name (`mesh-delivery.transition`).
The controller derives subscriptions by that rule.
- **What is not built by this record**: porting the walk itself into mesh-delivery (option 1 of *where the walk
runs*); the controller's self-check reading `stalled` and raising `delivery.<delivery>.stalled`, with H2
calling `close`, which is to-be 47's Phase B (the verbs exist and are tested, the probe does not); a web view
of deliveries beyond the pull request's comment; requiring `mesh/delivery-group` on the trunks, which is the
operator's setting through the forge module's protection tool; the scratch namespace, until a check builds
something.
## How it is checked
| Rule | Checked by |
|---|---|
| a transition not in the table is refused; the table is walked | mesh-delivery's test that walks every row and every pair not in the table; the machine-step table walked the same way |
| a pull request's head goes proposed → checked → ready, or rejected | mesh-delivery's test from a `pull.updated` and a `checked` |
| a merge publishes, delivers and is delivered; a failed gate fails it; a newer commit supersedes it | mesh-delivery's tests driven by the controller's `plan-moved` snapshots: done, gate failed with rollback, superseded |
| the trunk rule | mesh-delivery's test that an off-trunk commit never reaches published; the controller's publish-rule test (ADR 0238) |
| a group's membership, order, composed check and clean stop | mesh-delivery's group tests: by branch name, declared and inferred order, a cycle rejected, ready only when all are, a failed member stopping those after it and leaving those before it delivered |
| a restart resumes | mesh-delivery's test that a holder restarted over the same state resumes each delivery and reconciles a walk it missed |
| the bootstrap rule | the controller's tests: a walk moving a core module or mesh-delivery never waits; another waits only while the seat has a holder on record and asks no build while it waits, so a push carries nothing of it; `plans go` starts it with mesh-delivery down, with why, recorded as a hand act; unassigned, the mesh is back on the controller's own path |
| the inferred order | the controller's `delivery-order` test: built by, version skew, engine before controller, declared, by name the same on every reading, and a contradiction named as a cycle |
| a group is checked as one future state | the controller's gate test: a catalogue change needing a provision only another repository's change adds fails alone and passes composed with it; the builder's check test, on a container runtime, clones a group's other head and hands it to the gate, and runs no member's own script |
| a walk moved by a verb is said | the controller's test that `delivery go`, run as a command of its own, says `plan-moved` on the bus |
| every transition is persisted, said, noted and reflected | mesh-delivery's test that each transition is put before it is emitted; the forge module's note test (append-only, idempotent) and view test (statuses carry its link) |
| the seat is in the set and its holder is one | the controller's seat test; the catalogue's manifest test of mesh-delivery |
| live | the next merge with mesh-delivery held waits for `deliver`, and `deliveries` shows it from head to delivered; `git log --notes=mesh-plan` shows the note on its merge commit |
## References
- [ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md)
decision 6, which this record gives an owner and builds;
[ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)
§3, §4a and §5, whose release plan becomes the delivering stage and whose gate rules decide what stays;
[ADR 0218](0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md),
[ADR 0201](0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md),
[ADR 0231](0231-a-healer-acts-on-what-observation-raised-and-only-observation-says-it-worked.md).
- [To-be 47](../03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md), the design;
[to-be 10](../03-DESIGN/01-to-be/10-delivery.md), the delivery it carries out;
[playbook 07](../00-META/process/07-feature-branches.md), whose branch name is a group's membership.
- mesh-controller, mesh-catalog and mesh-host: the branches `feat/mesh-delivery`.
02-DECISIONS/README.md
+1
View File
@@ -207,6 +207,7 @@ python3 00-META/checks/index.py fail if stale
- **0236** — [A build is judged on its first machine and put back by something other than itself, and so it rolls out unattended](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)
- **0237** — [A change is judged against the mesh that runs, before it merges, on the build seat](0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md)
- **0238** — [A commit is the build at hand: one commit, one change plan, checked off the trunk and published only on it](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md)
- **0239** — [A delivery is owned by the mesh-delivery module and runs from commit to delivered](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)
### Its tiers, from the bottom up
03-DESIGN/01-to-be/10-delivery.md
+28 -2
View File
@@ -5,8 +5,9 @@ code:
- mesh-controller internal/builder
- mesh-controller cmd/mesh-controller (build, build --behind, push, status)
- mesh-controller internal/inventory/builds.go
updated: 2026-09-29
updated: 2026-10-06
decisions:
- 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
- 02-DECISIONS/0090-a-failure-that-repeats-is-said-to-be-stuck.md
- 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md
- 02-DECISIONS/0010-delivery.md
@@ -100,7 +101,9 @@ That is the same shape the host uses on a machine, one layer up:
| the host | machine state | declarations |
**There is no pipeline as a state machine.** No stage list something can be omitted from, and no
run to lose.
run to lose. *What* is built stays decided by this comparison. *One commit's journey* through the mesh is a
delivery with a state table of its own: it records and orders that journey and never decides what to build
(below, *A delivery has an owner*).
### An artifact is current, or it is not
@@ -257,3 +260,26 @@ A verification mechanism was drafted for this and withdrawn. It would have repor
would not have prevented it, and the part of it that was hard — deciding which network position to check
from — existed only because the rule was wrong. Whether the mesh should check that a grant works is still
open, in issue 145; it is not the remedy for a configuration error.
## A delivery has an owner
*2026-10-06, by [ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md);
designed in full in [to-be 47](47-delivery-from-commit-to-delivered.md).*
The comparison above decides what is built, and that does not change. What it never had is an owner for
**did my change go out?** for one commit: from its pull request's head through its check, the merge, its
builds and every machine's gate. Five records answered it in parts, and a person joined them.
**A delivery is one commit in one repository, and the module `mesh-delivery` owns it.** Its states are one
compiled table: proposed, checked, ready or rejected, published, delivering, held, and four final states.
A transition the table does not hold is refused. Two or more deliveries that share a head branch name are a
**delivery group**, delivered in an order the planner infers and the pull requests may declare, and checked
together as one future state of the mesh.
**This answers the first open question above.** A fit artifact does not declare itself. Off the trunk it is
only checked. On the trunk it is published, and its walk across the machines starts when its delivery says
so: by itself for the core, which cannot wait for anything to be repaired, and by `mesh-delivery` for
everything else while that module is held. A person can always deliver by hand.
The controller keeps the comparison, the planner, the gate, sending and rollback. `mesh-delivery` keeps the
record and the order, and asks.
03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md
+4
View File
@@ -737,6 +737,10 @@ the status's link to the kept plan; the notes under `refs/notes/mesh-plan`; chec
namespace (the check builds no module today, so there is none to keep apart yet); the statuses made
required — prepared as calls of the forge module's tool, applied by the operator.
**Given an owner 2026-10-06** ([ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)): the state machine, the
notes and the status's link are built as a delivery's, owned by the module `mesh-delivery`. They are designed
in [to-be 47](47-delivery-from-commit-to-delivered.md), and their progress is tracked there.
## What is not decided here
- The bus as a cluster of three, to upgrade it live.
03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md
+190
View File
@@ -0,0 +1,190 @@
---
layer: to-be
status: designed
code: []
updated: 2026-10-06
decisions:
- 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
- 02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md
- 02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md
- 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md
---
# 47 — Delivery: from a commit to delivered
**A delivery is one commit in one repository, from the moment its pull request's head is announced to the
moment every machine of its deploy plan runs it, or to the moment it fails, is superseded or is stopped. A
delivery group is two or more deliveries that share a head branch name, delivered as one unit in order. Both
are owned by one module, `mesh-delivery`, which records every step and asks the controller for each act.**
([ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md).)
This is the delivery of [to-be 10](10-delivery.md) carried out. To-be 10 says what is built: the comparison
of source and artifacts decides, and nothing here changes that. This design says who answers for one
commit's journey and in what order the steps are allowed.
## The parts
```
forge's holder ──pull.updated / .merged / .closed──► mesh-delivery ◄──checked / plan-moved── controller
(statuses, the view, (deliveries, groups, (planner, gate,
the notes) the table, its state) registration,
▲ │ │ sending, walk)
└── its tools: note, view, status ◄───────────────┘ └── its verbs: delivery-plan, -order,
-check, deliver, delivery-stop,
delivery-walks
```
- **The controller** stays the only writer of what it owns: machine declarations, bus objects, grants,
registration. It keeps the primitives: the planner (one function, ADR 0238 decision 3), the gate, the trunk
rule, a gated send, the rollback and the walk of one trunk commit's tiers across machines. It still answers
every pull request's head with its check itself, so a check never depends on mesh-delivery.
- **mesh-delivery** is a Go bundle holding the mesh-scoped seat `mesh-delivery`. It owns the delivery, the
group, the state table and their state, and nothing else.
- **The forge's holder** sets a commit's statuses, keeps the delivery's view as one comment on the pull
request, and appends to the commit's note under `refs/notes/mesh-plan`.
- **The build seat** builds and checks, asked by the controller as before.
## A delivery
Its id is `<owner>/<repository>@<twelve characters of its commit>`. It holds:
- the pull request (number, base, head branch, description), or none for a commit that reached the trunk
without one;
- its **delivery plan**: build plan, deploy plan and verdict, as the controller's planner computed them for
its diffset (the change plan of ADR 0238 under the glossary's name);
- its **state**, its **transitions** (the last hundred, each with when, the event, from, to and why), and its
**machine steps** while delivering;
- once merged, **the commit it landed on the trunk as**, and the walk that delivers it.
## The state table
One table, compiled into mesh-delivery, read by its code, its test, its `table` verb, the controller's
self-check and healer H2. Each state has a **bound**: how long it may be held before `stalled` lists it.
Each state also says **what H2 may do** after the bound, which is only ever a transition the table already
allows.
| State | Means | Bound | H2 after the bound |
|---|---|---|---|
| proposed | a head was announced and is being checked | 1 hour | nothing: the check's own watchdog (S6) speaks |
| checked | a verdict arrived | 1 minute | nothing: it is decided at once |
| ready | it passed; it waits to be merged | none | — |
| rejected | it failed its check | none | — |
| published | merged on the trunk its modules follow, its walk opened and asking nothing; it waits for its turn or its word | 30 minutes | `superseded` when a newer delivery took over its walk; `go` when its walk started |
| held | it waits for a person | 1 day | nothing: it is the operator's |
| delivering | its walk runs; its builds are asked and registered tier by tier | 2 hours | `delivered`, `failed` or `superseded`, as the walk's record says |
| delivered, failed, superseded, stopped | final | — | — |
The transitions are those of ADR 0239 decision 2. A delivery that is merged while not `ready` goes to
`held`. Its check did not pass, and only a person decides that it goes on. A delivery merged with no walk
opened for it within ten minutes is held too, saying so: nothing it moves follows that branch, or the merge
was not heard. Nothing waits silently. **Nothing of a published delivery is registered before its walk
starts**, because a send carries every registered build that waits on its machine (ADR 0236 §4a), and a
build registered before its turn would be carried by somebody else's send.
**The machine steps.** While a delivery is `delivering`, each machine its walk reaches has a row with the
module and the build: `sent` (the first machine's send, or the rest's), `judging` (the gate's healthy
readings so far), `passed`, `failed`, `rolled-back`. They are read from the walk's own record in the
controller's `plan-moved` event, never inferred from time.
## A delivery group
- **Membership**: the pull request's head branch name, across the mesh's repositories, among open pull
requests. Two or more make a group. A group that has started delivering takes no new member.
- **Order**: `after: <repository>` lines in a member's description, and the controller's `delivery-order`
over the graph (ADR 0239 decision 4). The plan shows the order and why each pair is ordered: *declared*,
*built by*, *version skew*, *engine before controller*, *by name*.
- **Check**: the members' heads composed together, one gate over every machine with all their definitions. It
is asked by mesh-delivery through `delivery-check` when the group forms or a member's head moves, and
posted on every member's head as `mesh/delivery-group`.
- **Delivering**: in order. A member waits in `published` until the one before is `delivered`.
- **A failed member**: the members after it are `stopped`, naming it. The members before it stay delivered.
Its own builds were put back at its gate.
A group's state is derived from its members and its composed check, and is shown, never stored apart from
them.
## What every transition does
In this order, each by its owner:
1. **kept**: mesh-delivery puts the delivery in its state (`deliveries`, one key per delivery; `groups`, one
key per group). Nothing is said before it is kept. What it owes outside is kept with it and retried until
done;
2. **said**: the event `mesh-delivery.transition` (and `mesh-delivery.group` when a group's derived state
changes), with the delivery's id, from, to, event, why and plan summary, and no secret or address;
3. **noted**: mesh-delivery asks the forge's holder (`gitea_note_append`) to append a line to the commit's
note: the head while the delivery is off the trunk, the commit it landed as once it is on it, and at its
end a line of what was executed there. The forge's holder writes it in its own repository as the forge's
own account, and a line already there is not appended again;
4. **reflected**: the delivery's view, one comment on the pull request with the state, the plan, the machine
steps and the group, kept current (`gitea_delivery_view`); the status `mesh/delivery` on the head, and
`mesh/delivery-group` on every member's head (`gitea_commit_status`). Every status links to the pull
request, `mesh/merge-gate` among them.
## The verbs
**mesh-delivery's seat** (served by its holder; the first five read):
| Verb | Answers or does |
|---|---|
| `deliveries` | every delivery not final, and the final ones of the last day, one line each; filtered by state, repository or group |
| `show` | one delivery or group whole: plan, transitions, machine steps, order, the walk |
| `groups` | every group with its members in order and its derived state |
| `what-if` | the delivery plan a diffset would have, asked of the controller's planner, kept nowhere |
| `table` | the state table and the machine-step table, with bounds and H2's repairs |
| `stalled` | every delivery held past its bound, with the transition H2 may take |
| `recheck` | a rejected or ready delivery back to proposed, with why |
| `release` | a held delivery to delivering, with why: a person's word |
| `stop` | any delivery not final to stopped, with why; its walk is ended through the controller |
| `close` | H2's only verb: the transition the table names for a stalled delivery, refused otherwise |
**The controller's verbs for it** (on the mesh-controller seat): `delivery-plan`, `delivery-order`,
`delivery-check` (a group's heads composed, or one head checked again), `deliver`, `delivery-stop`,
`delivery-walks`. A person has `plans go <plan> --why`, which does what `deliver` does under their name and
is recorded as a hand act. The controller says every walk it keeps as `plan-moved`, from a verb's command
too.
**The forge's holder's tools for it**: `gitea_note_append`, `gitea_delivery_view`, `gitea_commit_status`. It
also announces `pull.closed`, and a merge's head and the statuses the forge holds on it.
## The store
Key-value state on the bus, declared by the module (ADR 0201): `deliveries` and `groups`. One writer, the
seat's holder, whose code changes state from a single goroutine. A final delivery is kept for thirty days and
then removed. Its note on the commit keeps its record after that. The bus's own snapshot (ADR 0235) backs it
up.
## The bootstrap
- A walk that moves the controller, the node-engine, the node tools, the bus or mesh-delivery is the
controller's own, started by the merge and witnessed on the machine. mesh-delivery records it.
- Any other walk waits for `deliver` only while the seat has a holder on record. With none on record, the
controller starts it as it always did.
- With the holder down, waiting walks are `stalled` past their bound, said with the remedy: `push <machine>`
or `plans go`.
## The switch
1. The controller's change (the seat, its verbs, `plan-moved`, the wait for a holder's word, `plans go`) is
merged and rolls out as the core does. With no holder on record it behaves exactly as before.
2. The catalogue's change (mesh-delivery, the forge holder's notes and view) is merged. mesh-delivery is built
and published.
3. A person assigns mesh-delivery to the control node. The seat now has a holder on record. Walks that start
from then on wait for its word. Walks already open finish on their own and are adopted as deliveries in
`delivering`.
4. To step back, unassign it. Waiting walks are then started by the controller at its next pass.
## Phases
| Phase | Delivers | Done when |
|---|---|---|
| A — the owner | mesh-delivery with its table, state, verbs, events and adoption; the controller's seat, verbs, `plan-moved` and the wait; the forge holder's view, notes and statuses; the group's order and composed check | the tests of ADR 0239's *How it is checked* pass; a merge on the live mesh with mesh-delivery held is delivered by it and shown by `deliveries` |
| B — the conditions | the self-check reads `stalled`; `delivery.<id>.stalled`; H2 calling `close` (the verbs are built in A) | a delivery stalled on purpose is raised and closed by H2 through `close` |
| C — the walk itself | the per-tier walk moved into mesh-delivery behind single-send verbs, the controller's planner kept only for the built-in path | decided by its own record once Phase A has delivered for a while |
## What is not decided here
- A web view of deliveries beyond the pull request's comment.
- Whether `mesh/delivery-group` is required on the trunks. That is the operator's setting, through the forge
module's protection tool.
- Moving the per-tier walk out of the controller (Phase C).
03-DESIGN/01-to-be/README.md
+1
View File
@@ -47,6 +47,7 @@ document is written and this one's status becomes `implemented`.
| [`42-the-machines-modules-in-order.md`](42-the-machines-modules-in-order.md) | **In progress.** The order the machines' modules of research 026 and 027 are built and rolled out: every machine's first (sudo, localization, time sync, pacman, logrotate, avahi, systemd, docker, `~/.ssh`, scripts, kernel), then both workstations', then one machine model's; each proven on one workstation before the rest | [ADR 0173](../../02-DECISIONS/0173-the-operators-machine-is-the-meshs-and-a-module-is-what-it-declares.md), [ADR 0182](../../02-DECISIONS/0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md), [ADR 0205](../../02-DECISIONS/0205-software-the-distribution-does-not-package-ships-as-a-pinned-archive-of-the-module.md) |
| [`45-a-core-that-cannot-fail-silently.md`](45-a-core-that-cannot-fail-silently.md) | **Designed.** The core says when it is wrong, refuses what is stale or unreadable, heals what it knows, upgrades one machine at a time with a witness that rolls it back, and is checked against the real mesh before merge: the writers and signals tables, the condition store, `doctor`, the minimal output channel, healers, the lease and report order, staged upgrades, the facts snapshot and replays, in six phases | [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md), [ADR 0224](../../02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md), [ADR 0218](../../02-DECISIONS/0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md) |
| [`46-the-conversation-with-the-operator.md`](46-the-conversation-with-the-operator.md) | **Designed.** The mesh tells and asks its operator over channels that are holders of two kinded benches, `channel` and `intake`, declaring capabilities from a fixed vocabulary; the router orders them by work context and never lowers the bar; an answer that performs an action is checked and performed by the controller, on a TOTP code or a verified Telegram sender, never on a desk click alone; operator messages addressed or answered by the mesh's responder, untrusted input kept as data, references for what words may not carry, and a recoverable factor; Telegram first, in six phases | [ADR 0234](../../02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md), [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md) |
| [`47-delivery-from-commit-to-delivered.md`](47-delivery-from-commit-to-delivered.md) | **Designed.** A delivery is one commit in one repository, from its pull request's head to every machine; a delivery group is deliveries sharing a branch name, ordered and checked as one future state; both owned by the `mesh-delivery` module with one state table, its state on the bus, every transition said, noted on the commit and shown on the pull request; the controller keeps the planner, the gate, sending and the walk, and the core's own updates never wait for the module | [ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md), [ADR 0238](../../02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md), [ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) |
## Not yet written