Merge pull request 'Design 47's milestones and issues, and the as-is delivery as it runs today' (#393) from handoff/47-milestones into main

2026-10-16 22:09:55 +00:00
parent 73b9cc3257 748641dd22
commit dd5a065295
4 changed files with 91 additions and 84 deletions
@@ -30,7 +30,8 @@ A design becomes **milestones**, and each milestone becomes **issues**:
 4. **Break it into milestones.** Write them into the design, in a section *Milestones*: for each, its number,
   its one-sentence outcome, the acceptance tests of the design it proves, the repositories it touches, and
   what it depends on. Order them so each is usable as soon as it is delivered, the smallest useful one
-   first.
+   first. A milestone is named by what it lets someone do; how it ships is shaped by the plan its changes would
   have, asked through `mesh-delivery.what-if`: the modules it moves and their fan-out (ADR 0401 §6).
 5. **Ask the operator to confirm the milestones**, in a short table: number, outcome, the acceptance tests it
   proves, repositories, the parts each repository builds. Change what they change.
 6. **Open the issues.** For each confirmed milestone, `issue-tracker.open` one issue per repository part, its
@@ -140,12 +140,16 @@ never into the running one, unless it blocks the running one.
 5. **The reviewer marks each pull request ready as soon as the acceptance is shown** with
   `mesh-delivery.mark-ready` (the delivery of the head it approved, who reviewed it and why), which writes the
   label `mesh/ready` and then `ready at <full sha>`, the order the forge counts (issue 913); mesh-delivery merges it when its checks pass
-   (ADR 0364). The operator's word is waited for only when the step changes what the operator sees, removes
+   and the operator approved the merge into main on Telegram, the channel that proves who answered today (ADR 0364,
   ADR 0398 §3; an Approve at the terminal is issue 953). Beyond that
   Approve, the operator's word is waited for only when the step changes what the operator sees, removes
   data, or needs a step only the operator's terminal can take; the lead says which before the step starts.
 **One exception: a catalogue change that uses a new manifest word.** The catalogue's merge check reads
 manifests with the controller the mesh runs, so the controller's pull request is merged and delivered first;
 then `mesh-delivery.recheck` the catalogue's and merge it on the new verdict. Plan such a feature as two steps.
 The exception ends when milestone 3 of design 47 is delivered (issue 977, ADR 0402, ADR 0403): the group's composed
 check, judged by the group's controller, is then each member's gate, and the feature is one group.
 ## Phase 4 — Deliver
@@ -1,100 +1,90 @@
 ---
 layer: as-is
 status: implemented
-code: [hal]
+code: [mesh-controller, mesh-catalog modules/mesh-delivery, mesh-catalog modules/gitea, mesh-catalog modules/build-agent]
-updated: 2026-08-23
+updated: 2026-10-17
 decisions:
-  - "ADR 0010"
+  - "ADR 0239"
-  - "ADR 0010"
+  - "ADR 0267"
-  - "ADR 0010"
+  - "ADR 0276"
  - "ADR 0282"
  - "ADR 0289"
  - "ADR 0304"
  - "ADR 0350"
  - "ADR 0358"
  - "ADR 0360"
  - "ADR 0364"
  - "ADR 0382"
  - "ADR 0395"
  - "ADR 0398"
 ---
-# Delivery — from a push to a running node
+# Delivery — from a merge to the nodes, as it runs
-One trigger, three silos, and a fan-out point that is the most consequential boundary in the
+What runs on the mesh's main branches on 2026-10-16, read from the code and the live record by research 055
-mesh.
+(findings 1 to 12). It is what [to-be 47](../01-to-be/47-delivery-from-commit-to-delivered.md) replaces, milestone
 by milestone. Until 2026-10-16 this document described the predecessor's pipeline, which no longer runs.
-## The trigger
+## Who does what
-A push to the forge. The forge calls a webhook; the receiving module verifies its signature
+- **mesh-delivery** owns deliveries and delivery groups (ADR 0239): one delivery per commit, a group per branch
-and emits an event. The coordinator resolves which modules the pushed commits affect, orders
+  name shared across repositories. It records each transition in its state on the bus (key-value buckets
-them into dependency levels, and creates a pipeline per level.
+  `deliveries` and `groups`), marks pull requests, asks the controller to deliver, and merges a pull request marked
  ready at its head once both required checks pass against the trunk's head (ADR 0364) and the operator approved
  the merge into main on a channel that proves who answered (ADR 0398 §3).
 - **The controller** plans, orders, sends and judges. It computes a delivery plan for a repository and the files a
  merge changes, moves only modules whose build source changed with their build dependents (ADR 0267), and
  reuses a merge check's build of the same reuse key (ADR 0350).
 - **The builders** (holders of the build seat) take build requests and checks from a work-queue stream on the bus
  and build inside a toolchain image another module provides, with the network off; on 2026-10-13 no node had Go
  or npm installed (issue 626).
 - **The forge module** publishes every forge event on the bus through the forge's system webhook since 2026-10-14
  (issue 729), and still polls the forge every 30 s, or every 300 s once it hears events. It keeps files of what it
  announced so it does not announce twice.
-There is no second path. A manual trigger exists for a module the push detection missed, and
+## From a merge to a rollout
 using it routinely is a sign the detection is wrong rather than a workflow.
-Detection is the pipeline's most fragile input. It has failed for reasons that have nothing to
+- **The merge window.** The controller collects merges into an open batch; each merge restarts the window, which
-do with the change: a webhook truncating its commit list on a large merge, a forge address
+  closes after 90 seconds without a merge or 10 minutes after the batch's first merge (ADR 0276). A lone merge is
-whose port broke the module-path matching. The characteristic outcome is the bad one — **a
+  cut at once (ADR 0282 §2, as ADR 0304 §1 changed it). The batch and its merges are rows in the controller's
-merge that created no pipeline, and nothing said so**.
+  store, cut by a timer every 5 s, with a catch-up timer every 5 minutes for merges no batch holds.
 - **The rollout.** A closed batch becomes one rollout, at most one commit per repository, tiered along the module
  graph. Rollouts run one at a time: a batch cut while a rollout is open is queued behind it, whatever it shares
  with it. Running unrelated rollouts side by side (ADR 0282 §3, ADR 0304 §2) is decided and not built (issue 525). A rollout's builds go to every machine at once, with no first-machine gate, and the rollout ends on
  the machines' reports or after fifteen minutes of silence (ADR 0360). A controller timer advances rollouts every
  30 s.
 - **A merge during a rollout.** Its place in the next batch is a store row the open rollout's holder writes. On
  2026-10-16, when another controller worked the plans, a merge was not recorded and waited until a person released
  it (issue 921).
 - **"Release branches".** A pull request into a branch other than a trunk is judged by the controller built from that
  branch (ADR 0358); feature 57 shipped about 37 packages that way over about 2.5 days and held the controller for
  55.7 h.
-## Three silos
+## Checks
-Cardinality is the whole point, and the three differ
+- Every pull request's head gets `mesh/merge-gate` (the composed machines) and `mesh/repo-check` (the repository's
-(ADR 0010):
+  `merge-check.sh`), each judged against one trunk commit (ADR 0323). A trunk move asks the check of every active,
  cleanly merging pull request again (ADR 0395).
 - A delivery group's members are each checked beside their siblings' heads, and the group also gets a composed
  check, `mesh/delivery-group`; each member keeps its own gate beside it (ADR 0382).
 - The queue's order (delivery builds first, then ready checks, then the rest) is kept by pausing the worker and
  deleting and republishing a request (issue 558); an old head's check is ended after asking the forge for the
  current head (issue 726).
 - mesh-delivery ticks every 10 s, reconciles with the controller every 30 s and polls the forge for ready marks
  every minute, although the forge module already publishes labeled, commented and synchronized events.
-| Silo | Runs | Where | Does |
+## Where the record of what runs lives
 |---|---|---|---|
 | **build** | once per module feature | the build node | compile and bundle into a self-contained output |
 | **publish** | once per module feature | the build node | package that output and upload it; a package-registry feature publishes here |
 | **deploy** | once per module feature **per node** | every assigned node | install, configure, start, verify |
-Commands and events are addressed per feature, not per module.
+Builds and build requests, rollouts with their tiers, batches and merges, and assignments are rows in the
 controller's store; artifacts are blobs in the artifact store by digest (ADR 0251); packages are versions in the
 package registry. The controller writes three record mirrors to the bus: assignments, declared config and
 predefined keys; of the five ADR 0289 §7 decided, the nodes, rollouts, builds and settings mirrors are not in the
 code. An assignment names its module and node but not the build, artifact or rollout that put it there: which commit
 runs where, and which delivery brought it, takes three verbs and a join by hand.
-Build hands over a **staged tree**, not a package. Packaging belongs to publish, so a failed
+No backup is taken before a delivery moves a module that holds data; backups run nightly and on demand (ADR 0214).
 upload retries by re-packaging rather than by re-sending something stale, and build never needs
 to know how each module composes its artifact. The handover is a tree in a known location,
 because the build's own working directory is reference-counted and may be gone by the time a
 later stage runs.
 ## The artifact
 The artifact is **build output** — compiled and bundled with its dependency graph inlined —
 never a filtered copy of source (ADR 0010).
 A deploy is extract-and-run and touches no network.
 The consequence is the whole cost of the decision: **anything not in the build output does not
 ship.** Every file kind had to be brought into that rule separately, and each was discovered by
 something silently not happening after deploy — migrations reading a source layout, provisioning
 scripts reading a source layout, selection files never packaged at all.
 ## The fan-out, and the build node
 After publish, work fans out to every assigned node. The build node is the only node that has
 already passed through two silos when this happens, and that asymmetry has its own defect
 class: anything advancing a node's stage must account for **both** pre-fan-out stages. Code
 that knew only about the first parked the build node forever while every other node deployed
 cleanly — and the recovery sweeper, which knew the same subset, reported nothing to recover.
 A recovery mechanism that knows less than the thing it guards is worse than none, because it
 reports success over a stall it cannot see.
 ## Levels
 A level completes before the next begins, so a module builds against its dependencies as they
 were just published. The shared library is at level zero, which means anything that breaks it
 breaks the first module of every cascade.
 ## What green proves
 **A green pipeline proves transport, not effect.** The stages report that a message was
 dispatched and accepted, which is not the same as the thing being running, correct, or present.
 This is the mesh's most consistent failure shape and it is not incidental to the design — it is
 what the stage reporting currently measures. Documented instances include a service reported
 started when the container command merely returned, an image pull failure that did not fail the
 deploy, a package install that 404ed from every mirror while the job went green (see
 [`04-ISSUES/001`](../../04-ISSUES/001-failed-package-install-reports-success/00-report.md)),
 and a node left on old code after a failed artifact download with a version marker that had
 already advanced.
 A verify stage exists to close this gap. It has been built and, for a period, was never
 scheduled, because the coordinator's stage list did not include it.
 ## What is not covered
-The end-to-end harness for this pipeline has not built since the workspace was removed, and
+The integration tests run only in each repository's merge check; there are no end-to-end tests of delivery yet,
-nothing reports that nothing runs it
+and the lab is not used (the operator, 2026-10-16).
 ([`04-ISSUES/005`](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)). The
 pipeline's coverage is currently assumed rather than checked — which is the same class of claim
 this repository exists to make people stop making.
@@ -485,7 +485,19 @@ What the build proves before this design is implemented (research 055's directio
 ## Milestones
-Set by playbook 04 at the handoff, after the operator confirms them.
+Confirmed by the operator on 2026-10-17 (playbook 04). Each is one feature step of playbook 07; the next starts
 when the previous one's delivery is done. Milestones 5 and 6 are split into steps when they come up.
 | # | Outcome | Acceptance tests | Repositories | Issues |
 |---|---|---|---|---|
 | 1 | A merge during a running delivery is never dropped and needs no person's release; the merge window lives on the bus; a rollout that shares nothing with an open one runs beside it | 8, 9, 11 | mesh-controller | 973 (with the rest of 525) |
 | 2 | A ready mark is acted on within seconds, and each pull request has at most one waiting check | 5, 6, 7 | mesh-controller, mesh-catalog (mesh-delivery, gitea) | 974, 975, 976 |
 | 3 | A feature with a new seat verb across the controller and the catalogue is delivered as one group: no `after:` line, one composed check, no "release branches" | 1 | mesh-controller, mesh-catalog (mesh-delivery) | 977, 978 |
 | 4 | A delivery backs up the data it moves first, and a failed backup holds it and is said | 3 | mesh-controller, mesh-catalog (mesh-delivery, the backup seat's holder) | 980 |
 | 5 | One read of a module on a node names the commit, build and rollout that put it there | 10, 12 | mesh-controller, mesh-catalog (mesh-config, nats) | 981 |
 | 6 | Builds use installed toolchains with no container, builders fetch over SSH with their own key, and an SDK change ships in one group with its users | 2, 4 | mesh-catalog (build modules, build-agent, ssh-client, gitea), mesh-controller, mesh-sdk | 979, 971 |
 The issues are linked `depends-on` in this order; 976 depends on 975, and 979 also on 971.
 ## What is not decided here