diff --git a/00-META/glossary.md b/00-META/glossary.md index b99573dc..c2f9c89a 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -600,7 +600,28 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel registering and changing one are trusted acts (ADR 0666 §2). The mesh knows modules, and a repository only as a module's git origin. - **watched branch** — a branch a module's registration names, whose commits start that module's pipeline and which a - placement may choose for a node; registering one the forge does not protect is refused (ADR 0666 §3, §6). + placement may choose for a node; registering one the forge does not protect is refused (ADR 0666 §3, §6), a refusal + the operator removed on 2026-11-14 (design 47 G §5); a merge into it still needs its protection (ADR 0666 §11). +- **merge call** — one call of the seat `git`'s verb `merge-pull` naming one or more pull requests, each at the head + its caller names: from the delivery of design 47's milestone H the start of their delivery and, with two or more, + their group; the forge module judges every member before it merges any and says the call as its event + `merge-called`, from which the gate names the merged heads one source revision (ADR 0682). Not a merge made in the + forge's own screen, which delivers nothing and leaves an **untaken head**. + *Not:* ~~kick-off~~ (hq) +- **untaken head** — a watched branch's head that the forge bridge said moved (`head-moved`) and that no source revision + names within the gate's setting `untaken-bound`: a head moved outside the mesh, raised as the gate's finding + `head-untaken` and delivered by nothing but a later merge call or the gate's hand take (ADR 0682; design 47 H §740). + Not a pending revision, which is a source revision not yet taken in. + *Not:* ~~merge-untaken~~ (hq) +- **take-in** — the indexer's act on one source revision of a watched branch: the commits it names, the + build-input fingerprints of the modules registered for those repositories, and one catalog revision of that branch, + said as `take-in.taken` naming the **movers**; it validates no manifest (that is the pipeline step `validate`) and + holds nothing for a named indexer (ADR 0682; A-476). Not a merge: what is taken in is a source revision the gate named. + *Not:* ~~merge take-in~~ (hq) +- **mover** — a module whose build-input fingerprint changed in a take-in, or a code dependent of one (a module, in any + repository, that imports a package of one): the modules a take-in's one pipeline runs and delivers (A-476a; ADR + 0573 §3; ADR 0682). Not every module of the revision. + *Not:* ~~moving module~~ (hq) - **run on request** — a pipeline run for a named commit or group that delivers nothing, asked by the runner's verb `run`; the indexer builds its candidate from the runner's `run-started` (ADR 0666 §18). - **artifact reference** — an artifact written `:`, such as `mesh-grants:program`: the end of a @@ -816,7 +837,8 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel level: a group holds deliveries, never groups. Its state is derived from its members, never set. From milestone G's delivery: a shared branch name carrying its issue number, complete when each member's commit has landed on its modules' watched branch, taken in as one source revision of that branch and run as one pipeline per watched branch - (ADR 0666 §13). + (ADR 0666 §13). From milestone H's delivery: the pull requests named in one **merge call**; no branch name and no + pull request read after the call forms one (ADR 0682). - **hotfix** — one pull request in one repository, delivered at its merge (ADR 0401). - **feature** — one branch name across its repositories: in two or more, its pull requests are one delivery group, merged together and delivered member by member in the group's order; in one, a single pull request delivered as a @@ -890,7 +912,9 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel - **pending revision** — a source revision the running indexer has not taken in, because it names another build of the indexer or follows one that does: only the named indexer's own keys are written, the rest kept in the indexer's state `pending` until that indexer starts and takes it in (ADR 0633 §1 to §3). A group's members wait - there too until the group's end, as members still gathered (§4). + there too until the group's end, as members still gathered (§4). Ends at milestone H's delivery: a take-in holds + nothing for a named indexer, and one that cannot be written waits as its unacknowledged `source-moved` (ADR 0682; + design 47 H §744). - **canary** — the first node of a module's nodes, ordered by ring then name: a new fingerprint reaches it first and the module's other nodes only once it runs healthy there (ADR 0429 §6). Replaces sending to every machine at once from release 6 (§15); a fingerprint that fails there is held and the canary set back, or for diff --git a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md index 4e852694..891248ec 100644 --- a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md +++ b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md @@ -2,8 +2,10 @@ layer: to-be status: in-progress code: [mesh-controller, mesh-catalog modules/mesh-delivery-gate, mesh-catalog modules/gitea, mesh-delivery, mesh-host, mesh-sdk, mesh-vault, mesh-contracts, mesh-tools, mesh-networking, hq 00-META/checks] -updated: 2026-11-17 +updated: 2026-11-20 decisions: + - "ADR 0682" + - "ADR 0681" - "ADR 0673" - "ADR 0672" - "ADR 0668" @@ -195,6 +197,16 @@ acting parts, the kept data on the bus restored with a delivery, and the verbs a section this file names in italics and does not hold is there, under the same name. A sentence that holds a delivered line and G's change together stays here. +**From milestone H's delivery, a delivery starts at the merge through the mesh** (ADR 0682; the operator, +2026-11-20). One **merge call** of the seat `git`'s verb `merge-pull`, naming one or more pull requests, is a delivery's +start and its group; the gate names the merged heads one source revision per watched branch and opens its delivery +record, the indexer takes it in and names its **movers**, one pipeline runs per take-in and none in parallel, and a +want names the newest released version whose pipeline passed. The forge bridge's one message per move of a watched +branch's head is the gate's watchdog: a head no merge call delivered is an **untaken head**, said and delivered by +nothing. Every line milestone H builds is in its part, opened by the plan +[47-milestones/H-deliver-the-delivery-system.md](47-milestones/H-deliver-the-delivery-system.md) and cited as "design +47 H §n" (§734 on); where an H line and a G line differ, the H line holds and the G line says so. + This is the delivery of [to-be 10](10-delivery.md) carried out. To-be 10 says what is built: the comparison of source and artifacts decides, and nothing here changes that. This design says who answers for one commit's journey, in what order the steps are allowed, and where each fact of it is kept. It was rewritten on @@ -1508,6 +1520,22 @@ way is written down before the switch (ADR 0526, which ADR 0522 §6 asked of R4g running versions are kept outside the artifact store on two nodes, each by digest with its module, artifact, commit, Go version and the steps that put it back, because the store's nightly collection removes a blob the controller did not record (ADR 0526 §6). +## Every delivery part wakes on its inputs alone + +*ADR 0682 §9; issue 1982; the plan of milestone H, section 3.6.* A delivery part (the forge module, the forge +bridge, the gate, the indexer, mesh-config, the runner, the builder, the build modules, the assigner, the node-engine, +the grants part, the watchman, the condition keeper, the status view, the teller and the tool runner) keeps a watched +copy of each of its own states, through one shape in mesh-sdk's state package (the shape the held pull request +mesh-catalog #708 built for mesh-config, moved into mesh-sdk), and runs a pass only when an input it consumes changed, +taking only the keys that input names. It keeps one timer, set to its earliest real deadline, a bound it states; it +polls nothing, reads nothing of its own whole on a wake, and asks no call again on a timer. Which part moves onto the +shape in which release is design 47 H §762. How it is checked: the code review of every delivery module from milestone +H1 on refuses a timer that is no stated deadline, a poll and a whole re-read; mesh-sdk ships a test helper that counts +the requests a pass makes, and each delivery module's integration test that starts it whole against the bus (ADR +0548) asserts that a pass after one changed key requests that key alone; live, milestone H4 is proven by the bus's +slow consumers and consumers behind at 0 for an hour, read with `mesh-broker.rates` (H §760) and the bus module's +consumer reads. + ## Acceptance tests What the build proves before this design is implemented (research 055's direction): @@ -1664,7 +1692,7 @@ list: performance is fixed; no check starts real builds. 37. **Nothing is delivered ahead.** Research 085's milestone is delivered with no part delivered ahead of it, read from its delivery record (ADR 0656, How it is checked). -38. **A newer indexer is reached.** A merged revision naming a newer indexer gives, from the running indexer, a catalog +38. **A newer indexer is reached.** *(Retires at milestone H, design 47 H §764.)* A merged revision naming a newer indexer gives, from the running indexer, a catalog graph revision that is the previous one with only the indexer's entry replaced; no other want moves; the new indexer is wanted, granted, started, and takes the revision in. 39. **Revisions are kept, and stores say their fill.** A revision pinned by two records stays until both free it, one @@ -1706,7 +1734,7 @@ Each change ADR 0656 makes in an earlier record, and its test: are read by the indexer's candidate pass and by no running part (whole). 48. **A process reports its handed build** (ADR 0639 §7). A part started while its node's want differs from what runs reports the build it was started with, and the node-engine seat's verb `process-builds` reads it (whole, node-engine). -49. **The forge bridge reads a group from the gate's verb** (ADR 0633 §4, §5). The forge bridge sets the group on each +49. **The forge bridge reads a group from the gate's verb** *(Retires at milestone H, design 47 H §764.)* (ADR 0633 §4, §5). The forge bridge sets the group on each merged message and publishes a group's end again from the seat `mesh-delivery`'s verb `groups`, and its account holds no read of the gate's `merges` state, so a read of it is refused (whole, forge bridge). @@ -1728,11 +1756,9 @@ What milestone G's prepare found, and its test: (whole, gate, checker and forge module). 53. **What is owed is said, answered and kept.** An `owed` said while the forge bridge is stopped is performed once after its restart; the bridge stopped between an act and its `owed.ended` repeats it and the forge holds it once; an - undecodable act is said `undecodable`; an act unanswered 15 minutes is listed by `stalled`; the switch with a merge - handed and not yet merged opens no `forge-made` record; a merge made at the forge opens one, said as `transition` - with why `forge-made`, and a later merge in its repository follows it; the checker's `mesh/check` is performed by - the bridge from the checker's `owed`, and a checker status the forge refuses is recorded in the checker's own - state and listed in its stalled read (whole, gate, checker and forge bridge). + undecodable act is said `undecodable`; an act unanswered 15 minutes is listed by `stalled` (whole, gate and forge bridge). The rows of a `forge-made` record + and of the checker's `mesh/check` went: no record is marked `forge-made` (design 47 G §151), a head moved outside + the mesh is an untaken head (design 47 H §740), and the checker goes (corrected in place by design 47 H §763). 54. **One reader of the seat list per part, and one view machinery.** Every part that reads the seat list has the two tools of ADR 0636 §7 from the runtime and no wiring of its own; every part that takes a first view answers its held and awaited revision in `_first_views`. With the forge on its old build, serving no verb of the seat @@ -1865,9 +1891,9 @@ No module ends the milestone with more tests than it starts with, counted per ca The failure paths: -81. **A group that never completes** raises `take-in-waits` at 30 minutes, and none of its members is delivered (whole, +81. **A group that never completes** *(Retires at milestone H, design 47 H §764.)* raises `take-in-waits` at 30 minutes, and none of its members is delivered (whole, indexer and watchman). -82. **An indexer whose manifest its validator refuses** leaves its revision pending, holds only its own watched branch, +82. **An indexer whose manifest its validator refuses** *(Retires at milestone H, design 47 H §764.)* leaves its revision pending, holds only its own watched branch, raises `take-in-waits` at 60 minutes, a later merge on that branch waits behind it and is not taken in past it, and a revert taken in moves the branch on (whole, indexer). 83. **A declined registration change, and an unprotected branch,** each write nothing: the first after its Decline, the @@ -1953,13 +1979,13 @@ ADR 0667's checks, of milestone G itself: package rebuilds it, started by the runner after its `outdated` ask, one that leaves its digest as it was rebuilds nothing, and a change of the Go build module's own code rebuilds no module; a `beta` build's entries compare by digest (whole, indexer, builder and runner, mesh-delivery). -105. **One query finds what is behind** (design 47 G §375). With the runner stopped while mesh-sdk `0.1.16` is +105. **One query finds what is behind** *(Retires at milestone H, design 47 H §764.)* (design 47 G §375). With the runner stopped while mesh-sdk `0.1.16` is released, `outdated` names the module's kind-1 edge at `0.1.15` against `0.1.16`, and at its restart the runner starts that module's pipeline once, keyed by mesh-sdk's source revision; `runner_catch_up` does it by hand and writes the hand-act log (whole, indexer and runner, mesh-delivery). After the first fill (§377), a mesh running its newest builds is answered with nothing behind. -106. **A delivery is waited for mesh-wide** (design 47 G §376, §379, §380). A module ordered after a provider's +106. **A delivery is waited for mesh-wide** *(Retires at milestone H, design 47 H §764.)* (design 47 G §376, §379, §380). A module ordered after a provider's new version is wanted only once `version-delivered` says every present machine placing the provider runs it; with one machine away the module's status reads "waiting for delivery of" the provider, naming the machine, and it catches up when that machine is back; with every placing machine away the provider reads "waiting for @@ -1993,9 +2019,37 @@ ADR 0667's checks, of milestone G itself: and Go build module, mesh-delivery and mesh-catalog). Live: each module's test steps run once per level, read with `runner_test_runs`. A reader no part ends raises `pipeline-stands` within five minutes of its last sign of life (whole, watchman; design 47 G §732). +110. **One merge call is delivered end to end with no hand act** (design 47 H §734 to §738, §743, §746, §750). With + `pipelines-at-once` at 1, a merge call naming one pull request that changes one module says one `merge-called`; + the gate writes one source revision and opens one record; the indexer says one `take-in.taken` naming that module + alone; the runner runs one pipeline; once it passes and the version is released, the module is wanted on every + machine that places it, runs there, and the record ends (whole, forge module, gate, indexer, runner, assigner and + node-engine, mesh-catalog, mesh-delivery and `mesh-host`). Live: the plan's section 5, step 7, read with the tools + and quoted with its times. +111. **A head moved outside the mesh delivers nothing and is said** (design 47 H §739 to §742). A merge made in the + forge's screen gives one `head-moved`, no source revision and no pipeline; `head-untaken` is raised within + `untaken-bound`, naming the repository, branch and head; a later merge call in that repository clears it; and + `gate_head_take` delivers such a head by hand, written in the hand-act log (whole, forge bridge and gate, + mesh-delivery and mesh-catalog). +112. **Two repositories in one merge call are one pipeline, and pipelines run one at a time** (design 47 H §734, + §735, §743, §746 to §748). A merge call naming a mesh-sdk change and a module that imports it, both on one watched + branch, merges both or, with one member refused, neither; merged, it is one source revision, one `take-in.taken` + naming mesh-sdk, that module and mesh-sdk's other importers, and one pipeline in which every importer builds after + mesh-sdk's `package-published`; a second merge call made while it runs is said waiting and runs after its + `pipeline-ended`, and a restart of the runner starts no pipeline of its own (whole, forge module, gate, indexer and + runner). +113. **A want is the newest released version whose pipeline passed** (design 47 H §750, §751). A module released with + a passed pipeline is wanted on every placing machine at once, whatever build an acting part runs and whatever a + dependency's delivery; a released version whose pipeline failed is not wanted; an older version's end heard after + a newer one writes nothing; on one machine the bus module's and the backup holder's changed wants apply before the + others and the node-engine's last (whole, assigner and node-engine, mesh-delivery and `mesh-host`). ## What retires +- From milestone H's delivery: everything design 47 H §764 lists (the per-pull-request paths of the gate and the forge + bridge, the catch-up by comparison, the runner's sweep, the start-time `outdated`, the checker, the groups by branch + name, the want holds A-478 ends), and the stale state H §765 drops once. + - The "release branches" and the controller's check of them (ADR 0358, superseded by ADR 0401). - The toolchain image and its module (ADR 0405). - The window cutter, the merge catch-up timer, the rollout ticker, mesh-delivery's tick, reconcile and @@ -2089,6 +2143,7 @@ release reports the controller's code lines against research 063's baseline (acc | L7 | History and records leave the controller (ADR 0571): the builder keeps its build log, one entry at a build's start and one at its end, from which build history and durations are read and the module mesh-catalog asks the builds it missed; the artifact sweep is the seat `mesh-artifact-sweep`, held by artifact-store-tools beside the artifact store, its keep set from the store's own listing, the wants and the build log, sweeping at each build's end from the lease write and recording each deletion in the hand-act log; disk readings are disk-load's own state on every node; who holds which seat is the state `seat-holders` of the seat `mesh-placement` (now `mesh-module-assigner`), and tools are found by the tool runners' announcements; what a node runs is its want and its have; the grants part writes `grants..` and keeps memberships in its own state; root-free is a probe of the self-check; no manifest but the controller's reads, invokes or consumes `mesh-controller`, but a named list kept to the lease write, so the lease write's check passes (ADR 0189, ADR 0547 §11, ADR 0549 §6) | — | mesh-sdk, mesh-controller (its pin only), mesh-delivery (builder, indexer, assigner, grants part, view), mesh-catalog (artifact-store-tools, disk-load, the backup seat's holder, mounts, nfs-server, minio, mesh-watcher, the messenger, the self-check, the module mesh-catalog, docker), mesh-tools (the tool runner) | 1955 | | B0 | The bootstrap delivery: every check, a lone pull request's as a group of one and a verdict carried after a trunk move included, is judged by a checking program built from the change's own heads in the build seat's sandbox, which also makes the heads' one seat list that the check hands the manifest validator and the composer; it brings no seat definition and no manifest field, so the running checker checks it (ADR 0644; *The bootstrap delivery, and the one milestone after it*) | 26 needs it | mesh-sdk (`go/delivery`: the seat `node-build-go`'s ask `run` and the checker's record of which checking program judged; `go/composer`: the request's seat list), mesh-catalog (mesh-build-go: the run step, the refused answer, the kept programs' tools and sweep), mesh-delivery (checker: the checking program and its command), hq (this design, the glossary) | 1921 | | G | Two graphs and one revision view: the catalog graph and the assignment graph kept per revision, a change delivered as its graph difference, the acting parts' rules on their seats as bounded waits that hold their modules when the new version fails, the check on the change's own acting parts, the revision view and each process's own build, the gate's delivery record, and the trace's links (ADR 0656); no part reads a pull request's description (ADR 0659); every delivery part reaches the forge through the seat `git` and names no other module; nothing holds an approved merge for another check; and, one milestone with them, research 086's pipelines: a change of any module goes from a commit on a watched branch to the machines through its module's pipeline, its policy in its registration, ordered by edges between artifacts, with no Approve per merge and no hand act, the checker gone (ADR 0666, ADR 0667). Test counts at its start (G's prepare, 2026-11-09): mesh-delivery 665 unit and 68 integration, the node-engine (`mesh-host`) 1,742 unit; each other repository's and each catalog module's from the last check's verdict, read in the joint prepare | 26 to 62, 63 to 106 | mesh-sdk (the runtime: the fingerprint, the view, the definition reader, the seat list's one wiring), mesh-contracts and the domains' contracts modules (`acts`, the new verbs and events, the seat `mesh-grants`, the seat `git`'s verbs), mesh-delivery (indexer, assigner, checker until its removal, builder, grants part, gate's view, forge bridge, watchman, the new pipeline runner `mesh-delivery-pipeline`, `contracts` and `sdk` moved under `modules/`), mesh-catalog (the gate, the build modules, the forge module, mesh-issues, mesh-knowledge, the `node-backup` holder, the router, nats: a module's kept data on the bus as a backed-up item and the object store beside each state), mesh-sdk (also its small object store package, for values of any size), the node-engine (`mesh-host`), hq | 1966, 1961 (retargeted to this row's shape), 1660, 1972, 1721, 1783, 1785, 1794, 1791, 1373, 595; 1962 goes back with 1958, whose assignment entry it needs (ADR 0641 §3) | +| H | The build-and-delivery system delivers itself by its one model, in six releases (H0 to H5, the plan's section 6): one merge call of the seat `git`'s `merge-pull` naming one or more pull requests is a delivery's start and its group; the gate names the merged heads one source revision and opens its record; the indexer takes it in (a commit, the fingerprints of its repositories' modules, one catalog revision) and names the movers; one pipeline per take-in, none in parallel (`pipelines-at-once`, default 1); a want is the newest released version whose pipeline passed; the forge's event is the watchdog (`head-untaken`); every delivery part wakes on its inputs alone and reads no other part's state; the per-pull-request paths, the catch-up by comparison, the sweep and the checker go; one clean state placed by hand once, after which every release is delivered by its own pipeline (ADR 0682, ADR 0681, ADR 0666, ADR 0667; design 47 H §734 to §765) | 110 to 113 | mesh-delivery, mesh-catalog, `mesh-host`, mesh-tools, mesh-sdk, mesh-contracts, hq | opened by each release's prepare | An issue is in one row only (ADR 0493 §2): where a row named an issue another row names too, the last row keeps it and the earlier row says so in words (the operator, 2026-10-26). Milestone M1 makes each row a record of the issue tracker diff --git a/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md b/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md index 070eafa6..e6d9de97 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md +++ b/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md @@ -84,9 +84,10 @@ check and deliver milestone G itself (ADR 0667 §2); each of them says so in a l what moved); the verb `registrations` answers every registration, or one with its history. The indexer derives the modules it keeps from them, and the forge bridge its watched branches, each by a first view of `registrations` and then that event, never by reading the ledger (ADR 0666 §3, ADR 0596). -- §5 **An unprotected branch is refused.** `register` and `watch` read a branch's protection through the seat `git`'s read - verb `branch` (its head, and whether a rule refuses a direct push and admits only merges), and refuse a watched branch - the forge does not protect, naming it (ADR 0666 §3). +- §5 **An unprotected branch is not refused at registration** (corrected in place by design 47 H §763). `register` + and `watch` read no branch's protection and refuse no watched branch for it: the operator approved removing the + check on 2026-11-14 (G-23, *Met during G's start*), which ADR 0666 §3 still states. `merge-pull` still judges the + protection at every merge (ADR 0666 §11, §38), and §17's `branch-unprotected` stands. - §6 **The foundation parts are the manifest validator's list.** The manifest validator's list (ADR 0619 §3) holds exactly the foundation parts, ADR 0619 §1's, the runner, which joins them (ADR 0666 §4), and the forge and image registry modules (§584); the checker leaves the @@ -317,8 +318,10 @@ check and deliver milestone G itself (ADR 0667 §2); each of them says so in a l removed with the checker's buckets and bus account, after the first pipeline after G's delivery has delivered a change end to end and on the operator's word (ADR 0667 §12; *How milestone G itself is checked and delivered*); the forge bridge hears the landed commit as it hears every commit of a watched branch. -- §40 **The operator-channel hides no ask.** It keeps design 46's three open asks per asker, never holds the newest back, - and replaces an open ask by the same asker's ask with the same `about` (ADR 0666 §12; design 46 §6). +- §40 **The operator-channel hides no ask.** It keeps no count of open asks per asker and holds none back: every ask + is offered when it is raised (A-424, the operator, 2026-11-15 about 15:50 CET), and it replaces an open ask by the + same asker's ask with the same `about` (ADR 0666 §12, whose three open asks per asker A-424 ends; design 46 §6; + corrected in place by design 47 H §763). - §41 **How it is checked.** Whole (forge module, forge bridge): a session's `merge-pull` at its head merges with no run yet, and with a passed one, and asks no Approve; with the runner stopped a head merges; a head at whose fingerprint any run failed a `required` step, a head that moved and a branch whose protection was removed since its last move are diff --git a/03-DESIGN/01-to-be/47-milestones/G-10-develop-answers.md b/03-DESIGN/01-to-be/47-milestones/G-10-develop-answers.md index 14d93388..b25a1313 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-10-develop-answers.md +++ b/03-DESIGN/01-to-be/47-milestones/G-10-develop-answers.md @@ -242,8 +242,9 @@ each the design's choice with its reason, §468 to §479 those of the runner's, whole (forge module), a head that passed, then another commit at its fingerprint failed a `required` step, is refused; its `seen` key goes when the pull request closes. - §476 **`take-in.taken`'s body and its type.** (§493) The indexer's own `take-in.taken` carries, beside what it carries on - main, `branch` (the watched branch), `source` (the source revision taken in) and `entries`, one `{module, commit, - fingerprint}` per module of that revision. Its body is defined in the seat `mesh-catalog`'s definition, in its + main, `branch` (the watched branch), `source` (the source revision taken in) and `entries`, one + `{module, commit, fingerprint, repository}` per mover of that take-in (design 47 H §743; corrected in place by H §763, where this line + said one entry per module of the revision). Its body is defined in the seat `mesh-catalog`'s definition, in its section `bodies` (§439, §454), so the generated contract carries the type the indexer writes and the runner reads, and neither imports the other (§333). Why: a body said on a module's own stream and read by others is a `bodies` entry (§439), never a client's hand-written type (develop question Q-RN4). How it is checked: unit (seat generator), diff --git a/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md b/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md index 08ce754f..4997bbbf 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md +++ b/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md @@ -389,9 +389,10 @@ for several build maps is answered once, and the maps' names for it are given in (G-24 §720, §725), and from its `ask-taken` by its step's bound (A-466p). [mesh-delivery DQ-3.] How it is checked: whole (runner), a seat with no holder ends the pipeline at once, naming it; an ask a present holder has not taken past five minutes is not failed (test 109). -- §648 **A `take-in.taken` entry carries the pull request.** (§476, §542) An entry is `{module, commit, fingerprint, - repository, pull}`, the last two from a `merged` message and absent for a `pushed` one. [mesh-delivery DQ-5.] How it - is checked: unit (indexer's contract). +- §648 **A `take-in.taken` entry names its repository, never a pull request.** (§476) An entry is `{module, commit, + fingerprint, repository}`, the repository from the gate's `source-moved` (design 47 H §743); the field `pull` goes, + since nothing reads a pull request after the merge call (ADR 0682 §1; corrected in place by H §763). [mesh-delivery + DQ-5.] How it is checked: unit (indexer's contract). - §649 **`ParseStepKey` answers the asker.** (§484) `buildstep.ParseStepKey(key)` answers `(asker, node, id, err)`, `asker` empty for main's form. [mesh-delivery DQ-8.] How it is checked: unit, both forms. - §650 **The `mesh-builds` client's name.** (§586) It is `contracts/go/clients/meshbuildsclient`, with `Reserved`. @@ -408,9 +409,8 @@ for several build maps is answered once, and the maps' names for it are given in - §654 **The readers of the forge module's old events and tools move to the seat `git`.** (§310, §311, §313, §431; ADR 0659 §1) [mesh-catalog Q-F1.] - - **mesh-config** consumes `git`'s `pull-merged` and `pull-closed` in place of `gitea.pull.merged`, - `gitea.pull.closed` and `gitea.forge.pull-request.closed`, and invokes `seat:git.pull` in place of - `gitea.gitea_get_pull_request`; **the records module** consumes `git.pull-merged`. + - **mesh-config** consumes no pull request event and invokes no `seat:git.pull` (ADR 0681 §1, which replaces this + bullet's first form; corrected in place by design 47 H §763); **the records module** consumes `git.pull-merged`. - **The order.** Main's forge module says its own `pull.merged`, not the seat's events, so each consumer of `git.pull-merged` or `pull-closed` is ordered after the forge module's version that says them, by the edge of kind 6 its `consumes` gives (§558); its durable consumer is made from the stream's first message, so no merge said diff --git a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md index 3fa721d9..e560f934 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md +++ b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md @@ -1275,14 +1275,15 @@ the principle that settles it, cited as "design 47 G A-n". never at the revision its own view holds; while its view holds an older one, it waits for it (§200) and places no step. (Live, 2026-11-16: main.947 was started at revision 356 while its take-in made 357; its composition read 356 after the indexer had removed it.) -- **A-449** (§153, §579, design 47's "kept while a delivery record pins it or it is current"; principle 7) A pipeline - pins its catalog graph revision as a delivery record does: the runner says the seat `mesh-delivery`'s `pinned` when it - starts a pipeline and `freed` when the pipeline ends, under the record id `pipeline.`, so the indexer keeps the - revision while any pipeline runs on it. No new mechanism; who may say them is a grant of the runner's account. +- **A-449** (§153, §579, design 47's "kept while a delivery record pins it or it is current"; principle 7; corrected in + place by design 47 H §763, H §749) A pipeline pins its catalog graph revision: the indexer pins it in the take-in's + write (A-449a) or on the runner's `pipeline-started`, and frees it on the runner's `pipeline-ended`, so the indexer + keeps the revision while any pipeline runs on it. The runner says no `pinned` or `freed` of the seat + `mesh-delivery`; those stay the gate's alone (§153). - **A-449a** (completes A-449; principles 3 and 7) The indexer pins the revision a take-in made for each pipeline the - take-in starts, under `pipeline.` (`..` of each entry), in the same write as it keeps the - revision, so no later take-in can remove it before the runner hears of it. The runner's pin at a pipeline's start stays - (a set: saying it twice is harmless) and its `freed` at the pipeline's end removes it. + take-in starts, under `pipeline.` (`.`, the one pipeline of the take-in, design 47 H §746), in the + same write as it keeps the revision, so no later take-in can remove it before the runner hears of it. Its free is on + the runner's `pipeline-ended` (A-449 as corrected). - **A-226b** (completes A-226 and A-167; ADR 0593 §7; principles 7 and 9) The seat `node-engine`'s `compose-candidate` takes three key forms, parsed by one parser in the seat's client (`nodeengineclient`) that every caller builds its key with: a check's candidate `.` (as today), a branch pipeline's revision `.` and a @@ -1862,7 +1863,7 @@ the principle that settles it, cited as "design 47 G A-n". - **A-475a** (A-475; principles 7, 2 and 11) "In a row" is counted on the pending revision's validation itself (`stale`, the stale ends since its last ask that was not stale), the only place that needs it. The catch-up waits while a validation ask of any repository is out, since every take-in moves the one revision a request is built from; - it still compares the heads and raises A-463's merge-untaken condition meanwhile, and only its take-in waits. + it still compares the heads and raises A-463's `merge-untaken` condition meanwhile, and only its take-in waits. `validate-tries` defaults to 3 and is at most 20. (From the develop and review of A-475.) - **A-476** (the operator, 2026-11-17 ~17:30 CET, from the delivery walkthrough: "no manifest check needed at this point"; "for each module that is registered in our module graph, for the affected repo -> check the fingerprints"; diff --git a/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md b/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md index 99d5d9e4..bf36f2ae 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md +++ b/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md @@ -103,7 +103,9 @@ record says so in its Why. Round 3 of the review by architectural domain (2026-1 `*.findings-stated` are the first. The manifest validator takes `*` only as the whole emitter, and only before an event that a seat's definition or a contract in mesh-sdk defines, with its fields (for the finding events, the seat `mesh-conditions`' definition, §439); it refuses `*.>` and `*` in part of an emitter's name, and a `*` entry before any other event keeps its meaning on main (§474). The grants part, from the catalog graph as it derives every grant (§59, §442), makes on each declaring module's stream one durable consumer - for the consumer, named `_` as ADR 0363 §5 names one, with its grants to read and acknowledge it and + for each process of the consumer that hears it, named as A-397 names a process's durable (`_` for + its program, `__tools` for its tool runner, `__` for another; corrected in + place by design 47 H §763, where one per module stood), with its grants to read and acknowledge it and its membership's word to bind it; it drops that consumer when the module no longer declares the event, and makes one for a module that declares it later, with no change to the consumer's manifest. A `*` entry never reaches a restricted stream (one with a `grant`, ADR 0363 §4). Why: the keeper hears modules it finds in the catalog and its diff --git a/03-DESIGN/01-to-be/47-milestones/H-1-the-merge-call-and-the-take-in.md b/03-DESIGN/01-to-be/47-milestones/H-1-the-merge-call-and-the-take-in.md new file mode 100644 index 00000000..2f2f5f23 --- /dev/null +++ b/03-DESIGN/01-to-be/47-milestones/H-1-the-merge-call-and-the-take-in.md @@ -0,0 +1,242 @@ +--- +layer: to-be +status: in-progress +design: 47 +milestone: H +updated: 2026-11-20 +decisions: + - "ADR 0682" + - "ADR 0681" + - "ADR 0667" + - "ADR 0666" + - "ADR 0656" + - "ADR 0633" + - "ADR 0596" + - "ADR 0573" +--- +# 47, milestone H: the merge call, the watchdog, the take-in and one pipeline at a time + +One file of milestone H's part of [design 47](../47-delivery-from-commit-to-delivered.md); the plan that opens the part +is [H-deliver-the-delivery-system](H-deliver-the-delivery-system.md). Its design lines continue the numbering of +milestone G's part after §733 and are cited as "design 47 H §n". Each line says which release of the plan's section 6 +builds it (H1 to H5), how it is checked, and which line of G or lead's answer it replaces; where the code it lands in +is named, it is named by module and function, from the gap tables of 2026-11-20, for the builder to find it. + +*The operator, 2026-11-19 about 23:30 CET: "Nobody should 'take in a merge', only commits to the configured branch of +registered modules." 2026-11-20 about 01:10 CET: "parallel pipelines sounds like a very bad idea". 01:15 to 01:20 CET: +a delivery is started by hand by the merge it already needs, the lead named principle 3 (a start a person forgets is +a silent non-delivery), and the operator chose "A": the merge through the mesh is the start and the group, and the +forge's event is the watchdog. ADR 0682 records it. The chain these lines build, in one sentence: a **merge call** +(the seat `git`'s `merge-pull` naming one or more pull requests) merges the heads, the gate names them one source +revision per watched branch and opens its delivery record, the indexer takes that revision in and names the movers, +the runner runs one pipeline for it when no other runs, and the assigner wants the newest passed release; the forge +bridge's one message per head move is heard only by the gate's watchdog, which raises an **untaken head** for a head +no merge call delivered.* + +```mermaid +flowchart LR + S[session or operator] -->|git.merge-pull pulls| F[forge module, seat git] + F -->|merge-called| G[gate, seat mesh-delivery] + G -->|source-moved| I[indexer, seat mesh-catalog] + I -->|take-in.taken: movers| R[runner, seat mesh-pipeline-runner] + R -->|pipeline-ended| A[assigner, seat mesh-wants] + F -->|branch-moved| B[forge bridge] + B -->|head-moved| G + G -.->|head-untaken| K[condition keeper] +``` + +### The merge call + +- §734 **One merge call is a delivery's start and its group.** (ADR 0682 §1, ADR 0666 §11; the operator, 2026-11-20 + 01:20 CET; principle 3) The seat `git`'s verb `merge-pull` takes `pulls`, a list of one or more members + `{repository, number, head}` (the head the caller names, as ADR 0666 §11 has it), and `by`, the caller as who@node. + The members named in one call are one delivery and, when there are two or more, its group; nothing else makes a group. + The verb's single-pull form goes: a call naming one pull request is a list of one. Built in H1 in the forge module's + `merge-pull` handler and the seat `git`'s contract in mesh-delivery-contracts. How it is checked: whole (forge + module): a call naming two pull requests in two repositories merges both and says one `merge-called` (§736). + Replaces G §38's one-pull form. +- §735 **Every member is judged before any is merged.** (ADR 0666 §11; principle 3) The verb judges each member as + ADR 0666 §11 does (still the pull request's head, a branch protection that refuses a direct push, read again; no run + at the head's build-input fingerprint failed a `required` step, from the forge module's own `failed-steps`), all of + them first. When one member is refused, none is merged, and the answer names every refused member and why. Then it + merges the members in the order the call names them. Why this order: a group merged in part would land half a + delivery (ADR 0666 §13's "none delivered if one fails", kept for the call's group). What the call says when the + forge refuses a member after all were judged is question 1 of *Questions this file leaves open*. Built in H1 (forge + module). How it is checked: whole (forge module): a call whose second member's head moved merges neither and names + it. +- §736 **The merge call says one event.** (ADR 0682 §1; principle 1) When every member is merged, the forge module says + the seat `git`'s event `merge-called` on `mesh.seat.git.event.merge-called.`, kept on the stream + `SEAT_git_merge-called` for 30 days and up to 64 MiB, its body in the seat's `bodies`: `call` (an id the forge module + makes once per call: 16 lowercase hex), `by`, `at` (the time of the last merge) and `members`, one + `{repository, number, branch, head, commit}` per member, where `branch` is the pull request's base (a watched + branch) and `commit` the merge commit the forge records for that pull request. Only the forge module says it. Its + one consumer is the gate (§737), on a durable consumer of its program. Why 30 days: the gate takes in a call heard + after a long stop, and the forge module's `seen` keys are kept as long (G §585). Built in H1 (forge module, seat + `git`'s definition and contract). How it is checked: unit (the seat definition: the event and its body); + whole (forge module) as §734. +- §737 **The gate names the merged heads one source revision and opens the record.** (ADR 0682 §1; principles 1 + and 12) The gate consumes `merge-called`. For each watched branch its members name, it writes `source.` once, + by compare-and-set: the branch's previous heads with each member's repository at its `commit`, numbered one above + the branch's newest, carrying `call`. It opens one delivery record per such revision, keyed `.` + (A-33 stands), its members the call's members on that branch, `call` and `by` in its trace, and says `source-moved` + with `branch`, `source`, `call`, `start` (false; true only for A-401's start pass of a newly watched branch, which + stays) and `heads` (the repositories it moved, each `{repository, commit}`). A + `merge-called` heard again writes nothing: its record's key exists. A call whose members name two branches is two + source revisions and two records, each naming the same `call`. The gate numbers every source revision; no other part + keeps a number of its own. Built in H1 in the gate: its `moveSource` moves from the `take-in.taken` handler to a + `merge-called` handler; `ForgeMerged`, `moveHead` and the tool `gate_source_follow` go (§764). How it is checked: + whole (gate): a `merge-called` with two members on one branch gives one revision, one record and one + `source-moved`; the same event heard twice gives one. Replaces A-34 (the source moved on `take-in.taken`), G §291, + A-431's numbering by the indexer, and A-464 to A-464e. +- §738 **A delivery record follows its pipeline to the machines.** (ADR 0666 §9; G §141 to §147) The record opened by + §737 is the record of G §141 to §147, one per source revision: it consumes the runner's `pipeline-started` and + `pipeline-ended`, the parts' `wait-held` and `wait-ended`, and the node-engine seat's `running`, on the gate's own + durable consumer, writes its trace (the assignment graph revisions it moved between, from the assigner's `wanted`), + and ends as G §147 says. A take-in with no movers (§741) ends its record at once as `nothing to deliver`. Built in + H3 in the gate's consumer (`stepFamilies`) and record (`SetTrace`, `EndIfRun`). How it is checked: whole (gate): + a record of a one-module delivery ends when the module runs on every placing machine (acceptance test 110). + +### The forge bridge's one message, and the watchdog + +- §739 **The forge bridge says one message per move of a watched branch's head.** (ADR 0682 §2, ADR 0666 §6; G §11, + A-236) On every `git.branch-moved` of a watched branch, and at its start for every watched branch's head, the bridge + reads the head through `git.branch` (never the event's body) and writes one message `head-moved` on its stream + `forge`, subject `forge...head-moved`: `repository`, `branch`, `head`, `before` (the head it held + for that branch in its state `heads`, empty the first time) and `start` (true when said at its start). It writes it + whether the head moved by a merge call, a merge in the forge's screen or an admitted push: it reads no pull request + and no tree. A head equal to the one it holds writes nothing. An unprotected watched branch writes nothing and is + said as `branch-unprotected` (G §17 stands). Its one consumer is the gate (§740). Built in H1 in the bridge; the + messages `merged` and `pushed`, their `base`, their trees in its state `inputs`, and its call of `git.pull` for a + merged commit go (§764). How it is checked: whole (bridge): a merge, an admitted push and a restart each give one + `head-moved`; a head heard twice gives one. Replaces G §509 and §714's message form; this is item 6 of the plan's + section 4. +- §740 **A head no merge call delivered is an untaken head.** (ADR 0682 §2; principles 1 and 3) The gate consumes + `head-moved` and keeps, in its template state `heads` (one key `.`, value `{head, since}`, the + key deleted by the gate when `source.` names that head), each head that `source.` does not name for + that repository. When `untaken-bound` (a setting of the gate, default 10 minutes, at least 1) passes and + `source.` still does not name that head, the gate raises the finding `head-untaken`, scope `delivery`, id + `.`, naming the repository, the branch, the head, since when, and "moved outside the mesh: + delivered by nothing". It clears it when `source.` names that repository at the newest head `head-moved` + said (a later merge call delivers it). The gate keeps one deadline timer, set to the earliest `since` plus the bound, + and no other clock. Built in H1 in the gate. How it is checked: whole (gate, bridge): a head moved by a merge call + raises nothing; a head pushed outside the mesh raises `head-untaken` within the bound, starts no pipeline, and + clears when a later merge call takes its repository in (acceptance test 111). Replaces A-463's `merge-untaken` and + the head half of G §12 (§741 keeps its step half). +- §741 **`pipeline-stands` times a pipeline's steps, not a head.** (ADR 0666 §24; principle 7) The watchman's finding + `pipeline-stands` (G §12, §75) is raised for a pipeline step standing still five minutes; a watched branch's head + that starts no pipeline is the gate's `head-untaken` (§740) alone, so the watchman learns no head. A pipeline waiting + its turn (§746) is progress not owed and raises neither. Built in H1 in the watchman's `takein` and `signals` parts. + How it is checked: whole (watchman): a pipeline waiting behind another raises nothing. Answers question 6 of + specification A. +- §742 **Taking an untaken head in by hand.** (ADR 0682 §2; principle 4) The gate's tool `gate_heads` reads the state + `heads` with each head's since and finding; `gate_head_take`, with `repository`, `branch` and `why`, names that head + a source revision as §737 does, its `call` `hand..