diff --git a/00-META/glossary.md b/00-META/glossary.md index a40d471c..b99573dc 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -620,6 +620,15 @@ controller (the planner, the rollout) until release 6 of ADR 0429; from each rel - **composition step** — the pipeline step that composes each machine's declaration with a change, by the composer built from its heads in the sandbox, refusing a change in which two modules own one path; served by the seat `mesh-wants` (ADR 0666 §17). From milestone G's delivery it replaces the **composition check**. +- **composition run** — the one run of the composer for one candidate on one machine, started by the first + composition step of that candidate that needs the machine and kept by the assigner in its state `composition-runs`; + every composition step of the candidate whose module is placed on the machine reads its verdict (design 47 G §718). + Not a **composition**, one module's resources composed for one node; nor the **composition step**, the pipeline + step that reads one composition run per machine. +- **test run** — the one run of a lint, unit test or integration test pipeline step for one candidate, one level and + one set of tested packages at their digests, kept by the runner in its state `test-runs`; every artifact's pipeline step of that candidate + naming the same packages reads its result (design 47 G §724). Not a **pipeline step**, each artifact's own, nor a **run + on request**, a whole pipeline that delivers nothing. - **merge check** — what a pull request carries before it may merge, judging the merge of its head into a named **trunk commit**, made in a clone of its own on a build node and never pushed (ADR 0323). One status, `mesh/check`, written by the checker: the change's tests, run by the language's build module, and the diff --git a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md index cb1f1fe5..4e852694 100644 --- a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md +++ b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md @@ -2,7 +2,7 @@ layer: to-be status: in-progress code: [mesh-controller, mesh-catalog modules/mesh-delivery-gate, mesh-catalog modules/gitea, mesh-delivery, mesh-host, mesh-sdk, mesh-vault, mesh-contracts, mesh-tools, mesh-networking, hq 00-META/checks] -updated: 2026-11-11 +updated: 2026-11-17 decisions: - "ADR 0673" - "ADR 0672" @@ -1972,6 +1972,27 @@ ADR 0667's checks, of milestone G itself: `condition-raised` (whole, condition keeper and a fixture observer, mesh-catalog and mesh-sdk). Live: after G's delivery, a node-engine restarted with the full seat list clears its `seat-list-unread`, read with `mesh-conditions.conditions`. +108. **One composition run per candidate and machine** (design 47 G §718 to §723, §727 to §733). Two composition + steps of one candidate started at once make one key, one request and one run ask; three steps of one revision on + two machines start two runs, each step reading both verdicts. A machine whose verdict blames one module fails that + module's composition step alone and says it as a warning in the others; a verdict with no `blamed` is judged as + today, naming no culprits. A run with no `step-ended` within its bound ends `could not run` with its readers; steps + asked again together start one run per machine that did not pass and move every former reader, a reopened + pipeline saying `pipeline-ended` again; a run that keeps saying its progress past 30 minutes is not ended, and a + taken ask silent for 30 minutes ends `could not run`. Both drop tools end + the readers, and a reader-only account calling one is refused (whole, assigner and the node-engine's composer, + mesh-delivery and `mesh-host`). Live: in the first publish after G's delivery, the composition run asks of a + revision number at most its machines, read with `assigner_compositions` and `assigner_inputs`. +109. **One test run per level and tested packages** (design 47 G §724 to §733). A module whose program and tools + bundle test one directory runs its unit and integration tests once per level, both artifacts' pipeline steps + ending with that result and naming it in `shared`, a failing test failing once, and `pipeline-ended` counting the + module's tests once. A rerun that passes moves both artifacts' steps and the pipeline ends again; a runner + restarted during a run takes it up again; a replaced reader leaves the ask to the others; at the cap a step ends + `could not run` naming the fill; a second revision with the same digests runs a failed test again; a test run queued past five minutes with a + holder present is not failed (design 47 G A-466y) (whole, runner + and Go build module, mesh-delivery and mesh-catalog). Live: each module's test steps run once per level, read with + `runner_test_runs`. A reader no part ends raises `pipeline-stands` within five minutes of its last sign of life + (whole, watchman; design 47 G §732). ## What retires diff --git a/03-DESIGN/01-to-be/47-milestones/G-12-develop-answers-fourth.md b/03-DESIGN/01-to-be/47-milestones/G-12-develop-answers-fourth.md index c4696164..8fc8e87d 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-12-develop-answers-fourth.md +++ b/03-DESIGN/01-to-be/47-milestones/G-12-develop-answers-fourth.md @@ -384,12 +384,15 @@ that round are written into the lines they name, here. Questions that an earlier - §535 **A missing build seat and a seat with no holder, each read where it shows.** The runner reads that no build seat serves an artifact's language from the catalog half of its revision view: no entry whose `acts`, the acting rules of the seats it claims, hold a rule `build` with `on` naming that language (§323); the pipeline then ends - failed at once, naming the language and the missing build seat. It reads that a seat has no holder from the step - itself: an ask on that seat's work queue whose `step-started` is not heard within five minutes ends the pipeline - failed, naming the seat (§32). It reads neither from the seat list, which answers definitions alone (§527), nor - from any holding. Why: §32 already ends a pipeline whose step nothing starts, and the catalog half is the runner's - one view (develop question Q-RN7). How it is checked: whole (runner), a fixture language with no build seat ends at - once, naming it; a seat whose holder is stopped ends within five minutes, naming the seat. + failed at once, naming the language and the missing build seat. *Amended by A-466y and A-466z (2026-11-17):* it reads that + a seat has no holder only when the seat `mesh-module-assigner`'s view `seat-holders` names none for it (G-21 §717's + follower; never the seat list, which answers definitions alone, §527), and a seat with none ends the pipeline failed + at once, naming it; an ask waiting for an existing holder is timed by §86's `build-capacity-waits` and A-466s's + `run-untaken-bound` (G-24 §720, §725), never by five minutes from the ask. Why: §32 already ends a pipeline whose + step nothing starts, and the catalog half is the runner's one view (develop question Q-RN7); a queued step ended + failed for "no holder" while its holder is present would say what is not true (principle 3). How it is checked: + whole (runner), a fixture language with no build seat ends at once, naming it; a seat the seat-holders view names + no holder for ends at once, naming the seat; a step queued past five minutes with a holder present is not failed. - §539 **The runner's tools are named `runner_`.** The runner's module, `mesh-delivery-pipeline`, names its tools with the prefix `runner_`, so the design's `runner_catch_up`, `runner_waits`, `runner_release` and `runner_health` stand, and its first-view tool is `runner_first_views`; `_first_views` (§205) carries the prefix the diff --git a/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md b/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md index 92f675b1..08ce754f 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md +++ b/03-DESIGN/01-to-be/47-milestones/G-17-develop-answers-seventh.md @@ -378,9 +378,17 @@ for several build maps is answered once, and the maps' names for it are given in - §646 **The runner's bodies name `fingerprint`.** (§585) The runner's `step-ended`, pipeline and wait bodies name the build-input fingerprint `fingerprint`, not `fp`, as §585 reads it. [mesh-delivery DQ-2.] How it is checked: unit (runner's contract). -- §647 **The runner times an ask from `ask-taken`.** (§535, §602) §535's five minutes run from the ask until the build - seat holder's `ask-taken` (§602), which the runner consumes; `step-started` is the runner's own. [mesh-delivery - DQ-3.] How it is checked: whole (runner), an ask no holder takes ends its pipeline after the setting. +- §647 **The runner times an ask from `ask-taken`.** (§535, §602; A-466y) The runner consumes the build seat holder's + `ask-taken` (§602); `step-started` is the runner's own. *Amended by A-466y and A-466z (2026-11-17):* §535's five + minutes from the ask until `ask-taken` go. A seat for which the seat `mesh-module-assigner`'s view `seat-holders` + names no holder ends the pipeline failed at once, naming it, judged only once that view's first view is read, never + from the not-ready picture `FollowSeat` answers before it (principle 3). The runner's manifest gains the `consumes` + entry `seat:mesh-module-assigner.views.seat-holders` (on main it has none: its `invokes` names only + `seat:mesh-module-assigner.assignments`); an + ask waiting for an existing holder is timed by §86's `build-capacity-waits` and by A-466s's `run-untaken-bound` + (G-24 §720, §725), and from its `ask-taken` by its step's bound (A-466p). [mesh-delivery DQ-3.] How it is checked: + whole (runner), a seat with no holder ends the pipeline at once, naming it; an ask a present holder has not taken + past five minutes is not failed (test 109). - §648 **A `take-in.taken` entry carries the pull request.** (§476, §542) An entry is `{module, commit, fingerprint, repository, pull}`, the last two from a `merged` message and absent for a `pushed` one. [mesh-delivery DQ-5.] How it is checked: unit (indexer's contract). diff --git a/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md b/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md index 59f768e6..8d701d2f 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md +++ b/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md @@ -3,7 +3,7 @@ layer: to-be status: in-progress design: 47 milestone: G -updated: 2026-11-11 +updated: 2026-11-17 decisions: - "ADR 0667" - "ADR 0666" @@ -149,7 +149,7 @@ One file of milestone G's part of [design 47](../47-delivery-from-commit-to-deli | `pipeline-started`, `pipeline-ended` | a pipeline's start, and its end with every step's result, its test counts, the source revision and `widens` | | `group-ended` | a group's one pipeline's end, on which alone its wants are written | | `run-started` | a run on request's start, with its members at their commits | - | `step-started`, `step-ended` | each step's start and end: the step, its status in the policy, its outcome, its build-input fingerprint, its test counts, for a carried result what it was carried from, and for a rerun that it is one and the caller the runtime proved; a bypassed step too | + | `step-started`, `step-ended` | each step's start and end: the step, its status in the policy, its outcome, its build-input fingerprint, its test counts, for a carried result what it was carried from, and for a rerun that it is one and the caller the runtime proved, for a step that read a test run or a composition run another step asked its `shared` (G-24 §729); a bypassed step too | | `wait-held`, `wait-ended` | a step's wait: on a person (approval, code review), on a build node, on the revision view | - §74 **What a session hears after its merge.** Between its `merge-pull` and the delivery, a session waits with `mesh_wait` @@ -195,14 +195,14 @@ four lines a module's tools answer; every read tool of a store also answers its | Part (seat) | Read everything it holds | Do by hand what it does itself | Recover without help | |---|---|---|---| -| the runner (`mesh-pipeline-runner`) | `show` (one pipeline whole, its steps, step log and outputs), `list` (the pipelines of a module, a branch or a commit), `stands` (what stands still, since when), `why-carried` (where a step's result was carried from), `kept` (the kept results by digest) | `start` (a delivering pipeline for a watched branch's head), `run` (the run on request), `rerun` (one step again) | `accept` (a failed `optional` step, with why), `cancel` (a pipeline, dropping its asks), `ask-again` (a declined `approval`) | +| the runner (`mesh-pipeline-runner`) | `show` (one pipeline whole, its steps, step log and outputs), `list` (the pipelines of a module, a branch or a commit), `stands` (what stands still, since when), `why-carried` (where a step's result was carried from), `kept` (the kept results by digest), `runner_test_runs` (each test run, G-24 §728) | `start` (a delivering pipeline for a watched branch's head), `run` (the run on request), `rerun` (one step again) | `accept` (a failed `optional` step, with why), `cancel` (a pipeline, dropping its asks), `ask-again` (a declined `approval`), `runner_test_runs_drop` | | the indexer (`mesh-catalog`) | `registrations` (with each one's history); `indexer_edges` (an artifact's edges per watched branch), `indexer_order` (the order of a change, its cycles named), `indexer_catalog_difference` (two revisions); `indexer_pending` (a group with its missing members) | `register`, `policy`, `watch`, `unwatch` (each on the operator's Approve) | `indexer_seed_again` (the first registrations' ask raised again), `indexer_take_in_pending` (a group taken in as gathered), `indexer_drop_pending` (dropped) | | the gate (`mesh-delivery`) | `source` with a branch (`gate_source`: every branch's source revision) | — | `gate_source_again` (say `source-moved` again for a branch) | | the forge module (`git`) | `git_failed_steps` (the failed `required` steps it holds, by module and fingerprint, with the run that failed each) | — | `git_failed_steps_drop` (drop one, with why) | | the grants part (`mesh-grants`) | `grants_callers` (the callers each verb admits, read from the seat definitions) | — | `grants_why_refused` (for an account and a verb, the grant it lacks and the definition's line that leaves it out) | | the build seats | `steps` (the steps detected for an artifact) | `steps` with `again` (detect again) | `drop` (ADR 0664 §4) | | the builder (`mesh-builds`) | `builder_reserved` (each module's reserved version, §369); the released versions are the catalog's `versions` | `builder_publish_again` (publish the missing artifacts under the reserved version) | — | -| the assigner (`mesh-wants`) | `assigner_why_not` (why a module is not on a machine yet: the pipeline it waits for, its hold, the placement, the node's last `running`) | — | `assigner_release` | +| the assigner (`mesh-wants`) | `assigner_why_not` (why a module is not on a machine yet: the pipeline it waits for, its hold, the placement, the node's last `running`), `assigner_compositions` (each composition run, G-24 §728) | — | `assigner_release`, `assigner_compositions_drop` | | the watchman | `watchman_views` (how far behind each part's view is, from each part's `view-waits` and first-view tools) | — | each part's `_first_view_again` | | the operator-channel | `asks` (with each asker's asks held back) | — | `ask-again` (offer a held-back ask now) | | every module that imports mesh-sdk's small object store package | `_state_objects`, `_sent_objects` | — | `_state_objects_drop`, `_sent_objects_drop` | @@ -230,6 +230,8 @@ and an estimated 2 mesh-sdk publishes a day, each rebuilding about 120 artifacts | the builder | `reserved`: one key per module and `MAJOR.MINOR` (§369) | old `state` section | until its `version-released`, a release of another `MAJOR.MINOR` of the module, or `builder_reserved_drop` | a few KiB; at the cap the builder reserves nothing and says why | | the assigner | `written`: the last source revision and end count per module and watched branch (§352) | old `state` section | until the branch is unwatched | a few KiB | | the indexer and the assigner | `inputs`: a `run` ask's input by write time and sha256 (§354) | old `state` section | until the ask's `step-ended`, at most a day | 64 MiB; an estimated 300 asks a day of under 100 KiB each is under 30 MiB; at the cap the asker refuses the ask loudly, naming the store | +| the assigner | `composition-runs`: one key per candidate and machine, its program, input, ask, readers and verdict (G-24 §719) | old `state` section, `max-mib` the setting `composition-runs-cap-mib` | the setting `composition-runs-kept` (default 24 hours) after its end; deleted by the assigner (§719) | default 16 MiB; an estimated 30 candidates a day on four machines, 120 keys of at most 8 KiB, is under 1 MiB a day | +| the runner | `test-runs`: one key per candidate, level and tested packages, its ask, readers, outcome and counts (G-24 §725) | old `state` section, `max-mib` the setting `test-runs-cap-mib` | the setting `test-runs-kept` (default 7 days) after its end, deleted by the runner in the step log's order with its output object in `test-runs-objects` (§330) | default 16 MiB; an estimated 1,000 test runs a day of about 1 KiB is about 7 MiB at 7 days | | the runner and the assigner | `findings` (§355), read by their own read tools alone; the keeper hears the findings as events (§383) | old `state` section | an entry deleted when its finding clears | a few KiB | | every observer (§387) | the stream `-findings`: `finding-written`, `finding-cleared`, `findings-stated` (§383) | old `streams` section for a foundation part, the module's `streams` otherwise | 7 days, its oldest dropped when full; the next statement covers a lost one (§384) | 16 MiB; an estimate of a few hundred findings a day of about 1 KiB is under 2 MiB a week | | the condition keeper | `findings-views`: one key per observer and node, its last stated set, pending keys and open asks (§388) | the form of its `conditions` | deleted when the module leaves the catalog or the node (§365) | under 1 MiB; about 30 observers on four machines | diff --git a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md index 2f32cfb5..f6deae07 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md +++ b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md @@ -245,8 +245,9 @@ the principle that settles it, cited as "design 47 G A-n". `view_tiers` call for a source revision compares the kept assignment revision whose catalog for that branch is that source revision with the revision kept before it. The view keeps a state `waits` (key owner, step, node, module) written by its process from every part's `wait-held` and `wait-ended` it consumes, read by its own read tool. -- **A-243** (corrects A-226's joining; A-237, A-238; principle 7) The composition step runs one request per machine, - each carrying that machine's input; no inputs are joined. A-237 holds for every run's input (the assigner's and +- **A-243** (corrects A-226's joining; A-237, A-238; principle 7; amended by design 47 G §718 on 2026-11-17) The + composition step's machines are composed by one request per machine and candidate, shared by every composition step + of that candidate (§718), each request carrying that machine's input; no inputs are joined. A-237 holds for every run's input (the assigner's and `mesh-wants`' `input` too): any size, kept with an `-objects` store, answered in chunks. The composition run names no head (A-238); its program is the node-engine fingerprint's entry (repository, commit, directory) with the composer's package. @@ -284,7 +285,9 @@ the principle that settles it, cited as "design 47 G A-n". come from the step's catalog revision through mesh-catalog's `revision` (each entry's manifest digest and verdict, its `definitions`), depth compose, the one machine, one case `change`. A machine's node-engine build is what the assigner keeps from the node-engine seat's own events, never read from the node-engine's bucket; where the step's - revision's node-engine entry differs from the trunk's, that build serves every machine. One run ask per machine. + revision's node-engine entry differs from the trunk's, that build serves every machine. One run ask per machine and + candidate, which every composition step of that candidate placing a module on that machine reads (amended by design + 47 G §718 on 2026-11-17). - **A-253** (A-249; principle 8) The tool runner hands a bundle a chunked answer as the verb's own answer with the whole file added as the member `file` (base64). - **A-254** (§712, §354, A-240, A-247, ADR 0596; principles 1, 2 and 7) A build seat's `step-ended` of a run carries @@ -1535,6 +1538,78 @@ the principle that settles it, cited as "design 47 G A-n". time, and an input with its object once no ask's key names it, so no bookkeeping key of its own is kept. A key of the old form `