|
|
|
@@ -3,7 +3,7 @@ layer: to-be
|
|
|
|
|
status: in-progress
|
|
|
|
|
design: 47
|
|
|
|
|
milestone: G
|
|
|
|
|
updated: 2026-11-11
|
|
|
|
|
updated: 2026-11-17
|
|
|
|
|
decisions:
|
|
|
|
|
- "ADR 0667"
|
|
|
|
|
- "ADR 0666"
|
|
|
|
@@ -25,6 +25,7 @@ decisions:
|
|
|
|
|
- "ADR 0448"
|
|
|
|
|
- "ADR 0363"
|
|
|
|
|
- "ADR 0283"
|
|
|
|
|
- "ADR 0267"
|
|
|
|
|
- "ADR 0240"
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
@@ -112,6 +113,139 @@ One file of milestone G's part of [design 47](../47-delivery-from-commit-to-deli
|
|
|
|
|
pipelines up again. Live: once the checker is removed no module claims the seat `mesh-delivery-checker`. Tests 64,
|
|
|
|
|
84 and 88.
|
|
|
|
|
|
|
|
|
|
### One composition run per candidate and machine, one test run per tested packages
|
|
|
|
|
|
|
|
|
|
*§718 to §728 come from milestone G's live delivery of 2026-11-17. The 24 pipelines of one catalog revision asked 96
|
|
|
|
|
composition runs, one per pipeline and machine, and 80 more when they were asked again; each run kept its own copy of
|
|
|
|
|
a node's input of 1.1 to 1.5 MB in the assigner's state `inputs`, which reached its 64 MiB cap twice. The indexer's
|
|
|
|
|
and the teller's same failing tests failed once per artifact. The operator, 2026-11-17 about 10:20 CET: "In G, now".
|
|
|
|
|
Each line is a choice within ADR 0666 §7, §8 and §17, settled by the principle it names.*
|
|
|
|
|
|
|
|
|
|
- §718 **One composition run per candidate and machine.** (§354, A-226, A-243, A-252; ADR 0666 §17; principle 7) The
|
|
|
|
|
assigner, holding the seat `mesh-wants`, composes each machine once per candidate, whatever number of composition
|
|
|
|
|
steps name it. The candidate is named by its key (A-226): `<branch>.<revision>` for a branch pipeline, `run-<id>` for
|
|
|
|
|
a run on request. The first composition step of a candidate that needs a machine starts that machine's composition
|
|
|
|
|
run: one `compose-candidate` ask of the node, its request kept once in `inputs`, one `run` ask on `node-build-go`.
|
|
|
|
|
Every other composition step of that candidate whose module is placed on that machine (A-252) reads that run's
|
|
|
|
|
verdict. A run's request depends only on the candidate and the machine (the candidate's manifests, verdicts and
|
|
|
|
|
seat definitions, depth compose, the one machine, the case `change`), never on the module whose pipeline asked it,
|
|
|
|
|
so one verdict serves every module's pipeline. The composer program is chosen once, when the run starts, by A-252's
|
|
|
|
|
and A-451's rule, and recorded in the run. This replaces "one run ask per machine" for each composition step (A-243,
|
|
|
|
|
A-252) with one per candidate and machine.
|
|
|
|
|
- §719 **Kept in the assigner's state `composition-runs`.** (§354, §85; ADR 0666 §4, §7; principles 1, 2 and 7) Each
|
|
|
|
|
composition run is one key of the assigner's old `state` section `composition-runs`, whose `max-mib` is 16. The key
|
|
|
|
|
is `<candidate key>.<machine>`, each part escaped as a key's name (`keys.EscapeName`). The step that starts the run
|
|
|
|
|
writes the key create-only, so two steps of one candidate starting at once start one run and the second reads it.
|
|
|
|
|
The value holds:
|
|
|
|
|
- `candidate` and `machine`;
|
|
|
|
|
- `program`: repository, commit, directory, and the node-engine fingerprint it was chosen from;
|
|
|
|
|
- `input`: the digest of the kept request;
|
|
|
|
|
- `ask`: the id of the build seat's run ask;
|
|
|
|
|
- `started`;
|
|
|
|
|
- `readers`: the step keys of the composition steps reading the run, each added and removed by compare-and-set;
|
|
|
|
|
- at the run's end, `ended`, `outcome` (`passed`, `failed` or `could not run`, with its words) and `verdict` (that
|
|
|
|
|
machine's composer verdict, §720).
|
|
|
|
|
|
|
|
|
|
A step waiting on a running composition run hears its end through the assigner's own watch of that key (a module
|
|
|
|
|
watches its own state, principle 1). A restart of the assigner loses nothing: at its start the assigner takes up
|
|
|
|
|
again each key with no `ended`, by its `ask`'s `step-ended` (A-240), and ends `could not run` a key whose `ask` the
|
|
|
|
|
build seat no longer holds, naming the ask. The assigner deletes a key a day after its `ended`, and a key with no
|
|
|
|
|
`ended` a day after its `started`, ending each of its readers `could not run`; both are judged as the state `inputs`
|
|
|
|
|
is (A-465b). At the cap no run is started, and its steps end `could not run`, naming the fill (as A-465a).
|
|
|
|
|
- §720 **A machine's verdict names the modules it blames.** (ADR 0666 §17; A-452; principles 10 and 3) The composer's
|
|
|
|
|
verdict of a machine (mesh-delivery-sdk's `composer.Machine`, answered by the node-engine's `mesh-compose`) gains
|
|
|
|
|
`blamed`, a list of `{module, problem}`. It names each module that the candidate's composition of the machine holds
|
|
|
|
|
or refuses and the composition without the candidate does not (the modules in its input's `HeldBefore` are left
|
|
|
|
|
out). The culprits are found by the node-engine's own rule of holding one module at a time until the problem ends,
|
|
|
|
|
as the indexer's validator names a manifest whose removal ends a problem; two modules owning one path are each
|
|
|
|
|
blamed, with that path. `composes` stays false when the machine does not compose with every module of the
|
|
|
|
|
candidate. A machine that does not compose and blames no module fails on the node's own account (its own
|
|
|
|
|
composition, or no module composed at all, as A-452 has it).
|
|
|
|
|
- §721 **One module's fault fails only its own pipelines.** (§720; ADR 0666 §9, §17; A-452; principles 10 and 3) A
|
|
|
|
|
composition step judges each of its machines from that machine's verdict:
|
|
|
|
|
- `failed` when the verdict blames the step's module, or when the machine does not compose and blames no module;
|
|
|
|
|
- `could not run` when the composition run could not run;
|
|
|
|
|
- `passed` otherwise, with a warning for each module blamed there that is not the step's own (`<machine> holds
|
|
|
|
|
<module>: <problem>`). Another module's fault is so said in every pipeline it touches and fails none of them.
|
|
|
|
|
|
|
|
|
|
The step's outcome then follows from its machines, as today: failed before could not run before passed (§354). A
|
|
|
|
|
machine that is away passes with a warning and is not composed (§64).
|
|
|
|
|
- §722 **What asking a composition step again does.** (§83 `rerun`; ADR 0666 §11, §24; principle 7) The runner's
|
|
|
|
|
`rerun` of a composition step asks the assigner for the step again, marked as a rerun (§73). A machine that passed in
|
|
|
|
|
the step keeps its verdict. For each machine whose verdict in the step was not `passed`, the step:
|
|
|
|
|
- joins its candidate's and machine's composition run while that run is running;
|
|
|
|
|
- reads that run's verdict when the run started after the step's last end;
|
|
|
|
|
- otherwise starts a new run, replacing the ended key by compare-and-set.
|
|
|
|
|
|
|
|
|
|
So twenty composition steps of one revision asked again together start one run per machine. A new run's verdict
|
|
|
|
|
counts for no other step until that step is itself asked again or asked for the first time.
|
|
|
|
|
- §723 **How a composition run ends and is said.** (§73, §116, §496, §119, A-182; ADR 0666 §7, §21; principles 7 and 1)
|
|
|
|
|
A composition run says no event of its own. Its end is the build seat's `step-ended` of its `ask`, which the
|
|
|
|
|
assigner alone hears (A-240). Each composition step reading the run says its own `step-ended` on `mesh-wants`, as
|
|
|
|
|
today. That event's output names, per machine, the run it read (`shared`: the run's key and its `ask`) and the
|
|
|
|
|
verdict. So the runner, the forge module's `failed-steps` and the delivery record hear every pipeline's result from
|
|
|
|
|
the step's holder, as before (ADR 0666 §21). Each reading step says its `step-progress` once a minute while it waits
|
|
|
|
|
(A-262). A step that is superseded or cancelled leaves `readers`. The run's `ask` is dropped (§119, A-182) only when
|
|
|
|
|
no reader is left, so a superseded pipeline never takes away another pipeline's composition run.
|
|
|
|
|
- §724 **One test run per level and tested packages.** (§22, §25, §27, §416; ADR 0666 §8, §17; ADR 0267 §1;
|
|
|
|
|
principle 7) The runner asks a lint, unit test or integration test step of a build seat once per level and tested
|
|
|
|
|
packages, whatever number of artifacts' steps name them. A module's program and its tools bundle are built from
|
|
|
|
|
different import closures (ADR 0267 §1), but their test steps run the packages of one directory (the build seat's
|
|
|
|
|
`steps`, §22), so one set of tests is run. The share key is the level and the sha256 of the sorted lines
|
|
|
|
|
`<package>@<digest>`, one for each package the step would run after its carry (§25: those with no kept result for
|
|
|
|
|
their digest). The digests are those the runner already reads for its carry (§416). The first step of a key puts the
|
|
|
|
|
build seat's `step` ask; every other step with that key reads its result, and applies its own status from the policy
|
|
|
|
|
(§23). A later step of the same key reads the result of the key's ended run, a failed result too, so the same
|
|
|
|
|
failing tests fail once and not once per artifact.
|
|
|
|
|
- §725 **Kept in the runner's state `test-runs`.** (§27, §85, §330; ADR 0666 §4; principles 1, 2 and 7) Each test run is
|
|
|
|
|
one key of the runner's old `state` section `test-runs`, whose `max-mib` is 16. The key is `<level>.<digest>`, written
|
|
|
|
|
create-only by the step that asks the run. The value holds:
|
|
|
|
|
- `level`, and `packages` (each with its digest);
|
|
|
|
|
- `seat` and `ask`, and `node` once a holder has taken the ask;
|
|
|
|
|
- `started`;
|
|
|
|
|
- `readers`: each reading step's pipeline key and artifact, added and removed by compare-and-set;
|
|
|
|
|
- at the run's end, `ended`, `outcome`, its test counts, and the name of its output object in `step-outputs`.
|
|
|
|
|
|
|
|
|
|
The runner hears the run's end as the build seat's `step-ended` of its `ask`, as it hears every step (§24). At its
|
|
|
|
|
start the runner takes up again each key with no `ended`, by its `ask`. It deletes a key 7 days after its `ended`, in
|
|
|
|
|
the step log's order, as §330 ages its `pipelines`. A passed run's results go into `kept` per package, as today
|
|
|
|
|
(§27), so a step asked after the key is deleted carries them.
|
|
|
|
|
- §726 **How a test run ends, is said and asked again.** (§24, §73, §83 `rerun`, §119, §419, §496; ADR 0666 §8, §11,
|
|
|
|
|
§21; principles 3, 7 and 1)
|
|
|
|
|
- **Each reading step is said.** Each step reading a test run says its own `step-started` and `step-ended`. Its
|
|
|
|
|
`step-ended` carries the run's outcome and counts and `shared`: the key, its `ask`, and the pipeline and artifact of
|
|
|
|
|
the step that asked the run. Its `step-started` names the run's `ask`, so the watchman times it from that ask's
|
|
|
|
|
`step-progress` (§496).
|
|
|
|
|
- **Counted once.** `pipeline-ended` counts a catalog module's tests once per test run, never once per step reading
|
|
|
|
|
it (§24, §419).
|
|
|
|
|
- **Dropped only with its last reader.** A step that is superseded or cancelled leaves `readers`, and the run's ask
|
|
|
|
|
is dropped (§119) only when no reader is left.
|
|
|
|
|
- **Asked again.** A `rerun` of a test step joins a run of its key while that run is running, or reads one that
|
|
|
|
|
started after the step's last end, and otherwise asks a new run, replacing the ended key by compare-and-set. A
|
|
|
|
|
sibling step with the same key moves only by its own rerun, which then reads that run.
|
|
|
|
|
- §727 **The tools.** (§83; ADR 0666 §25; principle 4) Two for each new state, in §83's table:
|
|
|
|
|
- `assigner_compositions` takes a candidate, a machine, or neither, and answers each composition run with its
|
|
|
|
|
program, input digest, ask, readers, outcome and verdict, and the state's fill against its cap.
|
|
|
|
|
- `assigner_compositions_drop` takes `{candidate, machine, why}`. It writes the hand-act log first, drops the run's
|
|
|
|
|
ask, ends each reader `could not run` naming the drop, and deletes the key.
|
|
|
|
|
- `runner_test_runs` takes a level, a digest, a module, or none of them, and answers each test run with its packages,
|
|
|
|
|
ask, node, readers, outcome and counts, and the state's fill against its cap.
|
|
|
|
|
- `runner_test_runs_drop` takes `{key, why}` and does the same as `assigner_compositions_drop`.
|
|
|
|
|
- §728 **How it is checked.** Tests 108 and 109.
|
|
|
|
|
- Whole (assigner, mesh-delivery): three composition steps of one revision, on two machines, start two runs, keep
|
|
|
|
|
two requests and put two run asks, and each step reads both verdicts. A fixture machine whose verdict blames module
|
|
|
|
|
A fails A's composition step and passes B's, with a warning naming A. A machine that does not compose and blames no
|
|
|
|
|
module fails both. Three steps asked again at once start one run for each machine that did not pass. A superseded
|
|
|
|
|
step leaves the run running for its other readers. An assigner restarted during a run takes it up again by its
|
|
|
|
|
ask. At the cap, a step ends `could not run` naming the fill.
|
|
|
|
|
- Whole (the node-engine's composer, `mesh-host`): a fixture module whose manifest is refused on the machine is
|
|
|
|
|
blamed by name, and two modules owning one path are both blamed.
|
|
|
|
|
- Whole (runner and Go build module): a module whose program and bundle test one directory asks one unit step and
|
|
|
|
|
one integration step. Both artifacts' steps end with that result and `shared`, a failing test fails once, and
|
|
|
|
|
`pipeline-ended` counts the module's tests once.
|
|
|
|
|
- Live: in the first publish after G's delivery, the composition run asks of each revision number at most its
|
|
|
|
|
machines, and each module's test steps run once per level.
|
|
|
|
|
|
|
|
|
|
### Who may say what
|
|
|
|
|
|
|
|
|
|
- §68 **Callers per verb, in the seat's definition.** Each seat definition names, per verb, the callers allowed to call it,
|
|
|
|
@@ -149,7 +283,7 @@ One file of milestone G's part of [design 47](../47-delivery-from-commit-to-deli
|
|
|
|
|
| `pipeline-started`, `pipeline-ended` | a pipeline's start, and its end with every step's result, its test counts, the source revision and `widens` |
|
|
|
|
|
| `group-ended` | a group's one pipeline's end, on which alone its wants are written |
|
|
|
|
|
| `run-started` | a run on request's start, with its members at their commits |
|
|
|
|
|
| `step-started`, `step-ended` | each step's start and end: the step, its status in the policy, its outcome, its build-input fingerprint, its test counts, for a carried result what it was carried from, and for a rerun that it is one and the caller the runtime proved; a bypassed step too |
|
|
|
|
|
| `step-started`, `step-ended` | each step's start and end: the step, its status in the policy, its outcome, its build-input fingerprint, its test counts, for a carried result what it was carried from, and for a rerun that it is one and the caller the runtime proved, for a step that read a test run or a composition run another step asked its `shared` (§723, §726); a bypassed step too |
|
|
|
|
|
| `wait-held`, `wait-ended` | a step's wait: on a person (approval, code review), on a build node, on the revision view |
|
|
|
|
|
|
|
|
|
|
- §74 **What a session hears after its merge.** Between its `merge-pull` and the delivery, a session waits with `mesh_wait`
|
|
|
|
@@ -195,14 +329,14 @@ four lines a module's tools answer; every read tool of a store also answers its
|
|
|
|
|
|
|
|
|
|
| Part (seat) | Read everything it holds | Do by hand what it does itself | Recover without help |
|
|
|
|
|
|---|---|---|---|
|
|
|
|
|
| the runner (`mesh-pipeline-runner`) | `show` (one pipeline whole, its steps, step log and outputs), `list` (the pipelines of a module, a branch or a commit), `stands` (what stands still, since when), `why-carried` (where a step's result was carried from), `kept` (the kept results by digest) | `start` (a delivering pipeline for a watched branch's head), `run` (the run on request), `rerun` (one step again) | `accept` (a failed `optional` step, with why), `cancel` (a pipeline, dropping its asks), `ask-again` (a declined `approval`) |
|
|
|
|
|
| the runner (`mesh-pipeline-runner`) | `show` (one pipeline whole, its steps, step log and outputs), `list` (the pipelines of a module, a branch or a commit), `stands` (what stands still, since when), `why-carried` (where a step's result was carried from), `kept` (the kept results by digest), `runner_test_runs` (each test run, §727) | `start` (a delivering pipeline for a watched branch's head), `run` (the run on request), `rerun` (one step again) | `accept` (a failed `optional` step, with why), `cancel` (a pipeline, dropping its asks), `ask-again` (a declined `approval`), `runner_test_runs_drop` |
|
|
|
|
|
| the indexer (`mesh-catalog`) | `registrations` (with each one's history); `indexer_edges` (an artifact's edges per watched branch), `indexer_order` (the order of a change, its cycles named), `indexer_catalog_difference` (two revisions); `indexer_pending` (a group with its missing members) | `register`, `policy`, `watch`, `unwatch` (each on the operator's Approve) | `indexer_seed_again` (the first registrations' ask raised again), `indexer_take_in_pending` (a group taken in as gathered), `indexer_drop_pending` (dropped) |
|
|
|
|
|
| the gate (`mesh-delivery`) | `source` with a branch (`gate_source`: every branch's source revision) | — | `gate_source_again` (say `source-moved` again for a branch) |
|
|
|
|
|
| the forge module (`git`) | `git_failed_steps` (the failed `required` steps it holds, by module and fingerprint, with the run that failed each) | — | `git_failed_steps_drop` (drop one, with why) |
|
|
|
|
|
| the grants part (`mesh-grants`) | `grants_callers` (the callers each verb admits, read from the seat definitions) | — | `grants_why_refused` (for an account and a verb, the grant it lacks and the definition's line that leaves it out) |
|
|
|
|
|
| the build seats | `steps` (the steps detected for an artifact) | `steps` with `again` (detect again) | `drop` (ADR 0664 §4) |
|
|
|
|
|
| the builder (`mesh-builds`) | `builder_reserved` (each module's reserved version, §369); the released versions are the catalog's `versions` | `builder_publish_again` (publish the missing artifacts under the reserved version) | — |
|
|
|
|
|
| the assigner (`mesh-wants`) | `assigner_why_not` (why a module is not on a machine yet: the pipeline it waits for, its hold, the placement, the node's last `running`) | — | `assigner_release` |
|
|
|
|
|
| the assigner (`mesh-wants`) | `assigner_why_not` (why a module is not on a machine yet: the pipeline it waits for, its hold, the placement, the node's last `running`), `assigner_compositions` (each composition run, §727) | — | `assigner_release`, `assigner_compositions_drop` |
|
|
|
|
|
| the watchman | `watchman_views` (how far behind each part's view is, from each part's `view-waits` and first-view tools) | — | each part's `<module>_first_view_again` |
|
|
|
|
|
| the operator-channel | `asks` (with each asker's asks held back) | — | `ask-again` (offer a held-back ask now) |
|
|
|
|
|
| every module that imports mesh-sdk's small object store package | `<module>_state_objects`, `<module>_sent_objects` | — | `<module>_state_objects_drop`, `<module>_sent_objects_drop` |
|
|
|
|
@@ -230,6 +364,8 @@ and an estimated 2 mesh-sdk publishes a day, each rebuilding about 120 artifacts
|
|
|
|
|
| the builder | `reserved`: one key per module and `MAJOR.MINOR` (§369) | old `state` section | until its `version-released`, a release of another `MAJOR.MINOR` of the module, or `builder_reserved_drop` | a few KiB; at the cap the builder reserves nothing and says why |
|
|
|
|
|
| the assigner | `written`: the last source revision and end count per module and watched branch (§352) | old `state` section | until the branch is unwatched | a few KiB |
|
|
|
|
|
| the indexer and the assigner | `inputs`: a `run` ask's input by write time and sha256 (§354) | old `state` section | until the ask's `step-ended`, at most a day | 64 MiB; an estimated 300 asks a day of under 100 KiB each is under 30 MiB; at the cap the asker refuses the ask loudly, naming the store |
|
|
|
|
|
| the assigner | `composition-runs`: one key per candidate and machine, its program, input, ask, readers and verdict (§719) | old `state` section, `max-mib` 16 | a day after its end, or a day after its start with no end; deleted by the assigner (§719) | 16 MiB; an estimated 30 candidates a day on four machines, 120 keys of at most 8 KiB, is under 1 MiB a day |
|
|
|
|
|
| the runner | `test-runs`: one key per level and tested packages, its ask, readers, outcome and counts (§725) | old `state` section, `max-mib` 16 | 7 days after its end, deleted by the runner in the step log's order (§330) | 16 MiB; an estimated 1,000 test runs a day of about 1 KiB is about 7 MiB at 7 days |
|
|
|
|
|
| the runner and the assigner | `findings` (§355), read by their own read tools alone; the keeper hears the findings as events (§383) | old `state` section | an entry deleted when its finding clears | a few KiB |
|
|
|
|
|
| every observer (§387) | the stream `<module>-findings`: `finding-written`, `finding-cleared`, `findings-stated` (§383) | old `streams` section for a foundation part, the module's `streams` otherwise | 7 days, its oldest dropped when full; the next statement covers a lost one (§384) | 16 MiB; an estimate of a few hundred findings a day of about 1 KiB is under 2 MiB a week |
|
|
|
|
|
| the condition keeper | `findings-views`: one key per observer and node, its last stated set, pending keys and open asks (§388) | the form of its `conditions` | deleted when the module leaves the catalog or the node (§365) | under 1 MiB; about 30 observers on four machines |
|
|
|
|
|