novox/hq
6
0
Fork 0
Code Issues Pull Requests 6 Releases Wiki Activity

Merge pull request 'ADR 0282 (proposed): a leaf fix runs healthy on every node within five minutes of its merge, a core module within ten' (#289) from decisions/0282-a-leaf-fix-runs-healthy-everywhere-within-five-minutes-of-its-merge into main

Browse Source
This commit was merged in pull request #289.
This commit is contained in:
mesh-admin
2026-10-10 17:59:02 +00:00
parent e08aaae484 0a6172640c
commit 8eca7b921e
5 changed files with 308 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
00-META/glossary.md
+9
View File
@@ -366,6 +366,15 @@ planner, the walk). **Downstream of** Module, Core, Health and repair and Data.
*Not:* ~~rollout~~ (hq)
- **first-node gate** — the judgement on a delivery's first node: healthy three times, no witness put it
back, no condition raised since the send (ADR 0236 §2). Not *the gate* bare (see [Homonyms](#homonyms)).
- **core module** and **leaf module** — a core module is one on the controller's own path (the controller,
the node-engine, the tool runner, the bus, mesh-delivery) or the mesh's resolver, whose judgement protects
the mesh's ability to reach a node at all; a leaf module is every other module. Decided by
[ADR 0282](../02-DECISIONS/0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md).
Not a *core component* (ADR 0236's controller, node-engine, tool runner and bus, each with a health probe):
the core modules add mesh-delivery and the resolver.
- **delivery time** — from a delivery's merge to every node of its deploy plan that is heard from running its
modules' builds and reporting them healthy. Its **delivery budget** is what it is held to: five minutes for
a leaf module, ten for a core module (ADR 0282). Not the walk's time, which ends at the send.
- **release** — a person's word that a held delivery goes on; the verb `mesh-delivery.release`. Used in
this sense only.
- **rollback** — putting a node back on the build it ran before, by something other than the build being
02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md
+7
View File
@@ -111,6 +111,13 @@ gated: it is the module saying it must change everywhere at once.
> passes, so that when the send fails, a module healthy on its own for the passes asked keeps a pass and is
> not put back (issue 318).
> **The mechanism changed — 2026-10-10, by [ADR 0282](0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md).**
> What stands: a core module's three judgings, their spacing and bound, and every other point above. What
> moved: a leaf module (every module off the controller's own path and the mesh's resolver) is judged on two
> readings when it has a long-running resource and on one when its every resource is a tools bundle, and one
> tier's first machines are judged at once. Until ADR 0282's third slice is built, three judgings hold for every
> module.
**3. A build that fails its gate is put back there, once, and never sent again on its own.** The plan
stops; the build is marked failed at its gate; the module's registered build goes back to the build the
first machine ran before — the newest successful build of that commit still kept ([ADR 0189](0189-the-store-keeps-what-the-records-name.md)
02-DECISIONS/0276-merges-are-assembled-in-a-rolling-window-and-each-batch-is-delivered-by-one-walk-with-one-commit-per-repository.md
+7
View File
@@ -135,6 +135,13 @@ either, because **delivery group** already names deliveries that share a head br
are replaced by it. A merge heard after a later merge of its repository and branch was put in a walk is
answered by that walk: by the batch still assembling, by the open walk, or, when that walk was delivered,
it ends `covered` at once.
> **The mechanism changed — 2026-10-10, by [ADR 0282](0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md).**
> What stands: the window for a merge whose delivery group has a member not yet merged or that arrives while a
> batch assembles (decision 1), two walks of one repository never at once, and no walk superseded once it
> started. What moved: a lone merge is cut at once without the window, and walks that share no repository,
> module or module-graph edge run at once, one first-node judgement per machine at a time. Until ADR 0282's
> second and fourth slices are built, the window for every merge and one walk at a time hold.
5. **The order of a delivery group holds inside a walk** (ADR 0249 unchanged). The pairs its order rules
give (declared, built by, version skew, engine before controller, each with the rule that won) become
edges of the walk's tiers beside the module graph: the modules of a member ordered after another are in
02-DECISIONS/0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md
+234
View File
@@ -0,0 +1,234 @@
---
topic: the mesh
status: accepted
date: 2026-10-10
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0276-merges-are-assembled-in-a-rolling-window-and-each-batch-is-delivered-by-one-walk-with-one-commit-per-repository.md
---
# 282. A fix to a leaf module runs healthy on every node within five minutes of its merge, and a core module within ten
## Context
On 2026-10-10 the operator asked, in their words: "we definitely need to speed up our bug fixing process so a
bugfix can land deployed and rolled out in less than 5 minutes". Issue 382 in the issue tracker (a small fix took
45 minutes to reach a node) names the causes it saw: a 90-second merge window, one walk at a time, about four to
five minutes per tier, and three judgement passes for every module. Issue 338 (a module built from a shared
repository moves on every merge to it) names a fourth: every merge of the controller walks two catalogue modules
in two more tiers.
### What was measured
Every walk the controller keeps (`delivery-walks`) records, per module, when its build was asked and finished,
when its first node was sent it, when the first-node gate judged it and when the rest were sent. Read on
2026-10-10 for the 51 walks that finished between 2026-10-09 13:55 and 2026-10-10 17:32 (UTC), each walk's time
from its merge to its builds being sent to every node it targets splits into these phases. A walk is a **core**
walk when it moves a module of the controller's own path (the controller, the node-engine, the tool runner, the
bus, mesh-delivery) or the mesh's resolver, and a **leaf** walk otherwise. Nine walks that failed or were folded
into another are not counted.
| Phase, in minutes | Leaf: median | Leaf: worst | Leaf: share | Core: median | Core: worst | Core: share |
|---|---|---|---|---|---|---|
| the merge window: merge to walk opened (a wait over 2.2 counted as 1.8 of window, the rest as waiting) | 0.5 | 2.1 | 9 % | 0.4 | 2.1 | 7 % |
| waiting behind another walk | 0.3 | 59.1 | 47 % | 0.0 | 1.8 | 1 % |
| build | 0.4 | 1.6 | 5 % | 1.2 | 4.1 | 14 % |
| built to sent to the first node | 0.3 | 0.6 | 3 % | 0.5 | 3.0 | 5 % |
| first-node judgement, the longest of each tier | 2.6 | 6.3 | 27 % | 7.2 | 14.9 | 63 % |
| first nodes of one tier judged one after another | 0.0 | 4.2 | 8 % | 0.0 | 0.0 | 0 % |
| judged to sent to the rest | 0.0 | 0.9 | 2 % | 0.5 | 5.1 | 6 % |
| between tiers | 0.0 | 0.6 | 0 % | 0.0 | 1.0 | 3 % |
| **merge to sent to every node** | **6.0** | **66.0** | | **9.7** | **20.5** | |
- **Leaf walks** (20): median 6.0 minutes, 90th percentile 14.8, worst 66.0; **7 of 20 within five minutes**.
The worst is the incident of issue 362 (58 minutes waiting for a person's word), which ADR 0276 already
repaired; without it the worst wait behind another walk is 13.3 minutes. Since the merge window went live on
2026-10-10 at about 12:40, a lone merge waits 1.6 to 2.1 minutes for its window (median 1.8), and **4 of the 9
leaf walks since then waited behind another walk**, for 1.8, 4.9, 9.5 and 13.3 minutes, each behind a walk of
the controller's own path.
- **Core walks** (31): median 9.7, 90th percentile 17.0, worst 20.5; 4 of 31 within five minutes. They are two
populations. **13 core walks had one tier**: median 5.6, 90th percentile 7.2, worst 8.8. **18 walks
were controller merges that also walked the build agent and the route proxy** in tiers of their own (issue
338): median 14.2, worst 20.5, none within ten minutes.
- **The first-node judgement** is three healthy readings, at least forty seconds apart and two minutes after
the send (ADR 0236 decision 2). It took a median of 2.4 minutes for a core module other than the controller (10
judgements), 3.6 minutes for the controller (19 judgements, worst 4.3, its readiness includes its
self-check), and 2.6 minutes per tier of a leaf walk. **Five of the 20 leaf walks judged two first nodes one
after the other** within one tier (a module first on one node, another first on a second), adding 3.0 to 4.2
minutes each.
- **The build is small**: a median of 0.4 minutes for a leaf walk and 0.6 minutes for a core walk of one tier.
- **The rest are sent at once already**: a leaf walk sends the rest a median of 0.0 minutes after its judgement.
The walk ends when the rest are *sent*, not when they run the build. Applying a send takes a median of 10 to
17 seconds per node (the controller's `durations`, seven days, every node), so the measured times are short of
"healthy on every node" by about a quarter of a minute.
- **The merge gate does not build the module.** Its record for a controller pull request on 2026-10-10 shows it
clones the repositories, composes every node's declaration with the change, compiles the controller only to
judge the change with itself (3.2 seconds), and runs the repository's `merge-check.sh` (6 minutes 13 seconds,
before the merge). No artifact is built that the merge could reuse; the merge's build is the first.
Three limits of the measurement: a walk opens only when the walk before it ends, so time beyond the window's
own 1.6 to 2.1 minutes is counted as waiting; the walk keeps only the last judgement pass, so the time of a
first pass is estimated below, not measured; and the check before the merge is not counted, because the
operator's target starts at the merge.
**Checked against GENESIS.** *Anything requiring a human to notice it will be noticed late*: a fix that takes
fifteen minutes to land is noticed by the operator waiting for it. *Failure must be loud*: a delivery over its
budget raises a condition. *Evidence over assertion*: every lever below is weighed by what it would have saved
in the walks above. Nothing here conflicts with GENESIS.
**Words.** The glossary has no word for the time from a merge to its modules running healthy everywhere, nor for
the time it is held to, nor for the two kinds of module. This record proposes **delivery time**, **delivery
budget**, **core module** and **leaf module** for the Change and delivery domain, with this record.
## Considered Options
Each lever is weighed by what it saves in the 20 leaf walks measured, against what it costs.
1. **No merge window for a lone merge** (chosen). Saves the window's 1.8 minutes on every lone merge. The window
exists to put the parts of one change, merged together across repositories, into one walk (ADR 0276). A
merge whose delivery group has no member still unmerged has no part to wait for.
2. **Independent walks in parallel** (chosen). Saves the wait behind another walk: 1.8 to 13.3 minutes in 4 of
the 9 leaf walks since the window went live, 34.7 minutes over the 19 leaf walks outside the issue 362
incident. Its cost is two builds judged on one node at once, which the rules in decision 3 keep apart.
3. **Judgement passes by the module's class** (chosen). One healthy reading takes an estimated minute after the
send (a send applies in 10 to 17 seconds, and the controller acted on a build in a median of 0.3 minutes),
against 2.6 minutes for three. Saves about 1.5 minutes per tier of a leaf walk. The cost is a crash that shows
only after the first reading, which a second reading forty seconds later catches for a module with a
long-running resource. A module whose every resource is a tools bundle has no process of its own to crash: the
tool runner serves its tools or does not.
4. **The first nodes of one tier judged at once** (chosen; not in issue 382, found by the measurement). Saves
3.0 to 4.2 minutes in 5 of the 20 leaf walks.
5. **Reuse the merge gate's build of the pull request's head** when the merged tree equals the checked one.
Rejected: the merge gate builds no artifact today (see Context), so there is nothing to reuse. Building every
head's artifacts at its check, to save the merge's build, would save at most the build's median 0.4 minutes
for a leaf walk, cost a build on every push to a pull request, and need ADR 0238 decision 1 ("anything built to
check it is never registrable") reopened.
6. **A build cache.** Rejected as a lever of this record: the build is 5 % of a leaf walk's time, and the
artifact that did not change is already not rebuilt (ADR 0267's build source).
7. **Sending the rest at once after the first passes.** Already so: the rest are sent a median of 0.0 minutes
after the judgement. Nothing to gain.
8. **The plan says when it is expected to land** (chosen). Saves no time. It tells the operator, while waiting,
when to look, from the same measurements.
9. **One budget for every module, five minutes.** Rejected: a core module's judgement is what protects the
mesh's ability to reach a node at all. A broken node-engine, tool runner, bus or resolver cannot be put back
from outside the node, and the controller judges every other build. The controller's judgement alone takes 3.6
minutes; its three readings are kept.
With options 1 to 4 applied to the 20 measured leaf walks (no window, no wait, one judgement of about a minute
per tier with its first nodes at once, plus a quarter of a minute for the rest to apply), the median would be 2.1
minutes and the worst 5.3; **19 of 20 within five minutes**, against 7 of 20 measured. Keeping three readings
and applying only options 1, 2 and 4 gives a median of 3.9 and a worst of 9.6 (18 of 20).
## Decision
1. **The target, and how it is measured.** A delivery's **delivery time** runs from its merge to the moment
every node of its deploy plan that is heard from runs its modules' builds and reports them healthy. A node not
heard from (a laptop asleep) is listed and not waited for, as ADR 0236 leaves it. **A leaf module's delivery
budget is five minutes. A core module's is ten.** A delivery held for a person (a module that records, a hold,
a bus step) is shown and not held to a budget: its wait is the person's, and the controller's
condition for a walk left waiting (S16) and its probe of stalled deliveries (D14) already speak for it.
- A **core module** is one on the controller's own path (the controller, the node-engine, the tool runner, the
bus, mesh-delivery) or the mesh's resolver. A **leaf module** is every other module. A delivery's class is
that of the slowest-budgeted module it moves.
- The ten minutes for a core module are the measurement's: the 13 core walks of one tier took a median of
5.6 minutes and at worst 8.8. A controller merge that also walks the build agent and the route proxy (median
14.2) meets it once issue 338's path precision is built: that work is tracked by issue 338 and is the
planner's, not this record's.
2. **No merge window for a lone merge.** When a merge arrives, no batch is assembling, and its delivery belongs to
no delivery group with a member not yet merged, the controller cuts its batch at once. A merge of a group still
waiting for members, or a merge arriving while a batch assembles, waits for the window as ADR 0276 decided.
3. **Independent walks run at once.** A batch becomes a walk without waiting for an open walk when it shares with
every open walk no repository, no module, and no edge of the module graph between their modules. Where they
meet, they wait for each other:
- **one first-node judgement per node at a time**: a walk's first node is chosen among its module's nodes
where no other walk is judging; a send to a node where another walk's build is being judged waits for that
verdict, so ADR 0236 decision 4a (no build reaches a node without a judgement) holds as written;
- **a core walk keeps its ground**: no leaf build is sent to a node where a core build is being judged, and
while a new controller is judged, no leaf judgement counts a reading; those resume when it passes or is put
back. A core send never waits for a leaf judgement: a leaf judgement open on that node is set aside without
a verdict and starts again, from its first reading, after the core build's verdict, so a core delivery is
not held past its budget by a leaf one;
- two walks of one repository never run at once (a repository's later merge contains its earlier ones, ADR 0276 decision 2), and a merge into a repository with
an open walk joins that repository's next batch.
This replaces ADR 0276 decision 4's "one walk at a time"; its rule that a walk which started is never
superseded stands.
4. **Judgement by class.** The first-node gate's readings, each at least forty seconds after the one before:
- a **core module**: three, the first at least two minutes after the send, as ADR 0236 decision 2 decided;
- a **leaf module with a long-running resource** (a service, a container, a process the node-engine runs):
two, the first after the node-engine states every such resource healthy (ADR 0240);
- a **leaf module whose every resource is a tools bundle**: one, once the tool runner serves its tools.
Everything else ADR 0236 decided of the first-node gate stands: no witness put the build back, no condition
raised since the send, ten minutes at the most, and a rollback when it fails.
5. **One tier's first nodes are judged at once**, each by its own readings, and the tier passes when every one
has passed.
6. **Every walk keeps its phases, and every delivery its delivery time.**
- The controller records in the walk, and says in `plan-moved`, each phase of the table above with its start
and end, each judgement reading with its time, and the first healthy report of each node of the rest. Its
`durations` gains the kind `walk-phase`, per class.
- mesh-delivery records each delivery's delivery time, its class and its phases from the walk. A new verb,
`times`, answers each class's budget, the last seven days' median and worst delivery time, and every
delivery over its budget with where its time went; `deliveries` heads its list with one line of the same,
and `show` gives a delivery's phases.
- The controller's `status` shows that line, read from `times` by the self-check, beside what is wrong.
7. **A delivery over its budget is loud.** The self-check's probe reads `times` and raises the warning
`delivery.<class>.over-budget` when a delivery's time exceeds its class's budget, naming the delivery and its
longest phase. It clears when the next delivery of that class lands within its budget.
8. **The plan says when it will land.** `plans` and mesh-delivery's `show` give an open batch and an open walk an
expected landing time: now plus the remaining phases at the last seven days' median for its class, and once a
walk is judging, plus the readings still owed.
## Consequences
- **A leaf fix lands within five minutes when nothing else is in its way**, by the measurement's own estimate:
build 0.4, send 0.3, one or two readings at about a minute each, the rest a quarter of a minute.
- **A burst of unrelated merges no longer queues**; it walks in parallel, and one node judges one walk at a time.
The controller holds several open walks, and every reader of a walk (`plans`, the walk's conditions, `plan-moved`,
mesh-delivery) reads a list where it read one.
- **A leaf module is judged on less evidence.** A build that fails after its last reading reaches the rest. That
is bounded: the rest's first healthy report is recorded, a failure there is a condition on that node, and the
build that broke it is put back the ordinary way. The trade is the operator's stated choice of speed for leaf
modules; core modules keep every reading.
- **A core walk still holds its ground.** A leaf delivery whose node is judging a core build waits up to that
judgement (3.6 minutes for the controller) and may exceed its budget; the condition says so, and the week's
numbers show how often.
- **The core budget depends on issue 338.** Until the planner moves only the modules whose build source a merge
touched, a controller merge exceeds ten minutes and raises the condition each time. That is intended: the
condition counts what issue 338 costs.
- **What got harder:** the walk's record grows by the phases and readings, kept in the controller's store as a
numbered migration; the scheduler of walks gains the independence test and the per-node judgement rule, each a
place where two walks can disagree, and each tested.
- **On acceptance**, ADR 0236 (decision 2, three readings for every module) and ADR 0276 (decision 4, one walk at
a time; decision 1, the window for every merge) each gain a note saying the mechanism moved here, and to-be 47
lists this record among its decisions.
## How it is checked
| Rule | Checked by |
|---|---|
| a lone merge is cut at once; a group's member and a merge during assembly wait | the controller's tests: a lone merge asks its build with no window; a merge whose group has an unmerged member waits for the window; a second merge within the window joins the batch |
| independent walks run at once, dependent ones wait | the controller's tests: two catalogue merges of unrelated modules walk at once; two of one repository, two sharing a module, and two joined by a graph edge do not; two walks never judge on one node at once; a send to a node under another walk's judgement waits for its verdict; no leaf reading counts while a new controller is judged |
| judgement by class | the controller's tests: a tools-only leaf passes on one reading, a leaf with a service on two forty seconds apart, a core module on three two minutes after the send; each fails and is put back as before |
| one tier's first nodes at once | the controller's test: a tier whose modules are first on two nodes judges both at once and passes when both pass |
| every phase and the delivery time are recorded | the controller's test of the walk record and `plan-moved`; mesh-delivery's tests of the delivery time, its class, `times` and `show` |
| a delivery over budget is loud | the probe's test: a leaf delivery of 5 minutes 10 seconds raises `delivery.leaf.over-budget` naming its longest phase; the next within budget clears it |
| the target holds on the live mesh | `mesh-delivery.times`: the week's leaf median within five minutes and the condition raised by no more than one leaf delivery in ten; the same measurement as this record's Context, repeated a week after the last part is built |
## References
- Issue 382 (a small fix takes 45 minutes to reach a node) and issue 338 (a module built from a shared repository
moves on every merge to it), in the issue tracker.
- [ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md):
the first-node gate's three readings, kept for core modules.
- [ADR 0238](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md):
a check's build is never registrable, which option 5 would have reopened.
- [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md):
mesh-delivery owns the delivery, and so its delivery time.
- [ADR 0267](0267-an-artifact-is-rebuilt-only-when-its-build-source-changes-and-a-module-moves-only-along-build-dependencies-on-artifacts.md):
an unchanged artifact is not rebuilt.
- [ADR 0276](0276-merges-are-assembled-in-a-rolling-window-and-each-batch-is-delivered-by-one-walk-with-one-commit-per-repository.md):
the merge window and the walk this record changes in part.
- [Research 044](../01-RESEARCH/044-what-leaves-the-controller-next/00-overview.md), whose question 6 asks for
the time from merge to a node running the fix as a baseline; the table above is that baseline.
- To-be [47](../03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md), amended with this record.
03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md
+51
View File
@@ -4,6 +4,7 @@ status: designed
code: []
updated: 2026-10-10
decisions:
- 02-DECISIONS/0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md
- 02-DECISIONS/0276-merges-are-assembled-in-a-rolling-window-and-each-batch-is-delivered-by-one-walk-with-one-commit-per-repository.md
- 02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md
- 02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md
@@ -158,6 +159,56 @@ controller's `plan-moved` event, never inferred from time.
answers within ten minutes is held with the true reason (the open walks of its repository and that none
names its merge, or that the controller holds none) and listed by `stalled` at once.
## The delivery budget
*Decided 2026-10-10 by [ADR 0282](../../02-DECISIONS/0282-a-fix-to-a-leaf-module-runs-healthy-on-every-node-within-five-minutes-of-its-merge-and-a-core-module-within-ten.md).
Where it differs, this section replaces the window for every merge and the one walk at a time above, slice by
slice as each slice below is built; until a slice is built, the text above is what runs.*
**The target.** A delivery's **delivery time** runs from its merge to every node of its deploy plan that is heard
from running its modules' builds and reporting them healthy. A **leaf module's** delivery budget is five minutes;
a **core module's** (the controller's own path and the mesh's resolver) is ten. A delivery held for a person is
shown and held to no budget. Measured on 51 walks on 2026-10-09 and 10, a leaf walk took a median of 6.0 minutes
(7 of 20 within five), a core walk of one tier 5.6 (worst 8.8), and a controller walk dragging two catalogue tiers
14.2 (issue 338).
**What changes in the walk.**
- **A lone merge is cut at once**: no batch assembling, and no member of its delivery group still unmerged.
- **Independent walks run at once**: no shared repository, module or module-graph edge with any open walk. One
node judges one walk at a time; a send to a node under another walk's judgement waits for its verdict; no leaf
build goes to a node where a core build is judged, and no leaf reading counts while a new controller is judged.
- **The first-node gate reads by class**: three readings for a core module, as ADR 0236 decided; two for a leaf
module with a long-running resource; one for a leaf module whose every resource is a tools bundle.
- **A tier's first nodes are judged at once.**
- **Every phase is kept**: the window, the wait, the build, the send, each reading, the rest's first healthy
report; the plan shows an expected landing time from the last seven days' medians.
**Who builds what.**
| Part | Owner | What it adds |
|---|---|---|
| the walk's phases and readings | mesh-controller | each phase's start and end and each reading in the walk and in `plan-moved`, the rest's first healthy report per node, `durations` of kind `walk-phase`; a numbered migration |
| the delivery time | mesh-delivery | each delivery's delivery time, class and phases; the verb `times` (budgets, the week's median and worst per class, the deliveries over budget and their longest phase); a line atop `deliveries`; phases in `show` |
| the condition | mesh-controller | a self-check probe reading `times`, raising `delivery.<class>.over-budget` (a warning) and clearing it at the next delivery of that class within budget; the line in `status` |
| the lone merge | mesh-controller | the cut at once when nothing waits to join |
| walks at once | mesh-controller | the independence test, one judgement per node, a core walk's ground, a repository's next batch |
| judgement by class | mesh-controller | the readings per class in the first-node gate; a tier's first nodes judged at once |
| the landing time | mesh-controller, mesh-delivery | an expected landing time in `plans`, in `plan-moved`, and in `show` |
**Slices**, each one pull request per repository, each with its acceptance sentence:
| Slice | Delivers | Done when |
|---|---|---|
| 1 — the measure | the walk's phases and readings (controller); the delivery time, `times` and the line in `deliveries` (mesh-delivery); the probe and its condition (controller) | `mesh-delivery.times` shows the week's leaf and core median and worst delivery time, and a leaf delivery over five minutes raises `delivery.leaf.over-budget` naming its longest phase |
| 2 — no window for a lone merge | the cut at once | a lone leaf merge's build is asked within fifteen seconds of its merge, read from its walk's phases |
| 3 — judgement by class | the readings by class, a tier's first nodes at once | a tools-only leaf module passes its first-node gate on one reading, read from its walk |
| 4 — walks at once | the independence test and the per-node rule | two unrelated catalogue merges a minute apart are both delivered within five minutes, neither waiting for the other |
| 5 — the landing time | the expected landing time | `plans` shows it for an open walk, and the week's deliveries land within a minute of it at the median |
Issue 338's path precision is the planner's, tracked by issue 338; the core budget is met by controller merges
only once it is built.
## A delivery group
- **Membership**: the pull request's head branch name, across the mesh's repositories, among open pull