novox/hq
6
0
Fork 0
Code Issues Pull Requests 6 Releases Wiki Activity

Merge pull request 'ADR 0270 and 0271 (accepted): a held delivery a later merge contains, or that moves nothing, ends by itself; a send held for the bus's step is told at once' (#241) from decision/0270-0271-held-deliveries-end-and-bus-step-said into main

Browse Source
This commit was merged in pull request #241.
This commit is contained in:
mesh-admin
2026-10-09 07:35:21 +00:00
parent 9744ed7b9d 29bf2cae30
commit b43505d88b
8 changed files with 447 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
00-META/glossary.md
+10 -3
View File
@@ -278,7 +278,7 @@ and the operator act. **Recorded by** the controller's `identity` and `licences`
**Purpose.** To take one commit from its pull request to every node that should run it, and to put it
back when it fails. **Recorded by** the `mesh-delivery` module (deliveries) and the controller (the
planner, the walk). **Downstream of** Module, Core, Health and repair and Data.
**Decided by** ADR 0162, 0218, 0236, 0237, 0238, 0239, 0267.
**Decided by** ADR 0162, 0218, 0236, 0237, 0238, 0239, 0250, 0267, 0270.
**Uses:** module, node, declaration and send (Placement), health (Health and repair), person.
- **merge check** — the two statuses a pull request carries before it may merge: the **merge gate**
@@ -313,12 +313,19 @@ planner, the walk). **Downstream of** Module, Core, Health and repair and Data.
modules live in. `mesh-catalog` and `catalog_modules` are identifiers and keep their spelling.
*Not:* ~~catalog~~ (hq)
- **delivery** — one commit in one repository on its way to the nodes, from its pull request's head
being announced to delivered, failed, superseded or stopped: one pull request, one status, one note
being announced to one of its final states: one pull request, one status, one note
([ADR 0239](../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)).
Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`,
`held`, and the final four). Not *change* (a diff). *Pipeline* is the predecessor's build-and-deploy
`held`, and the final six: `delivered`, `failed`, `superseded`, `stopped`, `retired` and `covered`;
[ADR 0250](../02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md),
[ADR 0270](../02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md)). Not *change* (a diff). *Pipeline* is the predecessor's build-and-deploy
mechanism, which the as-is designs describe; it is never a word for a delivery.
*Not:* ~~deployment~~ (hq)
- **covered** — the end of a delivery that a later merge containing it delivered: every module it moves was
sent to every machine running it by the walk of a later merge into the same trunk, delivered on its own
passing check, whose merge commit the forge says contains this one. Not *superseded*, which is a delivery
whose walk a newer delivery took over before it ended; nor *delivered*, which its own walk or an empty plan
ends in ([ADR 0270](../02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md)).
- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's
repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One
level: a group holds deliveries, never groups. Its state is derived from its members, never set.
02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md
+203
View File
@@ -0,0 +1,203 @@
---
topic: the mesh
status: accepted
date: 2026-10-09
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
---
# 270. A held delivery that moves nothing, or that a later delivered merge contains, ends by itself, and a hold whose reason is gone lifts
## Context
[ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)
decision 2 gives a delivery one table of states. A delivery is `held` when only a person can decide whether
it goes on. A person then has four ways out: `release` walks the delivery's own commit, `stop` ends it as
stopped, `retire-history` ends history
([ADR 0250](0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md)),
and the walk can start on another word (`go`).
On 2026-10-09, fifteen deliveries were held, most for more than a day, each raising probe D14's
`delivery.<id>.stalled` (read with `mesh-delivery.deliveries` and `show`). None of them asked a question a
person could answer correctly:
- **Four merges of hq build nothing.** Their plan says "builds nothing: the change touches no module of the
mesh's graph". They were merged before their check answered, and the row *merged unchecked → held* is tried
before anything reads the plan.
- **Four catalogue merges only add a module.** The controller builds and registers a new module and sends it
nowhere, so it opens no walk ([issue 300](../04-ISSUES/300-a-new-module-is-not-built-at-its-merge/00-report.md)).
mesh-delivery counted the new module as something a walk moves, so the no-walk rule held each one ten
minutes after its merge.
- **Five controller merges had no walk of their own.** A later merge into the same trunk was delivered, and
its walk sent every module these move. The diagnosis of [issue 319](../04-ISSUES/319-a-held-delivery-a-later-commit-superseded-has-no-end-but-release/01-diagnosis.md)
checked each merge commit against the later delivered commit: each was contained in it, and the
controller's `status` listed none of their modules as behind.
- **Two catalogue merges were held for their group's composed check**
([issue 316](../04-ISSUES/316-a-group-member-held-before-its-composed-check-answered-stayed-held-after-it-passed/00-report.md)).
One was published while the check was asked and not yet answered. The code read "not answered" as "did
not pass" and held it. Two minutes later the check passed for exactly the heads that merged, and nothing
read that answer for the held member. The own walk of each of the two was since taken over by a later walk
of the same trunk that sent its module again.
`release` would walk an old commit over a newer one, or walk nothing. `retire-history` refuses each one,
because their heads were heard. `stop` is the only way out that works, and it is the wrong record: it sets
`mesh/delivery` to `error` on a pull request whose change a later delivered merge carried. Four were stopped that way on
2026-10-09 before the issue was found, and their pull requests now say "stopped: covered: …" as an error. A
condition that only a wrong act can clear teaches people to ignore the list it is in.
## Considered Options
1. **Stop them with why.** Rejected: it is what was done, and it records a delivered change as an error on
its pull request. Issue 313 ruled it out for the same reason.
2. **A person's act, `supersede`, that ends a held delivery as superseded.** Rejected as the main path. Every
fact the person would check can be read by the mesh: the plan, the walks' records and the forge's history.
A person asked to confirm a fact the mesh can read is a person asked for nothing. A person keeps `stop` and
`release` for what the facts do not settle.
3. **Trust merge order alone.** Rejected after review. On a trunk that takes no push, a commit the forge
merged later contains every commit merged before it, and mesh-delivery already holds the merge times. But
that rests on the trunk's protection, which is the forge's setting and which nothing in mesh-delivery
checks. A protection lifted once, or a merge time read wrongly, would end a delivery as covered when its
commit is not in the later one. Merge order stays, as the cheap first filter.
4. **Ask the forge whether the later merge commit contains this one** — chosen. The forge module gains a
read-only tool, `gitea_contains`, over the forge's compare of the two commits: none of this commit's history
missing from the later one means it is contained. mesh-delivery is not given the forge's general API tool,
which can write. The tool is TypeScript, as the rest of the forge module is; no Go route to the forge's API
exists for a module to call. Asking the controller what each machine runs was weighed and left: the
controller holds no clones, and what runs is already what the later walk's record says it sent.
5. **Read the facts, and end the delivery by observation**, with point 4 deciding containment — below.
For issue 316 the alternative was to require `mesh/delivery-group` on the trunks, so that no member can merge
before its composed check passed. That stays the operator's setting, as ADR 0239 left it. It does not settle
a check that answers after the merge.
## Decision
1. **A merge that moves nothing is delivered at once.** A delivery whose plan moves no module the mesh holds
is delivered as soon as it is merged and has no walk. That covers a plan that builds nothing and a plan
that only adds a module the controller builds and sends nowhere. This holds whatever its check said: from
`proposed`, `checked` and `rejected`, the row is tried before *merged unchecked → held*, and from `held`
it is tried on every tick. `ready → published → delivered` already did this for a merge that passed.
**A delivery that only adds a module is reported delivered whatever its build's outcome.** The build is
the controller's, asked at the merge, registered when it lands and sent nowhere. A failed build of it
raises no condition and is not this delivery's: the controller prints a line in its log, and nothing
else says it until a person assigns the module, which is refused while no build of it is registered.
This is a known gap, not closed here.
2. **A new final state, `covered`: a later merge that contains it was delivered.** A merged delivery is
covered when, for every module its plan moves, another delivery:
- is of the same repository and was merged into the same trunk, after it by the forge's merge times (the
cheap filter);
- is `delivered`, and its own check passed;
- was delivered by its own walk: the walk's commit is its merge commit, on that trunk;
- sent the module to every machine running it, as its machine steps say (sent to all at once, or to the
rest after its first machines passed), with no step of it failed or put back;
- has a merge commit that contains this one's, as the forge says through `gitea_contains`.
The forge is asked only once every module has a candidate, once per pair, outside the holder's lock, and its
answer is kept on the delivery. A forge that cannot answer means "not yet covered": the delivery stays where
it is and is asked about again a minute later. A delivery whose own walk is still open is never covered: its
walk delivers it.
The row is observed. It is tried from `held`, and from `ready` once the ten minutes for a walk to open have
passed, before the no-walk hold. From `held` it is tried before `go`. If a walk a person started fails
between two readings of a covered delivery, the delivery ends `covered`, not `delivering` then `failed`: the
later merge that contains it was delivered whatever its own walk did. The transition's why names each
covering delivery, the commit it merged as and its walk. On the pull request, `mesh/delivery` says
`covered: a later merge that contains it was delivered: …` as a **success**, never an error. A group counts
a covered member as delivered, so the member after it gets its turn and is not stopped.
`covered` says what was delivered, not what runs now: a revert merged after the covering merge is possible,
and is that revert's own delivery.
3. **A merge held without a passing check is covered without a person** when a later delivery whose own check
passed carries it, and only then. A later delivery's walk runs only on a passing check, so its check has
passed in every case this matters for. What a person would have been asked, "does this unchecked change go
on?", was answered by the later change, which was checked with this one's change in its history and
delivered.
4. **A group member is not held for a composed check that has not answered.** "Did not pass for the heads that
merged" means the check failed, ran for other heads, or was never asked. A member published while its
group's check is asked and not answered waits in `published`, within that state's bound.
5. **A hold whose reason is gone lifts.** A member held because its group's composed check had not passed is
taken back to `published` when that check passes or warns for the heads that merged, and its own walk is
still open. The event is *lifted*, observed, and the member then takes its turn in the group's order. Every
other hold stays a person's.
6. **Read again on every tick.** The holder reads the facts of every held delivery on each tick, not only when
an event arrives. A delivery that was held before these rows existed, or read back after a restart, ends by
itself. Once it is final, nothing moves it again.
The table of ADR 0239 decision 2 gains these rows:
| From | Event | To | Guard |
|---|---|---|---|
| proposed, checked, rejected, held | done | delivered | merged, no walk, and its plan moves no module the mesh holds |
| ready, held | covered | covered | a later merge that contains it was delivered: merged, no walk of its own open, and every module it moves sent to every machine running it by the walk of a later merge into the same trunk, delivered on its own passing check, whose merge commit the forge says contains this one |
| held | lifted | published | held only for its group's composed check, which has since passed or warned for the heads that merged, and its own walk still open |
`delivered`, `failed`, `superseded`, `stopped`, `retired` and `covered` are final.
## Consequences
- **The held deliveries end without a person.** When mesh-delivery with these rows is delivered, its first
ticks deliver the hq merges and the module-adding merges, and cover the controller and catalogue merges
that a later walk carried. Their D14 conditions clear at the self-check's next reading of `stalled`.
- **A pull request no longer says an error for a change a later merge delivered.** The four stopped on 2026-10-09 keep their
error: a final state is not undone. Their commit notes say why.
- **Containment is the forge's answer, not a premise.** The trunk's protection is no longer relied on: the
forge's compare decides. mesh-delivery gains one forge tool, which reads only.
- **The forge's tool must be delivered before `covered` can be reached.** Until `gitea_contains` answers, every
candidate is "not yet covered", and a held delivery stays held as before. Nothing ends wrongly while the forge
module is behind.
- **`covered` is only as complete as the walks' records.** A module the later walk held back for a person is
not sent, so it does not count. A delivery whose modules were only partly carried stays held for a person.
- **A module that runs on one machine only is never covered.** A walk records "the rest running it" only
after its first machines passed and the rest were sent. A module on one machine has no rest, so its steps
never say it reached every machine. mesh-delivery itself, on the control node alone, is such a module: a
held merge of it stays held for a person, as before this record. Reading "every machine running it" from
the first machines when they are all the machines would close it, and is left for a later change.
- **A group member waits longer, rather than being held too soon.** A composed check that never answers leaves
the member in `published`. Its bound of 30 minutes lists it as stalled. That is the right owner for a check
that does not answer.
**How it is checked.** mesh-delivery's tests, in `covered_test.go`:
- `TestAMergeThatMovesNothingIsDeliveredAtOnce`: builds nothing, or only a new module sent nowhere; merged
before its check answered, on a failed check, or on a passing one. Each is delivered, and its status is a
success.
- `TestAMergeALaterDeliveredWalkCarriedIsCovered`: covered whether it was held first or not. It stays held
when the later walk carried only one of its modules, merged before it, went into another branch, or failed,
and when the forge says the later merge does not contain it. With the forge away it stays held, and is
covered once the forge answers. The forge is asked once per pair, and never for a pair merge order rules
out. Its status is `success covered: a later merge that contains it was delivered` and names the commit
that covered it, never an error.
- `TestACoveredGroupMemberLetsTheNextOneGo`: the member after a covered one is let go, and the group ends
delivered.
- `TestHeldDeliveriesReadBackCloseByThemselvesOnce`: deliveries held before these rows existed are read back
from the store and end delivered or covered, or stay held when not covered. A merge held without a passing
check is covered by a later delivery whose own check passed, and stays held when that check did not pass. It
stays held when the later delivery's walk was of another commit or another branch, or sent its module to its
first machine only. A second restart moves nothing.
- The forge module's `test/checks.test.ts`: a compare with no commits ahead is contained, one with some is
not, an answer without a count is no answer, and only a sha reaches the compare's path. The forge module is
TypeScript. Its tests, and a type-check of every source, are run on the build seat by the catalogue's
TypeScript part of its check, `merge-check-typescript.sh` (mesh-catalog #151), in the TypeScript toolchain
with the SDK it holds. Before #151 no gate ran them: the catalogue's check ran only Go.
- `TestAGroupMemberWaitsForItsComposedCheckToAnswer` and `TestAMemberHeldForItsComposedCheckIsLiftedWhenItPasses`:
a pass or a warning lifts the member; a failure, or a pass for other heads, keeps it held.
The table's own tests take and refuse every new row. They also check that the only way to `published` other
than a ready delivery's merge is *lifted* from `held`, and that nothing observed ends a held delivery as
stopped or failed.
## References
- [Issue 319](../04-ISSUES/319-a-held-delivery-a-later-commit-superseded-has-no-end-but-release/00-report.md) and
[issue 316](../04-ISSUES/316-a-group-member-held-before-its-composed-check-answered-stayed-held-after-it-passed/00-report.md),
their reports and diagnoses.
- [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md),
decisions 2, 4 and 9; [ADR 0250](0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md),
the last state added to the table.
- [Issue 300](../04-ISSUES/300-a-new-module-is-not-built-at-its-merge/00-report.md), why a new
module opens no walk.
- [Design 47](../03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md), *The state table* and *A
delivery group*, which carry these rows.
- mesh-catalog #150, which builds it, and the forge module's `gitea_contains` with it; mesh-catalog #151,
which has the catalogue's check run its TypeScript modules.
02-DECISIONS/0271-a-send-held-for-the-buss-planned-step-is-told-to-the-operator-at-once-and-a-change-that-needs-it-says-so-before-it-merges.md
+122
View File
@@ -0,0 +1,122 @@
---
topic: the mesh
status: accepted
date: 2026-10-09
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md
---
# 271. A send held for the bus's planned step is told to the operator at once, and a change that needs it says so before it merges
## Context
A new build of the bus is a **planned step**
([ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md),
[to-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §8). Only a person's
`mesh-controller.bus upgrade` moves it. Until then, every send to the bus's machine is refused, because such a
send would carry the new bus with it. The controller's `sendToEach` refuses the send whole, and a walk keeps
the refusal as its note and tries again on the next tick.
On 2026-10-08 a catalogue merge built a new bus for the control-node
([issue 336](../04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/00-report.md)). For 28 minutes every
send to that machine was held: the controller's own walk, the node-engine's, and builds kept back for a gate.
`mesh-controller.conditions` listed nothing about it. The operator found the hold only because a walk looked
stuck, and asked where the approve button was.
Three things were missing, read from the controller's code
([diagnosis](../04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/01-diagnosis.md)):
- the refusal reaches only the walk's note, which nobody reads unless they already ask `plans`;
- the only watchdog that could fire, S3, fires no sooner than 30 minutes in. Its words say the delivery is late
and that nothing is lost. They name neither the bus nor the verb, and its resolver is not the operator;
- the merge gate and the delivery plan were green. The plan said "a bus step", but not that merging would
stop every later send to the bus's machine until a person acts.
## Considered Options
1. **Lower S3's bound for a walk refused for the bus.** Rejected: it would still say lateness, about one walk,
in words that name neither the cause nor the verb. A wait for a person is not a walk running late.
2. **Raise it from the self-check's probe DB, which already says a running bus step.** Rejected: the probe reads
only a step that has started, and it runs on the self-check's interval. The refusal happens in the walk,
which the watchdogs read every half minute.
3. **Let the walk send everything but the bus, and keep the bus at the build its machine runs.** Not decided
here. The diagnosis thinks it may now be possible, because every send keeps a recorded module at the build
its machine was last sent. That has to be checked against the bus's user list, which issue 249 sends first in
the same declaration. It would not remove the need to tell the operator: the step still waits for a person.
4. **Hold the merge for the operator's word when it builds a new bus.** Rejected for now. It would block a
change on a decision the operator can take after the merge as well. Asking the operator from the merge gate
belongs with the asks of ADR 0259 (an ask is a message on the bus that any channel answers), which is still
a pull request and not built.
5. **A condition of its own, raised at the first refusal, and a line in the delivery plan** — chosen.
## Decision
1. **Row S17 of the signals table: a send held for the bus's planned step is told to a person.** On the first
watchdog tick after a walk's send was refused for the bus, the controller raises
`bus.<bus module>.step-waiting`, kind `bus-step-waiting`. It is a warning, and its resolver is the operator.
Its summary names:
- the machines whose bus the new build would replace;
- each walk that waits, with its repository, commit and the modules of its tier;
- the bus build, from and to;
- since when: the first refusal this controller saw, or the save that kept the refusal after a restart;
- the call that ends it, as the operator makes it through the mesh MCP server:
`mesh_call mesh-controller.bus {"upgrade": "true", "why": "…", "reversible" or "irreversible": "true"}`.
Its plain words say that sends to the machine wait for a bus upgrade, what waits, for how long, and that the
operator starts the upgrade from the mesh MCP server. It clears once the bus's machine has been sent the build the
mesh holds, that is, once the step was taken, whatever the walks' notes still say. From then on no send is
refused for the bus, so nothing waits for a person; whether the new bus came up healthy is the step's own
condition, `bus-maintenance` and then `bus-upgrade-failed` (probe DB), not this one. A walk whose note names
an older bus build than the one held now does not count.
2. **S3 leaves out a walk held only by the bus's step**, as it leaves out a walk under a paused build seat. The
wait is said once, as itself, never as lateness.
3. **The delivery plan says it before the merge.** When a change builds the module that provides the bus and
that module runs somewhere, its plan has the step: "merging this needs a person's bus upgrade: nats is never
sent by an ordinary send, and until the bus upgrade runs, nothing else is sent to <its machines> either —
unless the new build turns out the same as the one running there (issue 280)". The plan cannot know that
before the build, so it says so rather than staying silent. Its summary says "merging this needs a person's bus
upgrade". The forge module writes that summary into the `mesh/merge-gate` status and the plan into the pull
request's comment. It is said, not enforced: the gate's verdict does not change.
## Consequences
- **The operator is told within half a minute of the first refusal**, not when a walk looks stuck. The
condition says what to run.
- **Merging a change that builds the bus is a known promise.** Its pull request says, before the merge, that a
person's step follows and what waits until then.
- **One more row in the signals table.** The generated test walks it like every other row.
- **The condition is about the bus, not about one walk.** Several walks waiting for the same step are one
condition, and the last one to be refused does not reopen it.
- Turning the condition into an ask, whose yes starts the step with the operator's word recorded, waits for
ADR 0259's asks to be built.
**How it is checked.** In the controller:
- the signals table's generated test for S17: with a new bus build waiting and no send refused, nothing is
raised; with a refused walk, the condition is raised; when the signal returns, it clears;
- `TestASendHeldForTheBusStepIsFoundFromTheWalksItHolds`: a walk refused for the bus is found, and since when.
A refusal for another reason, a refusal for an older bus build, a bus that already runs the build held, and a
walk waiting for its delivery's word are not;
- `TestASendHeldForTheBusStepIsSaidAtOnceAndNotAsLateness`: the condition's key, kind and operator resolver;
its summary names the machine, the module, the builds from and to, the verb and since when; its words are
plain. S3 says nothing of the walk past its bound, and still raises for a walk held for something else. Through
the keeper, the condition opens at the first tick and clears when the step was taken;
- the replay `TestReplay336`: a walk of the node-engine refused for a new bus build on the control-node raises
the condition on the next watchdog tick, through the store, the plans and the watchdogs' own gathering. It
clears once the machine is sent the new bus. It fails on the controller before the fix and passes on it;
- `TestAChangePlanSaysWhatEachMachineReceives`: a change that builds the bus says "merging this needs a
person's bus upgrade", the machine, and the verb.
## References
- [Issue 336](../04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/00-report.md), its report and
diagnosis.
- [ADR 0236](0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md),
the bus as a planned step; [ADR 0227](0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md)
rule 4, no failure is silent;
[ADR 0254](0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md),
the precedent of a wait for a person said as its own condition.
- [To-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §3, the signals table, and §8, the
bus as a planned step, which carry this row.
- mesh-controller #173, which builds it.
03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md
+7 -2
View File
@@ -2,8 +2,9 @@
layer: to-be
status: in-progress
code: [mesh-controller, mesh-host, mesh-tools, mesh-catalog, mesh-sdk, mesh-lab]
updated: 2026-10-08
updated: 2026-10-09
decisions:
- 02-DECISIONS/0271-a-send-held-for-the-buss-planned-step-is-told-to-the-operator-at-once-and-a-change-that-needs-it-says-so-before-it-merges.md
- 02-DECISIONS/0265-a-verb-that-reads-is-answered-on-the-serving-controllers-own-connection-and-every-connection-says-what-it-is.md
- 02-DECISIONS/0264-a-message-a-consumer-gives-up-on-is-kept-until-a-person-delivers-it-again-or-drops-it.md
- 02-DECISIONS/0261-an-assignment-made-while-its-modules-build-runs-is-kept-pending-and-the-controller-makes-it.md
@@ -233,6 +234,7 @@ signal and asserts its condition. `doctor signals` shows, for every row, the age
| S14 | facts snapshot exported | controller | when it moved, and daily | 2 days, or none kept by a controller up that long (Phase 5) | `facts-stale` | warning | — |
| S15 | a hand act with a cause already recorded | hand-act log | each act | the second within 14 days; clears when fewer than two remain within 14 days | `healer-wanted` (names the cause, and the healer that was not enough where one answers it) | warning | — |
| S16 | a walk waiting for its delivery's word is let go | mesh-delivery, through `deliver` (ADR 0239) | each merge whose walk waits | 30 min; urgent after 4 h | `waiting` (names the walk and `plans go`) | warning | — |
| S17 | a send held for the bus's planned step is told to a person (ADR 0271) | controller's plan | each send refused because it would replace the bus outside its step | none: the first refusal; cleared once the bus's machine has been sent the build the mesh holds | `bus-step-waiting`, for the operator (names the machines, what waits, the bus build from and to, since when and the `mesh-controller.bus` call); S3 leaves such a walk out | warning | — |
The bus advisories (S9) cost one read-only subscription: the server already publishes them. The
controller translates each into a condition naming the thing in the mesh's words, as the refused reply
@@ -562,7 +564,10 @@ bus machine's backup holder backing the bus module up now ([ADR 0235](../../02-D
or one a person took and names with `--snapshot-taken`; the step's bound is fifteen minutes; **no send
reaches the bus's machine while a new bus build waits for it** — a push, a plan's send for another
module, a healer's — after a plan's send restarted the bus unasked on 2026-10-06; a change of the bus's
user list is reloaded in place, not a restart. *2026-10-06:* the snapshot
user list is reloaded in place, not a restart. **A send held for the step is told at once** ([ADR 0271](../../02-DECISIONS/0271-a-send-held-for-the-buss-planned-step-is-told-to-the-operator-at-once-and-a-change-that-needs-it-says-so-before-it-merges.md)):
S17 raises `bus.<module>.step-waiting` for the operator on the first refusal, and a change that builds the bus
says in its delivery plan, and so in `mesh/merge-gate`'s description, that merging it needs a person's bus
upgrade. *2026-10-06:* the snapshot
and the way back from it exist — the bus image's own snapshot program, the same one the night's backup
runs, and a restore that builds a new store beside the live one for a person to swap in
([ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md),
03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md
+24 -8
View File
@@ -2,8 +2,9 @@
layer: to-be
status: designed
code: []
updated: 2026-10-08
updated: 2026-10-09
decisions:
- 02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md
- 02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md
- 02-DECISIONS/0249-a-delivery-groups-order-rules-have-a-precedence-and-a-declared-order-outranks-them-all.md
- 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
@@ -72,14 +73,26 @@ allows.
| ready | it passed; it waits to be merged | none | — |
| rejected | it failed its check | none | — |
| published | merged on the trunk its modules follow, its walk opened and asking nothing; it waits for its turn or its word | 30 minutes | `superseded` when a newer delivery took over its walk; `go` when its walk started |
| held | it waits for a person | 1 day | nothing: it is the operator's |
| held | it waits for a person; read again each tick, so it ends by itself when it moves nothing or a later merge covers it (ADR 0270) | 1 day | nothing: it is the operator's |
| delivering | its walk runs; its builds are asked and registered tier by tier | 2 hours | `delivered`, `failed` or `superseded`, as the walk's record says |
| delivered, failed, superseded, stopped, retired | final | — | — |
| delivered, failed, superseded, stopped, retired, covered | final | — | — |
The transitions are those of ADR 0239 decision 2. A delivery that is merged while not `ready` goes to
`held`. Its check did not pass, and only a person decides that it goes on. A delivery merged with no walk
opened for it within ten minutes is held too, saying so: nothing it moves follows that branch, or the merge
was not heard. Nothing waits silently.
The transitions are those of ADR 0239 decision 2 and [ADR 0270](../../02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md). A delivery that is
merged while not `ready` goes to `held`. Its check did not pass, and only a person decides that it goes on. A
delivery merged with no walk opened for it within ten minutes is held too, saying so: nothing it moves follows
that branch, or the merge was not heard. Nothing waits silently.
**What needs no person ends by itself** (ADR 0270):
- *a merge that moves nothing* (its plan builds nothing, or only adds a module the controller builds and
sends nowhere) is `delivered` as soon as it is merged, whatever its check said;
- *a merge a later delivered merge contains* is `covered`: every module it moves was sent to every machine
running it by the walk of a later merge into the same trunk, delivered on its own passing check, and the
forge says that later merge commit contains it (`gitea_contains`, asked once per pair and kept). Merge
order is only the first filter, and a forge that cannot answer means not yet covered. `mesh/delivery`
says `covered: a later merge that contains it was delivered` as a success. A module that runs on one
machine only gets no "rest" step and is never covered;
- the covered row is tried before `go`, from `held` and from `ready` past the ten minutes.
**History is retired, not held** ([ADR 0250](../../02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md)).
A delivery made from the forge's word at its merge, heard fifteen minutes or more after that merge, with no walk
@@ -108,7 +121,10 @@ controller's `plan-moved` event, never inferred from time.
- **Check**: the members' heads composed together, one gate over every machine with all their definitions. It
is asked by mesh-delivery through `delivery-check` when the group forms or a member's head moves, and
posted on every member's head as `mesh/delivery-group`.
- **Delivering**: in order. A member waits in `published` until the one before is `delivered`.
- **Delivering**: in order. A member waits in `published` until the one before is `delivered` or `covered`.
A member published while its group's composed check is asked and not answered waits there for the answer;
one held because that check had not passed is *lifted* back to `published` once it passes or warns for the
heads that merged and its own walk is still open (ADR 0270).
- **A failed member**: the members after it are `stopped`, naming it. The members before it stay delivered.
Its own builds were put back at its gate.
04-ISSUES/316-a-group-member-held-before-its-composed-check-answered-stayed-held-after-it-passed/01-diagnosis.md
+18
View File
@@ -47,3 +47,21 @@ The group's order put it second. Open question 4 in the report asks whether that
**Located** in mesh-delivery: `groupTurns` holds on a composed check that has not answered, and neither
`Checked` nor the table brings a held member back when the check later passes.
## 2026-10-09 — the fix, proposed
[ADR 0270](../../02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md)
(proposed) answers open questions 1 and 2, and mesh-catalog #150 builds it:
- **Asked and not answered is a reason to wait, not to hold.** `groupTurns` leaves a member in `published` while
its group's composed check is asked for the current heads and has no verdict.
- **A hold whose reason is gone lifts, by observation.** A member held because its group's composed check had not
passed goes back to `published` (the event *lifted*) once the check passes or warns for the heads that merged,
while its own walk is still open. It then takes its turn in order.
- A held member whose own walk was since taken over by a later walk is not lifted, since its walk can no longer
deliver it. It is `covered` when that later walk was delivered and sent its module (issue 319).
Open question 3, whether `mesh/delivery-group` is required on the trunks, stays the operator's setting. Open
question 4 is not touched.
**Accepted 2026-10-09.** The operator accepted the record named above; it is no longer proposed.
04-ISSUES/319-a-held-delivery-a-later-commit-superseded-has-no-end-but-release/01-diagnosis.md
+40
View File
@@ -107,3 +107,43 @@ remaining seven were left `held` for this issue's fix.
The seven show the two ends this issue asks for: four hold nothing to deliver, and three are covered by a
later walk.
## 2026-10-09 — the fix, proposed
[ADR 0270](../../02-DECISIONS/0270-a-held-delivery-that-moves-nothing-or-that-a-later-delivered-merge-contains-ends-by-itself-and-a-hold-whose-reason-is-gone-lifts.md)
(proposed) gives both ends this issue asks for, by observation, and mesh-catalog #150 builds them in
mesh-delivery:
- **A merge that moves nothing is delivered at once.** `MovesNothing()` no longer counts `Plan.New`: a module
the controller builds and sends nowhere opens no walk, so there is nothing to wait for. A merge whose plan moves
nothing goes from `proposed`, `checked`, `rejected` or `held` to `delivered`, before the row *merged unchecked
→ held*.
- **`covered`, a new final state: a later merge that contains it was delivered.** For every module its plan
moves, a delivered delivery of the same repository and trunk, merged after it, delivered on its own passing
check by the walk of its own merge commit, whose machine steps say the module reached every machine running
it. The forge then says, through a new read-only tool `gitea_contains` (its compare of the two commits),
whether the later merge commit contains this one; merge order is only the first filter. A forge that cannot
answer leaves it held. `mesh/delivery` says `covered: a later merge that contains it was delivered: …` as a
success.
- **Read on every tick**, so what is held now ends without an event.
This answers open question 1: mesh-delivery reads it from its own records and asks the forge for ancestry; the
controller is not asked. Open question 3, why the controller's own merges opened no walk, is not answered by this fix; it no
longer leaves anything held.
**How to verify after mesh-delivery is delivered with it.** Within a minute of its start:
1. `mesh-delivery.deliveries` with `state: held` lists none of the four hq merges, none of the module-adding
merges (`builds new: …, sent nowhere`), and none of the five controller merges held for "no walk opened".
The hq and module-adding ones are `delivered`; the controller ones are `covered`.
2. `mesh-delivery.show` of one of them, for example `novox/mesh-controller@b98fd0f396c8`, ends with a
transition `held → covered (covered)` whose why names the later delivery and its walk.
3. `mesh-delivery.checks` for its pull request shows `mesh/delivery` as `success covered: …`.
4. The forge module is delivered in the same pull request. Until its `gitea_contains` answers, nothing is
covered, and the held controller merges stay held.
5. At the self-check's next reading, `mesh-controller.conditions` no longer lists their
`delivery.<id>.stalled`.
6. The two catalogue merges held for their group (issue 316) are covered only if a later delivered walk sent
their module. One that stays held is a real question: its change may not run anywhere.
**Accepted 2026-10-09.** The operator accepted the record named above; it is no longer proposed.
04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/01-diagnosis.md
+23
View File
@@ -58,3 +58,26 @@ running build and carry the rest, so a pending bus step would stop holding every
control-node. That needs checking against the bus's user list, which [issue 249](../249-a-modules-new-state-is-refused-until-a-push-the-merge-did-not-make/00-report.md) sends first in the same
declaration, before it is proposed as a fix. It does not replace the condition: the step still waits for a
person, and the person must be told.
## 2026-10-09 — the fix, proposed
[ADR 0271](../../02-DECISIONS/0271-a-send-held-for-the-buss-planned-step-is-told-to-the-operator-at-once-and-a-change-that-needs-it-says-so-before-it-merges.md)
(proposed) takes suggested fixes 1 and 3. mesh-controller #173 builds them:
- **Row S17**, `bus.<bus module>.step-waiting`: raised on the first watchdog tick after a walk's send was refused
for the bus's planned step, while the new bus build would still replace the bus on its machine. It is a
warning, for the operator. It names the machines, each waiting walk and the modules of its tier, the bus build
from and to, since when, and the `mesh-controller.bus` call with `upgrade`. It clears once the bus's machine has
been sent the build the mesh holds.
- **S3 leaves out a walk held only by it.**
- **The delivery plan says it before the merge**: "merging this needs a person's bus upgrade: … nothing else is
sent to <machine> either", and its summary, which `mesh/merge-gate` carries, says the same.
The replay `TestReplay336` in the controller fails on the commit before the fix ("the first tick after the refusal
told the operator nothing about the bus's step: 0 condition(s)") and passes on it. It still has to be registered
in mesh-lab's replays register as R336 before this issue resolves with `replay: R336`.
Suggested fix 2 (an ask) waits for ADR 0259's asks. The guess about letting a send keep the bus at its running
build and carry the rest is not checked here.
**Accepted 2026-10-09.** The operator accepted the record named above; it is no longer proposed.