Merge pull request 'Issue 313: history adopted at the switch was held for a person; ADR 0250 retires it' (#197) from fix/313-retire-adopted-history into main
This commit was merged in pull request #197.
This commit is contained in:
+96
@@ -0,0 +1,96 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-08
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
|
||||
---
|
||||
|
||||
# 250. A merge heard only from the bus's history is retired, and owed work the forge cannot do ends
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)
|
||||
gives the delivery's owner one table of states. Decision 10, *the switch*, adopts the walks open when the
|
||||
owner is first assigned. It does not mention the merges the bus still holds from before. A new consumer of the
|
||||
forge's `pull.merged` hears them all again. The owner took each as a fresh merge whose head it had never
|
||||
heard, and its row *merged unchecked → held* held each one for a person.
|
||||
|
||||
At the first switch that was 688 deliveries, merged from four days to two hours earlier, all on the machines
|
||||
long before
|
||||
([issue 313](../04-ISSUES/313-history-was-held-for-a-person-and-its-views-retried-for-ever/00-report.md)).
|
||||
`held` could be left only by a release, which walks, or a stop, which says a person stopped a running
|
||||
delivery. Neither is true of history. The self-check listed every one past its bound.
|
||||
|
||||
Decision 5 says owed work is "retried until done". A view on a pull request the forge no longer holds can
|
||||
never be done. It was tried every ten seconds, 11,000 times each, and nothing said so.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **A final state `retired` for history, taken at adoption, and a person's act for what is already held.**
|
||||
Chosen.
|
||||
2. **Adopt history as `delivered`.** It did reach the machines, but this owner did not deliver it, and
|
||||
`delivered` is the word for a walk this owner saw done. Rejected.
|
||||
3. **Stop them.** `stopped` means a person ended a delivery, and its status is an error on the commit.
|
||||
Rejected.
|
||||
4. **Let healer H2's `close` take held history.** `held` is the operator's by design. A healer leaving it
|
||||
unasked would hide the case where the rule is wrong. Rejected for the deliveries already held. New history
|
||||
is not held at all, so no healer is needed for it.
|
||||
5. **Skip merges from the bus's history altogether.** Then the commit's note would keep no record that this
|
||||
owner heard them, and a merge heard late for another reason would go unrecorded. Rejected.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **History is a delivery made from the forge's word at its merge, heard fifteen minutes or more after
|
||||
that merge, and with no walk of it open.** The forge says a merge within seconds, so one heard that late was
|
||||
replayed. A walk the controller keeps open or waiting for it makes it a delivery, never history.
|
||||
2. **A new final state, `retired`.** It is reached in two ways:
|
||||
- *history*, an observed row from `proposed`, `checked`, `ready` and `rejected`, taken before any other.
|
||||
New history goes there at adoption and is never checked, held or walked;
|
||||
- *retire*, a person's act from `held`, with why, whose guard is the rule of point 1. It is never a release:
|
||||
nothing is walked, delivered or put back.
|
||||
3. **The seat gains `retire-history`.** It takes one delivery by id and is refused when the rule does not
|
||||
take it. Or it takes every held delivery, of one repository when named, and answers how many it retired and
|
||||
which it left held and why. `dry` answers the same and changes nothing. The verb is promised by the
|
||||
controller as optional before the holder serves it
|
||||
([ADR 0246](0246-a-seats-new-verb-is-promised-before-it-is-required.md)).
|
||||
4. **History is not shown on its pull request.** Its transitions are said on the bus and kept on the commit's
|
||||
note. No view or `mesh/delivery` status is asked: the forge may no longer hold that pull request, or may
|
||||
hold another under its number.
|
||||
5. **Owed work ends.** This replaces "retried until done" in decision 5:
|
||||
- the forge's own answer that what an effect is for does not exist (a 404, or "does not exist") is final.
|
||||
The effect is given up with one line among the owner's refusals;
|
||||
- any other failure is tried again later each time: ten seconds, doubling to an hour. After sixty tries,
|
||||
about two days, it is given up the same way.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The deliveries held at the first switch are retired by one call of `retire-history`, after a dry run says
|
||||
how many. The self-check's findings for them end.
|
||||
- A switch, a reassignment or a lost consumer position no longer turns the bus's history into work for a
|
||||
person.
|
||||
- A merge heard late that is not history, because its walk waits, is held as before.
|
||||
- An owed effect can now be lost. When it is, the owner says so, and the commit's note keeps whatever was
|
||||
already written.
|
||||
|
||||
**How it is checked.** mesh-delivery's tests: `TestAMergeLongPastHeardFromTheForgesWordIsRetiredAtAdoption`
|
||||
(history retired at adoption, with nothing asked of its pull request; a live merge from the forge's word and
|
||||
a merge whose walk is open are not history; a reused number does not merge an old delivery again),
|
||||
`TestRetireHistoryRetiresTheAdoptedHistoryInBulkAndLeavesEveryOtherHeldOne` (dry first, then the bulk retire;
|
||||
nothing asked of the controller; a retired delivery cannot be released),
|
||||
`TestRetireHistoryIsRefusedForARealHeldDelivery`, and `TestOwedWorkTheForgeSaysCanNeverBeDoneIsDropped` (a
|
||||
final answer is given up after one try and said; other failures back off and end). The table's own tests
|
||||
take and refuse both new rows. The controller's `TestTheDeliverySeatPromisesRetireHistoryOptionally` checks
|
||||
the seat promises the verb as optional.
|
||||
|
||||
## References
|
||||
|
||||
- [Issue 313](../04-ISSUES/313-history-was-held-for-a-person-and-its-views-retried-for-ever/00-report.md),
|
||||
its report and diagnosis.
|
||||
- [ADR 0239](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md),
|
||||
decisions 2, 5 and 10.
|
||||
- [ADR 0246](0246-a-seats-new-verb-is-promised-before-it-is-required.md), a verb promised before it is
|
||||
required.
|
||||
- [Design 47](../03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md), *The state table*, *What every
|
||||
transition does* and *The verbs*.
|
||||
@@ -4,6 +4,7 @@ status: designed
|
||||
code: []
|
||||
updated: 2026-10-08
|
||||
decisions:
|
||||
- 02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md
|
||||
- 02-DECISIONS/0249-a-delivery-groups-order-rules-have-a-precedence-and-a-declared-order-outranks-them-all.md
|
||||
- 02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md
|
||||
- 02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md
|
||||
@@ -73,12 +74,18 @@ allows.
|
||||
| published | merged on the trunk its modules follow, its walk opened and asking nothing; it waits for its turn or its word | 30 minutes | `superseded` when a newer delivery took over its walk; `go` when its walk started |
|
||||
| held | it waits for a person | 1 day | nothing: it is the operator's |
|
||||
| delivering | its walk runs; its builds are asked and registered tier by tier | 2 hours | `delivered`, `failed` or `superseded`, as the walk's record says |
|
||||
| delivered, failed, superseded, stopped | final | — | — |
|
||||
| delivered, failed, superseded, stopped, retired | final | — | — |
|
||||
|
||||
The transitions are those of ADR 0239 decision 2. A delivery that is merged while not `ready` goes to
|
||||
`held`. Its check did not pass, and only a person decides that it goes on. A delivery merged with no walk
|
||||
opened for it within ten minutes is held too, saying so: nothing it moves follows that branch, or the merge
|
||||
was not heard. Nothing waits silently. **Nothing of a published delivery is registered before its walk
|
||||
was not heard. Nothing waits silently.
|
||||
|
||||
**History is retired, not held** ([ADR 0250](../../02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md)).
|
||||
A delivery made from the forge's word at its merge, heard fifteen minutes or more after that merge, with no walk
|
||||
of it open, was replayed from the bus's history: it reached the machines before this owner was there. It goes
|
||||
to `retired` at adoption, before any check, hold or walk, and its pull request is not told. A held delivery that
|
||||
fits the same rule is retired by a person's `retire-history`, never released. **Nothing of a published delivery is registered before its walk
|
||||
starts**, because a send carries every registered build that waits on its machine (ADR 0236 §4a), and a
|
||||
build registered before its turn would be carried by somebody else's send.
|
||||
|
||||
@@ -113,15 +120,16 @@ them.
|
||||
In this order, each by its owner:
|
||||
|
||||
1. **kept**: mesh-delivery puts the delivery in its state (`deliveries`, one key per delivery; `groups`, one
|
||||
key per group). Nothing is said before it is kept. What it owes outside is kept with it and retried until
|
||||
done;
|
||||
key per group). Nothing is said before it is kept. What it owes outside is kept with it and retried, later
|
||||
each time, from ten seconds doubling to an hour. The forge's answer that what it is for does not exist ends
|
||||
it at once, and sixty failed tries end it too, each said among the owner's refusals (ADR 0250);
|
||||
2. **said**: the event `mesh-delivery.transition` (and `mesh-delivery.group` when a group's derived state
|
||||
changes), with the delivery's id, from, to, event, why and plan summary, and no secret or address;
|
||||
3. **noted**: mesh-delivery asks the forge's holder (`gitea_note_append`) to append a line to the commit's
|
||||
note: the head while the delivery is off the trunk, the commit it landed as once it is on it, and at its
|
||||
end a line of what was executed there. The forge's holder writes it in its own repository as the forge's
|
||||
own account, and a line already there is not appended again;
|
||||
4. **reflected**: the delivery's view, one comment on the pull request with the state, the plan, the machine
|
||||
4. **reflected** (not for history, whose pull request the forge may no longer hold): the delivery's view, one comment on the pull request with the state, the plan, the machine
|
||||
steps and the group, kept current (`gitea_delivery_view`); the status `mesh/delivery` on the head, and
|
||||
`mesh/delivery-group` on every member's head (`gitea_commit_status`). Every status links to the pull
|
||||
request, `mesh/merge-gate` among them.
|
||||
@@ -142,6 +150,7 @@ In this order, each by its owner:
|
||||
| `release` | a held delivery to delivering, with why: a person's word |
|
||||
| `stop` | any delivery not final to stopped, with why; its walk is ended through the controller |
|
||||
| `close` | H2's only verb: the transition the table names for a stalled delivery, refused otherwise |
|
||||
| `retire-history` | held history to retired, with why: one by id, refused when the rule does not take it, or every held one, answering how many and naming those it left; `dry` changes nothing |
|
||||
|
||||
**The controller's verbs for it** (on the mesh-controller seat): `delivery-plan`, `delivery-order`,
|
||||
`delivery-check` (a group's heads composed, or one head checked again), `deliver`, `delivery-stop`,
|
||||
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-08
|
||||
located-in: [mesh-catalog modules/mesh-delivery (holder.go, adoption of a merge from the forge's word; effects.go, the owed effects; table.go, the state table)]
|
||||
fixed-by:
|
||||
amended-design: 03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md
|
||||
---
|
||||
|
||||
# 313. History was held for a person, and its views were retried for ever
|
||||
|
||||
## Symptom
|
||||
|
||||
On 2026-10-08 the controller's self-check D14 failed with 689 findings of one kind:
|
||||
|
||||
> the delivery novox/hq@018ee359ae52 has been held for 33h58m, past its bound of 24h (it waits for the
|
||||
> operator)
|
||||
|
||||
They spread over ten repositories: the hq repository 233, the catalogue 174, the controller 160, the
|
||||
node-engine 58, the tool runner 37, the SDK 14, the media catalogue 7, two personal repositories 4 and the lab 2.
|
||||
688 of them say the same thing when shown with `mesh-delivery.show`:
|
||||
|
||||
- the pull request merged days before the delivery's owner existed — the one shown on 2026-10-02;
|
||||
- the delivery was made at the owner's first start, 2026-10-06 23:05, within two seconds of all the
|
||||
others, with the transition "merged; its head was not heard by this owner, so it is made from the
|
||||
forge's word";
|
||||
- then "merged-unchecked → held: merged without a passing check: only a person decides that it goes on".
|
||||
|
||||
Every one of those commits had reached the machines long before. Nothing was to be delivered, released or put
|
||||
back for any of them, and yet each waited for a person, and the self-check said so 688 times.
|
||||
|
||||
Beside them, 488 owed a **view** — the comment on the pull request — that failed every ten seconds: the one shown
|
||||
11,051 times, each time with the forge's answer "issue does not exist". The forge's numbers began again around
|
||||
2026-10-04, so the old pull requests are gone. Where a number had been reused, the view of an old delivery
|
||||
was asked of a new, unrelated pull request.
|
||||
|
||||
## Why it is a design issue
|
||||
|
||||
[ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)
|
||||
decision 10, *the switch*, adopts the walks open when mesh-delivery is assigned. It says nothing of the merges the
|
||||
bus still holds from before: the forge's `pull.merged` is kept on the bus and heard again by a new consumer.
|
||||
Each was taken as a merge just made whose head was never heard, so the table's row *merged unchecked → held*
|
||||
held each for a person. The table had no state for "this already happened", and `held`'s only ways on were
|
||||
a release, which would walk it, and a stop, which says a person stopped a delivery that was never running.
|
||||
Healer H2's `close` has no transition from `held`, rightly: a held delivery is the operator's.
|
||||
|
||||
Decision 5 says what a transition owes outside is "retried until done". An effect the forge answers can never
|
||||
be done was retried without end, every ten seconds, and nothing said so.
|
||||
|
||||
## Fix
|
||||
|
||||
- **A new final state, `retired`: history.** A delivery made from the forge's word at its merge, whose merge
|
||||
was heard fifteen minutes or more after it happened and whose walk, if the controller keeps one, is not
|
||||
open, is history. At adoption it goes straight to `retired` by the observed row *history*, before any check,
|
||||
hold or walk. Its pull request is not told: the forge may no longer hold it, or holds another under its
|
||||
number. Its transitions are still said on the bus and kept on the commit's note.
|
||||
- **A person's verb for the ones already held: `retire-history`**, with why. One delivery by id, refused when
|
||||
the rule does not take it, or every held one (of one repository when named), answering how many it retired
|
||||
and naming each one it left held. `dry` says what it would do and changes nothing. It is the act
|
||||
*held → retired* of the table, never a release: nothing is walked, delivered or put back.
|
||||
- **Owed work ends.** The forge's answer that what an effect is for does not exist (a 404, or "does not
|
||||
exist") is final: the effect is given up with one line, kept among the owner's refusals. Any other failure is
|
||||
tried again later each time, from ten seconds doubling to an hour, and given up after sixty tries, saying so.
|
||||
- A merge announced under a number an older delivery already merged with is not merged into that delivery
|
||||
again: a forge whose numbers began again reuses them.
|
||||
|
||||
The seat's protocol gains `retire-history` as an optional verb
|
||||
([ADR 0246](../../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md)): the controller
|
||||
promises it first, then the holder serves it. The decision is
|
||||
[ADR 0250](../../02-DECISIONS/0250-a-merge-heard-only-from-the-buss-history-is-retired-and-owed-work-the-forge-cannot-do-ends.md).
|
||||
|
||||
**Not core.** The defect and its fix are in the delivery's owner, a catalogue module that is not the bus, the
|
||||
forge or the build agent. The controller's part is one optional verb in a seat's protocol. Per `cycle.py`'s
|
||||
CORE rule, no replay is required.
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
# 313 — diagnosis
|
||||
|
||||
**2026-10-08.** Read from the delivery's owner, `mesh-delivery.show` and `deliveries`:
|
||||
|
||||
- 699 deliveries were `held`. 688 say "merged without a passing check", all made between 23:05:27 and
|
||||
23:05:29 on 2026-10-06, the owner's first start. A sample across repositories shows merges from
|
||||
2026-10-02 to two hours before that start, each with the transition "its head was not heard by this owner,
|
||||
so it is made from the forge's word", then *merged unchecked → held*. None has a walk.
|
||||
- The other 11 are held for other reasons: a merge with no walk opened within ten minutes (10), and a group
|
||||
whose composed check did not pass (1). They are not history, and this issue leaves them alone. One of them
|
||||
was past its bound, which makes the 689th finding.
|
||||
|
||||
**Where it comes from.** The holder subscribes to the forge's `pull.merged` on the bus after reading back its
|
||||
state and the controller's walks. The bus hands a new consumer the merges it still holds. For each, the holder
|
||||
finds no delivery for the head, so it makes one from the forge's word. The pull request's statuses at its
|
||||
merge do not include a passing `mesh/merge-gate` (the check did not exist for most of them), so the table's
|
||||
*merged unchecked* row holds it.
|
||||
|
||||
**What would close them, and why not.**
|
||||
|
||||
- `release` walks a held delivery: wrong, these reached the machines already.
|
||||
- `stop` takes it to `stopped`, which says a person stopped a delivery and sets `mesh/delivery` to error on
|
||||
the commit. Wrong in meaning, and it would ask the forge for a view on a pull request that is gone.
|
||||
- Healer H2's `close` reads `held`'s bound, which names no H2 transition, and refuses: "the state is the
|
||||
operator's". Checked in the code (`Close`, verbs.go) and in the table: it would not release them, and it would
|
||||
not retire them either.
|
||||
|
||||
**The view retried for ever.** `Flush` tries every owed effect on every tick, ten seconds apart, keeping only
|
||||
a count and the last error. A view on a pull request the forge no longer holds fails with the forge's
|
||||
"issue does not exist", and nothing in the holder treated any answer as final. Where a number was reused,
|
||||
the old view was asked of the new pull request under that number.
|
||||
|
||||
**Ruled out:** the forge's holder. It answers what the forge says. The forge's renumbering is outside the
|
||||
mesh: the holder must tolerate it, not prevent it.
|
||||
|
||||
**Located** in the delivery's owner: its adoption of a merge made from the forge's word (holder.go
|
||||
`PullMerged`), its table, which had no state for history, and its owed effects (effects.go `Flush`).
|
||||
Reference in New Issue
Block a user