Merge pull request 'Playbooks 07 and 08: lessons from the 2026-10-25 recovery' (#476) from pb/recovery-lessons-2026-10-25 into main

This commit was merged in pull request #476.
This commit is contained in:
2026-10-25 12:30:43 +00:00
2 changed files with 23 additions and 0 deletions
+16
View File
@@ -269,6 +269,12 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
One with no answer is a change asked. On 2026-10-23 the forge module's loop rewrote every repository's
branch protection every ten minutes from rules in its code and undid the operator's changes at the forge
(issue 1903).
9. **At the second instance of a fault kind in a review, the developer searches for every instance and names each in the
report.** On 2026-10-25 a "busy, return at once" guard that dropped a newer request was found in three places,
one review after another. The developer writes the kind in one line, searches the repository (`git grep`) for
its shape, fixes every hit in one change and lists each file and line in the report. How it is checked: a
report to the second round naming a kind and one place only, followed by a third round on the same kind, is
this rule skipped.
## Step 3 — Check (no review)
@@ -349,6 +355,16 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
the Approve and the merge, and wakes the lead. Beyond that
Approve, the operator's word is waited for only when the milestone changes what the operator sees, removes
data, or needs a step only the operator's terminal can take; the lead says which before the milestone starts.
**A pull request that depends on an earlier one outside its delivery group is opened only once the earlier one is delivered**, its build
running on every machine it should reach, not merely merged. A note in the description is no safeguard: the
operator's Approve merges whatever is asked. On 2026-10-25 a second change merged before the first was delivered
and machines lost modules. How it is checked: `mesh-delivery.deliveries` shows the earlier change's source
revision `delivered` before the later pull request exists (`gitea.gitea_get_pull_request`, its creation time);
a later one older than that is this rule skipped. Members of one delivery group are the exception: they open together, with one check, one Approve and one merge in one pass (item 1).
**A reviewed follow-up to a change the operator has not yet approved is folded into that pull request** (its head
moved to include it): one Approve, one delivery. The old Approve ask closes itself on the new head and the new
head is asked once its checks pass. How it is checked: a follow-up that sits in a second pull request while the
first still waits shows in `gitea.gitea_list_pull_requests` as two open pull requests of one lead.
**One exception: a catalogue change that uses a new manifest word.** The catalogue's merge check reads
manifests with the controller the mesh runs, so the controller's pull request is merged and delivered first;
+7
View File
@@ -76,6 +76,13 @@ one delivery**. Everything before the try is local and fast.
integration test starting the module whole against the real bus and store, run by `mesh-controller.try` and the checker's merge check
(ADR 0391 §1, §6); unit tests alone are not that proof, and the lab is not used. The operator's word is waited for only when the fix changes what the operator sees, removes data,
or needs the operator's terminal.
**A fix that depends on another pull request outside its delivery group opens only once that one is delivered**, not merely merged (members of one group open together, as playbook 07 step 3 item 1 says) (a note
in the description does not hold it back: the Approve merges whatever is asked; 2026-10-25, machines lost modules
when a second change merged first). **A reviewed follow-up to a fix still waiting for the Approve is folded into
that pull request** (head moved): one Approve, one delivery. Both are checked as playbook
[07](07-feature-branches.md) step 3 says.
**At the second instance of a fault kind found in a review, the developer searches for every instance** and names
each in the report (playbook [07](07-feature-branches.md) step 2, item 9).
7. **Follow its delivery to done**: a watcher waits on the status view's events (`mesh-delivery-view.moving`,
`.settled`, `.stuck` and `.delivered`) and reads the view's `assigner_wants` and `node_haves` after each wake, as
playbook [07](07-feature-branches.md) step 4 says; the delivery is done when the view says `delivered` for its