Merge pull request 'Playbooks 07 and 08: lessons of 2026-10-28 (the report's tree, its cadence, named local integration tests, member repositories during a check, delivered means what the nodes run, real-shaped fixtures)' (#489) from pb/lessons-2026-10-28 into main
This commit was merged in pull request #489.
This commit is contained in:
@@ -56,7 +56,7 @@ opened **once**, after the review, and **one check** (ADR 0548). It runs in five
|
||||
| Step | What happens | Bound |
|
||||
|---|---|---|
|
||||
| 0. Prepare | the lead settles the open questions and writes one context file per repository | 15 minutes |
|
||||
| 1. Develop | one fixing agent per repository builds every issue locally, the whole unit suite before every commit, the integration tests compiled; nothing is pushed | the milestone's issues, 30 minutes each |
|
||||
| 1. Develop | one fixing agent per repository builds every issue locally, the whole unit suite before every commit, the integration tests compiled (those that exercise the change are run by name before the push, step 3); nothing is pushed | the milestone's issues, 30 minutes each |
|
||||
| 2. Review | one reviewer reads all of it, the local commits, judging the integration tests by reading them against the code; reports go both ways; at most 3 rounds | 15 minutes |
|
||||
| 3. Check | push the reviewed heads once and open the pull requests together; the checker's one check runs the integration tests; no further review unless the check fails | one check run |
|
||||
| 4. Deliver | merge; the source revision moves, each module is built once and wanted on its nodes, its canary first | until the view says the source revision `delivered` |
|
||||
@@ -201,8 +201,8 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
||||
acceptance criteria**: one test or more per criterion at the level it names, each test's comment citing
|
||||
the criterion it proves by its label (`// 1118/AC2`), and no criterion left without its test; an integration criterion is a test
|
||||
that compiles and is run by the pull request's one check. The integration tests (those that start a store, a bus
|
||||
or a process) are compiled with `go vet -tags integration ./...` while developing and run by that check, nowhere
|
||||
else.
|
||||
or a process) are compiled with `go vet -tags integration ./...` while developing and run by that check; before the group's push only the tests that exercise the change are run by
|
||||
name (step 3), nothing else.
|
||||
- **Nothing is pushed and no pull request is opened in this step, and there is no try** (ADR 0548; the operator,
|
||||
2026-10-26: "every release should only run 1 check or 1 try"). When every issue is committed, the lead brings each
|
||||
feature branch up to the current trunk (fetch by name, then merge) and the review reads the local commits. Never call
|
||||
@@ -215,7 +215,8 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
||||
is a numbered migration and the consolidated schema; Go only unless the context file says otherwise; never
|
||||
move another issue; **leave the integration tests to the check** (no store or bus started by hand): compile them
|
||||
with `go vet -tags integration ./...`. A single integration test may be run locally against an embedded bus when
|
||||
it is the fastest way to find a cause, said in the review report and the pull request. After a failed check it
|
||||
it is the fastest way to find a cause, said in the review report and the pull request; before the group's push the
|
||||
tests that exercise the change are run this way, by name (step 3). After a failed check it
|
||||
usually pays to run that repository's whole integration level locally (`go test -tags integration -count=1 ./...` in each
|
||||
module), by the lead or by the fixing agent on the lead's word, before the one more push: a verdict names only the first failed
|
||||
test (issue 1921), and on 2026-10-23 R4g's faults surfaced one per run until a local run listed them all in a minute (a
|
||||
@@ -263,6 +264,9 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
||||
work-queue retention; three review rounds passed it, and only its try showed that every seat's work stream has
|
||||
that retention too, so dead asks across the mesh would no longer have been delivered again (issue 1178, fixed
|
||||
before its pull request by a mark only a declared module work queue carries).
|
||||
**A test fixture writes state in the form its real writer writes it** (for example a resolved config value with its
|
||||
source, as the controller stores it); a fixture that agrees with the mistake proves the mistake. How it is checked:
|
||||
the review asks it of every new fixture, and says which writer it compared with.
|
||||
6a. **The reviewer reads the milestone file's three answers** (step 0, item 8) against the change: it lists every bucket
|
||||
key and stream the diff creates or writes (search the diff for the bucket and stream names) and finds each in the
|
||||
lifetime answer with a deleter and event, or a time to live; it reads the switch-over answer and checks it was done as
|
||||
@@ -296,8 +300,17 @@ issue whether it is being fixed in between, after the delivery, or waits, and on
|
||||
and the integration level, through the build module of the repository's language and by that language's convention
|
||||
(ADR 0445, ADR 0529); the check waits on the language's check stream until a build node has a free slot
|
||||
(ADR 0525 §1). **No try:** never call `mesh-controller.try` or `mesh-delivery-checker.checker_try`, except when the
|
||||
operator asks for one. While developing, no `go test -tags integration` run of a whole level, and no store or bus started
|
||||
operator asks for one. While developing, no `go test -tags integration` run of a whole module or level, and no store or bus started
|
||||
by hand on the machine (the operator, 2026-10-18).
|
||||
**Before a group's single push, the integration tests that exercise the change are run locally against an embedded
|
||||
bus**, picked by name (`go test -tags integration -count=1 -run '<names>'`): the tests in the touched packages that
|
||||
start the part whole, not a whole module's or level's integration run, and not only a compile with `go vet`; a
|
||||
change in how a node composes or applies always has such a test (2026-10-28: a compiled-only group failed in the one
|
||||
check on what a local run shows). How it is checked: the pull request's description names those tests and their
|
||||
results.
|
||||
**While a group is being checked, nothing else is merged or pushed into a repository that is one of its members:**
|
||||
a moved main asks the group's check again. How it is checked: the lead's wait list names the group's members, and
|
||||
`mesh-delivery.show` shows one check per head.
|
||||
**A group is one check.** Before the push, every member pins the group's own mesh-sdk head exactly (the checker refuses an
|
||||
older commit of the same branch), then all members are pushed and opened together, so the group's check runs once. A head the review replaces
|
||||
counts for nothing against a running check; say so, and ask no other until the new head is reviewed.
|
||||
@@ -406,7 +419,10 @@ the source revision equals its fingerprint and every have its want). After each
|
||||
the view's wants page or its tools: `assigner_wants` and `assigner_want` (a want not yet at the newest fingerprint
|
||||
names, in `newest` and `rule`, the rule that holds it back, and its words say why), and `node_haves` (the node
|
||||
agent's haves: fingerprint, composition digest, phase and health). The delivery is done when the view says
|
||||
`delivered` for its source revision; then the lead resolves the feature's issues with `fixed-by` naming its pull
|
||||
`delivered` for its source revision; **"on the machines" from the delivery tools is not yet "delivered"**: the lead
|
||||
also reads what the nodes run (the view's haves per node) and the indexer's record of the module (its newest
|
||||
fingerprint) before saying a change is delivered. How it is checked: the report names each node's build.
|
||||
Then the lead resolves the feature's issues with `fixed-by` naming its pull
|
||||
requests, and removes what is left: no `feat/<slug>` branch and no `.work/<slug>` worktree survive.
|
||||
|
||||
**What a merge moves is read before the operator is told.** A change to mesh-sdk rebuilds every module that takes
|
||||
@@ -511,7 +527,7 @@ Approve, a Retry or a Release waits for them, it reads `operator-channel.asks` a
|
||||
on 2026-10-20 an ask was named to the operator that the router did not hold. A merge the gate passed but whose
|
||||
ask is not on the router is the fault of the Approve's wait in the table above.
|
||||
|
||||
**Every ten minutes (the operator, 2026-10-17), every two minutes while a delivery moves (the view says modules `moving`), and at once when something lands, stops or
|
||||
**Every five minutes (the operator, 2026-10-17, ten then; five since 2026-10-26), every two minutes while a delivery moves (the view says modules `moving`), and at once when something lands, stops or
|
||||
needs the operator,** the lead reports, in one of the two layouts below (the operator, 2026-10-16: an overview
|
||||
of the issues, often). Times are local and absolute, from the clock (`date`), never guessed; every number has its
|
||||
unit and its bound or baseline; an issue is named by its number and a few words, never by its number alone.
|
||||
@@ -543,12 +559,14 @@ Allowance: week <n> % (<+d> since the last report), session <n> %
|
||||
Next: <the next actions up to the delivery, in order>
|
||||
```
|
||||
|
||||
**The tree is read, not assembled.** `knowledge-base.progress` with `of: "design <nn>"` answers the tree's
|
||||
**Every report ends by scheduling the next** (a timer or a watcher set for five minutes ahead), so that agents'
|
||||
incoming reports never replace it. How it is checked: the operator sees a report at least every five minutes.
|
||||
**The report starts with the tree, drawn from the verb's answer and never by hand.** `knowledge-base.progress` with `of: "design <nn>"` answers the tree's
|
||||
milestones, issues, pull requests and held groups in one call, each with when it last moved: `detail: minimal` for the
|
||||
five-minute report, `detailed` about every thirty minutes and whenever something stops (issue 1947). What it cannot
|
||||
read for certain from the design's milestone table it says in `warnings`; the lead fixes the table, it does not
|
||||
correct the tree by hand. How it is checked: a report's tree that disagrees with the verb's answer of the same minute
|
||||
is a tree assembled by hand.
|
||||
is a tree assembled by hand, and a report that does not quote the verb's read-at time is the same.
|
||||
|
||||
**The short layout** (on a channel such as Telegram, or when nothing changed but time):
|
||||
|
||||
|
||||
@@ -53,7 +53,10 @@ processing power"). Everything before the push is local and fast, and no try is
|
||||
criterion it proves by its label (`<issue>/AC<n>`, playbook 07 step 0 item 5). Commit, naming the issue. For a playbook or design edit, run the repository's own checks (in hq,
|
||||
`go test ./...` at the root). The integration tests are compiled (`go vet -tags integration ./...`) and not run:
|
||||
nothing is pushed, no pull request is opened and no try (`mesh-controller.try`, `mesh-delivery-checker.checker_try`) is called, unless the operator asks for
|
||||
one. A single integration test may be run locally against an embedded bus when it is the fastest way to find a cause.
|
||||
one. A single integration test may be run locally against an embedded bus when it is the fastest way to find a cause;
|
||||
before the push, the integration tests that exercise the change are run that way, by name
|
||||
(playbook 07 step 3), and the pull request names them and their results. A test fixture writes state in the form its real
|
||||
writer writes it; the review asks it.
|
||||
4. **One review of the local commit** (a reviewer that did not write it, larger
|
||||
model; at most 3 rounds, as playbook 07; from round 2 only what is wrong; it reads each acceptance criterion
|
||||
against its test, as playbook 07 step 2 item 6; a finding outside the issue's done-when and its acceptance
|
||||
@@ -87,7 +90,8 @@ processing power"). Everything before the push is local and fast, and no try is
|
||||
7. **Follow its delivery to done**: a watcher waits on the status view's events (`mesh-delivery-view.moving`,
|
||||
`.settled`, `.stuck` and `.delivered`) and reads the view's `assigner_wants` and `node_haves` after each wake, as
|
||||
playbook [07](07-feature-branches.md) step 4 says; the delivery is done when the view says `delivered` for its
|
||||
source revision. Then prove the fix on the mesh (the symptom gone, measured), and resolve the issue with
|
||||
source revision, and the haves per node and the indexer's record of the module name the build (playbook 07 step 4;
|
||||
"on the machines" alone is not "delivered"). Then prove the fix on the mesh (the symptom gone, measured), and resolve the issue with
|
||||
`located-in` and `fixed-by` the same hour.
|
||||
|
||||
A fix that stops before its delivery moves its issue back to `open` with a note saying how far it got
|
||||
|
||||
Reference in New Issue
Block a user