Merge pull request 'Issue 297: two open decisions always conflict' (#177) from issues/297-two-open-decisions-always-conflict into main

04-ISSUES/297-two-open-decisions-always-conflict/00-report.md

@@ -0,0 +1,71 @@
+---
+status: located
+opened: 2026-10-07
+located-in: [hq (02-DECISIONS/README.md, 00-META/checks/index.py)]
+fixed-by:
+amended-design:
+---
+
+# 297. Two open decisions always conflict
+
+## Symptom
+
+`02-DECISIONS/README.md` holds a generated index between `<!-- index:start -->` and
+`<!-- index:end -->`. `00-META/checks/index.py --write` writes it, and `index.py` fails any pull request
+whose index is not current. So every pull request that adds a decision record also adds a line to that
+file. The index groups records by `topic:` and sorts each group by number, so a new record's line always
+lands at or near the end of its group. Two open decisions on the same topic insert into the same place.
+
+On 2026-10-07 four pull requests carrying decisions were open together: hq #168 (ADR 0241), #171 (issue
+295 and ADR 0242), #172 (ADR 0243) and #173 (ADR 0245). Each merge to `main` made the next open
+decision pull request unmergeable. Each one then had to be rebased, have its index written again, be
+pushed, and be checked again from the start by the build seat. The heads the build seat checked, from the
+controller's journal (times UTC):
+
+| PR | heads checked | merged |
+|---|---|---|
+| #168 | `297945e7` 17:06, `af225efc` 17:21, `c224cea3` 18:18, `b9caffaa` 18:26, `c462837f` 18:36 | 19:04 |
+| #171 | `1f161c3c` 17:41, `18774cbe` 18:15, `998980bb` 18:26 | 18:33 |
+| #173 | `71339dc4` 17:47, `666095c4` 17:54, `24aeeadf` 18:30, `10ab240e` 18:57, `6cacf772` 19:06 | open |
+
+That makes three rebases of #168, two of #171 and four of #173 in about two hours. Every one was
+followed by a full check.
+
+## Evidence that the index is what conflicted
+
+Each old head was replayed onto the `main` that came after it, with `git merge-tree`:
+
+- **#171** at `1f161c3c`, onto `main` after #172 merged (ADR 0243): conflict in
+  `02-DECISIONS/README.md`. The two sides are the 0243 line and the 0242 line, one after the other in
+  the same topic.
+- **#168** at `297945e7` and at `af225efc`, onto the same `main`: conflict in `02-DECISIONS/README.md`.
+- **#168** at `b9caffaa`, onto `main` after #171 merged (ADR 0242): conflict in
+  `02-DECISIONS/README.md`. 0241 sorts before 0242, so its line had to go above the line #171 had just
+  added.
+- **#173** at `666095c4`, onto `main` after #174 merged (ADR 0244): conflict in a design document,
+  and the index merged cleanly. Its first head, `71339dc4`, is not available to replay.
+
+Not every rebase that day was forced by the index. #168 and #171 were rebased once more after #174 merged
+because the words check that #174 introduced failed on their prose. #173 changed its number twice
+(0242, then 0244, then 0245) because the numbers it took were used first by other branches. Its last two
+heads merged cleanly. Even so, three of the forced rebases came from the index alone, and every
+decision pull request open at the same time as another is exposed to it.
+
+## Why it matters
+
+- Every rebase needs a full check from the build seat. With four decisions open, merging them one after
+  another costs about as many checks as there are pairs of them, not as many as there are decisions.
+- The rebased line is a copy of a fact the record already states in its own `topic:` and `status:`.
+  The conflict is between two copies of frontmatter, never between two decisions.
+- The index also marks status (`*(superseded)*`, `*(proposed)*`). ADR 0019 already says a decision
+  index is a status view generated from frontmatter "never written to disk", and in the same record it
+  says this index is written. The two statements disagree, and the merge conflicts come from the
+  second one.
+
+## How it is checked
+
+Today `index.py` checks that the written index is current, and that is what turns every concurrent
+decision into a conflict. A fix is checked by a replay of the 2026-10-07 case: two branches off one
+`main`, each adding a decision on the same topic, both merge after the first one lands with no rebase,
+and `sh merge-check.sh` passes on both and on the merge. [01-diagnosis.md](01-diagnosis.md) weighs the
+fixes.

04-ISSUES/297-two-open-decisions-always-conflict/01-diagnosis.md

@@ -0,0 +1,72 @@
+# 297 — diagnosis
+
+## 2026-10-07: where the conflict comes from
+
+`index.py` builds the index from every record's `topic:`, `status:` and heading. It groups the records
+under six topics and sorts each group by number. `--write` replaces the text between the markers in
+`02-DECISIONS/README.md`, and without `--write` the script fails when that text differs from what it
+would build. `merge-check.sh` runs it, so the build seat's `mesh/repo-check` fails any pull request whose
+index is stale. The rule that requires this is ADR 0019's: the index is written because "a reader looking
+at the folder on a forge sees the folder, not a command", and it is checked so that it cannot drift.
+
+New records get the highest numbers, so each new line lands at the end of its topic's group. Two branches
+that each add a record to the same topic edit the same lines of the same file, and a textual merge
+cannot settle that. A record numbered below one that merged first, as ADR 0241 was below 0242, has to go
+above the line just added, and that conflicts too. A decision that supersedes another also edits that
+older record's line to add the status mark.
+
+Ruled out:
+
+- **The forge being strict.** The protection on `main` requires only `mesh/repo-check`. It does not
+  require a branch to be current. The conflicts are real textual conflicts, replayed with
+  `git merge-tree` (see the report).
+- **The record numbers.** Number collisions forced rebases of their own (#173 was renumbered twice).
+  Issue 155 and playbook 03 already deal with those, and they are a separate problem. Without any
+  collision, the index still conflicts.
+
+## The fixes
+
+**A. Generate the index when it is read, and stop storing it.** Remove the markers and the list between
+them. `02-DECISIONS/README.md` keeps the six topics in their reading order with a sentence for each, and
+says where the generated list is. The list is generated from frontmatter by the `hq-status` skill, which
+already renders "the ADR index" and says that view is "always generated, always ephemeral". It is also
+generated by the records module, which reads this repository for the mesh and can serve the list as a
+read. `index.py` keeps the part of its check that still means something: every record has a topic, and
+the topic is one of the six. `records.py` already checks the `topics` part. With no line to add, no
+decision pull request touches a shared file.
+*Cost:* a reader on the forge no longer sees the whole list in one page. They see the topics and the
+folder, and each record names its topic. That is the reason ADR 0019 gave for writing the index, so this
+fix needs a decision that supersedes that paragraph.
+
+**B. One index file per decision.** For example `02-DECISIONS/index/NNNN.md`, each holding one line.
+Nothing conflicts, but nothing is gained either. The record's own frontmatter is already one file per
+decision with that fact in it. A folder of one-line files is no more readable on a forge than the folder
+of records, and it is a third copy of `topic:` that has to be checked against the record.
+
+**C. Let `index.py --write` run after each merge on `main`.** Pull requests stop touching the index,
+and a job rewrites it once a merge lands. `main` is protected and every merge needs a pull request and a
+person's approval, so the job opens an index pull request after every decision merges. That doubles the
+merges a person approves, and leaves the index on `main` stale until they do. `index.py` on every other
+pull request would then fail on the stale index, or would have to stop checking it, which gives up the
+reason for writing it.
+
+**D. A union merge for the README** (`merge=union` in `.gitattributes`). The forge's own merge may not
+read the attribute. Where it does, a union keeps both sides in the order the merge sees them, not in
+number order. The ADR 0241 and 0242 hunk of 2026-10-07 would have landed 0242 above 0241, and `index.py`
+would fail on `main` after the merge, which is worse than failing before it.
+
+## Recommendation: A
+
+A removes the conflict where it starts: a decision pull request no longer edits a file that every other
+decision pull request edits. It also ends ADR 0019's disagreement with itself, since a decision index
+with status marks is the kind of view that record says is never written to disk. The cost, a forge
+reader without the full list on one page, is real but small. The topics and their order stay written,
+and the list is one skill or one tool call away. B adds a copy without removing the shared file's
+purpose. C makes each decision cost two approvals. D trades a conflict before the merge for a broken
+`main` after it.
+
+A needs a decision record superseding the "the index is written" paragraph of ADR 0019, per playbook 02.
+After that, the work is a change to `index.py` and `02-DECISIONS/README.md` in this repository, and the
+`hq-status` skill's index view as the place the list is read. If the records module should serve the
+list, that is a separate change in the catalogue. Until the decision is taken, the cost of rebasing stays
+with whoever lands decisions in parallel.