Add a structural check over HQ's own records
Nothing in this repository was verified by anything but reading, which is how a superseded decision stayed live in the constitution and in the to-be README at the same time. Both were found by a person looking, and nothing stopped a third. Five checks: links resolve; `decisions:`/`extends:` name records that exist and are accepted; a governing document citing a superseded record must name its replacement in the same paragraph; supersession is symmetric; filename number matches heading number. Each was made to fail before it was made to pass. The live-citation check was verified against a reconstruction of the actual incident -- the to-be README citing ADR 0017 as live guidance -- and reports it with file and line. It found one thing nobody had noticed: ADR 0018 never declared that it superseded 0011, though 0011 has named 0018 as its superseder since August. Fixed. Deliberately not checked, and said so in the README: 02-DECISIONS and 01-RESEARCH may cite superseded records freely, because a decision record discusses history and research records what was observed. 00-as-is may rest on one, per 0056. Flagging those would put noise on correct documents, and a check that cries wolf gets suppressed -- which costs more than not having it. Two bugs found by running it: the frontmatter reader iterated an inline list as characters, and the as-is exemption was missing entirely.
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Checks
|
||||
|
||||
```
|
||||
python3 00-META/checks/records.py
|
||||
```
|
||||
|
||||
Non-zero exit on any problem, so it can be a gate rather than a report.
|
||||
|
||||
**Why this exists.** Until now nothing in this repository was verified by anything but reading,
|
||||
which is how a superseded decision stayed live in the constitution for days and in
|
||||
`01-to-be/README.md` alongside it. Both were found by a person looking. `how-we-build` §5 says
|
||||
*an unenforced rule is indistinguishable from a wrong one, and costs more, because people
|
||||
believe it* — this repository was carrying several.
|
||||
|
||||
**Every check here failed on something real before it passed.** A check that has never failed is
|
||||
indistinguishable from one that cannot.
|
||||
|
||||
| Check | Asserts | Found |
|
||||
|---|---|---|
|
||||
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
||||
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
||||
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0017 as live guidance |
|
||||
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0018 never declared that it superseded 0011 |
|
||||
| `numbering` | the number in the filename is the number in the heading | — |
|
||||
|
||||
## What is deliberately not checked
|
||||
|
||||
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
||||
discusses history; research records what was observed. Flagging those would produce noise on
|
||||
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
||||
having it.
|
||||
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
||||
runs was built under whatever was decided at the time
|
||||
([ADR 0056](../../02-DECISIONS/0056-the-authority-is-the-control-plane-not-a-database.md):
|
||||
*as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
||||
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
||||
|
||||
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
||||
what to do.
|
||||
|
||||
## Adding a check
|
||||
|
||||
State what incident it would have caught, and make it fail before you make it pass. A check
|
||||
whose failure has never been observed is a guess about its own correctness.
|
||||
Reference in New Issue
Block a user