jschoubben
03874f3fe2
Nothing in this repository was verified by anything but reading, which is how a superseded decision stayed live in the constitution and in the to-be README at the same time. Both were found by a person looking, and nothing stopped a third. Five checks: links resolve; `decisions:`/`extends:` name records that exist and are accepted; a governing document citing a superseded record must name its replacement in the same paragraph; supersession is symmetric; filename number matches heading number. Each was made to fail before it was made to pass. The live-citation check was verified against a reconstruction of the actual incident -- the to-be README citing ADR 0017 as live guidance -- and reports it with file and line. It found one thing nobody had noticed: ADR 0018 never declared that it superseded 0011, though 0011 has named 0018 as its superseder since August. Fixed. Deliberately not checked, and said so in the README: 02-DECISIONS and 01-RESEARCH may cite superseded records freely, because a decision record discusses history and research records what was observed. 00-as-is may rest on one, per 0056. Flagging those would put noise on correct documents, and a check that cries wolf gets suppressed -- which costs more than not having it. Two bugs found by running it: the frontmatter reader iterated an inline list as characters, and the as-is exemption was missing entirely.
Checks
python3 00-META/checks/records.py
Non-zero exit on any problem, so it can be a gate rather than a report.
Why this exists. Until now nothing in this repository was verified by anything but reading,
which is how a superseded decision stayed live in the constitution for days and in
01-to-be/README.md alongside it. Both were found by a person looking. how-we-build §5 says
an unenforced rule is indistinguishable from a wrong one, and costs more, because people
believe it — this repository was carrying several.
Every check here failed on something real before it passed. A check that has never failed is indistinguishable from one that cannot.
| Check | Asserts | Found |
|---|---|---|
links |
every relative link resolves | — (run ad hoc during authoring; now permanent) |
rests-on |
decisions: and extends: name records that exist and are accepted |
the class behind both incidents |
live-citation |
a governing document citing a superseded record names its replacement in the same paragraph | 01-to-be/README.md citing ADR 0017 as live guidance |
supersession |
if A says it was superseded by B, B says it supersedes A | ADR 0018 never declared that it superseded 0011 |
numbering |
the number in the filename is the number in the heading | — |
What is deliberately not checked
02-DECISIONS/and01-RESEARCH/may cite superseded records freely. A decision record discusses history; research records what was observed. Flagging those would produce noise on correct documents, and a check that cries wolf gets suppressed — which costs more than not having it.03-DESIGN/00-as-is/may rest on a superseded record. It describes what runs, and what runs was built under whatever was decided at the time (ADR 0056: as-is describing a superseded decision is exactly what as-is is for).- Whether a citation's prose is still true. Only whether the record it points at is live. A document can cite an accepted record and describe it wrongly, and nothing here notices.
So "governing" means 00-META/ and 03-DESIGN/01-to-be/ — the documents that tell somebody
what to do.
Adding a check
State what incident it would have caught, and make it fail before you make it pass. A check whose failure has never been observed is a guess about its own correctness.