jschoubben
61f61b5d06
An audit of the six code repositories found eleven open issues fixed on main with commits and beds to show (025, 027, 033, 036, 037, 040, 045, 047, 050, 052, 053), three partly (007, 026, 035), nine not (020, 031, 041, 046, 049, 054, 064, 065, 066) and one whose fix would live outside those repos (006). Resolved ones name their evidence; partly ones say what remains; 041 records that the exposure has widened since it was reported.
Checks
python3 00-META/checks/records.py structure: links, citations, supersession, topics
python3 00-META/checks/index.py the reading order in 02-DECISIONS/README.md is current
python3 00-META/checks/index.py --write regenerate it
Non-zero exit on any problem, so it can be a gate rather than a report.
Why this exists. Until now nothing in this repository was verified by anything but reading,
which is how a superseded decision stayed live in the constitution for days and in
01-to-be/README.md alongside it. Both were found by a person looking. how-we-build §5 says
an unenforced rule is indistinguishable from a wrong one, and costs more, because people
believe it — this repository was carrying several.
Every check here failed on something real before it passed. A check that has never failed is indistinguishable from one that cannot.
| Check | Asserts | Found |
|---|---|---|
links |
every relative link resolves | — (run ad hoc during authoring; now permanent) |
rests-on |
decisions: and extends: name records that exist and are accepted |
the class behind both incidents |
live-citation |
a governing document citing a superseded record names its replacement in the same paragraph | 01-to-be/README.md citing ADR 0022 as live guidance |
supersession |
if A says it was superseded by B, B says it supersedes A | ADR 0012 never declared that it superseded 0011 |
numbering |
the number in the filename is the number in the heading | — |
topics |
every record names a topic the index knows | — |
| (index.py) | the written reading order matches what the records say | — |
status-vs-code |
a to-be document naming specific code is not still designed |
ten documents, several with a What was built section, describing lab-proven code |
What is deliberately not checked
02-DECISIONS/and01-RESEARCH/may cite superseded records freely. A decision record discusses history; research records what was observed. Flagging those would produce noise on correct documents, and a check that cries wolf gets suppressed — which costs more than not having it.03-DESIGN/00-as-is/may rest on a superseded record. It describes what runs, and what runs was built under whatever was decided at the time (ADR 0006: as-is describing a superseded decision is exactly what as-is is for).- Whether a citation's prose is still true. Only whether the record it points at is live. A document can cite an accepted record and describe it wrongly, and nothing here notices.
So "governing" means 00-META/ and 03-DESIGN/01-to-be/ — the documents that tell somebody
what to do.
Adding a check
State what incident it would have caught, and make it fail before you make it pass. A check whose failure has never been observed is a guess about its own correctness.
cycle.py
The development cycle, checked (ADR 0080):
a to-be design names a decision, an in-progress/implemented design names its owning code, a
located/fixed issue names its owner, a fixed/resolved issue says what fixed it, a graduated
research overview says what it became. python3 00-META/checks/cycle.py