jschoubben
03874f3fe2
Nothing in this repository was verified by anything but reading, which is how a superseded decision stayed live in the constitution and in the to-be README at the same time. Both were found by a person looking, and nothing stopped a third. Five checks: links resolve; `decisions:`/`extends:` name records that exist and are accepted; a governing document citing a superseded record must name its replacement in the same paragraph; supersession is symmetric; filename number matches heading number. Each was made to fail before it was made to pass. The live-citation check was verified against a reconstruction of the actual incident -- the to-be README citing ADR 0017 as live guidance -- and reports it with file and line. It found one thing nobody had noticed: ADR 0018 never declared that it superseded 0011, though 0011 has named 0018 as its superseder since August. Fixed. Deliberately not checked, and said so in the README: 02-DECISIONS and 01-RESEARCH may cite superseded records freely, because a decision record discusses history and research records what was observed. 00-as-is may rest on one, per 0056. Flagging those would put noise on correct documents, and a check that cries wolf gets suppressed -- which costs more than not having it. Two bugs found by running it: the frontmatter reader iterated an inline list as characters, and the as-is exemption was missing entirely.
46 lines
2.4 KiB
Markdown
46 lines
2.4 KiB
Markdown
# Checks
|
|
|
|
```
|
|
python3 00-META/checks/records.py
|
|
```
|
|
|
|
Non-zero exit on any problem, so it can be a gate rather than a report.
|
|
|
|
**Why this exists.** Until now nothing in this repository was verified by anything but reading,
|
|
which is how a superseded decision stayed live in the constitution for days and in
|
|
`01-to-be/README.md` alongside it. Both were found by a person looking. `how-we-build` §5 says
|
|
*an unenforced rule is indistinguishable from a wrong one, and costs more, because people
|
|
believe it* — this repository was carrying several.
|
|
|
|
**Every check here failed on something real before it passed.** A check that has never failed is
|
|
indistinguishable from one that cannot.
|
|
|
|
| Check | Asserts | Found |
|
|
|---|---|---|
|
|
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
|
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
|
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0017 as live guidance |
|
|
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0018 never declared that it superseded 0011 |
|
|
| `numbering` | the number in the filename is the number in the heading | — |
|
|
|
|
## What is deliberately not checked
|
|
|
|
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
|
discusses history; research records what was observed. Flagging those would produce noise on
|
|
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
|
having it.
|
|
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
|
runs was built under whatever was decided at the time
|
|
([ADR 0056](../../02-DECISIONS/0056-the-authority-is-the-control-plane-not-a-database.md):
|
|
*as-is describing a superseded decision is exactly what as-is is for*).
|
|
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
|
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
|
|
|
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
|
what to do.
|
|
|
|
## Adding a check
|
|
|
|
State what incident it would have caught, and make it fail before you make it pass. A check
|
|
whose failure has never been observed is a guess about its own correctness.
|