Add a structural check over HQ's own records
Nothing in this repository was verified by anything but reading, which is how a superseded decision stayed live in the constitution and in the to-be README at the same time. Both were found by a person looking, and nothing stopped a third. Five checks: links resolve; `decisions:`/`extends:` name records that exist and are accepted; a governing document citing a superseded record must name its replacement in the same paragraph; supersession is symmetric; filename number matches heading number. Each was made to fail before it was made to pass. The live-citation check was verified against a reconstruction of the actual incident -- the to-be README citing ADR 0017 as live guidance -- and reports it with file and line. It found one thing nobody had noticed: ADR 0018 never declared that it superseded 0011, though 0011 has named 0018 as its superseder since August. Fixed. Deliberately not checked, and said so in the README: 02-DECISIONS and 01-RESEARCH may cite superseded records freely, because a decision record discusses history and research records what was observed. 00-as-is may rest on one, per 0056. Flagging those would put noise on correct documents, and a check that cries wolf gets suppressed -- which costs more than not having it. Two bugs found by running it: the frontmatter reader iterated an inline list as characters, and the as-is exemption was missing entirely.
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Checks
|
||||
|
||||
```
|
||||
python3 00-META/checks/records.py
|
||||
```
|
||||
|
||||
Non-zero exit on any problem, so it can be a gate rather than a report.
|
||||
|
||||
**Why this exists.** Until now nothing in this repository was verified by anything but reading,
|
||||
which is how a superseded decision stayed live in the constitution for days and in
|
||||
`01-to-be/README.md` alongside it. Both were found by a person looking. `how-we-build` §5 says
|
||||
*an unenforced rule is indistinguishable from a wrong one, and costs more, because people
|
||||
believe it* — this repository was carrying several.
|
||||
|
||||
**Every check here failed on something real before it passed.** A check that has never failed is
|
||||
indistinguishable from one that cannot.
|
||||
|
||||
| Check | Asserts | Found |
|
||||
|---|---|---|
|
||||
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
||||
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
||||
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0017 as live guidance |
|
||||
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0018 never declared that it superseded 0011 |
|
||||
| `numbering` | the number in the filename is the number in the heading | — |
|
||||
|
||||
## What is deliberately not checked
|
||||
|
||||
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
||||
discusses history; research records what was observed. Flagging those would produce noise on
|
||||
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
||||
having it.
|
||||
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
||||
runs was built under whatever was decided at the time
|
||||
([ADR 0056](../../02-DECISIONS/0056-the-authority-is-the-control-plane-not-a-database.md):
|
||||
*as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
||||
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
||||
|
||||
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
||||
what to do.
|
||||
|
||||
## Adding a check
|
||||
|
||||
State what incident it would have caught, and make it fail before you make it pass. A check
|
||||
whose failure has never been observed is a guess about its own correctness.
|
||||
@@ -0,0 +1,275 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Structural checks over HQ's own records.
|
||||
|
||||
Every check here exists because the thing it checks for actually happened. See README.md
|
||||
for which incident is behind which check. Run from the repository root:
|
||||
|
||||
python3 00-META/checks/records.py
|
||||
|
||||
Exits non-zero if anything fails, so it can be a gate rather than a report.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Where a citation is *guidance* rather than history. A document here tells somebody what to
|
||||
# do, so a link to a superseded record is an instruction to follow a withdrawn decision.
|
||||
# 02-DECISIONS and 01-RESEARCH are deliberately absent: they record what was decided and what
|
||||
# was observed, and both legitimately discuss superseded records at length.
|
||||
GOVERNING = ("00-META/", "03-DESIGN/01-to-be/")
|
||||
|
||||
LINK = re.compile(r"\[[^\]]*\]\((?!https?:|mailto:)([^)]+)\)")
|
||||
ADR_FILE = re.compile(r"^(\d{4})-")
|
||||
|
||||
|
||||
def markdown_files():
|
||||
for base, dirs, files in os.walk(ROOT):
|
||||
dirs[:] = [d for d in dirs if d not in (".git", ".claude")]
|
||||
for name in sorted(files):
|
||||
if name.endswith(".md"):
|
||||
yield os.path.join(base, name)
|
||||
|
||||
|
||||
def rel(path):
|
||||
return os.path.relpath(path, ROOT)
|
||||
|
||||
|
||||
def read(path):
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
return handle.read()
|
||||
|
||||
|
||||
def frontmatter(text):
|
||||
"""Minimal frontmatter reader — enough for the fields these checks use.
|
||||
|
||||
Not a YAML parser on purpose: a dependency in a repository that has none, to read four
|
||||
scalar fields and one list, would cost more than it returns.
|
||||
"""
|
||||
if not text.startswith("---\n"):
|
||||
return {}
|
||||
end = text.find("\n---", 4)
|
||||
if end == -1:
|
||||
return {}
|
||||
fields, key = {}, None
|
||||
for line in text[4:end].split("\n"):
|
||||
item = re.match(r"^\s+-\s+(.*)$", line)
|
||||
if item and key:
|
||||
fields.setdefault(key, []).append(item.group(1).strip())
|
||||
continue
|
||||
pair = re.match(r"^([A-Za-z_-]+):\s*(.*)$", line)
|
||||
if pair:
|
||||
key = pair.group(1)
|
||||
value = pair.group(2).strip()
|
||||
if value.startswith("[") and value.endswith("]"):
|
||||
# inline list: `decisions: [a, b]`, `code: []`
|
||||
inner = value[1:-1].strip()
|
||||
fields[key] = [v.strip() for v in inner.split(",") if v.strip()]
|
||||
elif value:
|
||||
fields[key] = value
|
||||
else:
|
||||
fields[key] = []
|
||||
return fields
|
||||
|
||||
|
||||
def load_records():
|
||||
"""Every decision record, by its four-digit number."""
|
||||
records = {}
|
||||
folder = os.path.join(ROOT, "02-DECISIONS")
|
||||
for name in sorted(os.listdir(folder)):
|
||||
match = ADR_FILE.match(name)
|
||||
if not name.endswith(".md") or not match:
|
||||
continue
|
||||
path = os.path.join(folder, name)
|
||||
text = read(path)
|
||||
records[match.group(1)] = {
|
||||
"number": match.group(1),
|
||||
"name": name,
|
||||
"path": path,
|
||||
"text": text,
|
||||
"front": frontmatter(text),
|
||||
}
|
||||
return records
|
||||
|
||||
|
||||
class Failures:
|
||||
def __init__(self):
|
||||
self.items = []
|
||||
|
||||
def add(self, check, location, message):
|
||||
self.items.append((check, location, message))
|
||||
|
||||
def report(self):
|
||||
if not self.items:
|
||||
print("records: all checks passed")
|
||||
return 0
|
||||
by_check = {}
|
||||
for check, location, message in self.items:
|
||||
by_check.setdefault(check, []).append((location, message))
|
||||
for check in sorted(by_check):
|
||||
print(f"\n{check} — {len(by_check[check])} problem(s)")
|
||||
for location, message in by_check[check]:
|
||||
print(f" {location}\n {message}")
|
||||
print(f"\nrecords: {len(self.items)} problem(s)")
|
||||
return 1
|
||||
|
||||
|
||||
def check_links(failures):
|
||||
"""Every relative link resolves to something that exists."""
|
||||
for path in markdown_files():
|
||||
folder = os.path.dirname(path)
|
||||
for number, line in enumerate(read(path).split("\n"), 1):
|
||||
for target in LINK.findall(line):
|
||||
target = target.split("#")[0].strip()
|
||||
if not target:
|
||||
continue
|
||||
if not os.path.exists(os.path.normpath(os.path.join(folder, target))):
|
||||
failures.add("links", f"{rel(path)}:{number}", f"link does not resolve: {target}")
|
||||
|
||||
|
||||
def check_rests_on(failures, records):
|
||||
"""A document's `decisions:` and a record's `extends:` must name a live record.
|
||||
|
||||
These are the load-bearing citations: the document declares that it rests on that
|
||||
decision. Resting on a withdrawn one is the defect this whole check set exists for.
|
||||
"""
|
||||
for path in markdown_files():
|
||||
front = frontmatter(read(path))
|
||||
cited = list(front.get("decisions", []) or [])
|
||||
extends = front.get("extends")
|
||||
if isinstance(extends, str) and extends:
|
||||
cited.append(extends)
|
||||
|
||||
for entry in cited:
|
||||
match = ADR_FILE.match(os.path.basename(entry))
|
||||
if not match:
|
||||
failures.add("rests-on", rel(path), f"not a decision record: {entry}")
|
||||
continue
|
||||
number = match.group(1)
|
||||
if number not in records:
|
||||
failures.add("rests-on", rel(path), f"no such record: {entry}")
|
||||
continue
|
||||
status = records[number]["front"].get("status")
|
||||
if status != "accepted":
|
||||
# An as-is document describes what runs, and what runs was built under
|
||||
# whatever was decided at the time. ADR 0056: "as-is describing a superseded
|
||||
# decision is exactly what as-is is for."
|
||||
if rel(path).startswith("03-DESIGN/00-as-is/"):
|
||||
continue
|
||||
# An extension that supersedes legitimately names what it replaced.
|
||||
this = ADR_FILE.match(os.path.basename(path))
|
||||
supersedes = records[number]["front"].get("superseded-by", "")
|
||||
if this and supersedes and os.path.basename(path) in str(supersedes):
|
||||
continue
|
||||
failures.add(
|
||||
"rests-on",
|
||||
rel(path),
|
||||
f"rests on ADR {number}, which is '{status}' — a document may not rest on a "
|
||||
f"record that is not accepted",
|
||||
)
|
||||
|
||||
|
||||
def check_live_citations(failures, records):
|
||||
"""In a governing document, a link to a superseded record must name its replacement.
|
||||
|
||||
The reader of a rule needs to know the rule was withdrawn, and needs somewhere to go.
|
||||
Naming the superseder in the same paragraph is both, and it is what a person would
|
||||
write anyway.
|
||||
"""
|
||||
superseded = {
|
||||
number: record["front"].get("superseded-by", "")
|
||||
for number, record in records.items()
|
||||
if record["front"].get("status") == "superseded"
|
||||
}
|
||||
|
||||
for path in markdown_files():
|
||||
if not any(rel(path).startswith(prefix) for prefix in GOVERNING):
|
||||
continue
|
||||
text = read(path)
|
||||
offset = 0
|
||||
for paragraph in text.split("\n\n"):
|
||||
line_no = text[:offset].count("\n") + 1
|
||||
offset += len(paragraph) + 2
|
||||
targets = LINK.findall(paragraph)
|
||||
for target in targets:
|
||||
match = ADR_FILE.match(os.path.basename(target.split("#")[0]))
|
||||
if not match or match.group(1) not in superseded:
|
||||
continue
|
||||
number = match.group(1)
|
||||
replacement = os.path.basename(str(superseded[number]))
|
||||
if not replacement:
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number}, which names no superseder",
|
||||
)
|
||||
continue
|
||||
if not any(replacement in t for t in targets):
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number} without naming its replacement "
|
||||
f"({replacement}) in the same paragraph",
|
||||
)
|
||||
|
||||
|
||||
def check_supersession_symmetry(failures, records):
|
||||
"""If A says it was superseded by B, B must say it supersedes A."""
|
||||
for number, record in records.items():
|
||||
front = record["front"]
|
||||
status = front.get("status")
|
||||
by = os.path.basename(str(front.get("superseded-by", "")))
|
||||
|
||||
if status == "superseded" and not by:
|
||||
failures.add("supersession", rel(record["path"]), "marked superseded but names no superseder")
|
||||
continue
|
||||
if by and status != "superseded":
|
||||
failures.add("supersession", rel(record["path"]), f"names a superseder but status is '{status}'")
|
||||
if not by:
|
||||
continue
|
||||
|
||||
match = ADR_FILE.match(by)
|
||||
if not match or match.group(1) not in records:
|
||||
failures.add("supersession", rel(record["path"]), f"superseder does not exist: {by}")
|
||||
continue
|
||||
other = records[match.group(1)]
|
||||
claims = os.path.basename(str(other["front"].get("supersedes", "")))
|
||||
if claims != record["name"]:
|
||||
failures.add(
|
||||
"supersession",
|
||||
rel(other["path"]),
|
||||
f"ADR {number} says this supersedes it; this record does not say so "
|
||||
f"(supersedes: {claims or 'absent'})",
|
||||
)
|
||||
|
||||
|
||||
def check_numbering(failures, records):
|
||||
"""The number in the filename is the number in the heading."""
|
||||
for number, record in records.items():
|
||||
heading = re.search(r"^# (\d+)\.", record["text"], re.M)
|
||||
if not heading:
|
||||
failures.add("numbering", rel(record["path"]), "no '# N. Title' heading")
|
||||
elif heading.group(1) != number.lstrip("0"):
|
||||
failures.add(
|
||||
"numbering",
|
||||
rel(record["path"]),
|
||||
f"filename says {number}, heading says {heading.group(1)}",
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
failures = Failures()
|
||||
records = load_records()
|
||||
check_links(failures)
|
||||
check_rests_on(failures, records)
|
||||
check_live_citations(failures, records)
|
||||
check_supersession_symmetry(failures, records)
|
||||
check_numbering(failures, records)
|
||||
print(f"records: {len(records)} decision records checked")
|
||||
return failures.report()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -3,6 +3,7 @@ status: accepted
|
||||
date: 2026-08-23
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 0011-the-installer-owns-linking.md
|
||||
extends: 0011-the-installer-owns-linking.md
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user