Add a structural check over HQ's own records
Nothing in this repository was verified by anything but reading, which is how a superseded decision stayed live in the constitution and in the to-be README at the same time. Both were found by a person looking, and nothing stopped a third. Five checks: links resolve; `decisions:`/`extends:` name records that exist and are accepted; a governing document citing a superseded record must name its replacement in the same paragraph; supersession is symmetric; filename number matches heading number. Each was made to fail before it was made to pass. The live-citation check was verified against a reconstruction of the actual incident -- the to-be README citing ADR 0017 as live guidance -- and reports it with file and line. It found one thing nobody had noticed: ADR 0018 never declared that it superseded 0011, though 0011 has named 0018 as its superseder since August. Fixed. Deliberately not checked, and said so in the README: 02-DECISIONS and 01-RESEARCH may cite superseded records freely, because a decision record discusses history and research records what was observed. 00-as-is may rest on one, per 0056. Flagging those would put noise on correct documents, and a check that cries wolf gets suppressed -- which costs more than not having it. Two bugs found by running it: the frontmatter reader iterated an inline list as characters, and the as-is exemption was missing entirely.
This commit is contained in:
@@ -0,0 +1,45 @@
|
|||||||
|
# Checks
|
||||||
|
|
||||||
|
```
|
||||||
|
python3 00-META/checks/records.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Non-zero exit on any problem, so it can be a gate rather than a report.
|
||||||
|
|
||||||
|
**Why this exists.** Until now nothing in this repository was verified by anything but reading,
|
||||||
|
which is how a superseded decision stayed live in the constitution for days and in
|
||||||
|
`01-to-be/README.md` alongside it. Both were found by a person looking. `how-we-build` §5 says
|
||||||
|
*an unenforced rule is indistinguishable from a wrong one, and costs more, because people
|
||||||
|
believe it* — this repository was carrying several.
|
||||||
|
|
||||||
|
**Every check here failed on something real before it passed.** A check that has never failed is
|
||||||
|
indistinguishable from one that cannot.
|
||||||
|
|
||||||
|
| Check | Asserts | Found |
|
||||||
|
|---|---|---|
|
||||||
|
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
||||||
|
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
||||||
|
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0017 as live guidance |
|
||||||
|
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0018 never declared that it superseded 0011 |
|
||||||
|
| `numbering` | the number in the filename is the number in the heading | — |
|
||||||
|
|
||||||
|
## What is deliberately not checked
|
||||||
|
|
||||||
|
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
||||||
|
discusses history; research records what was observed. Flagging those would produce noise on
|
||||||
|
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
||||||
|
having it.
|
||||||
|
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
||||||
|
runs was built under whatever was decided at the time
|
||||||
|
([ADR 0056](../../02-DECISIONS/0056-the-authority-is-the-control-plane-not-a-database.md):
|
||||||
|
*as-is describing a superseded decision is exactly what as-is is for*).
|
||||||
|
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
||||||
|
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
||||||
|
|
||||||
|
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
||||||
|
what to do.
|
||||||
|
|
||||||
|
## Adding a check
|
||||||
|
|
||||||
|
State what incident it would have caught, and make it fail before you make it pass. A check
|
||||||
|
whose failure has never been observed is a guess about its own correctness.
|
||||||
@@ -0,0 +1,275 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Structural checks over HQ's own records.
|
||||||
|
|
||||||
|
Every check here exists because the thing it checks for actually happened. See README.md
|
||||||
|
for which incident is behind which check. Run from the repository root:
|
||||||
|
|
||||||
|
python3 00-META/checks/records.py
|
||||||
|
|
||||||
|
Exits non-zero if anything fails, so it can be a gate rather than a report.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
# Where a citation is *guidance* rather than history. A document here tells somebody what to
|
||||||
|
# do, so a link to a superseded record is an instruction to follow a withdrawn decision.
|
||||||
|
# 02-DECISIONS and 01-RESEARCH are deliberately absent: they record what was decided and what
|
||||||
|
# was observed, and both legitimately discuss superseded records at length.
|
||||||
|
GOVERNING = ("00-META/", "03-DESIGN/01-to-be/")
|
||||||
|
|
||||||
|
LINK = re.compile(r"\[[^\]]*\]\((?!https?:|mailto:)([^)]+)\)")
|
||||||
|
ADR_FILE = re.compile(r"^(\d{4})-")
|
||||||
|
|
||||||
|
|
||||||
|
def markdown_files():
|
||||||
|
for base, dirs, files in os.walk(ROOT):
|
||||||
|
dirs[:] = [d for d in dirs if d not in (".git", ".claude")]
|
||||||
|
for name in sorted(files):
|
||||||
|
if name.endswith(".md"):
|
||||||
|
yield os.path.join(base, name)
|
||||||
|
|
||||||
|
|
||||||
|
def rel(path):
|
||||||
|
return os.path.relpath(path, ROOT)
|
||||||
|
|
||||||
|
|
||||||
|
def read(path):
|
||||||
|
with open(path, encoding="utf-8") as handle:
|
||||||
|
return handle.read()
|
||||||
|
|
||||||
|
|
||||||
|
def frontmatter(text):
|
||||||
|
"""Minimal frontmatter reader — enough for the fields these checks use.
|
||||||
|
|
||||||
|
Not a YAML parser on purpose: a dependency in a repository that has none, to read four
|
||||||
|
scalar fields and one list, would cost more than it returns.
|
||||||
|
"""
|
||||||
|
if not text.startswith("---\n"):
|
||||||
|
return {}
|
||||||
|
end = text.find("\n---", 4)
|
||||||
|
if end == -1:
|
||||||
|
return {}
|
||||||
|
fields, key = {}, None
|
||||||
|
for line in text[4:end].split("\n"):
|
||||||
|
item = re.match(r"^\s+-\s+(.*)$", line)
|
||||||
|
if item and key:
|
||||||
|
fields.setdefault(key, []).append(item.group(1).strip())
|
||||||
|
continue
|
||||||
|
pair = re.match(r"^([A-Za-z_-]+):\s*(.*)$", line)
|
||||||
|
if pair:
|
||||||
|
key = pair.group(1)
|
||||||
|
value = pair.group(2).strip()
|
||||||
|
if value.startswith("[") and value.endswith("]"):
|
||||||
|
# inline list: `decisions: [a, b]`, `code: []`
|
||||||
|
inner = value[1:-1].strip()
|
||||||
|
fields[key] = [v.strip() for v in inner.split(",") if v.strip()]
|
||||||
|
elif value:
|
||||||
|
fields[key] = value
|
||||||
|
else:
|
||||||
|
fields[key] = []
|
||||||
|
return fields
|
||||||
|
|
||||||
|
|
||||||
|
def load_records():
|
||||||
|
"""Every decision record, by its four-digit number."""
|
||||||
|
records = {}
|
||||||
|
folder = os.path.join(ROOT, "02-DECISIONS")
|
||||||
|
for name in sorted(os.listdir(folder)):
|
||||||
|
match = ADR_FILE.match(name)
|
||||||
|
if not name.endswith(".md") or not match:
|
||||||
|
continue
|
||||||
|
path = os.path.join(folder, name)
|
||||||
|
text = read(path)
|
||||||
|
records[match.group(1)] = {
|
||||||
|
"number": match.group(1),
|
||||||
|
"name": name,
|
||||||
|
"path": path,
|
||||||
|
"text": text,
|
||||||
|
"front": frontmatter(text),
|
||||||
|
}
|
||||||
|
return records
|
||||||
|
|
||||||
|
|
||||||
|
class Failures:
|
||||||
|
def __init__(self):
|
||||||
|
self.items = []
|
||||||
|
|
||||||
|
def add(self, check, location, message):
|
||||||
|
self.items.append((check, location, message))
|
||||||
|
|
||||||
|
def report(self):
|
||||||
|
if not self.items:
|
||||||
|
print("records: all checks passed")
|
||||||
|
return 0
|
||||||
|
by_check = {}
|
||||||
|
for check, location, message in self.items:
|
||||||
|
by_check.setdefault(check, []).append((location, message))
|
||||||
|
for check in sorted(by_check):
|
||||||
|
print(f"\n{check} — {len(by_check[check])} problem(s)")
|
||||||
|
for location, message in by_check[check]:
|
||||||
|
print(f" {location}\n {message}")
|
||||||
|
print(f"\nrecords: {len(self.items)} problem(s)")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
def check_links(failures):
|
||||||
|
"""Every relative link resolves to something that exists."""
|
||||||
|
for path in markdown_files():
|
||||||
|
folder = os.path.dirname(path)
|
||||||
|
for number, line in enumerate(read(path).split("\n"), 1):
|
||||||
|
for target in LINK.findall(line):
|
||||||
|
target = target.split("#")[0].strip()
|
||||||
|
if not target:
|
||||||
|
continue
|
||||||
|
if not os.path.exists(os.path.normpath(os.path.join(folder, target))):
|
||||||
|
failures.add("links", f"{rel(path)}:{number}", f"link does not resolve: {target}")
|
||||||
|
|
||||||
|
|
||||||
|
def check_rests_on(failures, records):
|
||||||
|
"""A document's `decisions:` and a record's `extends:` must name a live record.
|
||||||
|
|
||||||
|
These are the load-bearing citations: the document declares that it rests on that
|
||||||
|
decision. Resting on a withdrawn one is the defect this whole check set exists for.
|
||||||
|
"""
|
||||||
|
for path in markdown_files():
|
||||||
|
front = frontmatter(read(path))
|
||||||
|
cited = list(front.get("decisions", []) or [])
|
||||||
|
extends = front.get("extends")
|
||||||
|
if isinstance(extends, str) and extends:
|
||||||
|
cited.append(extends)
|
||||||
|
|
||||||
|
for entry in cited:
|
||||||
|
match = ADR_FILE.match(os.path.basename(entry))
|
||||||
|
if not match:
|
||||||
|
failures.add("rests-on", rel(path), f"not a decision record: {entry}")
|
||||||
|
continue
|
||||||
|
number = match.group(1)
|
||||||
|
if number not in records:
|
||||||
|
failures.add("rests-on", rel(path), f"no such record: {entry}")
|
||||||
|
continue
|
||||||
|
status = records[number]["front"].get("status")
|
||||||
|
if status != "accepted":
|
||||||
|
# An as-is document describes what runs, and what runs was built under
|
||||||
|
# whatever was decided at the time. ADR 0056: "as-is describing a superseded
|
||||||
|
# decision is exactly what as-is is for."
|
||||||
|
if rel(path).startswith("03-DESIGN/00-as-is/"):
|
||||||
|
continue
|
||||||
|
# An extension that supersedes legitimately names what it replaced.
|
||||||
|
this = ADR_FILE.match(os.path.basename(path))
|
||||||
|
supersedes = records[number]["front"].get("superseded-by", "")
|
||||||
|
if this and supersedes and os.path.basename(path) in str(supersedes):
|
||||||
|
continue
|
||||||
|
failures.add(
|
||||||
|
"rests-on",
|
||||||
|
rel(path),
|
||||||
|
f"rests on ADR {number}, which is '{status}' — a document may not rest on a "
|
||||||
|
f"record that is not accepted",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def check_live_citations(failures, records):
|
||||||
|
"""In a governing document, a link to a superseded record must name its replacement.
|
||||||
|
|
||||||
|
The reader of a rule needs to know the rule was withdrawn, and needs somewhere to go.
|
||||||
|
Naming the superseder in the same paragraph is both, and it is what a person would
|
||||||
|
write anyway.
|
||||||
|
"""
|
||||||
|
superseded = {
|
||||||
|
number: record["front"].get("superseded-by", "")
|
||||||
|
for number, record in records.items()
|
||||||
|
if record["front"].get("status") == "superseded"
|
||||||
|
}
|
||||||
|
|
||||||
|
for path in markdown_files():
|
||||||
|
if not any(rel(path).startswith(prefix) for prefix in GOVERNING):
|
||||||
|
continue
|
||||||
|
text = read(path)
|
||||||
|
offset = 0
|
||||||
|
for paragraph in text.split("\n\n"):
|
||||||
|
line_no = text[:offset].count("\n") + 1
|
||||||
|
offset += len(paragraph) + 2
|
||||||
|
targets = LINK.findall(paragraph)
|
||||||
|
for target in targets:
|
||||||
|
match = ADR_FILE.match(os.path.basename(target.split("#")[0]))
|
||||||
|
if not match or match.group(1) not in superseded:
|
||||||
|
continue
|
||||||
|
number = match.group(1)
|
||||||
|
replacement = os.path.basename(str(superseded[number]))
|
||||||
|
if not replacement:
|
||||||
|
failures.add(
|
||||||
|
"live-citation",
|
||||||
|
f"{rel(path)}:{line_no}",
|
||||||
|
f"cites superseded ADR {number}, which names no superseder",
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if not any(replacement in t for t in targets):
|
||||||
|
failures.add(
|
||||||
|
"live-citation",
|
||||||
|
f"{rel(path)}:{line_no}",
|
||||||
|
f"cites superseded ADR {number} without naming its replacement "
|
||||||
|
f"({replacement}) in the same paragraph",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def check_supersession_symmetry(failures, records):
|
||||||
|
"""If A says it was superseded by B, B must say it supersedes A."""
|
||||||
|
for number, record in records.items():
|
||||||
|
front = record["front"]
|
||||||
|
status = front.get("status")
|
||||||
|
by = os.path.basename(str(front.get("superseded-by", "")))
|
||||||
|
|
||||||
|
if status == "superseded" and not by:
|
||||||
|
failures.add("supersession", rel(record["path"]), "marked superseded but names no superseder")
|
||||||
|
continue
|
||||||
|
if by and status != "superseded":
|
||||||
|
failures.add("supersession", rel(record["path"]), f"names a superseder but status is '{status}'")
|
||||||
|
if not by:
|
||||||
|
continue
|
||||||
|
|
||||||
|
match = ADR_FILE.match(by)
|
||||||
|
if not match or match.group(1) not in records:
|
||||||
|
failures.add("supersession", rel(record["path"]), f"superseder does not exist: {by}")
|
||||||
|
continue
|
||||||
|
other = records[match.group(1)]
|
||||||
|
claims = os.path.basename(str(other["front"].get("supersedes", "")))
|
||||||
|
if claims != record["name"]:
|
||||||
|
failures.add(
|
||||||
|
"supersession",
|
||||||
|
rel(other["path"]),
|
||||||
|
f"ADR {number} says this supersedes it; this record does not say so "
|
||||||
|
f"(supersedes: {claims or 'absent'})",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def check_numbering(failures, records):
|
||||||
|
"""The number in the filename is the number in the heading."""
|
||||||
|
for number, record in records.items():
|
||||||
|
heading = re.search(r"^# (\d+)\.", record["text"], re.M)
|
||||||
|
if not heading:
|
||||||
|
failures.add("numbering", rel(record["path"]), "no '# N. Title' heading")
|
||||||
|
elif heading.group(1) != number.lstrip("0"):
|
||||||
|
failures.add(
|
||||||
|
"numbering",
|
||||||
|
rel(record["path"]),
|
||||||
|
f"filename says {number}, heading says {heading.group(1)}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
failures = Failures()
|
||||||
|
records = load_records()
|
||||||
|
check_links(failures)
|
||||||
|
check_rests_on(failures, records)
|
||||||
|
check_live_citations(failures, records)
|
||||||
|
check_supersession_symmetry(failures, records)
|
||||||
|
check_numbering(failures, records)
|
||||||
|
print(f"records: {len(records)} decision records checked")
|
||||||
|
return failures.report()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -3,6 +3,7 @@ status: accepted
|
|||||||
date: 2026-08-23
|
date: 2026-08-23
|
||||||
deciders: jochen
|
deciders: jochen
|
||||||
reconstructed: false
|
reconstructed: false
|
||||||
|
supersedes: 0011-the-installer-owns-linking.md
|
||||||
extends: 0011-the-installer-owns-linking.md
|
extends: 0011-the-installer-owns-linking.md
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user