005 resolved: a suite that cannot run on every push says when it last ran

Retired in favour of the lab rather than repaired — that answers the
first open question. The second finding is the one that generalises:
"nothing runs it, and nothing reports that nothing runs it" is not a
fact about that harness, it is a fact about any suite too expensive to
run on every push. The replacement inherited the fault it was replacing.

Records the three rules that now hold, and what the fix taught twice:
the remedy rebuilt the symptom inside itself, and the code that counts
results passed every test while reading nothing.
This commit is contained in:
2026-08-31 15:02:26 +02:00
parent f3ffdae909
commit 345bbe0552
2 changed files with 86 additions and 6 deletions
+33 -2
View File
@@ -2,9 +2,8 @@
layer: to-be
status: in-progress
code: [mesh-lab]
updated: 2026-08-28
updated: 2026-08-31
decisions:
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0019-how-this-repository-works.md
---
@@ -390,6 +389,38 @@ Everything a node itself does is real, because a node is a real machine.
---
## A suite too expensive to run on every push says when it last ran
*Written 2026-08-31, from resolving [04-ISSUES/005](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md).*
This suite needs a machine with a hypervisor. It therefore cannot run on every push, and a suite
that does not run on every push runs **when somebody remembers**. Remembering is not a mechanism,
and the harness this one replaces proves it: it had not built for two and a half months, nothing
said so, and the coverage was assumed rather than checked.
**The danger is not that the suite breaks. It is that nobody notices it stopped running** — and
that danger belongs to *this* design, not to the harness it retired.
So three rules, each held by a test:
**A run leaves a receipt** — when, what passed, what it ran, and the commit each repository was
at. Kept **outside version control**: the question is *has this machine run it*, and a receipt in
git would be a claim about everybody's machine made by whoever committed last.
**A receipt says why it does not count.** Old, failed, taken against commits the repositories have
moved past, or a run that never raised a machine. Something can be asked, and answers non-zero.
**A receipt that says nothing about something is not a receipt that clears it** — including a
receipt written before it recorded a given fact, which claims nothing rather than everything.
**The run rebuilds what it tests.** The suite consumes artifacts from other repositories, and an
artifact rebuilt from memory is one rebuilt sometimes. A stale binary reporting success against
rules that have since changed is the same fault wearing different clothes.
**The general rule, which outlives this suite:** *silence and success must never look alike.*
It is the same rule the host follows about a service that does not exist
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable
from a failure to answer — applied to coverage instead of to a machine.
## Consequences
**Bringing a node into being is part of the framework.** A test creates its own nodes — one