005 resolved: a suite that cannot run on every push says when it last ran
Retired in favour of the lab rather than repaired — that answers the first open question. The second finding is the one that generalises: "nothing runs it, and nothing reports that nothing runs it" is not a fact about that harness, it is a fact about any suite too expensive to run on every push. The replacement inherited the fault it was replacing. Records the three rules that now hold, and what the fix taught twice: the remedy rebuilt the symptom inside itself, and the code that counts results passed every test while reading nothing.
This commit is contained in:
@@ -2,9 +2,8 @@
|
||||
layer: to-be
|
||||
status: in-progress
|
||||
code: [mesh-lab]
|
||||
updated: 2026-08-28
|
||||
updated: 2026-08-31
|
||||
decisions:
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0019-how-this-repository-works.md
|
||||
---
|
||||
@@ -390,6 +389,38 @@ Everything a node itself does is real, because a node is a real machine.
|
||||
|
||||
---
|
||||
|
||||
## A suite too expensive to run on every push says when it last ran
|
||||
|
||||
*Written 2026-08-31, from resolving [04-ISSUES/005](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md).*
|
||||
|
||||
This suite needs a machine with a hypervisor. It therefore cannot run on every push, and a suite
|
||||
that does not run on every push runs **when somebody remembers**. Remembering is not a mechanism,
|
||||
and the harness this one replaces proves it: it had not built for two and a half months, nothing
|
||||
said so, and the coverage was assumed rather than checked.
|
||||
|
||||
**The danger is not that the suite breaks. It is that nobody notices it stopped running** — and
|
||||
that danger belongs to *this* design, not to the harness it retired.
|
||||
|
||||
So three rules, each held by a test:
|
||||
|
||||
**A run leaves a receipt** — when, what passed, what it ran, and the commit each repository was
|
||||
at. Kept **outside version control**: the question is *has this machine run it*, and a receipt in
|
||||
git would be a claim about everybody's machine made by whoever committed last.
|
||||
|
||||
**A receipt says why it does not count.** Old, failed, taken against commits the repositories have
|
||||
moved past, or a run that never raised a machine. Something can be asked, and answers non-zero.
|
||||
**A receipt that says nothing about something is not a receipt that clears it** — including a
|
||||
receipt written before it recorded a given fact, which claims nothing rather than everything.
|
||||
|
||||
**The run rebuilds what it tests.** The suite consumes artifacts from other repositories, and an
|
||||
artifact rebuilt from memory is one rebuilt sometimes. A stale binary reporting success against
|
||||
rules that have since changed is the same fault wearing different clothes.
|
||||
|
||||
**The general rule, which outlives this suite:** *silence and success must never look alike.*
|
||||
It is the same rule the host follows about a service that does not exist
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable
|
||||
from a failure to answer — applied to coverage instead of to a machine.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Bringing a node into being is part of the framework.** A test creates its own nodes — one
|
||||
|
||||
Reference in New Issue
Block a user