2026-09-05 10:27:11 +00:00
2 changed files with 86 additions and 6 deletions
@@ -2,9 +2,8 @@
 layer: to-be
 status: in-progress
 code: [mesh-lab]
-updated: 2026-08-28
+updated: 2026-08-31
 decisions:
  - 02-DECISIONS/0016-the-lab.md
  - 02-DECISIONS/0016-the-lab.md
  - 02-DECISIONS/0019-how-this-repository-works.md
 ---
@@ -390,6 +389,38 @@ Everything a node itself does is real, because a node is a real machine.
 ---
 ## A suite too expensive to run on every push says when it last ran
 *Written 2026-08-31, from resolving [04-ISSUES/005](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md).*
 This suite needs a machine with a hypervisor. It therefore cannot run on every push, and a suite
 that does not run on every push runs **when somebody remembers**. Remembering is not a mechanism,
 and the harness this one replaces proves it: it had not built for two and a half months, nothing
 said so, and the coverage was assumed rather than checked.
 **The danger is not that the suite breaks. It is that nobody notices it stopped running** — and
 that danger belongs to *this* design, not to the harness it retired.
 So three rules, each held by a test:
 **A run leaves a receipt** — when, what passed, what it ran, and the commit each repository was
 at. Kept **outside version control**: the question is *has this machine run it*, and a receipt in
 git would be a claim about everybody's machine made by whoever committed last.
 **A receipt says why it does not count.** Old, failed, taken against commits the repositories have
 moved past, or a run that never raised a machine. Something can be asked, and answers non-zero.
 **A receipt that says nothing about something is not a receipt that clears it** — including a
 receipt written before it recorded a given fact, which claims nothing rather than everything.
 **The run rebuilds what it tests.** The suite consumes artifacts from other repositories, and an
 artifact rebuilt from memory is one rebuilt sometimes. A stale binary reporting success against
 rules that have since changed is the same fault wearing different clothes.
 **The general rule, which outlives this suite:** *silence and success must never look alike.*
 It is the same rule the host follows about a service that does not exist
 ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable
 from a failure to answer — applied to coverage instead of to a machine.
 ## Consequences
 **Bringing a node into being is part of the framework.** A test creates its own nodes — one
@@ -1,9 +1,9 @@
 ---
-status: open
+status: resolved
 opened: 2026-08-22
-located-in: [hal]
+located-in: [hal, mesh-lab]
-fixed-by:
+fixed-by: mesh-lab — a run leaves a receipt, and the receipt says what it covered
-amended-design:
+amended-design: 03-DESIGN/01-to-be/01-end-to-end-testing.md
 ---
 # 005 — The end-to-end pipeline harness has not built since the workspace was removed
@@ -45,3 +45,52 @@ unmentioned: until the lab exists, this is the coverage the pipeline is presumed
 - Repair, or retire in favour of the lab? Leaving it in the repository unbuilt is the one
  option that keeps the false impression of coverage.
 - Was anything relying on it, or had it already stopped running before the workspace removal?
 ## Resolution
 *2026-08-31.* **Retired in favour of the lab, and the reason it went unnoticed was fixed
 separately from the harness itself.**
 The old harness is not repaired. What replaced it is the end-to-end suite on a lab mesh, which
 raises real machines and proves the pipeline against them. That answers the first open question.
 The second finding is the one worth keeping. *Nothing runs it, and nothing reports that nothing
 runs it* is not a fact about that harness — it is a fact about **any** suite too expensive to run
 on every push, and the lab suite is exactly that: it needs a machine with a hypervisor, so it runs
 when somebody remembers. **Remembering is not a mechanism**, and the replacement inherited the
 fault it was replacing.
 So three things now hold, each checked by a test that was confirmed to fail without it:
 - **A run leaves a receipt** — when it ran, what passed, and the commit each repository was at.
  Kept outside version control, because the question is *has this machine run it*, and a receipt in
  git would be a claim about everybody's machine made by whoever committed last.
 - **The receipt can be judged, and says why it does not count.** Old, failed, taken against code
  the repositories have since moved past, or a run that never raised a machine — each reads
  differently, and only the last of those is new. **A receipt that says nothing about something is
  not a receipt that clears it.**
 - **The artifacts are rebuilt by the run, not beside it.** The suite consumes three artifacts from
  two repositories. They were rebuilt by hand, from memory, and a rename that needed two of them
  got one — leaving a binary eleven hours old refusing a field the mesh had just renamed, found by
  a full run. That step now lives in the repository rather than in a terminal history.
 ### What this issue taught twice
 **The fix reintroduced the fault, in miniature, and the second time was caught by running it.**
 The suite takes paths, so it can be pointed at one quick unit file — and the receipt from that run
 was, at first, indistinguishable from a receipt for the real thing. A green record standing for a
 run that raised no machines is this issue's own symptom, rebuilt inside its remedy. The receipt now
 records what it ran, and a run that did not include the end-to-end file is not coverage.
 Separately, the code that decides *no receipt rather than a guessed one* — the rule that keeps the
 record meaning something — was first written where no test could reach it. Writing "0 failed"
 because nothing said otherwise is how a green record comes to mean nothing.
 And the counting itself **passed every test while reading nothing**: the test runner colours its
 summary even into a pipe, so the anchored pattern never matched, and the fixtures it was checked
 against were output that had been imagined rather than captured. **A fixture that agrees with the
 mistake proves the mistake.** It is now checked against the runner's real bytes.
 Each of these was found by running the thing, not by reading it — which is the same argument this
 issue makes about the pipeline.