--- status: resolved opened: 2026-08-22 located-in: [hal, mesh-lab] fixed-by: mesh-lab — a run leaves a receipt, and the receipt says what it covered amended-design: 03-DESIGN/01-to-be/01-end-to-end-testing.md --- # 005 — The end-to-end pipeline harness has not built since the workspace was removed ## Symptom The repository's end-to-end pipeline test harness depends on a workspace that no longer exists. It has not been buildable since 2026-06-04. Nothing runs it, and nothing reports that nothing runs it. ## Why this matters The delivery pipeline is the mesh's most consequential machinery — every module reaches every node through it — and its only end-to-end coverage has been silently dead for over two and a half months. That interval is not incidental. Several of the pipeline's most expensive defects landed inside it: packaging that was never actually split from build, migrations and provisioning scripts reading a source layout that no longer ships, selection files never packaged at all. Whether this harness would have caught any of them is unknown — which is itself the point. The coverage was assumed, not checked. ## Evidence - The workspace was removed by pull request #240 on 2026-06-04 ([ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md)). - The harness has not built since that date. - Recorded in the knowledge base as a standing entry, not as a fixed incident. ## Relationship to the lab [`03-DESIGN/01-to-be/01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md) designs end-to-end testing on a lab mesh, which would replace this harness rather than repair it. That is a reason to decide its fate deliberately, not a reason to leave it broken and unmentioned: until the lab exists, this is the coverage the pipeline is presumed to have. ## Open questions - Repair, or retire in favour of the lab? Leaving it in the repository unbuilt is the one option that keeps the false impression of coverage. - Was anything relying on it, or had it already stopped running before the workspace removal? ## Resolution *2026-08-31.* **Retired in favour of the lab, and the reason it went unnoticed was fixed separately from the harness itself.** The old harness is not repaired. What replaced it is the end-to-end suite on a lab mesh, which raises real machines and proves the pipeline against them. That answers the first open question. The second finding is the one worth keeping. *Nothing runs it, and nothing reports that nothing runs it* is not a fact about that harness — it is a fact about **any** suite too expensive to run on every push, and the lab suite is exactly that: it needs a machine with a hypervisor, so it runs when somebody remembers. **Remembering is not a mechanism**, and the replacement inherited the fault it was replacing. So three things now hold, each checked by a test that was confirmed to fail without it: - **A run leaves a receipt** — when it ran, what passed, and the commit each repository was at. Kept outside version control, because the question is *has this machine run it*, and a receipt in git would be a claim about everybody's machine made by whoever committed last. - **The receipt can be judged, and says why it does not count.** Old, failed, taken against code the repositories have since moved past, or a run that never raised a machine — each reads differently, and only the last of those is new. **A receipt that says nothing about something is not a receipt that clears it.** - **The artifacts are rebuilt by the run, not beside it.** The suite consumes three artifacts from two repositories. They were rebuilt by hand, from memory, and a rename that needed two of them got one — leaving a binary eleven hours old refusing a field the mesh had just renamed, found by a full run. That step now lives in the repository rather than in a terminal history. ### What this issue taught twice **The fix reintroduced the fault, in miniature, and the second time was caught by running it.** The suite takes paths, so it can be pointed at one quick unit file — and the receipt from that run was, at first, indistinguishable from a receipt for the real thing. A green record standing for a run that raised no machines is this issue's own symptom, rebuilt inside its remedy. The receipt now records what it ran, and a run that did not include the end-to-end file is not coverage. Separately, the code that decides *no receipt rather than a guessed one* — the rule that keeps the record meaning something — was first written where no test could reach it. Writing "0 failed" because nothing said otherwise is how a green record comes to mean nothing. And the counting itself **passed every test while reading nothing**: the test runner colours its summary even into a pipe, so the anchored pattern never matched, and the fixtures it was checked against were output that had been imagined rather than captured. **A fixture that agrees with the mistake proves the mistake.** It is now checked against the runner's real bytes. Each of these was found by running the thing, not by reading it — which is the same argument this issue makes about the pipeline.