jschoubben
5dcb9dfbf6
The end-to-end design held 'the run rebuilds what it tests' for binaries and images and not for manifests. The beds' inline copies fell into three kinds; only the first is a stale copy. The other two are named: a mesh test wearing a catalogue module's name (074) and a stocked runtime image the run never rebuilds (075).
1.9 KiB
status, opened, located-in, fixed-by, amended-design
| status | opened | located-in | fixed-by | amended-design | |
|---|---|---|---|---|---|
| located | 2026-09-21 |
|
03-DESIGN/01-to-be/01-end-to-end-testing.md |
Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere
Symptom, as observed
The per-module beds (assigned-catalogue-small, assigned-catalogue-apps, assigned-grafana,
assigned-model-usage, assigned-redis, and others) build the manifests they install inline, as
JSON in the test, rather than reading modules/<name>/module.json from the catalogue checkout the
lab is pointed at. The copies were taken when each bed was written and have not moved since.
Converting six modules to file-delivered secrets (ADR 0086) therefore changed nothing any of those beds run. Worse: the copies still deliver secrets through an env-file without the declared exception, which the catalogue engine now refuses, so the beds would fail against a current controller for a reason that has nothing to do with what they test.
Why it matters beyond this instance
- It is issue 072 again, one copy per bed: a manifest with two sources of truth, and the one the proof runs against is the stale one.
- "Proven in the lab" is the standard playbook 06 sets for a module. A bed that installs a copy proves the copy.
- The genesis bed and the vault bed already read the catalogue (
--catalogand acatalogueManifest()helper that swaps the build artifact for the image the lab built); the pattern exists and is one helper away for the rest.
What would close it
Every bed that installs a catalogue module reads its manifest from the catalogue checkout, rewriting only what the lab must (the built artifact's image, a lab-local address). The inline copies go. Until then, each bed's copy is updated by hand alongside the catalogue, which is how the six conversions were proven.