From 5dcb9dfbf6a7b11d454973b7d43408e7b9881e7d Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 14:35:08 +0200 Subject: [PATCH] ADR 0089: a bed reads the catalogue it proves; issue 073 diagnosed; issues 074 and 075 opened The end-to-end design held 'the run rebuilds what it tests' for binaries and images and not for manifests. The beds' inline copies fell into three kinds; only the first is a stale copy. The other two are named: a mesh test wearing a catalogue module's name (074) and a stocked runtime image the run never rebuilds (075). --- ...089-a-bed-reads-the-catalogue-it-proves.md | 69 +++++++++++++++++++ 02-DECISIONS/README.md | 1 + 03-DESIGN/01-to-be/01-end-to-end-testing.md | 30 +++++++- .../00-report.md | 2 +- .../01-diagnosis.md | 33 +++++++++ .../00-report.md | 41 +++++++++++ .../00-report.md | 36 ++++++++++ 7 files changed, 210 insertions(+), 2 deletions(-) create mode 100644 02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md create mode 100644 04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/01-diagnosis.md create mode 100644 04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md create mode 100644 04-ISSUES/075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md diff --git a/02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md b/02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md new file mode 100644 index 0000000..967d879 --- /dev/null +++ b/02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md @@ -0,0 +1,69 @@ +--- +topic: checking it +status: accepted +date: 2026-09-21 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0016-the-lab.md +--- + +# 89. A bed reads the catalogue it proves + +## Context + +A lab bed installs a catalogue module and asserts what the mesh does with it; "proven in the +lab" is the standard a module must meet before it ships. The beds built the manifests they +install inline — a literal copied from the catalogue when each bed was written, and never since. +Six modules were converted to file-delivered secrets ([ADR 0086](0086-a-secret-reaches-a-process-as-a-file.md)) +and not one bed ran the converted shape; each ran its copy, and the copies still delivered +secrets the way the catalogue engine now refuses +([issue 073](../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md)). A run's +receipt named the commits of the host, the controller and the lab, and said nothing about the +catalogue, so a catalogue change and a proven catalogue change were indistinguishable. + +The end-to-end design already has the rule this breaks — *the run rebuilds what it tests; an +artifact rebuilt from memory is one rebuilt sometimes* — for binaries and images. A manifest is +an artifact too. + +## Considered Options + +1. **Keep the copies and check them against the catalogue** — a test that diffs each literal + against the module's manifest, ignoring what the lab must rewrite. Rejected: it keeps two + sources of truth and adds a third thing that can drift, the list of what to ignore. +2. **Read the catalogue, rewriting only what the lab must.** Adopted. + +## Decision + +A bed that installs a catalogue module reads that module's manifest from the catalogue checkout +the run was pointed at. It may rewrite what the lab must and nothing else: a build artifact +becomes the image the machine holds, an image is pinned to what the machine holds, a host port +is remapped where one machine carries colliding modules, and an address may point at a stand-in +the bed raises in place of an upstream. Everything else is the catalogue's, verbatim. + +A bed that needs less than the catalogue declares — no upstream server, a secret in the +environment, a requirement edge removed — is not testing that module. It is a mesh test, and it +carries a name of its own (see [issue 074](../04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md)). + +The run's receipt names the catalogue's commit alongside the other repositories', so a receipt +taken before a manifest changed says so. + +## Consequences + +A catalogue change is proven by the beds that install the module, or it is not proven, and the +receipt says which. What got harder: a bed can no longer trim a module to the shape it finds +convenient; it meets the module's declared requirements or gives its fixture another name. +The beds that still carry a copy are declared, each with its reason, and the declared list +only shrinks. + +## How it is checked + +A unit test in the lab refuses an inline manifest literal that names a catalogue module unless +the bed is declared, with its reason, in the test's own list; a declaration for a bed that no +longer carries the copy is refused too, so the list cannot outlive the debt. The receipt test +asserts the catalogue is claimed whenever the run is pointed at one. + +## References + +- [issue 073](../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md), [issue 074](../04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md) +- [ADR 0016](0016-the-lab.md), [ADR 0086](0086-a-secret-reaches-a-process-as-a-file.md) +- [`03-DESIGN/01-to-be/01-end-to-end-testing.md`](../03-DESIGN/01-to-be/01-end-to-end-testing.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 1bdb3ba..839a162 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -161,6 +161,7 @@ python3 00-META/checks/index.py fail if stale - **0017** — [A test defends a decision](0017-a-test-defends-a-decision.md) - **0018** — [A picture of a system is read from the system, never from what asked for it](0018-a-picture-is-read-from-what-runs.md) +- **0089** — [A bed reads the catalogue it proves](0089-a-bed-reads-the-catalogue-it-proves.md) ### How we work diff --git a/03-DESIGN/01-to-be/01-end-to-end-testing.md b/03-DESIGN/01-to-be/01-end-to-end-testing.md index 2032147..733172a 100644 --- a/03-DESIGN/01-to-be/01-end-to-end-testing.md +++ b/03-DESIGN/01-to-be/01-end-to-end-testing.md @@ -2,10 +2,11 @@ layer: to-be status: in-progress code: [mesh-lab] -updated: 2026-08-31 +updated: 2026-09-21 decisions: - 02-DECISIONS/0016-the-lab.md - 02-DECISIONS/0019-how-this-repository-works.md + - 02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md --- # End-to-end testing @@ -424,6 +425,33 @@ It is the same rule the host follows about a service that does not exist ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable from a failure to answer — applied to coverage instead of to a machine. +## A bed reads the catalogue it proves + +*Written 2026-09-21, from resolving [04-ISSUES/073](../../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md); +decided in [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md).* + +The rule above — the run rebuilds what it tests — was held for binaries and images and not for +manifests. Beds built the manifests they install inline, as literals copied from the catalogue +when each bed was written; the copies did not move when the catalogue did, and a catalogue change +was proven by no bed at all, while every bed stayed green against its copy. + +**A bed that installs a catalogue module reads that module's manifest from the catalogue the run +was pointed at.** It rewrites what the lab must — a build artifact becomes the image the machine +holds, an image is pinned, a host port is remapped where one machine carries colliding modules, +an address may point at a stand-in the bed raises — and nothing else. + +**A bed that needs less than the module declares is not testing that module.** No upstream +server, a secret in the environment, a requirement edge cut so no second provider is needed: +that is a mesh test, and it carries a fixture with a name of its own, never a catalogue module's. + +**The receipt names the catalogue's commit** with the others', so a run taken before a manifest +changed says so — the same rule as for the binaries, for the same reason. + +*How it is checked:* a unit test in the lab refuses an inline manifest literal that names a +catalogue module unless the bed is declared, with its reason, in the test's own list, and refuses +a declaration for a copy that is gone; the receipt test asserts the catalogue is claimed whenever +the run is pointed at one. + ## Consequences **Bringing a node into being is part of the framework.** A test creates its own nodes — one diff --git a/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md index 78d676b..2bd1d9d 100644 --- a/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md +++ b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md @@ -3,7 +3,7 @@ status: located opened: 2026-09-21 located-in: [mesh-lab test/integration] fixed-by: -amended-design: +amended-design: 03-DESIGN/01-to-be/01-end-to-end-testing.md --- # Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere diff --git a/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/01-diagnosis.md b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/01-diagnosis.md new file mode 100644 index 0000000..1516089 --- /dev/null +++ b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/01-diagnosis.md @@ -0,0 +1,33 @@ +# Diagnosis — 2026-09-21 + +1. Every bed was read against the catalogue. Of forty-five, thirteen already read a manifest + from the catalogue checkout, twenty-one built one or more inline, and eleven install no + catalogue module. The thirteen readers used five private copies of one loader, and the copies + had drifted: one never resolved a runtime artifact to the image the lab stocks, so a module + the mesh builds travelled to the machine unresolved; one still asked the catalogue for two + modules by names it no longer uses and recorded the miss as "not assigned". +2. The inline copies fall into three kinds, and only the first is what the report assumed: + - copies that differ from the catalogue only in what the lab must rewrite — an image, a + build section, an optional key dropped. Ten modules across eight beds; + - a second provider raised beside the foundation's. The catalogue's postgres and lavinmq + *claim* the foundation's store and broker and adopt them in place; four beds raise a + second one next to it, renamed and on a private network. Reading the catalogue there + changes what the bed raises, and its assertions with it; + - a module cut down to the shape a mesh mechanism needs — no upstream server, a secret in + the environment, a requirement edge removed so no second provider is wanted — under a + catalogue module's name. Twelve beds. These are mesh tests wearing a module's name, and + the honest fix is a name of their own, not a catalogue read + ([issue 074](../074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md)). +3. The receipt claimed the host, the controller and the lab and never the catalogue, so even a + bed that read the catalogue could not be told apart from one that had read it last week. +4. While converting, the images the beds stock for the modules' runtimes were found to date + from two weeks before the manifests they serve — built by hand, by a script the run never + calls. The run rebuilds the host and the controller and not these + ([issue 075](../075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md)). + +**Located in:** mesh-lab, the integration beds and their harness. The fix gives the harness one +loader that reads the catalogue and rewrites only what the lab must, converts the first kind of +copy to it, folds the five private loaders onto it, makes the receipt claim the catalogue, and +adds a unit test that refuses an inline copy naming a catalogue module unless the bed is declared +with its reason. The second and third kinds are declared there; each declaration names the work +that removes it. Decided in [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md). diff --git a/04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md b/04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md new file mode 100644 index 0000000..04fe590 --- /dev/null +++ b/04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md @@ -0,0 +1,41 @@ +--- +status: located +opened: 2026-09-21 +located-in: [mesh-lab test/integration] +fixed-by: +amended-design: +--- + +# A mesh test wears a catalogue module's name + +## Symptom, as observed + +Twelve lab beds install a module named `redis`, `grafana`, `plex`, `sonarr`, `minio`, `postgres`, +`route-proxy` or `hello-web` that is not the catalogue's module of that name. Each is cut down to +what the bed's mechanism needs: the sidecar runtime without the upstream server it manages; a +token or an API key in the container's environment where the catalogue delivers a file; a +requirement on a route or a certificate authority removed so no second provider is needed; a +vault-granted secret turned into one the module mints itself; a route contribution in the shape a +decision replaced. One bed's header says its manifests are "verbatim from the catalogue" and its +manifest adds three resources the catalogue has not got. + +Found while diagnosing [issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md); +the beds are listed, each with what it cuts, in the lab's `beds-read-the-catalogue` unit test. + +## Why it matters beyond this instance + +- **A green bed named for a module reads as that module proven.** The status view counts a bed + by the module it names; a bed proving a grant mechanism with a `redis` that mints its own + secret proves nothing about the catalogue's redis, which requires the vault's. +- **The cut is invisible.** Nothing distinguishes "this is redis" from "this is a redis-shaped + fixture" except reading the literal against the catalogue, which is what issue 073 found nobody + had done. +- [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md) now refuses the + copy; until each bed is renamed or made to read the catalogue, it is declared debt. + +## What would close it + +Each of the twelve beds either reads the catalogue's module and meets its declared requirements +(a route needs the route module beside it; a secret needs the vault), or gives its fixture a name +that is no catalogue module's — `a-cache`, `a-store`, `a-sidecar` — so a green run claims exactly +what it proved. The declared list in the unit test is empty of the `WEARING` reason. diff --git a/04-ISSUES/075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md b/04-ISSUES/075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md new file mode 100644 index 0000000..11b23e5 --- /dev/null +++ b/04-ISSUES/075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md @@ -0,0 +1,36 @@ +--- +status: open +opened: 2026-09-21 +located-in: [] +fixed-by: +amended-design: +--- + +# A stocked runtime image is never rebuilt by the run + +## Symptom, as observed + +A per-module bed stocks the module's runtime image — the tool runtime carrying that module's +code — from the workstation's image store, by tag. The image is built by hand, by a script in the +lab repository that the suite never calls. On the day issue 073 was worked, the images for six +modules whose beds were about to run dated from two weeks before the manifests they were to be +installed with; the catalogue's code for those modules had changed since, and every bed would have +passed against the old image. The suite's own rule — *the run rebuilds what it tests* — is held +for the host binary and the controller image and not for these. + +## Why it matters beyond this instance + +- **Silence and success look alike again.** A bed that passes against a stale runtime reports + the module proven; nothing says the image predates the code. +- **It is the same fault [issue 005](../005-pipeline-test-harness-unbuildable/00-report.md) + named**, one layer down: a stale artifact reporting success against code that moved. +- The receipt now names the catalogue's commit ([ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md)), + which makes this sharper, not better: the receipt claims a commit whose runtime code was never + built into what ran. + +## What would close it + +The suite builds, or refuses to stock, a module runtime whose image is older than the module's +source in the catalogue the run is pointed at — the same treatment the host binary and the +controller image get. Or the scenario says which images it stocks stale, and the receipt says +so too. -- 2.54.0