ADR 0089: a bed reads the catalogue it proves; issue 073 diagnosed; issues 074 and 075 opened #62
@@ -0,0 +1,69 @@
|
|||||||
|
---
|
||||||
|
topic: checking it
|
||||||
|
status: accepted
|
||||||
|
date: 2026-09-21
|
||||||
|
deciders: jochen
|
||||||
|
reconstructed: false
|
||||||
|
extends: 02-DECISIONS/0016-the-lab.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# 89. A bed reads the catalogue it proves
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
A lab bed installs a catalogue module and asserts what the mesh does with it; "proven in the
|
||||||
|
lab" is the standard a module must meet before it ships. The beds built the manifests they
|
||||||
|
install inline — a literal copied from the catalogue when each bed was written, and never since.
|
||||||
|
Six modules were converted to file-delivered secrets ([ADR 0086](0086-a-secret-reaches-a-process-as-a-file.md))
|
||||||
|
and not one bed ran the converted shape; each ran its copy, and the copies still delivered
|
||||||
|
secrets the way the catalogue engine now refuses
|
||||||
|
([issue 073](../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md)). A run's
|
||||||
|
receipt named the commits of the host, the controller and the lab, and said nothing about the
|
||||||
|
catalogue, so a catalogue change and a proven catalogue change were indistinguishable.
|
||||||
|
|
||||||
|
The end-to-end design already has the rule this breaks — *the run rebuilds what it tests; an
|
||||||
|
artifact rebuilt from memory is one rebuilt sometimes* — for binaries and images. A manifest is
|
||||||
|
an artifact too.
|
||||||
|
|
||||||
|
## Considered Options
|
||||||
|
|
||||||
|
1. **Keep the copies and check them against the catalogue** — a test that diffs each literal
|
||||||
|
against the module's manifest, ignoring what the lab must rewrite. Rejected: it keeps two
|
||||||
|
sources of truth and adds a third thing that can drift, the list of what to ignore.
|
||||||
|
2. **Read the catalogue, rewriting only what the lab must.** Adopted.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
A bed that installs a catalogue module reads that module's manifest from the catalogue checkout
|
||||||
|
the run was pointed at. It may rewrite what the lab must and nothing else: a build artifact
|
||||||
|
becomes the image the machine holds, an image is pinned to what the machine holds, a host port
|
||||||
|
is remapped where one machine carries colliding modules, and an address may point at a stand-in
|
||||||
|
the bed raises in place of an upstream. Everything else is the catalogue's, verbatim.
|
||||||
|
|
||||||
|
A bed that needs less than the catalogue declares — no upstream server, a secret in the
|
||||||
|
environment, a requirement edge removed — is not testing that module. It is a mesh test, and it
|
||||||
|
carries a name of its own (see [issue 074](../04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md)).
|
||||||
|
|
||||||
|
The run's receipt names the catalogue's commit alongside the other repositories', so a receipt
|
||||||
|
taken before a manifest changed says so.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
A catalogue change is proven by the beds that install the module, or it is not proven, and the
|
||||||
|
receipt says which. What got harder: a bed can no longer trim a module to the shape it finds
|
||||||
|
convenient; it meets the module's declared requirements or gives its fixture another name.
|
||||||
|
The beds that still carry a copy are declared, each with its reason, and the declared list
|
||||||
|
only shrinks.
|
||||||
|
|
||||||
|
## How it is checked
|
||||||
|
|
||||||
|
A unit test in the lab refuses an inline manifest literal that names a catalogue module unless
|
||||||
|
the bed is declared, with its reason, in the test's own list; a declaration for a bed that no
|
||||||
|
longer carries the copy is refused too, so the list cannot outlive the debt. The receipt test
|
||||||
|
asserts the catalogue is claimed whenever the run is pointed at one.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [issue 073](../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md), [issue 074](../04-ISSUES/074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md)
|
||||||
|
- [ADR 0016](0016-the-lab.md), [ADR 0086](0086-a-secret-reaches-a-process-as-a-file.md)
|
||||||
|
- [`03-DESIGN/01-to-be/01-end-to-end-testing.md`](../03-DESIGN/01-to-be/01-end-to-end-testing.md)
|
||||||
@@ -161,6 +161,7 @@ python3 00-META/checks/index.py fail if stale
|
|||||||
|
|
||||||
- **0017** — [A test defends a decision](0017-a-test-defends-a-decision.md)
|
- **0017** — [A test defends a decision](0017-a-test-defends-a-decision.md)
|
||||||
- **0018** — [A picture of a system is read from the system, never from what asked for it](0018-a-picture-is-read-from-what-runs.md)
|
- **0018** — [A picture of a system is read from the system, never from what asked for it](0018-a-picture-is-read-from-what-runs.md)
|
||||||
|
- **0089** — [A bed reads the catalogue it proves](0089-a-bed-reads-the-catalogue-it-proves.md)
|
||||||
|
|
||||||
### How we work
|
### How we work
|
||||||
|
|
||||||
|
|||||||
@@ -2,10 +2,11 @@
|
|||||||
layer: to-be
|
layer: to-be
|
||||||
status: in-progress
|
status: in-progress
|
||||||
code: [mesh-lab]
|
code: [mesh-lab]
|
||||||
updated: 2026-08-31
|
updated: 2026-09-21
|
||||||
decisions:
|
decisions:
|
||||||
- 02-DECISIONS/0016-the-lab.md
|
- 02-DECISIONS/0016-the-lab.md
|
||||||
- 02-DECISIONS/0019-how-this-repository-works.md
|
- 02-DECISIONS/0019-how-this-repository-works.md
|
||||||
|
- 02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md
|
||||||
---
|
---
|
||||||
|
|
||||||
# End-to-end testing
|
# End-to-end testing
|
||||||
@@ -424,6 +425,33 @@ It is the same rule the host follows about a service that does not exist
|
|||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — absence must be distinguishable
|
||||||
from a failure to answer — applied to coverage instead of to a machine.
|
from a failure to answer — applied to coverage instead of to a machine.
|
||||||
|
|
||||||
|
## A bed reads the catalogue it proves
|
||||||
|
|
||||||
|
*Written 2026-09-21, from resolving [04-ISSUES/073](../../04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md);
|
||||||
|
decided in [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md).*
|
||||||
|
|
||||||
|
The rule above — the run rebuilds what it tests — was held for binaries and images and not for
|
||||||
|
manifests. Beds built the manifests they install inline, as literals copied from the catalogue
|
||||||
|
when each bed was written; the copies did not move when the catalogue did, and a catalogue change
|
||||||
|
was proven by no bed at all, while every bed stayed green against its copy.
|
||||||
|
|
||||||
|
**A bed that installs a catalogue module reads that module's manifest from the catalogue the run
|
||||||
|
was pointed at.** It rewrites what the lab must — a build artifact becomes the image the machine
|
||||||
|
holds, an image is pinned, a host port is remapped where one machine carries colliding modules,
|
||||||
|
an address may point at a stand-in the bed raises — and nothing else.
|
||||||
|
|
||||||
|
**A bed that needs less than the module declares is not testing that module.** No upstream
|
||||||
|
server, a secret in the environment, a requirement edge cut so no second provider is needed:
|
||||||
|
that is a mesh test, and it carries a fixture with a name of its own, never a catalogue module's.
|
||||||
|
|
||||||
|
**The receipt names the catalogue's commit** with the others', so a run taken before a manifest
|
||||||
|
changed says so — the same rule as for the binaries, for the same reason.
|
||||||
|
|
||||||
|
*How it is checked:* a unit test in the lab refuses an inline manifest literal that names a
|
||||||
|
catalogue module unless the bed is declared, with its reason, in the test's own list, and refuses
|
||||||
|
a declaration for a copy that is gone; the receipt test asserts the catalogue is claimed whenever
|
||||||
|
the run is pointed at one.
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
**Bringing a node into being is part of the framework.** A test creates its own nodes — one
|
**Bringing a node into being is part of the framework.** A test creates its own nodes — one
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ status: located
|
|||||||
opened: 2026-09-21
|
opened: 2026-09-21
|
||||||
located-in: [mesh-lab test/integration]
|
located-in: [mesh-lab test/integration]
|
||||||
fixed-by:
|
fixed-by:
|
||||||
amended-design:
|
amended-design: 03-DESIGN/01-to-be/01-end-to-end-testing.md
|
||||||
---
|
---
|
||||||
|
|
||||||
# Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere
|
# Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Diagnosis — 2026-09-21
|
||||||
|
|
||||||
|
1. Every bed was read against the catalogue. Of forty-five, thirteen already read a manifest
|
||||||
|
from the catalogue checkout, twenty-one built one or more inline, and eleven install no
|
||||||
|
catalogue module. The thirteen readers used five private copies of one loader, and the copies
|
||||||
|
had drifted: one never resolved a runtime artifact to the image the lab stocks, so a module
|
||||||
|
the mesh builds travelled to the machine unresolved; one still asked the catalogue for two
|
||||||
|
modules by names it no longer uses and recorded the miss as "not assigned".
|
||||||
|
2. The inline copies fall into three kinds, and only the first is what the report assumed:
|
||||||
|
- copies that differ from the catalogue only in what the lab must rewrite — an image, a
|
||||||
|
build section, an optional key dropped. Ten modules across eight beds;
|
||||||
|
- a second provider raised beside the foundation's. The catalogue's postgres and lavinmq
|
||||||
|
*claim* the foundation's store and broker and adopt them in place; four beds raise a
|
||||||
|
second one next to it, renamed and on a private network. Reading the catalogue there
|
||||||
|
changes what the bed raises, and its assertions with it;
|
||||||
|
- a module cut down to the shape a mesh mechanism needs — no upstream server, a secret in
|
||||||
|
the environment, a requirement edge removed so no second provider is wanted — under a
|
||||||
|
catalogue module's name. Twelve beds. These are mesh tests wearing a module's name, and
|
||||||
|
the honest fix is a name of their own, not a catalogue read
|
||||||
|
([issue 074](../074-a-mesh-test-wears-a-catalogue-modules-name/00-report.md)).
|
||||||
|
3. The receipt claimed the host, the controller and the lab and never the catalogue, so even a
|
||||||
|
bed that read the catalogue could not be told apart from one that had read it last week.
|
||||||
|
4. While converting, the images the beds stock for the modules' runtimes were found to date
|
||||||
|
from two weeks before the manifests they serve — built by hand, by a script the run never
|
||||||
|
calls. The run rebuilds the host and the controller and not these
|
||||||
|
([issue 075](../075-a-stocked-runtime-image-is-never-rebuilt-by-the-run/00-report.md)).
|
||||||
|
|
||||||
|
**Located in:** mesh-lab, the integration beds and their harness. The fix gives the harness one
|
||||||
|
loader that reads the catalogue and rewrites only what the lab must, converts the first kind of
|
||||||
|
copy to it, folds the five private loaders onto it, makes the receipt claim the catalogue, and
|
||||||
|
adds a unit test that refuses an inline copy naming a catalogue module unless the bed is declared
|
||||||
|
with its reason. The second and third kinds are declared there; each declaration names the work
|
||||||
|
that removes it. Decided in [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md).
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
---
|
||||||
|
status: located
|
||||||
|
opened: 2026-09-21
|
||||||
|
located-in: [mesh-lab test/integration]
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# A mesh test wears a catalogue module's name
|
||||||
|
|
||||||
|
## Symptom, as observed
|
||||||
|
|
||||||
|
Twelve lab beds install a module named `redis`, `grafana`, `plex`, `sonarr`, `minio`, `postgres`,
|
||||||
|
`route-proxy` or `hello-web` that is not the catalogue's module of that name. Each is cut down to
|
||||||
|
what the bed's mechanism needs: the sidecar runtime without the upstream server it manages; a
|
||||||
|
token or an API key in the container's environment where the catalogue delivers a file; a
|
||||||
|
requirement on a route or a certificate authority removed so no second provider is needed; a
|
||||||
|
vault-granted secret turned into one the module mints itself; a route contribution in the shape a
|
||||||
|
decision replaced. One bed's header says its manifests are "verbatim from the catalogue" and its
|
||||||
|
manifest adds three resources the catalogue has not got.
|
||||||
|
|
||||||
|
Found while diagnosing [issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md);
|
||||||
|
the beds are listed, each with what it cuts, in the lab's `beds-read-the-catalogue` unit test.
|
||||||
|
|
||||||
|
## Why it matters beyond this instance
|
||||||
|
|
||||||
|
- **A green bed named for a module reads as that module proven.** The status view counts a bed
|
||||||
|
by the module it names; a bed proving a grant mechanism with a `redis` that mints its own
|
||||||
|
secret proves nothing about the catalogue's redis, which requires the vault's.
|
||||||
|
- **The cut is invisible.** Nothing distinguishes "this is redis" from "this is a redis-shaped
|
||||||
|
fixture" except reading the literal against the catalogue, which is what issue 073 found nobody
|
||||||
|
had done.
|
||||||
|
- [ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md) now refuses the
|
||||||
|
copy; until each bed is renamed or made to read the catalogue, it is declared debt.
|
||||||
|
|
||||||
|
## What would close it
|
||||||
|
|
||||||
|
Each of the twelve beds either reads the catalogue's module and meets its declared requirements
|
||||||
|
(a route needs the route module beside it; a secret needs the vault), or gives its fixture a name
|
||||||
|
that is no catalogue module's — `a-cache`, `a-store`, `a-sidecar` — so a green run claims exactly
|
||||||
|
what it proved. The declared list in the unit test is empty of the `WEARING` reason.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-21
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# A stocked runtime image is never rebuilt by the run
|
||||||
|
|
||||||
|
## Symptom, as observed
|
||||||
|
|
||||||
|
A per-module bed stocks the module's runtime image — the tool runtime carrying that module's
|
||||||
|
code — from the workstation's image store, by tag. The image is built by hand, by a script in the
|
||||||
|
lab repository that the suite never calls. On the day issue 073 was worked, the images for six
|
||||||
|
modules whose beds were about to run dated from two weeks before the manifests they were to be
|
||||||
|
installed with; the catalogue's code for those modules had changed since, and every bed would have
|
||||||
|
passed against the old image. The suite's own rule — *the run rebuilds what it tests* — is held
|
||||||
|
for the host binary and the controller image and not for these.
|
||||||
|
|
||||||
|
## Why it matters beyond this instance
|
||||||
|
|
||||||
|
- **Silence and success look alike again.** A bed that passes against a stale runtime reports
|
||||||
|
the module proven; nothing says the image predates the code.
|
||||||
|
- **It is the same fault [issue 005](../005-pipeline-test-harness-unbuildable/00-report.md)
|
||||||
|
named**, one layer down: a stale artifact reporting success against code that moved.
|
||||||
|
- The receipt now names the catalogue's commit ([ADR 0089](../../02-DECISIONS/0089-a-bed-reads-the-catalogue-it-proves.md)),
|
||||||
|
which makes this sharper, not better: the receipt claims a commit whose runtime code was never
|
||||||
|
built into what ran.
|
||||||
|
|
||||||
|
## What would close it
|
||||||
|
|
||||||
|
The suite builds, or refuses to stock, a module runtime whose image is older than the module's
|
||||||
|
source in the catalogue the run is pointed at — the same treatment the host binary and the
|
||||||
|
controller image get. Or the scenario says which images it stocks stale, and the receipt says
|
||||||
|
so too.
|
||||||
Reference in New Issue
Block a user