Issue 113: retract the diagnosis table — the original report was right

The diagnosis carried a table headed "claims that could not be substantiated", denying a
module.json, a digest pin, and an all-zeros runtime digest. All three exist. The table is
withdrawn in full and replaced with what is actually true, plus the two claims that remain
genuinely unverified rather than disproven.

The cause: one repository was searched and absence in it was written up as absence. The
catalogue of the mesh being built is a separate repository, not checked out where the search
ran, and all four claims were about that repository. Compounding it, the predecessor's
object-store module and the one being cut over to were treated as one thing — they are
different files in different repositories, one pinning a tag with no sidecar, the other a
digest with two container resources.

Also corrected in the report: located-in named the wrong repository; the "pins a tag" passage
described the predecessor; the open question about pinning by digest is struck, because this
module already does and it made no difference — a deleted digest resolves to nothing either
way. The section on why nothing broke is now scoped explicitly to the predecessor's
machinery.

The lesson kept in the record: "zero occurrences anywhere in the tree" is only as strong as
the tree searched, and a diagnosis must say which tree. A confident rebuttal of a correct
report is worse than no diagnosis — it sends the next person to the wrong place with a
written record behind them.
This commit is contained in:
jochen
2026-09-24 18:46:47 +02:00
parent 48abc36b5b
commit 999636e2a8
2 changed files with 59 additions and 21 deletions
@@ -1,7 +1,7 @@
---
status: located
opened: 2026-09-24
located-in: [hal modules/minio]
located-in: [mesh-catalog modules/minio]
fixed-by:
amended-design:
---
@@ -19,11 +19,14 @@ pull with `401 UNAUTHORIZED`:
<registry>/minio/mc 401 <registry>/minio/console 200
```
The module pins a **tag**, not a digest, and the default is four and a half years old:
The module pins the server image **by digest**, in its `module.json`:
```
image: <registry>/minio/minio:${MINIO_VERSION:-RELEASE.2022-01-07T01-53-23Z}
```
"image": "<registry>/minio/minio@sha256:14cea493…"
*Corrected 2026-09-24. This first said the module pinned a **tag** whose default was four and a
half years old. That described the **predecessor** mesh's object-store module — a different file
in a different repository — not the module being cut over to. The conflation, and what it cost,
is retracted in full in [the diagnosis](01-diagnosis.md).*
The cause is upstream and outside the mesh: the vendor **deleted** the community server and client
repositories. It is not an access policy that a credential could answer, and nothing about the
@@ -54,7 +57,12 @@ vendor namespace*. That was wrong in a way worth keeping, because the evidence l
Sibling repositories in the same namespace pulling normally is what rules out a namespace-wide
policy, and the registries' own APIs — `404` against `200` — are what establish deletion.
## The mesh was not blocked, which is the other half
## The predecessor mesh was not blocked, which is the other half
*Scope, corrected 2026-09-24: everything in this section describes the **predecessor's** delivery
machinery and its object-store module. It is what made the instance harmless, and it is why
dropping the module from the queue was unnecessary. It says nothing about how the mesh being built
resolves images, which is a different mechanism.*
A node that already holds the images runs the module normally. The node carrying the cutover holds
the pinned server image, the client, and the load-balancer image the module composes with, all
@@ -96,9 +104,10 @@ The instance is harmless; the standing condition is not.
warning is not a rule. A mesh cannot state that its modules are installable while the only
evidence is that they are already installed.
The third point is the general one, and it is not specific to this vendor: an image pinned by tag
against a registry the mesh does not control is a dependency with no guarantee behind it, and the
mesh currently learns it has lost one only by trying to use it.
The third point is the general one, and it is not specific to this vendor: an image pinned against
a registry the mesh does not control — **by tag or by digest, it makes no difference** — is a
dependency with no guarantee behind it, and the mesh currently learns it has lost one only by
trying to use it.
## Open questions
@@ -107,8 +116,11 @@ mesh currently learns it has lost one only by trying to use it.
goodwill? That is the fix that generalises. It costs storage and a policy about what to mirror,
and it is a deliberate move **away** from references-over-payload for third-party images
specifically — so it should be decided as such, not smuggled in as a fix.
- Should a module's images be pinned **by digest** rather than by tag? It makes the artifact
exact and auditable, but does nothing about withdrawal — a deleted digest is just as gone.
- ~~Should a module's images be pinned **by digest** rather than by tag?~~ **Answered, and the
premise was wrong.** This module already pins by digest, and it made no difference: the
repository was deleted, so the digest resolves to nothing. A digest buys an exact, auditable
artifact; it buys no protection whatever against withdrawal. Struck rather than deleted, because
the question was asked from a mistaken reading of the manifest and that is worth seeing.
- What **checks** that every module in the catalogue is still obtainable from a node that holds
nothing? Nothing does today. A periodic cold-pull of the catalogue would have caught this on
2026-09-11 rather than thirteen days later, mid-cutover.
@@ -86,23 +86,49 @@ in question.
So a deploy of this module on that node succeeds today.
## Claims in the original report that could not be substantiated
## RETRACTED — the four claims this diagnosis called unsubstantiated
Recorded because they were specific and load-bearing, and acting on them would have wasted time.
*Added 2026-09-24, the same day, after the error was pointed out.*
| Claim | Finding |
This diagnosis originally carried a table headed *"Claims in the original report that could not be
substantiated"*, asserting that a `module.json` did not exist, that no digest pin existed, that an
all-zeros runtime digest appeared nowhere, and that a readiness document was not on disk. **The
table was wrong and it is withdrawn in full.** The original report was accurate.
| Claim, as reported | Actual finding |
|---|---|
| The module's manifest is a `module.json`, pinning the server image by digest at line 83 | There is **no `module.json` anywhere** in the monorepo. The manifest is YAML and pins a **tag**. No digest pin exists. |
| The module's own runtime artifact is a placeholder with an all-zero digest | **Zero occurrences** of that image name or of an all-zero digest anywhere in the tree. The module declares no runtime or sidecar artifact. |
| A dated readiness document records six modules with placeholder digests | **No such file exists.** |
| The server image is cached locally at an older release than the module pins | The cached tag is **exactly** the pinned one, not an older release. |
| The manifest is a `module.json` pinning the server image by digest at line 83 | **True, and exactly.** `modules/minio/module.json`, digest pin, line 83. |
| The module's own runtime artifact carries an all-zeros placeholder digest | **True.** A second container resource pins a runtime sidecar at an all-zeros digest, meaning nothing was ever published for it. |
| A dated readiness document records several modules with placeholder digests | **Unverified, not disproven.** It is not on the machine searched. The migration record is a separate private repository that is not checked out there, so its absence locally is not evidence. |
| The cached server image is an older release than the module pins | **Unverified.** What was checked was the *predecessor's* tag pin against the cache, which did match. Whether the cached image is the digest this module pins was never checked. |
None of these change the real finding, which stands: the images are gone upstream.
### Why it went wrong, stated plainly
**One repository was searched, and absence in it was reported as absence.** The catalogue of the
mesh being built is a **separate repository**, not checked out on the machine where the search ran.
Every one of the four claims was about that repository. The searches were real and their output was
reported honestly; the inference drawn from them was not warranted.
Compounding it, the predecessor's object-store module and the one being cut over to were treated as
the same thing. They are different files, in different repositories, with different shapes: the
predecessor's is a compose file pinning a **tag** with a version variable, and it declares no
sidecar; the one being cut over to is a JSON manifest pinning a **digest**, and it declares two
container resources. Findings about the first were written up as findings about the second.
**The lesson worth keeping, because it is not specific to this issue:** *"zero occurrences anywhere
in the tree"* is only ever as strong as the tree that was searched, and a diagnosis must name which
tree that was. This one did not, which is what let a one-repository search read as a mesh-wide fact.
A confident rebuttal of a correct report is worse than no diagnosis, because it sends the next
person looking in the wrong place with the authority of a written record behind them.
What none of this changes: the images are gone upstream, and that finding stands on the registries'
own APIs.
## What is located, and what is not
**Located:** the module in the monorepo's catalogue — it pins, by tag, an image that no longer
exists anywhere public.
**Located:** the object-store module in the catalogue of the mesh being built — it pins, by digest,
a server image that no longer exists anywhere public, and a runtime sidecar that was never
published.
**Not located, and deliberately left open:** the general condition. The mesh has no mirror of the
third-party images its modules depend on and no check that a module is obtainable by a node