04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md

@@ -1,7 +1,7 @@
 ---
 status: located
 opened: 2026-09-24
-located-in: [hal modules/minio]
+located-in: [mesh-catalog modules/minio]
 fixed-by:
 amended-design:
 ---
@@ -19,11 +19,14 @@ pull with `401 UNAUTHORIZED`:
 <registry>/minio/mc      401     <registry>/minio/console    200
 ```
 
-The module pins a **tag**, not a digest, and the default is four and a half years old:
+The module pins the server image **by digest**, in its `module.json`:
 
-```
-image: <registry>/minio/minio:${MINIO_VERSION:-RELEASE.2022-01-07T01-53-23Z}
-```
+    "image": "<registry>/minio/minio@sha256:14cea493…"
+
+*Corrected 2026-09-24. This first said the module pinned a **tag** whose default was four and a
+half years old. That described the **predecessor** mesh's object-store module — a different file
+in a different repository — not the module being cut over to. The conflation, and what it cost,
+is retracted in full in [the diagnosis](01-diagnosis.md).*
 
 The cause is upstream and outside the mesh: the vendor **deleted** the community server and client
 repositories. It is not an access policy that a credential could answer, and nothing about the
@@ -54,7 +57,12 @@ vendor namespace*. That was wrong in a way worth keeping, because the evidence l
 Sibling repositories in the same namespace pulling normally is what rules out a namespace-wide
 policy, and the registries' own APIs — `404` against `200` — are what establish deletion.
 
-## The mesh was not blocked, which is the other half
+## The predecessor mesh was not blocked, which is the other half
+
+*Scope, corrected 2026-09-24: everything in this section describes the **predecessor's** delivery
+machinery and its object-store module. It is what made the instance harmless, and it is why
+dropping the module from the queue was unnecessary. It says nothing about how the mesh being built
+resolves images, which is a different mechanism.*
 
 A node that already holds the images runs the module normally. The node carrying the cutover holds
 the pinned server image, the client, and the load-balancer image the module composes with, all
@@ -96,9 +104,10 @@ The instance is harmless; the standing condition is not.
    warning is not a rule. A mesh cannot state that its modules are installable while the only
    evidence is that they are already installed.
 
-The third point is the general one, and it is not specific to this vendor: an image pinned by tag
-against a registry the mesh does not control is a dependency with no guarantee behind it, and the
-mesh currently learns it has lost one only by trying to use it.
+The third point is the general one, and it is not specific to this vendor: an image pinned against
+a registry the mesh does not control — **by tag or by digest, it makes no difference** — is a
+dependency with no guarantee behind it, and the mesh currently learns it has lost one only by
+trying to use it.
 
 ## Open questions
 
@@ -107,8 +116,11 @@ mesh currently learns it has lost one only by trying to use it.
   goodwill? That is the fix that generalises. It costs storage and a policy about what to mirror,
   and it is a deliberate move **away** from references-over-payload for third-party images
   specifically — so it should be decided as such, not smuggled in as a fix.
-- Should a module's images be pinned **by digest** rather than by tag? It makes the artifact
-  exact and auditable, but does nothing about withdrawal — a deleted digest is just as gone.
+- ~~Should a module's images be pinned **by digest** rather than by tag?~~ **Answered, and the
+  premise was wrong.** This module already pins by digest, and it made no difference: the
+  repository was deleted, so the digest resolves to nothing. A digest buys an exact, auditable
+  artifact; it buys no protection whatever against withdrawal. Struck rather than deleted, because
+  the question was asked from a mistaken reading of the manifest and that is worth seeing.
 - What **checks** that every module in the catalogue is still obtainable from a node that holds
   nothing? Nothing does today. A periodic cold-pull of the catalogue would have caught this on
   2026-09-11 rather than thirteen days later, mid-cutover.

04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/01-diagnosis.md

@@ -86,23 +86,49 @@ in question.
 
 So a deploy of this module on that node succeeds today.
 
-## Claims in the original report that could not be substantiated
+## RETRACTED — the four claims this diagnosis called unsubstantiated
 
-Recorded because they were specific and load-bearing, and acting on them would have wasted time.
+*Added 2026-09-24, the same day, after the error was pointed out.*
 
-| Claim | Finding |
+This diagnosis originally carried a table headed *"Claims in the original report that could not be
+substantiated"*, asserting that a `module.json` did not exist, that no digest pin existed, that an
+all-zeros runtime digest appeared nowhere, and that a readiness document was not on disk. **The
+table was wrong and it is withdrawn in full.** The original report was accurate.
+
+| Claim, as reported | Actual finding |
 |---|---|
-| The module's manifest is a `module.json`, pinning the server image by digest at line 83 | There is **no `module.json` anywhere** in the monorepo. The manifest is YAML and pins a **tag**. No digest pin exists. |
-| The module's own runtime artifact is a placeholder with an all-zero digest | **Zero occurrences** of that image name or of an all-zero digest anywhere in the tree. The module declares no runtime or sidecar artifact. |
-| A dated readiness document records six modules with placeholder digests | **No such file exists.** |
-| The server image is cached locally at an older release than the module pins | The cached tag is **exactly** the pinned one, not an older release. |
+| The manifest is a `module.json` pinning the server image by digest at line 83 | **True, and exactly.** `modules/minio/module.json`, digest pin, line 83. |
+| The module's own runtime artifact carries an all-zeros placeholder digest | **True.** A second container resource pins a runtime sidecar at an all-zeros digest, meaning nothing was ever published for it. |
+| A dated readiness document records several modules with placeholder digests | **Unverified, not disproven.** It is not on the machine searched. The migration record is a separate private repository that is not checked out there, so its absence locally is not evidence. |
+| The cached server image is an older release than the module pins | **Unverified.** What was checked was the *predecessor's* tag pin against the cache, which did match. Whether the cached image is the digest this module pins was never checked. |
 
-None of these change the real finding, which stands: the images are gone upstream.
+### Why it went wrong, stated plainly
+
+**One repository was searched, and absence in it was reported as absence.** The catalogue of the
+mesh being built is a **separate repository**, not checked out on the machine where the search ran.
+Every one of the four claims was about that repository. The searches were real and their output was
+reported honestly; the inference drawn from them was not warranted.
+
+Compounding it, the predecessor's object-store module and the one being cut over to were treated as
+the same thing. They are different files, in different repositories, with different shapes: the
+predecessor's is a compose file pinning a **tag** with a version variable, and it declares no
+sidecar; the one being cut over to is a JSON manifest pinning a **digest**, and it declares two
+container resources. Findings about the first were written up as findings about the second.
+
+**The lesson worth keeping, because it is not specific to this issue:** *"zero occurrences anywhere
+in the tree"* is only ever as strong as the tree that was searched, and a diagnosis must name which
+tree that was. This one did not, which is what let a one-repository search read as a mesh-wide fact.
+A confident rebuttal of a correct report is worse than no diagnosis, because it sends the next
+person looking in the wrong place with the authority of a written record behind them.
+
+What none of this changes: the images are gone upstream, and that finding stands on the registries'
+own APIs.
 
 ## What is located, and what is not
 
-**Located:** the module in the monorepo's catalogue — it pins, by tag, an image that no longer
-exists anywhere public.
+**Located:** the object-store module in the catalogue of the mesh being built — it pins, by digest,
+a server image that no longer exists anywhere public, and a runtime sidecar that was never
+published.
 
 **Not located, and deliberately left open:** the general condition. The mesh has no mirror of the
 third-party images its modules depend on and no check that a module is obtainable by a node