1 changed files with 68 additions and 0 deletions
@@ -0,0 +1,68 @@
+---
+status: located
+opened: 2026-09-24
+located-in: [mesh-catalog modules/minio]
+fixed-by:
+amended-design:
+---
+
+# 114 — quay.io disabled anonymous pulls for minio's own images, breaking the module's pin
+
+## What was observed
+
+On the control-node, 2026-09-24, checking `minio` before assigning it — the next module in the
+`novox` queue after the forge and analytics. `scripts/compare-module-with-running.py` reported no
+name conflict; separately, the module needs a data migration from a 4-node HA cluster HAL runs
+into the module's single-node server, which [`DATA-CUTOVER.md`](../../../migration/DATA-CUTOVER.md)
+already has a rehearsed recipe for (`scripts/rehearse-minio-cutover.sh`, rehearsed 2026-09-20).
+
+Pulling `quay.io/minio/mc` to run that recipe failed: `unauthorized: access to the requested
+resource is not authorized`. Not a `novox`-side problem — DNS resolves quay.io correctly, the
+daemon's `insecure-registries` entries are unrelated, and the auth handshake completes. The token
+quay.io itself issues is the tell:
+
+```json
+{
+  "access": [{"type": "repository", "name": "minio/mc", "actions": []}],
+  "context": {"com.apostille.roots": {"minio/mc": "$disabled"}, "com.apostille.root": "$disabled"}
+}
+```
+
+Checked the module's own pinned server image too, not just the tool used to test it —
+`modules/minio/module.json`'s `quay.io/minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`
+gets the identical answer: token issues, zero actions, `$disabled`. Anonymous pull of **both**
+images `minio` publishes on quay.io is disabled, upstream, sometime after the 2026-09-20 rehearsal
+— four days is the outside bound on when this broke.
+
+A second, already-known gap sits under this one and is not this issue: the module's `runtime`
+resource still carries a placeholder image
+(`mesh-runtime-minio@sha256:0000000000000000000000000000000000000000000000000000000000000000`),
+tracked since `MIGRATION-READINESS-2026-09-21.md` as deferred under
+[issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md).
+
+## Why it matters beyond this instance
+
+[Issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md) answered
+*can the mesh's build environment reach a declared vendor image* — a network-policy question,
+resolved by requiring the image be declared as a build input. It assumed the image, once declared,
+stays fetchable. This is the case that assumption doesn't cover: the vendor disabled the image
+itself, for everyone, regardless of network policy or declaration. A pinned digest does not protect
+against this — the digest still resolves, the registry just refuses to serve it anonymously.
+
+Any module pinning a third-party image by digest carries this risk silently until the day something
+needs to pull it fresh — a rebuild, a new node, a cache eviction. Nothing in the catalogue today
+distinguishes "pinned and known-good" from "pinned and quietly unfetchable."
+
+## Open questions
+
+- Is there an authenticated path to these images (a quay.io account, a mirror), or does `minio`
+  need to move to a different distribution channel entirely? HAL's own running cluster is on an
+  older release (`RELEASE.2022-01-07T01-53-23Z`, present in `novox`'s local image cache) — worth
+  checking whether that tag or a similar one is still fetchable from anywhere before assuming the
+  module must repin to whatever's locally cached by accident.
+- Should the catalogue periodically re-verify that a pinned vendor image is still fetchable, rather
+  than discovering it at the moment a cutover needs it?
+- Same question issue 109 asked about addresses, asked of images: is a locally-cached copy on one
+  node a fact the mesh can rely on, or a trap for the next node that doesn't have it?
+
+Handed off for a separate session to work; not blocking the rest of the `novox` module queue.