1 changed files with 68 additions and 0 deletions
@@ -0,0 +1,68 @@
 ---
 status: located
 opened: 2026-09-24
 located-in: [mesh-catalog modules/minio]
 fixed-by:
 amended-design:
 ---
 # 114 — quay.io disabled anonymous pulls for minio's own images, breaking the module's pin
 ## What was observed
 On the control-node, 2026-09-24, checking `minio` before assigning it — the next module in the
 `novox` queue after the forge and analytics. `scripts/compare-module-with-running.py` reported no
 name conflict; separately, the module needs a data migration from a 4-node HA cluster HAL runs
 into the module's single-node server, which [`DATA-CUTOVER.md`](../../../migration/DATA-CUTOVER.md)
 already has a rehearsed recipe for (`scripts/rehearse-minio-cutover.sh`, rehearsed 2026-09-20).
 Pulling `quay.io/minio/mc` to run that recipe failed: `unauthorized: access to the requested
 resource is not authorized`. Not a `novox`-side problem — DNS resolves quay.io correctly, the
 daemon's `insecure-registries` entries are unrelated, and the auth handshake completes. The token
 quay.io itself issues is the tell:
 ```json
 {
  "access": [{"type": "repository", "name": "minio/mc", "actions": []}],
  "context": {"com.apostille.roots": {"minio/mc": "$disabled"}, "com.apostille.root": "$disabled"}
 }
 ```
 Checked the module's own pinned server image too, not just the tool used to test it —
 `modules/minio/module.json`'s `quay.io/minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`
 gets the identical answer: token issues, zero actions, `$disabled`. Anonymous pull of **both**
 images `minio` publishes on quay.io is disabled, upstream, sometime after the 2026-09-20 rehearsal
 — four days is the outside bound on when this broke.
 A second, already-known gap sits under this one and is not this issue: the module's `runtime`
 resource still carries a placeholder image
 (`mesh-runtime-minio@sha256:0000000000000000000000000000000000000000000000000000000000000000`),
 tracked since `MIGRATION-READINESS-2026-09-21.md` as deferred under
 [issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md).
 ## Why it matters beyond this instance
 [Issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md) answered
 *can the mesh's build environment reach a declared vendor image* — a network-policy question,
 resolved by requiring the image be declared as a build input. It assumed the image, once declared,
 stays fetchable. This is the case that assumption doesn't cover: the vendor disabled the image
 itself, for everyone, regardless of network policy or declaration. A pinned digest does not protect
 against this — the digest still resolves, the registry just refuses to serve it anonymously.
 Any module pinning a third-party image by digest carries this risk silently until the day something
 needs to pull it fresh — a rebuild, a new node, a cache eviction. Nothing in the catalogue today
 distinguishes "pinned and known-good" from "pinned and quietly unfetchable."
 ## Open questions
 - Is there an authenticated path to these images (a quay.io account, a mirror), or does `minio`
  need to move to a different distribution channel entirely? HAL's own running cluster is on an
  older release (`RELEASE.2022-01-07T01-53-23Z`, present in `novox`'s local image cache) — worth
  checking whether that tag or a similar one is still fetchable from anywhere before assuming the
  module must repin to whatever's locally cached by accident.
 - Should the catalogue periodically re-verify that a pinned vendor image is still fetchable, rather
  than discovering it at the moment a cutover needs it?
 - Same question issue 109 asked about addresses, asked of images: is a locally-cached copy on one
  node a fact the mesh can rely on, or a trap for the next node that doesn't have it?
 Handed off for a separate session to work; not blocking the rest of the `novox` module queue.