Issue 114: quay.io disabled anonymous pulls for minio's own images

Found checking minio before assigning it on novox. Not a novox-side
problem - DNS, daemon.json and the auth handshake are all fine; quay.io's
own token for both quay.io/minio/mc and quay.io/minio/minio (the module's
own pinned server image) comes back with actions:[] and $disabled. Worked
four days ago at the 2026-09-20 rehearsal.

Distinct from issue 064: that one was about the mesh's build environment
reaching a declared vendor image. This is the vendor disabling the image
for everyone, which a declared, pinned digest does nothing to protect
against.

Checks pass.
This commit is contained in:
2026-09-24 15:10:47 +02:00
parent 183b22997c
commit 17fb7c26d2
@@ -0,0 +1,68 @@
---
status: located
opened: 2026-09-24
located-in: [mesh-catalog modules/minio]
fixed-by:
amended-design:
---
# 114 — quay.io disabled anonymous pulls for minio's own images, breaking the module's pin
## What was observed
On the control-node, 2026-09-24, checking `minio` before assigning it — the next module in the
`novox` queue after the forge and analytics. `scripts/compare-module-with-running.py` reported no
name conflict; separately, the module needs a data migration from a 4-node HA cluster HAL runs
into the module's single-node server, which [`DATA-CUTOVER.md`](../../../migration/DATA-CUTOVER.md)
already has a rehearsed recipe for (`scripts/rehearse-minio-cutover.sh`, rehearsed 2026-09-20).
Pulling `quay.io/minio/mc` to run that recipe failed: `unauthorized: access to the requested
resource is not authorized`. Not a `novox`-side problem — DNS resolves quay.io correctly, the
daemon's `insecure-registries` entries are unrelated, and the auth handshake completes. The token
quay.io itself issues is the tell:
```json
{
"access": [{"type": "repository", "name": "minio/mc", "actions": []}],
"context": {"com.apostille.roots": {"minio/mc": "$disabled"}, "com.apostille.root": "$disabled"}
}
```
Checked the module's own pinned server image too, not just the tool used to test it —
`modules/minio/module.json`'s `quay.io/minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`
gets the identical answer: token issues, zero actions, `$disabled`. Anonymous pull of **both**
images `minio` publishes on quay.io is disabled, upstream, sometime after the 2026-09-20 rehearsal
— four days is the outside bound on when this broke.
A second, already-known gap sits under this one and is not this issue: the module's `runtime`
resource still carries a placeholder image
(`mesh-runtime-minio@sha256:0000000000000000000000000000000000000000000000000000000000000000`),
tracked since `MIGRATION-READINESS-2026-09-21.md` as deferred under
[issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md).
## Why it matters beyond this instance
[Issue 064](../064-a-mesh-build-cannot-fetch-a-modules-external-dependencies/00-report.md) answered
*can the mesh's build environment reach a declared vendor image* — a network-policy question,
resolved by requiring the image be declared as a build input. It assumed the image, once declared,
stays fetchable. This is the case that assumption doesn't cover: the vendor disabled the image
itself, for everyone, regardless of network policy or declaration. A pinned digest does not protect
against this — the digest still resolves, the registry just refuses to serve it anonymously.
Any module pinning a third-party image by digest carries this risk silently until the day something
needs to pull it fresh — a rebuild, a new node, a cache eviction. Nothing in the catalogue today
distinguishes "pinned and known-good" from "pinned and quietly unfetchable."
## Open questions
- Is there an authenticated path to these images (a quay.io account, a mirror), or does `minio`
need to move to a different distribution channel entirely? HAL's own running cluster is on an
older release (`RELEASE.2022-01-07T01-53-23Z`, present in `novox`'s local image cache) — worth
checking whether that tag or a similar one is still fetchable from anywhere before assuming the
module must repin to whatever's locally cached by accident.
- Should the catalogue periodically re-verify that a pinned vendor image is still fetchable, rather
than discovering it at the moment a cutover needs it?
- Same question issue 109 asked about addresses, asked of images: is a locally-cached copy on one
node a fact the mesh can rely on, or a trap for the next node that doesn't have it?
Handed off for a separate session to work; not blocking the rest of the `novox` module queue.