The registry's public name is a second module beside the store, locked by the registry itself #47

2026-09-23T21:19:58Z

jschoubben commented

2026-09-23 21:19:58 +00:00

The registry's hand-over (hq ADR 0082/0104). Pairs with novox/mesh-controller PR #48, which adds the max-request-body route field the manifests here use — merge that first.

What changes

modules/distribution: unchanged in what it is (ADR 0082: reached by name over the private network, no account). Gains a config file carried from the predecessor's registry.yml where it changed behaviour — no per-process blob-descriptor cache (two processes now share the store), CORS headers for the retired UI dropped — mounted into the store's container with restart-on. Delete is not enabled on this door: the whole overlay reaches it with no account, and GC was not carried, so nothing needs it here. Provides artifact-storage at node scope so the gate lands beside it. The the-artifact-store seat is now mesh-scoped: a second store anywhere is refused by name, so a gate assigned to a machine without the store fails loudly instead of raising an empty second store behind the public name.
modules/distribution-gate (new): a second registry process, same pinned image, on the same mesh-registry-data volume, listening on 5001, behind the registry's own htpasswd basic auth (realm basic-realm, as the predecessor), storage.delete.enabled: true (tag retention runs through this door). Own secret htpasswd — the operator carries the predecessor's file in with secret accept. Contributes route {label: registry-api, port: 5001, max-request-body: 21474836480}.
modules/route-adapter: writes max-request-body as the predecessor's buffering.maxRequestBodyBytes middleware, named after the router, only when asked; skips a route whose limit it cannot read (a JSON null included). README updated; npm test 10/10, tsc clean.

Why a second module, not a route on the store: contributing a route is requiring one, and the store is raised at genesis on a node with no proxy (the lab's genesis set is distribution, builder). The controller's own artifact-store rule already says: "one that wants more beside it is a second module".

Hand-over on the node — in this order, and the order matters

Merge #48, then this; wait for the pipeline.
Accept the secret before assigning — required, not recommended. secret accept <node> distribution-gate htpasswd --from <the predecessor's auth/htpasswd>. Until accepted the mesh mints random bytes into the file; the pinned image runs with it and the door is locked either way — anonymous requests get 401 and any credentials get 400 (the file fails to parse per request; reload is lazy on mtime) — but the container names the secret in restart-on, so accepting it after assigning recreates the gate and, with http.secret unset, invalidates every in-flight upload token. Accept first, then nothing restarts on the cutover.
Port: the gate declares 5001:5001; the predecessor's UI holds 5001 until it retires → settings set distribution-gate <json> --node <node> with {"ports": {"5001": <free port>}}. The route follows the setting (tested: 5001→5101).
The cutover window: stop the predecessor's registry container BEFORE assign + take, and never let the two routers overlap. traefik gives the predecessor's label router and the adapter's file router identical priority and sorts them unstably; while both exist a docker push is split across two backends and fails with an invalid upload state. So: stop the predecessor's registry (its route disappears with the container), accept a short window of 401/404 on the public name, then assign + take distribution-gate. Also remove the predecessor's registry-api labels from its compose so a later restart of that stack cannot bring the router back.
Stop docker-registry-maintenance.timer (the runbook already says so). docker login registry-api.<domain> works unchanged (same realm, same file).

Not carried — filed as an hq issue by the coordinator: the predecessor's weekly maintenance (tag retention, then garbage-collect --delete-untagged with the registry stopped, then restart). With two processes over one filesystem, GC must stop both containers, and the store's own door has no delete, so retention runs through the gate. Until that lands, no GC runs on the mesh's registry; the volume only grows.

Version: mesh image is 2.8.3, the predecessor runs 2.7 — same storage layout and htpasswd format; an upgrade, noted.

The registry's hand-over (hq ADR 0082/0104). Pairs with novox/mesh-controller PR #48, which adds the `max-request-body` route field the manifests here use — merge that first. **What changes** - `modules/distribution`: unchanged in what it is (ADR 0082: reached by name over the private network, no account). Gains a `config` file carried from the predecessor's registry.yml where it changed behaviour — no per-process blob-descriptor cache (two processes now share the store), CORS headers for the retired UI dropped — mounted into the store's container with `restart-on`. **Delete is not enabled on this door**: the whole overlay reaches it with no account, and GC was not carried, so nothing needs it here. Provides `artifact-storage` at node scope so the gate lands beside it. **The `the-artifact-store` seat is now mesh-scoped**: a second store anywhere is refused by name, so a gate assigned to a machine without the store fails loudly instead of raising an empty second store behind the public name. - `modules/distribution-gate` (new): a second registry process, same pinned image, on the same `mesh-registry-data` volume, listening on 5001, behind the registry's own htpasswd basic auth (realm `basic-realm`, as the predecessor), `storage.delete.enabled: true` (tag retention runs through this door). Own secret `htpasswd` — the operator carries the predecessor's file in with `secret accept`. Contributes `route` `{label: registry-api, port: 5001, max-request-body: 21474836480}`. - `modules/route-adapter`: writes `max-request-body` as the predecessor's `buffering.maxRequestBodyBytes` middleware, named after the router, only when asked; skips a route whose limit it cannot read (a JSON `null` included). README updated; `npm test` 10/10, `tsc` clean. **Why a second module, not a route on the store**: contributing a route is requiring one, and the store is raised at genesis on a node with no proxy (the lab's genesis set is `distribution, builder`). The controller's own artifact-store rule already says: "one that wants more beside it is a second module". **Hand-over on the node — in this order, and the order matters** 1. Merge #48, then this; wait for the pipeline. 2. **Accept the secret before assigning — required, not recommended.** `secret accept <node> distribution-gate htpasswd --from <the predecessor's auth/htpasswd>`. Until accepted the mesh mints random bytes into the file; the pinned image runs with it and the door is locked either way — anonymous requests get 401 and any credentials get 400 (the file fails to parse per request; reload is lazy on mtime) — but the container names the secret in `restart-on`, so accepting it *after* assigning recreates the gate and, with `http.secret` unset, invalidates every in-flight upload token. Accept first, then nothing restarts on the cutover. 3. Port: the gate declares `5001:5001`; the predecessor's UI holds 5001 until it retires → `settings set distribution-gate <json> --node <node>` with `{"ports": {"5001": <free port>}}`. The route follows the setting (tested: 5001→5101). 4. **The cutover window: stop the predecessor's `registry` container BEFORE assign + take, and never let the two routers overlap.** traefik gives the predecessor's label router and the adapter's file router identical priority and sorts them unstably; while both exist a `docker push` is split across two backends and fails with an invalid upload state. So: stop the predecessor's registry (its route disappears with the container), accept a short window of 401/404 on the public name, then `assign` + `take distribution-gate`. Also remove the predecessor's `registry-api` labels from its compose so a later restart of that stack cannot bring the router back. 5. Stop `docker-registry-maintenance.timer` (the runbook already says so). `docker login registry-api.<domain>` works unchanged (same realm, same file). **Not carried — filed as an hq issue by the coordinator**: the predecessor's weekly maintenance (tag retention, then `garbage-collect --delete-untagged` with the registry stopped, then restart). With two processes over one filesystem, GC must stop **both** containers, and the store's own door has no delete, so retention runs through the gate. Until that lands, no GC runs on the mesh's registry; the volume only grows. **Version**: mesh image is 2.8.3, the predecessor runs 2.7 — same storage layout and htpasswd format; an upgrade, noted.

jschoubben added 1 commit 2026-09-23 21:19:59 +00:00

The registry's public name is a second module beside the store, locked by the registry itself 3249b9a0cc

The predecessor serves the registry under a public name, behind htpasswd basic auth, with a
twenty-gigabyte body limit for layer pushes. The mesh's registry has no name, no lock and no
limit — by design inside the mesh, where the private network is the boundary and every node
pulls without an account (hq ADR 0082). Taking the name over must not change that.

A route on `distribution` itself would: contributing a route is requiring one, and the store
is raised at genesis on a node with no proxy. So the public door is `distribution-gate`, a
second registry process on the same volume, behind the registry's own htpasswd (the
predecessor's realm, the predecessor's file, carried in with `secret accept`), with the
route and its limit. It requires the store's storage as a node-scoped provision, so it can
only land beside the store. The store's own door is untouched — no auth, no htpasswd — which
is what keeps the builder's pushes and every node's pulls working.

Both processes read the predecessor's configuration where it changed behaviour: delete
enabled, which tag retention depends on; no per-process descriptor cache, which two
processes over one store cannot share; the CORS headers for the retired interface dropped.

route-adapter writes the limit as the predecessor's own buffering middleware, named after
the router, only when asked for — and skips a route whose limit it cannot read rather than
carrying what the module said not to.

hq ADR 0082/0104, the registry hand-over.

jschoubben added 1 commit 2026-09-23 21:35:33 +00:00

One store per mesh, and no DELETE on the door nothing authenticates to be3d2ccaa5

Review of the registry hand-over. The store's seat was node-scoped, so a gate assigned to a
machine without the store pulled a second, empty store in beside it — behind the real
credentials and the public name, and offering `artifact-store` a second time so every
consumer elsewhere refused. `the-artifact-store` is one per mesh: a second store anywhere,
however it got there, is refused by name.

storage.delete.enabled was carried onto the store's own door, which the whole private
network reaches with no account (hq ADR 0082); anything on the overlay could have deleted a
manifest. Nothing needs it there — garbage collection was not carried. It stays on the gate
only, behind the registry's own auth, where tag retention runs.

hq ADR 0082/0104, the registry hand-over.

jschoubben commented

2026-09-23 21:38:40 +00:00

Closed without merging: the operator decided the mesh will not take over the predecessor's registry name — that registry holds only the predecessor's images and simply retires when the last of its apps has become a mesh-built module. The one real fix the review found (the store's seat is one per mesh) is re-submitted on its own as fix/the-artifact-store-is-one-per-mesh.

Closed without merging: the operator decided the mesh will not take over the predecessor's registry name — that registry holds only the predecessor's images and simply retires when the last of its apps has become a mesh-built module. The one real fix the review found (the store's seat is one per mesh) is re-submitted on its own as `fix/the-artifact-store-is-one-per-mesh`.

jschoubben closed this pull request

2026-09-23 21:38:49 +00:00

jschoubben referenced this pull request

2026-09-23 21:41:10 +00:00

The artifact store's seat is one per mesh #48

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#47