Commit Graph
15 Commits
Author SHA1 Message Date
jschoubben 973d80aaa2 minio: publish both public routes now that a module can answer route twice
files-api.novox.be (port 9000, the S3 data API) and files.novox.be (port
9001, the console) — same two names HAL routes today, via nginx's own
upstream split. Needed mesh-controller#55 (a module answering one
requirement several times) to exist first; it's merged and deployed.
2026-09-24 18:45:52 +02:00
jschoubben 945390e59a minio: run the real 4-node/8-drive erasure-coded cluster, not a single container
The single standalone instance from the first pass didn't match HAL's actual
topology: HAL runs minio1-4, two drives each, behind an nginx load balancer
on 9000 (S3) and 9001 (console). This rewrite mirrors that exactly — same
node count, same erasure-coding command, same LB config — so the migration
is a real like-for-like move, not a simplification.

Only the two images that had to change did: the minio server (dead upstream,
already fixed in the prior commit) and nginx (1.19.2-alpine is long EOL;
repinned to current stable-alpine by digest). Data still lands on a fresh,
empty, mesh-owned path, never HAL's live drives.

The OIDC-wait entrypoint wrapper HAL used is dropped: it's a no-op when
MINIO_IDENTITY_OPENID_CONFIG_URL is unset (it always is here — no OIDC
integration was ever wired to minio itself), and this catalogue has no
container resource field for overriding a container's entrypoint anyway —
every converted module relies on the image's own entrypoint plus args,
which is exactly what the original single-node version already did.
2026-09-24 18:31:38 +02:00
jschoubben 35e8a3fe19 minio: repin to pgsty's fork, build the runtime sidecar, move data off HAL's live drive
minio/minio and minio/mc were pulled from all public registries on 2026-09-11;
pgsty's fork is the working replacement (hq issue 113). The runtime sidecar
had no Dockerfile and no build section at all — added, following the same
tsc-over-client/tools/provisioner shape every other converted module uses.

The data resource pointed straight at /services/minio/data/data1-1, one of
HAL's live 8-drive erasure-coded array — starting this module would have
written into production storage the mesh doesn't own. Moved to a fresh,
empty, mesh-owned directory; the actual data migration happens over the S3
API (rclone), not by sharing a disk path.
2026-09-24 17:55:49 +02:00
jschoubben 1a28e5aec6 Six modules take their secrets from files; the rest say precisely why not
From the survey of every env-file secret (ADR 0086, issue 041): amqp-ping,
minio, mongodb and grafana use the _FILE twin their software honours;
mesh-catalog and model-usage read DATABASE_URL_FILE (a file the mesh
templates, mounted where only the runtime reads it); grafana's secret files
belong to its own account. Two dead deliveries removed: a line nothing read
in amqp-email-forwarder, and mailu's secret.env on four containers that
never read it. The 25 exceptions that remain carry the surveyed reason —
convertible and awaiting a bed, convertible through a generated config file,
the application's own code, or not convertible.
2026-09-21 12:29:25 +02:00
jschoubben 32dec5f0c2 The controller reads its credentials from files; every other env-file secret says why
ADR 0086. mesh-controller mounts its six own secrets and names them with
_FILE twins, so no credential of its own reaches its environment. The 35
containers that still read a secret through an env-file carry
secrets-in-environment with the reason; converting each where its software
accepts a path is the per-module work of issue 041.
2026-09-21 10:10:33 +02:00
jschoubben 2ef7eb2a27 minio: pull image from quay.io (docker.io denies anonymous pulls)
Same digest, a registry that serves anonymous pulls. Unblocks raising minio in
the lab and the object-store cutover.
2026-09-20 22:01:20 +02:00
jschoubben 0d2c2dd989 minio: pin the manifest these machines can actually run
The pinned digest named the arm64 manifest, so an x86 machine pulled it and the
container died with 'exec format error' on every restart. It never showed while
the lab ran its own registry: the harness pushed the WORKSTATION's copy, which
is amd64, and every machine then pulled that under a digest the registry had
just assigned. The registry was quietly correcting the architecture too.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-11 02:51:34 +02:00
jschoubben 1f0c3ac69e fix: db/cache/store providers must serve their port; drop baserow's empty redis contribution
The breadth install (catalogue-broad) surfaced two latent resolution bugs that
single-module compile checks never caught (compile != resolve):

1. postgres/redis/mongodb/minio served no "port", yet seven consumers
   (baserow, letta, invoicing, gitea, umami, keycloak, nextcloud) reference
   ${bound:<provision>:port}. A binding auto-carries at/from/as; the port is the
   provider's half of the answer and must be declared in `serves` (manifest.go:
   "Serves is what a consumer needs to know ... a port, a path, a realm"). Added
   port to each: postgres 5432, redis 6379, mongodb 27017, minio 9000. Without it
   no DB consumer could resolve, let alone deploy.

2. baserow declared an empty contribution `contributes: {"redis-cache": {}}`.
   redis's serves spec for the cache is empty (a cache takes no per-consumer
   payload), so a consumer only `requires` it; an empty contribution is refused.
   Dropped it (requires/binds/secrets unchanged).

Found by mesh-lab catalogue-broad; the same fixes are mirrored in that bed.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-06 02:38:22 +02:00
jschoubben 6e8c18afff Resync hq ADR references 0044-0054 -> 0039-0049 after the hq record reconciliation 2026-09-05 12:49:16 +02:00
jschoubben 08bdd0e456 Providers: broker-bound runtime container replaces the old provisioner (ADR 0052/0053)
Each provider's separate provisioner container becomes a runtime container that
serves the module's tools and runs its provisioner under the module's scoped broker
account: mesh-runtime-<module>, on the backend's own network (reaching the backend
by name and the broker by NAT), with MESH_BROKER_FILE + MESH_RECEIVES replacing
GRANTS. umami gains the broker own-secret it lacked. cloudflare-dns's adapter is
re-pointed at the ADR 0053 contract (a data provision, like umami — its record
return is the scoped-out concern).

Proven: provider-on-backend-network green — redis's runtime, on the private redis
network, binds the broker and provisions a consumer with the mesh's credential.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-05 00:52:09 +02:00
jschoubben 3b03fcd8f7 Providers create the credential the mesh minted, sealing nothing (ADR 0053)
redis, postgres and minio adapters drop generatePassword + the returned credential:
each creates the resource under the login the mesh derived (`as`) with the password
the mesh minted (`p.password`). minio's client gains a secret-key argument so it sets
the mesh's secret rather than generating one. umami (analytics) is re-pointed at the
new contract too; its siteId return is a data-provision concern ADR 0053 scopes out.

Proven: mesh-lab provider-uses-mesh-credential green — redis creates the consumer's
login with the mesh's password, the consumer authenticates (PONG), no seal key set.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-05 00:27:37 +02:00
jschoubben 67d8d0dcbd catalogue: one mesh-state namespace — /var/lib/mesh/<module>/ (consistency)
The review found broker own-secret paths drifting: mostly /var/lib/<module>/
broker, but grafana/redis/icecast/nextcloud carried a '-module' suffix to dodge
a collision with the service's own /var/lib/<name> data, and photos sat under
/etc. Normalized to one collision-free namespace a service never owns:
/var/lib/mesh/<module>/broker, with a mesh-state directory resource for the
parent, across 24 modules. audit-logger is grandfathered (lab-proven, referenced
by the assigned test, and it has no service to collide with). All 33 manifests
parse; no client hardcoded a path, so nothing in code moved.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 21:28:55 +02:00
jschoubben 5fd78a77ad minio: provisioner emits best-effort, so a missing broker can't fail provisioning
The review found minio's provisioner emitting with a bare await, which throws
when no broker is bound (a provisioner is not yet a runtime — ADR 0052) and so
fails create/remove. Wrapped like the other providers: the event is logged and
dropped, the bucket still made. The real fix — the provisioner carrying a broker
credential — is ADR 0052.
2026-09-04 21:25:42 +02:00
jschoubben f689b7dfa6 minio: full nox module — client, tools, provisioner and events (ADR 0044/0045/0046)
Object store, an s3-bucket provider. Client ported with no npm deps: S3 data
plane over fetch + SigV4 (node:crypto), scoped access keys via the mc CLI (the
admin API needs an Argon2 payload node built-ins can't make — the honest port
hal also used). Tools: list buckets/objects, bucket info, presigned url. The
provisioner makes a bucket + scoped key per grant and emits
module.minio.bucket.created/removed (the secret stays off the bus). Typechecks;
manifest parses.

(Trimmed the generated self-consuming ledger: a provider need not subscribe to
its own emits.)
2026-09-04 02:35:51 +02:00
jschoubben c5efbd53ca A module is a package, not a bare manifest
Each module becomes modules/<name>/ holding module.json, with room for
the rest of what a module is — its tools, health checks, provisioning,
lifecycle — which the conversion from hal still has to bring across.
The flat <name>.json was only the resource-declaration half.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-03 00:47:14 +02:00