Scenarios: egress where images are needed, and images: cut to what is ours
Every scenario that places a container runtime gives each of its machines `egress: true` — a node that runs modules pulls images from the internet, which is what a node does. The underlay-only scenarios (bootstrap-single, behind-nat, segmented-and-unforwardable, the-ordinary-shape, two-on-a-segment) stay sealed on purpose: an extra NIC would change the very reachability they are asserting about. `images:` keeps only the mesh's own — 55 third-party entries leave whole-mesh-full alone, and the machine fetches them itself by the digest its module.json already pins. The whole-mesh beds also say per machine which of ours they get: novox the substrate control plane and its own fifteen runtimes, ace its twenty-four, the two workstations one each. That is not a lab economy. An operator's workstation holds the images its own modules need, and giving these two the union would put some thirty gigabytes onto a thirty-gigabyte disk. bootstrap-with-registry.yml is deleted. It existed only to demonstrate the lab's registry, nothing referenced it, and there is nothing left for it to demonstrate. Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
@@ -14,9 +14,21 @@
|
||||
#
|
||||
# The `home` gateway masquerades v4 outbound and forwards inbound (an ordinary household router).
|
||||
# Home nodes reach novox's public 192.0.2.20 by dialling OUT through it: the substrate broker (5671),
|
||||
# the registry, and — the thing this bed exists to prove — the WireGuard overlay hub (51820/udp).
|
||||
# The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox cannot
|
||||
# initiate to a home node at all; every home↔novox path is either the overlay or a forwarded port.
|
||||
# the mesh's own artifact store, and — the thing this bed exists to prove — the WireGuard overlay hub
|
||||
# (51820/udp). The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox
|
||||
# cannot initiate to a home node at all; every home↔novox path is either the overlay or a forwarded
|
||||
# port.
|
||||
#
|
||||
# EVERY NODE HAS EGRESS, which is not a convenience. Third-party images — postgres, the whole Mailu
|
||||
# stack, Plex, everything the modules actually run — are pulled from the internet, because that is
|
||||
# where a real node gets them. The lab used to serve them from a registry it raised inside the
|
||||
# scenario; no production mesh has one, so a bootstrap that could only work against it went green
|
||||
# here and would have failed anywhere else. What is loaded onto a machine now is only what exists in
|
||||
# no registry at all: the mesh's own images, named per machine below.
|
||||
#
|
||||
# Egress is a SECOND path, not a replacement for the topology. Each node still reaches the rest of
|
||||
# the scenario through its declared gateway — that is where the overlay handshake has to survive a
|
||||
# masquerade — and the uplink carries only what leaves the scenario entirely.
|
||||
#
|
||||
# THE UNPROVEN THING (what the flat beds never tested): does the overlay tunnel FORM across the
|
||||
# access point — a home node dialling novox's public hub endpoint, the handshake completing through
|
||||
@@ -30,14 +42,13 @@
|
||||
# network on the container port, so the host side is free to move). Reported as a topology finding.
|
||||
#
|
||||
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
|
||||
# The images are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
|
||||
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set. Every
|
||||
# one is already built/pulled by the per-server bed prerequisites.
|
||||
# The images: are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
|
||||
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set, and
|
||||
# every one of them must already be BUILT on the workstation — nothing can fetch them.
|
||||
scenario: whole-mesh-full
|
||||
|
||||
segments:
|
||||
# The routable segment. novox lives here; the lab raises the image registry here too (a public
|
||||
# IPv4 segment is what serves the images), and the overlay hub endpoint is a public address here.
|
||||
# The routable segment. novox lives here, and the overlay hub endpoint is a public address here.
|
||||
hosting:
|
||||
kind: public
|
||||
cidr: [192.0.2.0/24]
|
||||
@@ -60,19 +71,69 @@ machines:
|
||||
# public ingress. Bigger than the flat bed's novox, because it now carries the substrate too.
|
||||
novox:
|
||||
at: { segment: hosting, address: [192.0.2.20] }
|
||||
egress: true
|
||||
inbound: allow
|
||||
memory: 24GiB
|
||||
cpus: 8
|
||||
disk: 130GiB
|
||||
# The substrate's control plane, the proxy, and a runtime per module novox is assigned.
|
||||
# step-ca, photos, invoicing, novox.be, only-office, de-spiegel, amqp-email-forwarder, registry,
|
||||
# firewall and fail2ban are not here because they carry no runtime image of their own — what
|
||||
# they run is third-party or is the node itself.
|
||||
images:
|
||||
- mesh-control:development
|
||||
- mesh-route-proxy:development
|
||||
- mesh-runtime-postgres:development
|
||||
- mesh-runtime-redis:development
|
||||
- mesh-runtime-minio:development
|
||||
- mesh-runtime-mongodb:development
|
||||
- mesh-runtime-mssql:development
|
||||
- mesh-runtime-lavinmq:development
|
||||
- mesh-runtime-keycloak:development
|
||||
- mesh-runtime-gitea:development
|
||||
- mesh-runtime-nextcloud:development
|
||||
- mesh-runtime-umami:development
|
||||
- mesh-runtime-verdaccio:development
|
||||
- mesh-runtime-portainer:development
|
||||
- mesh-runtime-mailu:development
|
||||
|
||||
# The home server: the whole ace media/home set — 24 modules, ~50 containers, several heavy
|
||||
# (Plex, Home Assistant, Letta, Baserow, the UniFi JVM, mssql). Behind the gateway.
|
||||
ace:
|
||||
at: { segment: home, address: [192.168.1.10] }
|
||||
egress: true
|
||||
inbound: allow
|
||||
memory: 18GiB
|
||||
cpus: 6
|
||||
disk: 120GiB
|
||||
# A runtime per module ace is assigned, and nothing of novox's. This is the point of saying it
|
||||
# per machine: ace has no business holding a Keycloak runtime, and an operator's home server
|
||||
# would not.
|
||||
images:
|
||||
- mesh-runtime-postgres:development
|
||||
- mesh-runtime-redis:development
|
||||
- mesh-runtime-mssql:development
|
||||
- mesh-runtime-portainer:development
|
||||
- mesh-runtime-sonarr:development
|
||||
- mesh-runtime-radarr:development
|
||||
- mesh-runtime-lidarr:development
|
||||
- mesh-runtime-plex:development
|
||||
- mesh-runtime-bazarr:development
|
||||
- mesh-runtime-nzbget:development
|
||||
- mesh-runtime-qbittorrent:development
|
||||
- mesh-runtime-jackett:development
|
||||
- mesh-runtime-ombi:development
|
||||
- mesh-runtime-tautulli:development
|
||||
- mesh-runtime-bookshelf:development
|
||||
- mesh-runtime-home-assistant:development
|
||||
- mesh-runtime-mosquitto:development
|
||||
- mesh-runtime-influxdb:development
|
||||
- mesh-runtime-grafana:development
|
||||
- mesh-runtime-baserow:development
|
||||
- mesh-runtime-letta:development
|
||||
- mesh-runtime-nodered:development
|
||||
- mesh-runtime-searxng:development
|
||||
- mesh-runtime-unifi:development
|
||||
|
||||
# Two workstations on the same home LAN. Light on purpose: they enrol, join the overlay, and run
|
||||
# one small module (portainer) so a real module converges on each without heavy load. Same-LAN
|
||||
@@ -80,77 +141,30 @@ machines:
|
||||
# is the normal case and is fine.
|
||||
shanks:
|
||||
at: { segment: home, address: [192.168.1.20] }
|
||||
egress: true
|
||||
inbound: allow
|
||||
memory: 3GiB
|
||||
cpus: 2
|
||||
disk: 30GiB
|
||||
# One module, one runtime. Handing these two the whole union would put roughly thirty gigabytes
|
||||
# of images onto a thirty-gigabyte disk, which is how you learn that "the lab loads everything
|
||||
# everywhere" was never a description of anything real.
|
||||
images: [mesh-runtime-portainer:development]
|
||||
g14:
|
||||
at: { segment: home, address: [192.168.1.30] }
|
||||
egress: true
|
||||
inbound: allow
|
||||
memory: 3GiB
|
||||
cpus: 2
|
||||
disk: 30GiB
|
||||
images: [mesh-runtime-portainer:development]
|
||||
|
||||
# The union of what the machines above ask for. This is the list the workstation must be able to
|
||||
# export — every entry is one of the mesh's own images, built from source and published nowhere, so
|
||||
# a machine holds it because it was handed it. Everything else the modules run comes from the
|
||||
# internet and is not named here at all.
|
||||
images:
|
||||
# --- substrate + shared (novox & ace both run postgres/redis/mssql/portainer) ---
|
||||
- postgres:17-alpine
|
||||
- cloudamqp/lavinmq:latest
|
||||
- mesh-control:development
|
||||
- redis:7-alpine
|
||||
- mcr.microsoft.com/mssql/server:2022-latest
|
||||
- portainer/portainer-ce:latest
|
||||
# --- novox server images ---
|
||||
- minio/minio:latest
|
||||
- mongo:7
|
||||
- quay.io/keycloak/keycloak:mesh
|
||||
- gitea/gitea:1.22
|
||||
- nextcloud:stable
|
||||
- ghcr.io/umami-software/umami:postgresql-latest
|
||||
- alpine:latest
|
||||
- verdaccio/verdaccio:6
|
||||
- registry:2
|
||||
- registry-api.novox.be/novox/invoicing-app:latest
|
||||
- registry-api.novox.be/novox/invoicing-api:latest
|
||||
- onlyoffice/documentserver:mesh
|
||||
- registry-api.novox.be/novox/de-spiegel:latest
|
||||
- registry-api.novox.be/novox/www:latest
|
||||
- registry-api.novox.be/novox/amqp-email-forwarder:latest
|
||||
- registry-api.novox.be/novox/photos-server:latest
|
||||
- registry-api.novox.be/novox/photos-admin-client:latest
|
||||
- registry-api.novox.be/novox/photos-client:latest
|
||||
# The full Mailu 1.9 stack (mailu-redis reuses redis:7-alpine above).
|
||||
- ghcr.io/mailu/unbound:1.9
|
||||
- ghcr.io/mailu/admin:1.9
|
||||
- ghcr.io/mailu/dovecot:1.9
|
||||
- ghcr.io/mailu/postfix:1.9
|
||||
- ghcr.io/mailu/rspamd:1.9
|
||||
- ghcr.io/mailu/clamav:1.9
|
||||
- ghcr.io/mailu/roundcube:1.9
|
||||
- ghcr.io/mailu/radicale:1.9
|
||||
- ghcr.io/mailu/fetchmail:1.9
|
||||
- ghcr.io/mailu/nginx:1.9
|
||||
# --- ace server images ---
|
||||
- lscr.io/linuxserver/sonarr:mesh
|
||||
- lscr.io/linuxserver/radarr:mesh
|
||||
- lscr.io/linuxserver/lidarr:mesh
|
||||
- lscr.io/linuxserver/bazarr:mesh
|
||||
- lscr.io/linuxserver/nzbget:mesh
|
||||
- lscr.io/linuxserver/qbittorrent:mesh
|
||||
- lscr.io/linuxserver/jackett:mesh
|
||||
- lscr.io/linuxserver/ombi:mesh
|
||||
- lscr.io/linuxserver/tautulli:mesh
|
||||
- lscr.io/linuxserver/unifi-controller:mesh
|
||||
- plexinc/pms-docker:mesh
|
||||
- ghcr.io/pennydreadful/bookshelf:mesh
|
||||
- ghcr.io/home-assistant/home-assistant:mesh
|
||||
- eclipse-mosquitto:mesh
|
||||
- influxdb:mesh
|
||||
- grafana/grafana:mesh
|
||||
- baserow/baserow:mesh
|
||||
- letta/letta:mesh
|
||||
- nodered/node-red:mesh
|
||||
- searxng/searxng:mesh
|
||||
- valkey/valkey:mesh
|
||||
# --- per-module runtimes (union) ---
|
||||
- mesh-runtime-postgres:development
|
||||
- mesh-runtime-redis:development
|
||||
@@ -166,9 +180,6 @@ images:
|
||||
- mesh-runtime-verdaccio:development
|
||||
- mesh-runtime-mailu:development
|
||||
- mesh-route-proxy:development
|
||||
# ADR 0056: the internal ACME authority. route-proxy now REQUIRES an `acme-ca`, so a bed that does
|
||||
# not serve this image has an unresolvable proxy — and with it every routed module on the mesh.
|
||||
- smallstep/step-ca:latest
|
||||
- mesh-runtime-sonarr:development
|
||||
- mesh-runtime-radarr:development
|
||||
- mesh-runtime-lidarr:development
|
||||
|
||||
Reference in New Issue
Block a user