novox/mesh-lab
3
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity
Files
5c91c0ecd288967d1cdc4c8b9b749b104d169a20
mesh-lab/scenarios/whole-mesh-full.yml
T
jschoubben 5c91c0ecd2 Scenarios: egress where images are needed, and images: cut to what is ours 
Every scenario that places a container runtime gives each of its machines
`egress: true` — a node that runs modules pulls images from the internet, which is what
a node does. The underlay-only scenarios (bootstrap-single, behind-nat,
segmented-and-unforwardable, the-ordinary-shape, two-on-a-segment) stay sealed on
purpose: an extra NIC would change the very reachability they are asserting about.

`images:` keeps only the mesh's own — 55 third-party entries leave whole-mesh-full
alone, and the machine fetches them itself by the digest its module.json already pins.
The whole-mesh beds also say per machine which of ours they get: novox the substrate
control plane and its own fifteen runtimes, ace its twenty-four, the two workstations
one each. That is not a lab economy. An operator's workstation holds the images its own
modules need, and giving these two the union would put some thirty gigabytes onto a
thirty-gigabyte disk.

bootstrap-with-registry.yml is deleted. It existed only to demonstrate the lab's
registry, nothing referenced it, and there is nothing left for it to demonstrate.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-10 23:16:23 +02:00

206 lines
9.6 KiB
YAML
Raw Blame History

# The FULL mesh in its REAL production shape: two segments, one access point, one overlay.
#
# This is the first multi-segment whole-mesh bed. The earlier flat whole-mesh-full sat every node
# on one public segment with a SEPARATE `anchor` carrying the substrate. Production is not flat, and
# there is no separate anchor: `novox` IS the anchor. It sits on the routable `hosting` segment,
# runs the substrate (store, broker, control) AND its own service set AND is the overlay hub and the
# public ingress. `ace`, `shanks` and `g14` sit on the household `home` segment BEHIND a NAT gateway
# — the access point — reachable from the outside only through what they dial out to.
#
# hosting (public, routable) home (private, behind the access point)
# novox 192.0.2.20 ── anchor ace 192.168.1.10 home server, media/IoT set
# substrate + novox set shanks 192.168.1.20 workstation (light)
# overlay hub, ingress g14 192.168.1.30 workstation (light)
#
# The `home` gateway masquerades v4 outbound and forwards inbound (an ordinary household router).
# Home nodes reach novox's public 192.0.2.20 by dialling OUT through it: the substrate broker (5671),
# the mesh's own artifact store, and — the thing this bed exists to prove — the WireGuard overlay hub
# (51820/udp). The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox
# cannot initiate to a home node at all; every home↔novox path is either the overlay or a forwarded
# port.
#
# EVERY NODE HAS EGRESS, which is not a convenience. Third-party images — postgres, the whole Mailu
# stack, Plex, everything the modules actually run — are pulled from the internet, because that is
# where a real node gets them. The lab used to serve them from a registry it raised inside the
# scenario; no production mesh has one, so a bootstrap that could only work against it went green
# here and would have failed anywhere else. What is loaded onto a machine now is only what exists in
# no registry at all: the mesh's own images, named per machine below.
#
# Egress is a SECOND path, not a replacement for the topology. Each node still reaches the rest of
# the scenario through its declared gateway — that is where the overlay handshake has to survive a
# masquerade — and the uplink carries only what leaves the scenario entirely.
#
# THE UNPROVEN THING (what the flat beds never tested): does the overlay tunnel FORM across the
# access point — a home node dialling novox's public hub endpoint, the handshake completing through
# the gateway's masquerade? The driving test verifies the WireGuard handshake and cross-segment
# reachability over the overlay explicitly, and reports form-vs-break as its headline.
#
# Substrate-on-novox collides on two host ports the separate-anchor beds never hit: the substrate
# store binds 127.0.0.1:5432 and novox's postgres provider publishes 5432; the substrate broker binds
# 5671 + 127.0.0.1:5672 and novox's lavinmq provider publishes 5672. The driving test REMAPS those two
# provider host publishes off the substrate's ports (consumers reach the providers over the mesh
# network on the container port, so the host side is free to move). Reported as a topology finding.
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# The images: are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set, and
# every one of them must already be BUILT on the workstation — nothing can fetch them.
scenario: whole-mesh-full
segments:
# The routable segment. novox lives here, and the overlay hub endpoint is a public address here.
hosting:
kind: public
cidr: [192.0.2.0/24]
# The household segment behind the access point. Its gateway is an ordinary home router: it
# masquerades v4 outbound, forwards inbound, and expires idle mappings after two minutes — which
# is exactly the NAT hole a WireGuard keepalive has to hold open.
home:
kind: private
cidr: [192.168.1.0/24]
gateway:
to: hosting
address: [192.0.2.50] # what the world sees the household as
nat: [v4]
forwardable: true
mapping_ttl: 120s
machines:
# The anchor: substrate (store, broker, control) + the whole novox service set + overlay hub +
# public ingress. Bigger than the flat bed's novox, because it now carries the substrate too.
novox:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 24GiB
cpus: 8
disk: 130GiB
# The substrate's control plane, the proxy, and a runtime per module novox is assigned.
# step-ca, photos, invoicing, novox.be, only-office, de-spiegel, amqp-email-forwarder, registry,
# firewall and fail2ban are not here because they carry no runtime image of their own — what
# they run is third-party or is the node itself.
images:
- mesh-control:development
- mesh-route-proxy:development
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-minio:development
- mesh-runtime-mongodb:development
- mesh-runtime-mssql:development
- mesh-runtime-lavinmq:development
- mesh-runtime-keycloak:development
- mesh-runtime-gitea:development
- mesh-runtime-nextcloud:development
- mesh-runtime-umami:development
- mesh-runtime-verdaccio:development
- mesh-runtime-portainer:development
- mesh-runtime-mailu:development
# The home server: the whole ace media/home set — 24 modules, ~50 containers, several heavy
# (Plex, Home Assistant, Letta, Baserow, the UniFi JVM, mssql). Behind the gateway.
ace:
at: { segment: home, address: [192.168.1.10] }
egress: true
inbound: allow
memory: 18GiB
cpus: 6
disk: 120GiB
# A runtime per module ace is assigned, and nothing of novox's. This is the point of saying it
# per machine: ace has no business holding a Keycloak runtime, and an operator's home server
# would not.
images:
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-mssql:development
- mesh-runtime-portainer:development
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
- mesh-runtime-lidarr:development
- mesh-runtime-plex:development
- mesh-runtime-bazarr:development
- mesh-runtime-nzbget:development
- mesh-runtime-qbittorrent:development
- mesh-runtime-jackett:development
- mesh-runtime-ombi:development
- mesh-runtime-tautulli:development
- mesh-runtime-bookshelf:development
- mesh-runtime-home-assistant:development
- mesh-runtime-mosquitto:development
- mesh-runtime-influxdb:development
- mesh-runtime-grafana:development
- mesh-runtime-baserow:development
- mesh-runtime-letta:development
- mesh-runtime-nodered:development
- mesh-runtime-searxng:development
- mesh-runtime-unifi:development
# Two workstations on the same home LAN. Light on purpose: they enrol, join the overlay, and run
# one small module (portainer) so a real module converges on each without heavy load. Same-LAN
# nodes with no overlay endpoint of their own hairpin the hub rather than peering directly, which
# is the normal case and is fine.
shanks:
at: { segment: home, address: [192.168.1.20] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
disk: 30GiB
# One module, one runtime. Handing these two the whole union would put roughly thirty gigabytes
# of images onto a thirty-gigabyte disk, which is how you learn that "the lab loads everything
# everywhere" was never a description of anything real.
images: [mesh-runtime-portainer:development]
g14:
at: { segment: home, address: [192.168.1.30] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
disk: 30GiB
images: [mesh-runtime-portainer:development]
# The union of what the machines above ask for. This is the list the workstation must be able to
# export — every entry is one of the mesh's own images, built from source and published nowhere, so
# a machine holds it because it was handed it. Everything else the modules run comes from the
# internet and is not named here at all.
images:
- mesh-control:development
# --- per-module runtimes (union) ---
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-mssql:development
- mesh-runtime-portainer:development
- mesh-runtime-minio:development
- mesh-runtime-mongodb:development
- mesh-runtime-lavinmq:development
- mesh-runtime-keycloak:development
- mesh-runtime-gitea:development
- mesh-runtime-nextcloud:development
- mesh-runtime-umami:development
- mesh-runtime-verdaccio:development
- mesh-runtime-mailu:development
- mesh-route-proxy:development
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
- mesh-runtime-lidarr:development
- mesh-runtime-plex:development
- mesh-runtime-bazarr:development
- mesh-runtime-nzbget:development
- mesh-runtime-qbittorrent:development
- mesh-runtime-jackett:development
- mesh-runtime-ombi:development
- mesh-runtime-tautulli:development
- mesh-runtime-bookshelf:development
- mesh-runtime-home-assistant:development
- mesh-runtime-mosquitto:development
- mesh-runtime-influxdb:development
- mesh-runtime-grafana:development
- mesh-runtime-baserow:development
- mesh-runtime-letta:development
- mesh-runtime-nodered:development
- mesh-runtime-searxng:development
- mesh-runtime-unifi:development
place:
all: [host, runtime]
Reference in New Issue View Git Blame Copy Permalink