mesh-lab/scenarios/whole-mesh-full.yml

# The FULL mesh in its REAL production shape: two segments, one access point, one overlay.
#
# This is the first multi-segment whole-mesh bed. The earlier flat whole-mesh-full sat every node
# on one public segment with a SEPARATE `anchor` carrying the foundation. Production is not flat, and
# there is no separate anchor: `novox` IS the anchor. It sits on the routable `hosting` segment,
# runs the foundation (store, broker, control) AND its own service set AND is the overlay hub and the
# public ingress. `ace`, `shanks` and `g14` sit on the household `home` segment BEHIND a NAT gateway
# — the access point — reachable from the outside only through what they dial out to.
#
#   hosting (public, routable)         home (private, behind the access point)
#     novox  192.0.2.20  ── anchor       ace     10.99.1.10  home server, media/IoT set
#              foundation + novox set     shanks  10.99.1.20  workstation (light)
#              overlay hub, ingress      g14     10.99.1.30  workstation (light)
#
# The `home` gateway masquerades v4 outbound and forwards inbound (an ordinary household router).
# Home nodes reach novox's public 192.0.2.20 by dialling OUT through it: the foundation broker (5671),
# the mesh's own artifact store, and — the thing this bed exists to prove — the WireGuard overlay hub
# (51820/udp). The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox
# cannot initiate to a home node at all; every home↔novox path is either the overlay or a forwarded
# port.
#
# EVERY NODE HAS EGRESS, which is not a convenience. Third-party images — postgres, the whole Mailu
# stack, Plex, everything the modules actually run — are pulled from the internet, because that is
# where a real node gets them. The lab used to serve them from a registry it raised inside the
# scenario; no production mesh has one, so a bootstrap that could only work against it went green
# here and would have failed anywhere else. What is loaded onto a machine now is only what exists in
# no registry at all: the mesh's own images, named per machine below.
#
# Egress is a SECOND path, not a replacement for the topology. Each node still reaches the rest of
# the scenario through its declared gateway — that is where the overlay handshake has to survive a
# masquerade — and the uplink carries only what leaves the scenario entirely.
#
# THE UNPROVEN THING (what the flat beds never tested): does the overlay tunnel FORM across the
# access point — a home node dialling novox's public hub endpoint, the handshake completing through
# the gateway's masquerade? The driving test verifies the WireGuard handshake and cross-segment
# reachability over the overlay explicitly, and reports form-vs-break as its headline.
#
# Foundation-on-novox collides on two host ports the separate-anchor beds never hit: the foundation
# store binds 127.0.0.1:5432 and novox's postgres provider publishes 5432; the foundation broker binds
# 5671 + 127.0.0.1:5672 and novox's lavinmq provider publishes 5672. The driving test REMAPS those two
# provider host publishes off the foundation's ports (consumers reach the providers over the mesh
# network on the container port, so the host side is free to move). Reported as a topology finding.
#
#   MESH_LAB_HOST_BINARY=.../mesh-host   MESH_LAB_BUNDLE=.../examples/foundation-first-node.lock
#   MESH_LAB_BOOTSTRAP_BINARY=.../mesh-bootstrap   MESH_LAB_CATALOG=.../mesh-catalog/modules
#
# GENESIS AND JOINING ARE TWO DIFFERENT ACTS, and this bed distinguishes them. novox is brought
# into existence by `mesh-bootstrap` — the same program a bare machine runs — and is afterwards a
# working mesh of one, with a registry and a control plane that is an ordinary module pinned to an
# image that registry serves. ace, shanks and g14 then JOIN it: host binary, token, enrol, run. No
# bootstrap, no foundation, no registry. novox is never enrolled twice, because the installer
# already did it.
#
# The images: are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set, and
# every one of them must already be BUILT on the workstation — nothing can fetch them.
scenario: whole-mesh-full

segments:
  # The routable segment. novox lives here, and the overlay hub endpoint is a public address here.
  hosting:
    kind: public
    cidr: [192.0.2.0/24]

  # The household segment behind the access point. Its gateway is an ordinary home router: it
  # masquerades v4 outbound, forwards inbound, and expires idle mappings after two minutes — which
  # is exactly the NAT hole a WireGuard keepalive has to hold open.
  home:
    kind: private
    cidr: [10.99.1.0/24]
    gateway:
      to: hosting
      address: [192.0.2.50]       # what the world sees the household as
      nat: [v4]
      forwardable: true
      mapping_ttl: 120s

machines:
  # The anchor: foundation (store, broker, control) + the whole novox service set + overlay hub +
  # public ingress. Bigger than the flat bed's novox, because it now carries the foundation too.
  novox:
    at: { segment: hosting, address: [192.0.2.20] }
    egress: true
    inbound: allow
    memory: 24GiB
    cpus: 8
    disk: 130GiB
    # The proxy, and a runtime per module novox is assigned. step-ca, photos, invoicing, novox.be,
    # only-office, de-spiegel, amqp-email-forwarder, registry, firewall and fail2ban are not here
    # because they carry no runtime image of their own — what they run is third-party or is the
    # node itself.
    #
    # **mesh-controller is NOT here, and its absence is the point** (novox/hq ADR 0067). The anchor is
    # brought into existence by the installer, and the installer carries the control plane's image
    # inside itself — that is the whole reason a machine that can reach no registry can still raise
    # a mesh. Handing it over from the workstation as well would mean the bed never found out
    # whether the installer really carries it: the load would say "already held" and the fiction
    # would be invisible, which is exactly the class of thing the lab's own registry used to hide.
    images:
      - mesh-route-proxy:development
      - mesh-runtime-postgres:development
      - mesh-runtime-redis:development
      - mesh-runtime-minio:development
      - mesh-runtime-mongodb:development
      - mesh-runtime-mssql:development
      - mesh-runtime-lavinmq:development
      - mesh-runtime-keycloak:development
      - mesh-runtime-gitea:development
      - mesh-runtime-nextcloud:development
      - mesh-runtime-umami:development
      - mesh-runtime-verdaccio:development
      - mesh-runtime-portainer:development
      - mesh-runtime-mailu:development

  # The home server: the whole ace media/home set — 24 modules, ~50 containers, several heavy
  # (Plex, Home Assistant, Letta, Baserow, the UniFi JVM, mssql). Behind the gateway.
  ace:
    at: { segment: home, address: [10.99.1.10] }
    egress: true
    inbound: allow
    memory: 18GiB
    cpus: 6
    disk: 120GiB
    # A runtime per module ace is assigned, and nothing of novox's. This is the point of saying it
    # per machine: ace has no business holding a Keycloak runtime, and an operator's home server
    # would not.
    images:
      - mesh-runtime-postgres:development
      - mesh-runtime-redis:development
      - mesh-runtime-mssql:development
      - mesh-runtime-portainer:development
      - mesh-runtime-sonarr:development
      - mesh-runtime-radarr:development
      - mesh-runtime-lidarr:development
      - mesh-runtime-plex:development
      - mesh-runtime-bazarr:development
      - mesh-runtime-nzbget:development
      - mesh-runtime-qbittorrent:development
      - mesh-runtime-jackett:development
      - mesh-runtime-ombi:development
      - mesh-runtime-tautulli:development
      - mesh-runtime-bookshelf:development
      - mesh-runtime-home-assistant:development
      - mesh-runtime-mosquitto:development
      - mesh-runtime-influxdb:development
      - mesh-runtime-grafana:development
      - mesh-runtime-baserow:development
      - mesh-runtime-letta:development
      - mesh-runtime-nodered:development
      - mesh-runtime-searxng:development
      - mesh-runtime-unifi:development

  # Two workstations on the same home LAN. Light on purpose: they enrol, join the overlay, and run
  # one small module (portainer) so a real module converges on each without heavy load. Same-LAN
  # nodes with no overlay endpoint of their own hairpin the hub rather than peering directly, which
  # is the normal case and is fine.
  shanks:
    at: { segment: home, address: [10.99.1.20] }
    egress: true
    inbound: allow
    memory: 3GiB
    cpus: 2
    disk: 30GiB
    # One module, one runtime. Handing these two the whole union would put roughly thirty gigabytes
    # of images onto a thirty-gigabyte disk, which is how you learn that "the lab loads everything
    # everywhere" was never a description of anything real.
    images: [mesh-runtime-portainer:development]
  g14:
    at: { segment: home, address: [10.99.1.30] }
    egress: true
    inbound: allow
    memory: 3GiB
    cpus: 2
    disk: 30GiB
    images: [mesh-runtime-portainer:development]

# The union of what the machines above ask for. This is the list the workstation must be able to
# export — every entry is one of the mesh's own images, built from source and published nowhere, so
# a machine holds it because it was handed it. Everything else the modules run comes from the
# internet and is not named here at all.
images:
  # --- per-module runtimes (union) ---
  - mesh-runtime-postgres:development
  - mesh-runtime-redis:development
  - mesh-runtime-mssql:development
  - mesh-runtime-portainer:development
  - mesh-runtime-minio:development
  - mesh-runtime-mongodb:development
  - mesh-runtime-lavinmq:development
  - mesh-runtime-keycloak:development
  - mesh-runtime-gitea:development
  - mesh-runtime-nextcloud:development
  - mesh-runtime-umami:development
  - mesh-runtime-verdaccio:development
  - mesh-runtime-mailu:development
  - mesh-route-proxy:development
  - mesh-runtime-sonarr:development
  - mesh-runtime-radarr:development
  - mesh-runtime-lidarr:development
  - mesh-runtime-plex:development
  - mesh-runtime-bazarr:development
  - mesh-runtime-nzbget:development
  - mesh-runtime-qbittorrent:development
  - mesh-runtime-jackett:development
  - mesh-runtime-ombi:development
  - mesh-runtime-tautulli:development
  - mesh-runtime-bookshelf:development
  - mesh-runtime-home-assistant:development
  - mesh-runtime-mosquitto:development
  - mesh-runtime-influxdb:development
  - mesh-runtime-grafana:development
  - mesh-runtime-baserow:development
  - mesh-runtime-letta:development
  - mesh-runtime-nodered:development
  - mesh-runtime-searxng:development
  - mesh-runtime-unifi:development

place:
  all: [host, runtime]