jschoubben
94e617915c
It is the commonest home LAN range there is, so on an ordinary workstation the lab's private segment and the machine's own network are the same addresses. The scenario routes an egress machine explicitly and marks the rest unreachable, so nothing leaked — but that guard was carrying the whole weight of a collision nobody chose, and a guard is a bad place for that. 10.99.1.0/24 is still RFC 1918, so the bed still models a home LAN behind an access point. It is simply far from what this kind of machine already has: 192.168.1 is the LAN, 172.16-31 and 192.168.16-95 are container bridges, and 10.10/10.42/10.208 are a tunnel, the mesh overlay and the virtualisation daemon. Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
206 lines
9.6 KiB
YAML
206 lines
9.6 KiB
YAML
# The FULL mesh in its REAL production shape: two segments, one access point, one overlay.
|
|
#
|
|
# This is the first multi-segment whole-mesh bed. The earlier flat whole-mesh-full sat every node
|
|
# on one public segment with a SEPARATE `anchor` carrying the substrate. Production is not flat, and
|
|
# there is no separate anchor: `novox` IS the anchor. It sits on the routable `hosting` segment,
|
|
# runs the substrate (store, broker, control) AND its own service set AND is the overlay hub and the
|
|
# public ingress. `ace`, `shanks` and `g14` sit on the household `home` segment BEHIND a NAT gateway
|
|
# — the access point — reachable from the outside only through what they dial out to.
|
|
#
|
|
# hosting (public, routable) home (private, behind the access point)
|
|
# novox 192.0.2.20 ── anchor ace 10.99.1.10 home server, media/IoT set
|
|
# substrate + novox set shanks 10.99.1.20 workstation (light)
|
|
# overlay hub, ingress g14 10.99.1.30 workstation (light)
|
|
#
|
|
# The `home` gateway masquerades v4 outbound and forwards inbound (an ordinary household router).
|
|
# Home nodes reach novox's public 192.0.2.20 by dialling OUT through it: the substrate broker (5671),
|
|
# the mesh's own artifact store, and — the thing this bed exists to prove — the WireGuard overlay hub
|
|
# (51820/udp). The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox
|
|
# cannot initiate to a home node at all; every home↔novox path is either the overlay or a forwarded
|
|
# port.
|
|
#
|
|
# EVERY NODE HAS EGRESS, which is not a convenience. Third-party images — postgres, the whole Mailu
|
|
# stack, Plex, everything the modules actually run — are pulled from the internet, because that is
|
|
# where a real node gets them. The lab used to serve them from a registry it raised inside the
|
|
# scenario; no production mesh has one, so a bootstrap that could only work against it went green
|
|
# here and would have failed anywhere else. What is loaded onto a machine now is only what exists in
|
|
# no registry at all: the mesh's own images, named per machine below.
|
|
#
|
|
# Egress is a SECOND path, not a replacement for the topology. Each node still reaches the rest of
|
|
# the scenario through its declared gateway — that is where the overlay handshake has to survive a
|
|
# masquerade — and the uplink carries only what leaves the scenario entirely.
|
|
#
|
|
# THE UNPROVEN THING (what the flat beds never tested): does the overlay tunnel FORM across the
|
|
# access point — a home node dialling novox's public hub endpoint, the handshake completing through
|
|
# the gateway's masquerade? The driving test verifies the WireGuard handshake and cross-segment
|
|
# reachability over the overlay explicitly, and reports form-vs-break as its headline.
|
|
#
|
|
# Substrate-on-novox collides on two host ports the separate-anchor beds never hit: the substrate
|
|
# store binds 127.0.0.1:5432 and novox's postgres provider publishes 5432; the substrate broker binds
|
|
# 5671 + 127.0.0.1:5672 and novox's lavinmq provider publishes 5672. The driving test REMAPS those two
|
|
# provider host publishes off the substrate's ports (consumers reach the providers over the mesh
|
|
# network on the container port, so the host side is free to move). Reported as a topology finding.
|
|
#
|
|
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
|
|
# The images: are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
|
|
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set, and
|
|
# every one of them must already be BUILT on the workstation — nothing can fetch them.
|
|
scenario: whole-mesh-full
|
|
|
|
segments:
|
|
# The routable segment. novox lives here, and the overlay hub endpoint is a public address here.
|
|
hosting:
|
|
kind: public
|
|
cidr: [192.0.2.0/24]
|
|
|
|
# The household segment behind the access point. Its gateway is an ordinary home router: it
|
|
# masquerades v4 outbound, forwards inbound, and expires idle mappings after two minutes — which
|
|
# is exactly the NAT hole a WireGuard keepalive has to hold open.
|
|
home:
|
|
kind: private
|
|
cidr: [10.99.1.0/24]
|
|
gateway:
|
|
to: hosting
|
|
address: [192.0.2.50] # what the world sees the household as
|
|
nat: [v4]
|
|
forwardable: true
|
|
mapping_ttl: 120s
|
|
|
|
machines:
|
|
# The anchor: substrate (store, broker, control) + the whole novox service set + overlay hub +
|
|
# public ingress. Bigger than the flat bed's novox, because it now carries the substrate too.
|
|
novox:
|
|
at: { segment: hosting, address: [192.0.2.20] }
|
|
egress: true
|
|
inbound: allow
|
|
memory: 24GiB
|
|
cpus: 8
|
|
disk: 130GiB
|
|
# The substrate's control plane, the proxy, and a runtime per module novox is assigned.
|
|
# step-ca, photos, invoicing, novox.be, only-office, de-spiegel, amqp-email-forwarder, registry,
|
|
# firewall and fail2ban are not here because they carry no runtime image of their own — what
|
|
# they run is third-party or is the node itself.
|
|
images:
|
|
- mesh-control:development
|
|
- mesh-route-proxy:development
|
|
- mesh-runtime-postgres:development
|
|
- mesh-runtime-redis:development
|
|
- mesh-runtime-minio:development
|
|
- mesh-runtime-mongodb:development
|
|
- mesh-runtime-mssql:development
|
|
- mesh-runtime-lavinmq:development
|
|
- mesh-runtime-keycloak:development
|
|
- mesh-runtime-gitea:development
|
|
- mesh-runtime-nextcloud:development
|
|
- mesh-runtime-umami:development
|
|
- mesh-runtime-verdaccio:development
|
|
- mesh-runtime-portainer:development
|
|
- mesh-runtime-mailu:development
|
|
|
|
# The home server: the whole ace media/home set — 24 modules, ~50 containers, several heavy
|
|
# (Plex, Home Assistant, Letta, Baserow, the UniFi JVM, mssql). Behind the gateway.
|
|
ace:
|
|
at: { segment: home, address: [10.99.1.10] }
|
|
egress: true
|
|
inbound: allow
|
|
memory: 18GiB
|
|
cpus: 6
|
|
disk: 120GiB
|
|
# A runtime per module ace is assigned, and nothing of novox's. This is the point of saying it
|
|
# per machine: ace has no business holding a Keycloak runtime, and an operator's home server
|
|
# would not.
|
|
images:
|
|
- mesh-runtime-postgres:development
|
|
- mesh-runtime-redis:development
|
|
- mesh-runtime-mssql:development
|
|
- mesh-runtime-portainer:development
|
|
- mesh-runtime-sonarr:development
|
|
- mesh-runtime-radarr:development
|
|
- mesh-runtime-lidarr:development
|
|
- mesh-runtime-plex:development
|
|
- mesh-runtime-bazarr:development
|
|
- mesh-runtime-nzbget:development
|
|
- mesh-runtime-qbittorrent:development
|
|
- mesh-runtime-jackett:development
|
|
- mesh-runtime-ombi:development
|
|
- mesh-runtime-tautulli:development
|
|
- mesh-runtime-bookshelf:development
|
|
- mesh-runtime-home-assistant:development
|
|
- mesh-runtime-mosquitto:development
|
|
- mesh-runtime-influxdb:development
|
|
- mesh-runtime-grafana:development
|
|
- mesh-runtime-baserow:development
|
|
- mesh-runtime-letta:development
|
|
- mesh-runtime-nodered:development
|
|
- mesh-runtime-searxng:development
|
|
- mesh-runtime-unifi:development
|
|
|
|
# Two workstations on the same home LAN. Light on purpose: they enrol, join the overlay, and run
|
|
# one small module (portainer) so a real module converges on each without heavy load. Same-LAN
|
|
# nodes with no overlay endpoint of their own hairpin the hub rather than peering directly, which
|
|
# is the normal case and is fine.
|
|
shanks:
|
|
at: { segment: home, address: [10.99.1.20] }
|
|
egress: true
|
|
inbound: allow
|
|
memory: 3GiB
|
|
cpus: 2
|
|
disk: 30GiB
|
|
# One module, one runtime. Handing these two the whole union would put roughly thirty gigabytes
|
|
# of images onto a thirty-gigabyte disk, which is how you learn that "the lab loads everything
|
|
# everywhere" was never a description of anything real.
|
|
images: [mesh-runtime-portainer:development]
|
|
g14:
|
|
at: { segment: home, address: [10.99.1.30] }
|
|
egress: true
|
|
inbound: allow
|
|
memory: 3GiB
|
|
cpus: 2
|
|
disk: 30GiB
|
|
images: [mesh-runtime-portainer:development]
|
|
|
|
# The union of what the machines above ask for. This is the list the workstation must be able to
|
|
# export — every entry is one of the mesh's own images, built from source and published nowhere, so
|
|
# a machine holds it because it was handed it. Everything else the modules run comes from the
|
|
# internet and is not named here at all.
|
|
images:
|
|
- mesh-control:development
|
|
# --- per-module runtimes (union) ---
|
|
- mesh-runtime-postgres:development
|
|
- mesh-runtime-redis:development
|
|
- mesh-runtime-mssql:development
|
|
- mesh-runtime-portainer:development
|
|
- mesh-runtime-minio:development
|
|
- mesh-runtime-mongodb:development
|
|
- mesh-runtime-lavinmq:development
|
|
- mesh-runtime-keycloak:development
|
|
- mesh-runtime-gitea:development
|
|
- mesh-runtime-nextcloud:development
|
|
- mesh-runtime-umami:development
|
|
- mesh-runtime-verdaccio:development
|
|
- mesh-runtime-mailu:development
|
|
- mesh-route-proxy:development
|
|
- mesh-runtime-sonarr:development
|
|
- mesh-runtime-radarr:development
|
|
- mesh-runtime-lidarr:development
|
|
- mesh-runtime-plex:development
|
|
- mesh-runtime-bazarr:development
|
|
- mesh-runtime-nzbget:development
|
|
- mesh-runtime-qbittorrent:development
|
|
- mesh-runtime-jackett:development
|
|
- mesh-runtime-ombi:development
|
|
- mesh-runtime-tautulli:development
|
|
- mesh-runtime-bookshelf:development
|
|
- mesh-runtime-home-assistant:development
|
|
- mesh-runtime-mosquitto:development
|
|
- mesh-runtime-influxdb:development
|
|
- mesh-runtime-grafana:development
|
|
- mesh-runtime-baserow:development
|
|
- mesh-runtime-letta:development
|
|
- mesh-runtime-nodered:development
|
|
- mesh-runtime-searxng:development
|
|
- mesh-runtime-unifi:development
|
|
|
|
place:
|
|
all: [host, runtime]
|