novox/mesh-lab
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Scenarios: egress where images are needed, and images: cut to what is ours

Browse Source 
Every scenario that places a container runtime gives each of its machines
`egress: true` — a node that runs modules pulls images from the internet, which is what
a node does. The underlay-only scenarios (bootstrap-single, behind-nat,
segmented-and-unforwardable, the-ordinary-shape, two-on-a-segment) stay sealed on
purpose: an extra NIC would change the very reachability they are asserting about.

`images:` keeps only the mesh's own — 55 third-party entries leave whole-mesh-full
alone, and the machine fetches them itself by the digest its module.json already pins.
The whole-mesh beds also say per machine which of ours they get: novox the substrate
control plane and its own fifteen runtimes, ace its twenty-four, the two workstations
one each. That is not a lab economy. An operator's workstation holds the images its own
modules need, and giving these two the union would put some thirty gigabytes onto a
thirty-gigabyte disk.

bootstrap-with-registry.yml is deleted. It existed only to demonstrate the lab's
registry, nothing referenced it, and there is nothing left for it to demonstrate.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
This commit is contained in:
jschoubben
2026-09-10 23:16:23 +02:00
parent 751948f0f9
commit 5c91c0ecd2
32 changed files with 208 additions and 310 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scenarios/a-provider.yml
+1 -3
View File
@@ -13,10 +13,8 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
images:
- postgres:17-alpine
place:
all: [runtime]
scenarios/a-public-name.yml
+1 -3
View File
@@ -17,10 +17,8 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
images:
- ghcr.io/letsencrypt/pebble:2.5.0
place:
all: [runtime]
scenarios/an-object-store.yml
+1 -6
View File
@@ -19,13 +19,8 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
images:
- minio/minio:RELEASE.2025-09-07T16-13-09Z
# The vendor's client, stocked so the provisioner has the thing it drives without reaching a
# public registry from a documentation range.
- minio/mc:RELEASE.2025-08-13T08-35-41Z
place:
all: [runtime]
scenarios/anthropic-bed.yml
+2 -4
View File
@@ -29,17 +29,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The two model-access runtimes, built by scripts/build-module-runtime.sh into the local daemon and
# stocked into the scenario's own registry, which is where the host pulls them from.
# loaded onto the machine, which holds them by their own image IDs.
- mesh-runtime-anthropic-manager:development
- mesh-runtime-anthropic-consumer:development
scenarios/audit-node.yml
+2 -3
View File
@@ -14,16 +14,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The tool runtime with the audit-logger, built by scripts/build-runtime-image.sh into the local
# daemon and stocked into the scenario's own registry, which is where the host pulls it from.
# daemon and loaded onto the machine, which holds it by its own image ID.
- mesh-runtime-audit:development
place:
scenarios/bootstrap-with-registry.yml
-23
View File
@@ -1,23 +0,0 @@
# One machine and a registry, which is the smallest scenario that can exercise a container.
#
# A sealed machine cannot reach a registry and an image placed from an archive cannot keep its
# digest (novox/hq 04-ISSUES/009), so the lab raises one inside the scenario and serves the
# images below from it. What a declaration pins is reported when this is raised — the digest
# belongs to this registry, not to the one the image came from.
scenario: bootstrap-with-registry
segments:
hosting:
kind: public
cidr: [192.0.2.0/24]
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
inbound: allow
images:
- alpine:3.20
place:
all: [host, runtime]
scenarios/catalogue-apps.yml
+1 -7
View File
@@ -28,6 +28,7 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
# Sized up past catalogue-small's 6GiB: this wave carries two heavy JVM/embedded-DB service
# containers (the UniFi controller and MaryTTS) on top of the substrate, mongodb, postgres and
@@ -36,14 +37,7 @@ machines:
cpus: 4
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module server images.
- mongo:7
- lscr.io/linuxserver/unifi-controller:latest
- synesthesiam/marytts:lab
# The runtimes built by scripts/build-module-runtime.sh and stocked here. marrytts needs none.
- mesh-runtime-mongodb:development
- mesh-runtime-unifi:development
scenarios/catalogue-media.yml
+1 -6
View File
@@ -30,6 +30,7 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
# Two *arr apps (server + runtime each) on top of the first-node substrate — seven containers.
# The Servarr images are lighter than catalogue-apps' JVM pair, so catalogue-small's 6GiB is
@@ -38,13 +39,7 @@ machines:
cpus: 4
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module server images.
- lscr.io/linuxserver/sonarr:latest
- lscr.io/linuxserver/radarr:latest
# The two runtimes built by scripts/build-module-runtime.sh and stocked here.
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
scenarios/catalogue-mqtt.yml
+2 -9
View File
@@ -14,7 +14,7 @@
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# scripts/build-module-runtime.sh mosquitto builds mesh-runtime-mosquitto:development (carrying
# mosquitto_ctrl and the compiled bootstrap entrypoint) into the local daemon, which this scenario
# stocks and serves by digest from its own registry. eclipse-mosquitto:2 must be in the local
# pulls from the internet over its uplink. eclipse-mosquitto:2 must be in the local
# daemon to be stocked.
scenario: catalogue-mqtt
@@ -26,20 +26,13 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# mosquitto's broker (the service image) and its runtime, the latter built by
# scripts/build-module-runtime.sh mosquitto into the local daemon and stocked into the scenario's
# own registry, which is where the host pulls it from. The runtime image is reused for the
# run-once bootstrap step and for the serve container.
- eclipse-mosquitto:2
- mesh-runtime-mosquitto:development
place:
scenarios/catalogue-small.yml
+2 -7
View File
@@ -13,7 +13,7 @@
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# scripts/build-module-runtime.sh {postgres,redis,minio,plex} build the four runtime images into the
# local daemon (postgres carries psql, minio carries mc), which this scenario stocks and serves by
# digest from its own registry. The service images must be in the local daemon to be stocked.
# the internet over its uplink. Only the mesh's own images come from the local daemon.
scenario: catalogue-small
segments:
@@ -24,6 +24,7 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
# Sized up: this anchor runs the substrate (store, broker, control) plus four modules — three of
# which are a server container and a runtime container each — so a dozen containers at once. The
@@ -32,13 +33,7 @@ machines:
cpus: 4
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module server images.
- redis:7-alpine
- minio/minio:latest
# The four per-module runtimes, built by scripts/build-module-runtime.sh and stocked here. plex
# needs no server image in the lab — its runtime serves tools with no Plex to reach.
- mesh-runtime-postgres:development
scenarios/first-node.yml
+8 -7
View File
@@ -1,8 +1,8 @@
# One machine, raising a substrate from the bundle its host carries.
#
# This is the bootstrap class (novox/hq ADR 0009): no forge, no control plane to talk to, no
# delivery. It exists to develop the steps of raising a mesh on a machine with no route out —
# a container runtime, a store, the control plane's schema in it, and the broker.
# delivery. It exists to develop the steps of raising a mesh on a machine that has nothing but a
# connection — a container runtime, a store, the control plane's schema in it, and the broker.
#
# It stops before the control plane *runs*, because there is nothing for it to serve yet.
scenario: first-node
@@ -15,14 +15,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
# Placed into a registry the scenario raises, which is what a real node pulls from anyway. The
# digests below are the ones that registry assigns, and that satisfies pinning: what is required
# is a reference that is exact and cannot move (novox/hq ADR 0006).
# The control plane's own image exists in no registry — it is built from source, and until this
# mesh has a registry of its own there is nowhere to have pushed it. So it is loaded onto the
# machine and named by the digest of its own configuration, which is exact and cannot move, which
# is what pinning asks for (novox/hq ADR 0006). The store and the broker are ordinary third-party
# images, and the machine pulls them from the internet like anything else.
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
place:
scenarios/grafana-node.yml
+1 -2
View File
@@ -14,13 +14,12 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
- mesh-runtime-grafana:development
scenarios/growing-mesh.yml
+3 -2
View File
@@ -16,17 +16,18 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
laptop:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
workstation:
at: { segment: hosting, address: [192.0.2.30] }
egress: true
inbound: allow
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
place:
scenarios/lavinmq-bed.yml
+5 -6
View File
@@ -22,8 +22,8 @@
# scripts/build-module-runtime.sh lavinmq /tmp/lavinmq.tar
# scripts/build-module-runtime.sh amqp-ping /tmp/amqp-ping.tar
# The service image cloudamqp/lavinmq:latest must be in the local daemon too — it is already stocked as
# the substrate's own broker image; each node pulls what it runs from the scenario's own registry by
# digest.
# the substrate's own broker image; each node pulls what it runs from the internet, over its own
# uplink.
scenario: lavinmq-bed
segments:
@@ -34,23 +34,22 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
laptop:
at: { segment: hosting, address: [192.0.2.11] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker (itself lavinmq), control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The lavinmq provider's runtime (reused for its run-once bootstrap and its provisioner) and the
# amqp-ping consumer's runtime, both built by scripts/build-module-runtime.sh into the local daemon
# and stocked into the scenario's own registry, which is where the hosts pull them from by digest.
# and loaded onto the machines, which hold them by their own image IDs.
# The lavinmq SERVICE image is cloudamqp/lavinmq:latest, already stocked above.
- mesh-runtime-lavinmq:development
- mesh-runtime-amqp-ping:development
scenarios/local-model-bed.yml
+1 -7
View File
@@ -24,20 +24,14 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
disk: 40GiB
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The local model server the ollama provider runs. Pulled by raise() and stocked into the scenario's
# own registry, which is where the host pulls it from. No model is pulled into it — the bed proves the
# mesh routes a consumer to the server's endpoint, not that the server generates tokens.
- ollama/ollama:latest
place:
all: [host, runtime]
scenarios/minio-node.yml
+3 -5
View File
@@ -15,17 +15,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
- minio/minio:latest
# minio's runtime, built by scripts/build-module-runtime.sh minio (it carries mc), stocked into the
# scenario's own registry.
# minio's runtime, built by scripts/build-module-runtime.sh minio (it carries mc), loaded onto
# the machine.
- mesh-runtime-minio:development
place:
scenarios/model-usage-bed.yml
+3 -5
View File
@@ -25,25 +25,23 @@ machines:
# The substrate ONLY: store, broker, control — three containers.
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
# The postgres PROVIDER (server + broker-bound runtime) and the model-usage CONSUMER (its run-once
# migrate and its long-lived event runtime). The 5432-vs-substrate conflict is gone because the
# substrate store is on the OTHER node. The runtime images plus postgres:17-alpine are pulled from
# the scenario's own registry by digest; forty gigabytes holds them with room to spare.
# the internet over the uplink; forty gigabytes holds them with room to spare.
laptop:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
disk: 40GiB
images:
# The first-node substrate: store, broker, control. postgres:17-alpine doubles as postgres's own
# service image.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The per-module runtimes, built by scripts/build-module-runtime.sh and stocked here. Each carries
# its module's code — postgres its provisioner, model-usage its consumer, tools and run-once migrate.
scenarios/openai-bed.yml
+2 -4
View File
@@ -24,17 +24,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The static-key consumer runtime, built by scripts/build-module-runtime.sh into the local daemon and
# stocked into the scenario's own registry, which is where the host pulls it from.
# loaded onto the machine, which holds it by its own image ID.
- mesh-runtime-openai-consumer:development
place:
scenarios/plex-node.yml
+2 -3
View File
@@ -15,16 +15,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# Plex's tool runtime, built by scripts/build-module-runtime.sh plex into the local daemon and
# stocked into the scenario's own registry, which is where the host pulls it from.
# loaded onto the machine, which holds it by its own image ID.
- mesh-runtime-plex:development
place:
scenarios/postgres-node.yml
+3 -4
View File
@@ -14,16 +14,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# postgres's runtime, built by scripts/build-module-runtime.sh postgres (it carries psql), stocked
# into the scenario's own registry.
# postgres's runtime, built by scripts/build-module-runtime.sh postgres (it carries psql), loaded
# onto the machine.
- mesh-runtime-postgres:development
place:
scenarios/redis-node.yml
+2 -4
View File
@@ -14,17 +14,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
- redis:7-alpine
# Redis's tool+provisioner runtime, built by scripts/build-module-runtime.sh redis into the local
# daemon and stocked into the scenario's own registry, which is where the host pulls it from.
# daemon and loaded onto the machine, which holds it by its own image ID.
- mesh-runtime-redis:development
place:
scenarios/route-forwarding.yml
+2 -5
View File
@@ -29,21 +29,18 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The route-proxy's image, built from the canonical Go proxy in mesh-control by
# scripts/build-route-proxy-image.sh, and hello-web's backend, a bare alpine nc loop.
- mesh-route-proxy:development
- alpine:latest
place:
# Only the host — neither module carries a mesh-runtime. Both service images are served by the
# scenario's registry and pulled by the host, not placed inside the machine.
# internet and pulled by the host over its uplink, not placed inside the machine.
all: [host]
scenarios/schedule-tick.yml
+5 -11
View File
@@ -15,8 +15,8 @@
# - it fires AGAIN on the following minute — recurrence, not a one-shot.
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_MODULES=.../mesh-control/examples/modules
# alpine:latest must be in the local daemon; the scenario stocks it into its own registry and
# serves it by digest, which is what the scheduled container declares (via pinned("alpine")).
# alpine:latest must be in the local daemon; the machine pulls it from the internet over its
# uplink, and the scheduled container declares it exactly as the catalogue writes it.
# There is no runtime image: schedtest carries no code of its own — the scheduled container is a
# bare alpine that runs `date >> /data/runs.log` and exits.
scenario: schedule-tick
@@ -29,21 +29,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The tick container's image. schedtest has no runtime of its own — its scheduled container is a
# bare alpine that appends a timestamp and exits. alpine:latest must be in the local daemon; the
# scenario stocks it and serves it by digest, which is what the manifest pins via pinned("alpine").
- alpine:latest
place:
# Only the host — schedtest has no mesh-runtime to place. The tick image is served by the
# scenario's registry and pulled by the host, not placed inside the machine.
# Only the host — schedtest has no mesh-runtime to place. The tick image is pulled from the
# internet by the host over its uplink, not placed inside the machine.
all: [host]
scenarios/sonarr-node.yml
+1 -2
View File
@@ -14,13 +14,12 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
- mesh-runtime-sonarr:development
scenarios/tools-confluence.yml
+3 -5
View File
@@ -11,7 +11,7 @@
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# scripts/build-module-runtime.sh confluence builds mesh-runtime-confluence:development into the
# local daemon, which this scenario stocks and serves by digest from its own registry. confluence
# local daemon, which the machine pulls from the internet over its uplink. confluence
# needs no service image — it is tools-only.
scenario: tools-confluence
@@ -23,17 +23,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# confluence's runtime, built by scripts/build-module-runtime.sh confluence into the local daemon
# and stocked into the scenario's own registry, which is where the host pulls it from. There is no
# and loaded onto the machine, which holds it by its own image ID. There is no
# service image: confluence is tools-only and outbound-only.
- mesh-runtime-confluence:development
scenarios/tools-gitlab.yml
+3 -5
View File
@@ -12,7 +12,7 @@
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# scripts/build-module-runtime.sh gitlab builds mesh-runtime-gitlab:development into the local
# daemon, which this scenario stocks and serves by digest from its own registry. gitlab needs no
# daemon, which the machine pulls from the internet over its uplink. gitlab needs no
# service image — it is tools-only.
scenario: tools-gitlab
@@ -24,17 +24,15 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
images:
# The first-node substrate: store, broker, control.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# gitlab's runtime, built by scripts/build-module-runtime.sh gitlab into the local daemon and
# stocked into the scenario's own registry, which is where the host pulls it from. There is no
# loaded onto the machine, which holds it by its own image ID. There is no
# service image: gitlab is tools-only and outbound-only.
- mesh-runtime-gitlab:development
scenarios/two-node-db.yml
+3 -9
View File
@@ -21,6 +21,7 @@ machines:
# containers on a node, which this one never does.
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
@@ -31,25 +32,18 @@ machines:
# so — and convergence would present as "the mesh hangs". Six gigabytes gives it room.
laptop:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 6GiB
cpus: 4
# The runtime images (280MB–600MB each) plus the service images — two of them heavy app images
# (Baserow ~1.5GB, Letta ~1.8GB) — are pulled from the scenario's own registry by digest, so the
# (Baserow ~1.5GB, Letta ~1.8GB) — are pulled from the internet over the uplink, so the
# same bytes land on this node twice. The pool default root disk exhausts mid-apply ("no space
# left on device"); sixty gigabytes holds the whole chain.
disk: 60GiB
images:
# The first-node substrate: store, broker, control. postgres:17-alpine doubles as postgres's own
# service image.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module service images.
- redis:7-alpine
- baserow/baserow:latest
- letta/letta:latest
# The per-module runtimes, built by scripts/build-module-runtime.sh and stocked here. Each carries
# its module's provisioner, so no separate mesh-provision-* image is listed — the runtime is the
# provisioner (ADR 0048).
scenarios/two-nodes.yml
+2 -17
View File
@@ -15,6 +15,7 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
# The whole substrate, the registry, the builder, an adopted workload and the modules under
# test all land here — eleven containers before the forge arrives. At the 1GiB default this
@@ -24,21 +25,12 @@ machines:
cpus: 4
laptop:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 2GiB
images:
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# So a module can mirror one into a registry of the mesh's own. The scenario's registry serves
# what the mesh's registry is built from — the same chicken-and-egg the bootstrap has, resolved
# the same way.
- registry:2
# A real third-party workload, for adopting one the way the conversion will. Its database is
# the substrate's postgres image rather than its own: what is under test is the mesh delivering
# a module, not which postgres it delivers.
- ghcr.io/umami-software/umami:postgresql-latest
# And the builder, because it is a module the mesh assigns rather than a program somebody
# starts by hand — which is the only way its credential can be one the mesh delivered.
- mesh-builder:development
@@ -47,17 +39,10 @@ images:
- mesh-provision-postgres:development
# And the proxy, which is what turns a route grant into traffic actually arriving.
- mesh-route-proxy:development
# And the cache, with its provisioner — the third provision after a database and a bucket,
# and the first whose tenancy is a keyspace rather than a namespace something else enforces.
- redis:7-alpine
- mesh-provision-redis:development
# And the object store's provisioner, so the module describing it can be planned. Without it
# that module still names an image nothing serves, and planning it is refused — correctly.
- mesh-provision-objectstore:development
# And a forge, so one of the real module descriptions can be started rather than only planned.
# It is the first of them to run: it needs a database from another module, a credential it did
# not choose, and a connection string it could not have written itself.
- gitea/gitea:1.22
place:
all: [host, runtime]
scenarios/whole-mesh-ace.yml
+36 -34
View File
@@ -10,11 +10,11 @@
# the mesh confirms the paths exist and mounts them, but creates and chowns none of it.
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# The runtimes are built by scripts/build-module-runtime.sh (one per module); every server image must
# be in the local daemon to be stocked. The media/app images are pulled by their pinned digests and
# tagged :mesh so repositoryFor matches the module.json paths (postgres/redis/portainer/mssql reuse
# their existing local tags). The test loads each committed module.json from mesh-catalog and rewrites
# its image references to what this scenario's own registry serves by digest.
# The runtimes are built by scripts/build-module-runtime.sh (one per module) and must be in the local
# daemon, because nothing serves them and nothing can. Every media/app image is pulled from the
# internet by the node itself, over its uplink, by the digest its module.json already pins. The test
# loads each committed module.json from mesh-catalog and rewrites only OUR image references, to the
# ID the machine holds each one under.
scenario: whole-mesh-ace
segments:
@@ -25,50 +25,52 @@ segments:
machines:
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
disk: 20GiB
# The substrate only, so the control plane's image only. The runtimes belong on the node that
# runs the modules, and a 20GiB disk has no room for them anyway.
images: [mesh-control:development]
# The whole ace service set — 24 modules, ~50 containers, several heavy (Plex, Home Assistant,
# Letta ~1.8GiB, Baserow ~1.5GiB, the UniFi controller's JVM, mssql ~2GiB). Sized past novox.
ace:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 18GiB
cpus: 8
disk: 120GiB
# Every runtime. Not mesh-control: the control plane runs on the anchor.
images:
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-mssql:development
- mesh-runtime-portainer:development
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
- mesh-runtime-lidarr:development
- mesh-runtime-plex:development
- mesh-runtime-bazarr:development
- mesh-runtime-nzbget:development
- mesh-runtime-qbittorrent:development
- mesh-runtime-jackett:development
- mesh-runtime-ombi:development
- mesh-runtime-tautulli:development
- mesh-runtime-bookshelf:development
- mesh-runtime-home-assistant:development
- mesh-runtime-mosquitto:development
- mesh-runtime-influxdb:development
- mesh-runtime-grafana:development
- mesh-runtime-baserow:development
- mesh-runtime-letta:development
- mesh-runtime-nodered:development
- mesh-runtime-searxng:development
- mesh-runtime-unifi:development
images:
# The first-node substrate.
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module server images. Reused local tags where the exact version does not matter for a boot
# (postgres/redis/portainer/mssql); pinned-digest :mesh tags for the media/app images.
- redis:7-alpine
- lscr.io/linuxserver/sonarr:mesh
- lscr.io/linuxserver/radarr:mesh
- lscr.io/linuxserver/lidarr:mesh
- lscr.io/linuxserver/bazarr:mesh
- lscr.io/linuxserver/nzbget:mesh
- lscr.io/linuxserver/qbittorrent:mesh
- lscr.io/linuxserver/jackett:mesh
- lscr.io/linuxserver/ombi:mesh
- lscr.io/linuxserver/tautulli:mesh
- lscr.io/linuxserver/unifi-controller:mesh
- plexinc/pms-docker:mesh
- ghcr.io/pennydreadful/bookshelf:mesh
- ghcr.io/home-assistant/home-assistant:mesh
- eclipse-mosquitto:mesh
- influxdb:mesh
- grafana/grafana:mesh
- baserow/baserow:mesh
- letta/letta:mesh
- nodered/node-red:mesh
- searxng/searxng:mesh
- valkey/valkey:mesh
- portainer/portainer-ce:latest
- mcr.microsoft.com/mssql/server:2022-latest
# The per-module runtimes (built by scripts/build-module-runtime.sh).
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
scenarios/whole-mesh-full.yml
+80 -69
View File
@@ -14,9 +14,21 @@
#
# The `home` gateway masquerades v4 outbound and forwards inbound (an ordinary household router).
# Home nodes reach novox's public 192.0.2.20 by dialling OUT through it: the substrate broker (5671),
# the registry, and — the thing this bed exists to prove — the WireGuard overlay hub (51820/udp).
# The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox cannot
# initiate to a home node at all; every home↔novox path is either the overlay or a forwarded port.
# the mesh's own artifact store, and — the thing this bed exists to prove — the WireGuard overlay hub
# (51820/udp). The hub keepalive holds the NAT hole open so the tunnel, once formed, stays up. novox
# cannot initiate to a home node at all; every home↔novox path is either the overlay or a forwarded
# port.
#
# EVERY NODE HAS EGRESS, which is not a convenience. Third-party images — postgres, the whole Mailu
# stack, Plex, everything the modules actually run — are pulled from the internet, because that is
# where a real node gets them. The lab used to serve them from a registry it raised inside the
# scenario; no production mesh has one, so a bootstrap that could only work against it went green
# here and would have failed anywhere else. What is loaded onto a machine now is only what exists in
# no registry at all: the mesh's own images, named per machine below.
#
# Egress is a SECOND path, not a replacement for the topology. Each node still reaches the rest of
# the scenario through its declared gateway — that is where the overlay handshake has to survive a
# masquerade — and the uplink carries only what leaves the scenario entirely.
#
# THE UNPROVEN THING (what the flat beds never tested): does the overlay tunnel FORM across the
# access point — a home node dialling novox's public hub endpoint, the handshake completing through
@@ -30,14 +42,13 @@
# network on the container port, so the host side is free to move). Reported as a topology finding.
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# The images are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set. Every
# one is already built/pulled by the per-server bed prerequisites.
# The images: are the UNION of the novox set (feat/novox-conversions @ 431310f: the slug + roundcube
# fixes, so only-office/de-spiegel/amqp-email-forwarder now resolve) and the ace media/home set, and
# every one of them must already be BUILT on the workstation — nothing can fetch them.
scenario: whole-mesh-full
segments:
# The routable segment. novox lives here; the lab raises the image registry here too (a public
# IPv4 segment is what serves the images), and the overlay hub endpoint is a public address here.
# The routable segment. novox lives here, and the overlay hub endpoint is a public address here.
hosting:
kind: public
cidr: [192.0.2.0/24]
@@ -60,19 +71,69 @@ machines:
# public ingress. Bigger than the flat bed's novox, because it now carries the substrate too.
novox:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 24GiB
cpus: 8
disk: 130GiB
# The substrate's control plane, the proxy, and a runtime per module novox is assigned.
# step-ca, photos, invoicing, novox.be, only-office, de-spiegel, amqp-email-forwarder, registry,
# firewall and fail2ban are not here because they carry no runtime image of their own — what
# they run is third-party or is the node itself.
images:
- mesh-control:development
- mesh-route-proxy:development
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-minio:development
- mesh-runtime-mongodb:development
- mesh-runtime-mssql:development
- mesh-runtime-lavinmq:development
- mesh-runtime-keycloak:development
- mesh-runtime-gitea:development
- mesh-runtime-nextcloud:development
- mesh-runtime-umami:development
- mesh-runtime-verdaccio:development
- mesh-runtime-portainer:development
- mesh-runtime-mailu:development
# The home server: the whole ace media/home set — 24 modules, ~50 containers, several heavy
# (Plex, Home Assistant, Letta, Baserow, the UniFi JVM, mssql). Behind the gateway.
ace:
at: { segment: home, address: [192.168.1.10] }
egress: true
inbound: allow
memory: 18GiB
cpus: 6
disk: 120GiB
# A runtime per module ace is assigned, and nothing of novox's. This is the point of saying it
# per machine: ace has no business holding a Keycloak runtime, and an operator's home server
# would not.
images:
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-mssql:development
- mesh-runtime-portainer:development
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
- mesh-runtime-lidarr:development
- mesh-runtime-plex:development
- mesh-runtime-bazarr:development
- mesh-runtime-nzbget:development
- mesh-runtime-qbittorrent:development
- mesh-runtime-jackett:development
- mesh-runtime-ombi:development
- mesh-runtime-tautulli:development
- mesh-runtime-bookshelf:development
- mesh-runtime-home-assistant:development
- mesh-runtime-mosquitto:development
- mesh-runtime-influxdb:development
- mesh-runtime-grafana:development
- mesh-runtime-baserow:development
- mesh-runtime-letta:development
- mesh-runtime-nodered:development
- mesh-runtime-searxng:development
- mesh-runtime-unifi:development
# Two workstations on the same home LAN. Light on purpose: they enrol, join the overlay, and run
# one small module (portainer) so a real module converges on each without heavy load. Same-LAN
@@ -80,77 +141,30 @@ machines:
# is the normal case and is fine.
shanks:
at: { segment: home, address: [192.168.1.20] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
disk: 30GiB
# One module, one runtime. Handing these two the whole union would put roughly thirty gigabytes
# of images onto a thirty-gigabyte disk, which is how you learn that "the lab loads everything
# everywhere" was never a description of anything real.
images: [mesh-runtime-portainer:development]
g14:
at: { segment: home, address: [192.168.1.30] }
egress: true
inbound: allow
memory: 3GiB
cpus: 2
disk: 30GiB
images: [mesh-runtime-portainer:development]
# The union of what the machines above ask for. This is the list the workstation must be able to
# export — every entry is one of the mesh's own images, built from source and published nowhere, so
# a machine holds it because it was handed it. Everything else the modules run comes from the
# internet and is not named here at all.
images:
# --- substrate + shared (novox & ace both run postgres/redis/mssql/portainer) ---
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
- redis:7-alpine
- mcr.microsoft.com/mssql/server:2022-latest
- portainer/portainer-ce:latest
# --- novox server images ---
- minio/minio:latest
- mongo:7
- quay.io/keycloak/keycloak:mesh
- gitea/gitea:1.22
- nextcloud:stable
- ghcr.io/umami-software/umami:postgresql-latest
- alpine:latest
- verdaccio/verdaccio:6
- registry:2
- registry-api.novox.be/novox/invoicing-app:latest
- registry-api.novox.be/novox/invoicing-api:latest
- onlyoffice/documentserver:mesh
- registry-api.novox.be/novox/de-spiegel:latest
- registry-api.novox.be/novox/www:latest
- registry-api.novox.be/novox/amqp-email-forwarder:latest
- registry-api.novox.be/novox/photos-server:latest
- registry-api.novox.be/novox/photos-admin-client:latest
- registry-api.novox.be/novox/photos-client:latest
# The full Mailu 1.9 stack (mailu-redis reuses redis:7-alpine above).
- ghcr.io/mailu/unbound:1.9
- ghcr.io/mailu/admin:1.9
- ghcr.io/mailu/dovecot:1.9
- ghcr.io/mailu/postfix:1.9
- ghcr.io/mailu/rspamd:1.9
- ghcr.io/mailu/clamav:1.9
- ghcr.io/mailu/roundcube:1.9
- ghcr.io/mailu/radicale:1.9
- ghcr.io/mailu/fetchmail:1.9
- ghcr.io/mailu/nginx:1.9
# --- ace server images ---
- lscr.io/linuxserver/sonarr:mesh
- lscr.io/linuxserver/radarr:mesh
- lscr.io/linuxserver/lidarr:mesh
- lscr.io/linuxserver/bazarr:mesh
- lscr.io/linuxserver/nzbget:mesh
- lscr.io/linuxserver/qbittorrent:mesh
- lscr.io/linuxserver/jackett:mesh
- lscr.io/linuxserver/ombi:mesh
- lscr.io/linuxserver/tautulli:mesh
- lscr.io/linuxserver/unifi-controller:mesh
- plexinc/pms-docker:mesh
- ghcr.io/pennydreadful/bookshelf:mesh
- ghcr.io/home-assistant/home-assistant:mesh
- eclipse-mosquitto:mesh
- influxdb:mesh
- grafana/grafana:mesh
- baserow/baserow:mesh
- letta/letta:mesh
- nodered/node-red:mesh
- searxng/searxng:mesh
- valkey/valkey:mesh
# --- per-module runtimes (union) ---
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
@@ -166,9 +180,6 @@ images:
- mesh-runtime-verdaccio:development
- mesh-runtime-mailu:development
- mesh-route-proxy:development
# ADR 0056: the internal ACME authority. route-proxy now REQUIRES an `acme-ca`, so a bed that does
# not serve this image has an unresolvable proxy — and with it every routed module on the mesh.
- smallstep/step-ca:latest
- mesh-runtime-sonarr:development
- mesh-runtime-radarr:development
- mesh-runtime-lidarr:development
scenarios/whole-mesh-novox.yml
+26 -32
View File
@@ -17,9 +17,10 @@
#
# MESH_LAB_HOST_BINARY=.../mesh-host MESH_LAB_BUNDLE=.../examples/substrate-first-node.lock
# The runtimes are built by scripts/build-module-runtime.sh (one per module that has code) and the
# route-proxy image by scripts/build-route-proxy-image.sh; every server image must be in the local
# daemon to be stocked. The test loads each committed module.json from mesh-catalog and rewrites its
# image references to what this scenario's own registry serves by digest.
# route-proxy image by scripts/build-route-proxy-image.sh; those must be in the local daemon,
# because nothing serves them and nothing can. Every third-party image is pulled from the internet
# over each node's uplink. The test loads each committed module.json from mesh-catalog and rewrites
# only OUR image references, to the ID the machine holds each one under.
scenario: whole-mesh-novox
segments:
@@ -31,55 +32,48 @@ machines:
# The substrate ONLY: store, broker, control. Nothing else lands here.
anchor:
at: { segment: hosting, address: [192.0.2.10] }
egress: true
inbound: allow
memory: 4GiB
cpus: 4
disk: 20GiB
# The substrate only, so the control plane's image only. Handing this machine the whole set of
# runtimes would fill a 20GiB disk with images nothing on it will ever start.
images: [mesh-control:development]
# The whole novox service set — ~38 containers (five providers with runtimes, six consumers with
# runtimes, portainer/verdaccio/registry/route-proxy, the nine-container Mailu stack and its
# runtime) plus two node-level modules. mssql alone wants ~2GiB; Mailu, Nextcloud and Keycloak are
# each heavy. Sized well past the two-node-db bed's second node.
novox:
at: { segment: hosting, address: [192.0.2.20] }
egress: true
inbound: allow
memory: 16GiB
cpus: 8
# ~14GiB of images are pulled from the scenario's own registry by digest, several of them large
# ~14GiB of images are pulled from the internet over the uplink, several of them large
# (mssql 1.7GiB, invoicing-api 1.9GiB, nextcloud 1.5GiB, umami/mongo ~0.9GiB), plus writable
# layers and the runtimes. A hundred gigabytes holds the whole set without exhausting the disk
# mid-apply.
disk: 100GiB
# Every runtime, and the proxy. Not mesh-control: the control plane runs on the anchor.
images:
- mesh-runtime-postgres:development
- mesh-runtime-redis:development
- mesh-runtime-minio:development
- mesh-runtime-mongodb:development
- mesh-runtime-mssql:development
- mesh-runtime-keycloak:development
- mesh-runtime-gitea:development
- mesh-runtime-nextcloud:development
- mesh-runtime-umami:development
- mesh-runtime-photos:development
- mesh-runtime-portainer:development
- mesh-runtime-verdaccio:development
- mesh-runtime-mailu:development
- mesh-route-proxy:development
images:
# The first-node substrate: store, broker, control. postgres:17-alpine doubles as the postgres
# provider's own service image (and Mailu's internal admin DB).
- postgres:17-alpine
- cloudamqp/lavinmq:latest
- mesh-control:development
# The module server images. Each is stocked under the repository path its module.json names, so the
# test's rewrite (pinned(repositoryFor(image))) finds it.
- redis:7-alpine
- minio/minio:latest
- mongo:7
- mcr.microsoft.com/mssql/server:2022-latest
- quay.io/keycloak/keycloak:mesh
- gitea/gitea:1.22
- nextcloud:stable
- ghcr.io/umami-software/umami:postgresql-latest
- alpine:latest
- portainer/portainer-ce:latest
- verdaccio/verdaccio:6
- registry:2
- registry-api.novox.be/novox/invoicing-app:latest
- registry-api.novox.be/novox/invoicing-api:latest
# The Mailu stack (pulled by digest, tagged :mesh so repositoryFor matches the module.json paths).
- ghcr.io/mailu/unbound:mesh
- ghcr.io/mailu/admin:mesh
- ghcr.io/mailu/dovecot:mesh
- ghcr.io/mailu/postfix:mesh
- ghcr.io/mailu/rspamd:mesh
- ghcr.io/mailu/webmail:mesh
- ghcr.io/mailu/nginx:mesh
# The per-module runtimes (built by scripts/build-module-runtime.sh). registry, route-proxy,
# invoicing, firewall and fail2ban carry no mesh-runtime image; route-proxy ships its own.
- mesh-runtime-postgres:development