jschoubben
0e382f1db7
The complete recipe, found across five runs: adopt BOTH the store (postgres) and broker (lavinmq) on the control-node — the broker's `listens` is what opens 5671 in the firewall for cross-node bus access; deliver the store's genesis superuser via `secret accept` (else the module mints a random one that cannot log in to the running store); inject the provider seal key (open hq issue 022 workaround); push the provider node again after the remote consumers (issue 057); and tolerate provisioner-runtime startup churn — a cross-node provisioner exits until the overlay tunnel is up, then settles. baserow and letta on the joined node get their databases from the one foundation store over the overlay. https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
58 lines
2.7 KiB
YAML
58 lines
2.7 KiB
YAML
# The DB-consumer chain a single node cannot host, proved across two machines.
|
||
#
|
||
# The app-postgres provider and the mesh's own foundation store both want host port 5432, so they
|
||
# cannot share a machine — the collision that blocked this chain single-node. Here the foundation
|
||
# (store, broker, control) lives on `anchor` and NOTHING else; `laptop` runs the whole chain —
|
||
# postgres and redis PROVIDERS plus the baserow and letta CONSUMERS that require them. Both
|
||
# machines sit on one shared segment and enrol into the one mesh; only enrolment crosses to anchor,
|
||
# over the underlay both machines already share. Provider and consumers are co-located on laptop, so
|
||
# no cross-node module comms and no overlay are needed — and the 5432-vs-foundation conflict is gone
|
||
# because the foundation store is on the OTHER node.
|
||
scenario: two-node-db
|
||
|
||
segments:
|
||
hosting:
|
||
kind: public
|
||
cidr: [192.0.2.0/24]
|
||
|
||
machines:
|
||
# The foundation ONLY: store, broker, control — three containers. Four gigabytes is plenty for a
|
||
# node that hosts no modules; the thrash the two-nodes bed warns of comes from stacking eleven
|
||
# containers on a node, which this one never does.
|
||
anchor:
|
||
at: { segment: hosting, address: [192.0.2.10] }
|
||
egress: true
|
||
inbound: allow
|
||
memory: 4GiB
|
||
cpus: 4
|
||
# The whole DB-consumer chain: postgres + redis providers, each a server and a broker-bound
|
||
# runtime, plus the baserow and letta consumer services and their tools runtimes — a dozen
|
||
# containers, two of them memory-hungry app servers (the Baserow all-in-one and the Letta server).
|
||
# At the 2GiB the two-nodes bed gives this machine it would thrash — its own anchor comment says
|
||
# so — and convergence would present as "the mesh hangs". Six gigabytes gives it room.
|
||
laptop:
|
||
at: { segment: hosting, address: [192.0.2.20] }
|
||
egress: true
|
||
inbound: allow
|
||
memory: 6GiB
|
||
cpus: 4
|
||
# The runtime images (280MB–600MB each) plus the service images — two of them heavy app images
|
||
# (Baserow ~1.5GB, Letta ~1.8GB) — are pulled from the internet over the uplink, so the
|
||
# same bytes land on this node twice. The pool default root disk exhausts mid-apply ("no space
|
||
# left on device"); sixty gigabytes holds the whole chain.
|
||
disk: 60GiB
|
||
|
||
images:
|
||
- mesh-controller:development
|
||
# The per-module runtimes, built by scripts/build-module-runtime.sh and stocked here. Each carries
|
||
# its module's provisioner, so no separate mesh-provision-* image is listed — the runtime is the
|
||
# provisioner (ADR 0048).
|
||
- mesh-runtime-postgres:development
|
||
- mesh-runtime-lavinmq:development
|
||
- mesh-runtime-redis:development
|
||
- mesh-runtime-baserow:development
|
||
- mesh-runtime-letta:development
|
||
|
||
place:
|
||
all: [host, runtime]
|