Re-runs the capstone from main after the dry-run fixes merged.
fail2ban: added to the novox set. The capability fix (intrusion-prevention ->
firewall) makes it HOSTABLE — it is now assigned, not refused — which is the
gate. Its service reaching active is a host concern the offline lab cannot meet
(the VM ships nftables but not fail2ban, and the isolated segment has no route to
the package mirror, so pacman cannot fetch it), so fail2ban joins GAPS_NOVOX: its
failed package resource is tolerated like firewall's oneshot nftables.service.
7 credential sidecars: before the push, a FAKE app credential is delivered for
each (plex/bazarr/ombi/home-assistant/nzbget/qbittorrent on ace, umami on novox)
through the real operator path — `secret accept <node> <module> <name> --from`.
The bed asserts each sidecar advances PAST its old "no credential" crash (it reads
the delivered value); app-auth failure against the real app with a bogus value is
expected and not gated.
Result: SUITE_EXIT=0. Both node-plans converge on one substrate (novox 13/13
core, ace 17/17 core), fail2ban hostable, all 7 sidecars past their crash.
Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
Combine the novox (17-module) and ace (24-module) sets on ONE substrate and
prove both node-plans converge together. anchor runs the substrate only;
novox and ace each run their own self-contained set (own postgres/redis), so
nothing crosses a node boundary except enrolment and the shared broker/store.
The four modules both nodes run (postgres, redis, mssql, portainer) are added
once and assigned to each node, each getting its own per-node broker account.
An overlay is placed across all three nodes.
Proven green: both nodes converge together on the one substrate. ace reaches
applied+current with all 17 of its CORE up (and letta too this run); novox
reaches all 13 CORE up with its only failed resource the known firewall.load
oneshot gap. The two node-plans share one broker without collision — distinct
novox-<mod> and ace-<mod> accounts for the modules both run. No new cross-node
bug (overlay/DNS/identity/port) surfaced; ports are per-VM and the sets are
node-self-contained. Tolerates the same nine credential-sidecar gaps and
firewall's nftables.service oneshot documented in the per-server beds.
Resource envelope: 3 VMs (anchor 4GiB, novox 16GiB, ace 18GiB) + registry
scenery, ~79 union images (~35GB) stocked to one registry VM and pulled
concurrently by both nodes; fit within 125GiB host RAM and the 180GiB lab pool.
Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
Install the real ace server's converted service set (24 modules) together on
one node behind the substrate — sibling of the whole-mesh-novox bed, the
media/home-automation half. Loads each committed module.json from
mesh-catalog, rewrites image refs to the scenario registry's digests, remaps
the co-located host-port collisions (qbittorrent/searxng/unifi :8080,
nzbget/unifi :6789), and pre-creates the ADR-0051 operator-owned media
library dirs under /services/media so the media stack's `accesses` resolve.
Proven green: the whole 24-module set RESOLVES and applies (214 resources,
node applied+current) — the ADR-0051 shared-dir `accesses` mechanism works
cleanly across eight co-accessing media modules. The CORE 17 converge whole:
postgres/redis/mssql, sonarr/radarr/lidarr/jackett/tautulli/bookshelf,
mosquitto/influxdb/grafana/baserow/nodered/searxng/unifi/portainer.
Reported as escalated gaps (do not gate green): six tool-runtime sidecars
crash-loop because the committed manifest does not wire the app credential
they need (plex MESH_PLEX_TOKEN, bazarr MESH_BAZARR_API_KEY, nzbget
MESH_NZBGET_URL/PASSWORD, qbittorrent MESH_QBITTORRENT_URL/PASSWORD, ombi
MESH_OMBI_API_KEY, home-assistant MESH_HOMEASSISTANT_TOKEN) — the umami/photos
class from novox; each server is up, only the sidecar is down. sonarr/radarr/
lidarr/jackett/tautulli self-configure from the app's config file and their
runtimes come up. letta's app has a first-boot postgres migration race
(pgvector the deeper blocker, per two-node-db).
Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
Install the real novox server's converted service set together on one node
behind the substrate — the whole-catalogue install this rebuild never ran.
The bed loads each committed module.json from mesh-catalog (no hand-written
manifests), rewrites image refs to the scenario registry's digests, and
remaps the co-located host-port collisions (nextcloud/invoicing/route-proxy
:80, minio/invoicing :9000, gitea/umami :3000).
Proven green: the whole set of 17 modules RESOLVES and applies (191
resources); the CORE 13 converge whole — all five providers (postgres,
redis, minio, mongodb, mssql) plus keycloak, gitea, nextcloud and invoicing
reaching their providers and staying up, plus portainer, verdaccio, registry
and route-proxy.
Reported as escalated gaps (do not gate green): fail2ban (declares
capability intrusion-prevention that no host detector provides, and an
unappliable assignment blocks whole-node resolution), umami/photos/mailu
(catalog manifests do not wire the runtime/app env the images need; photos'
server image is an alpine placeholder), and firewall (nftables.service is a
oneshot that exits, but the module declares state running so mesh-host marks
it failed).
Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
jschoubben