n8n: its own image built from source, placed data, and what its workflows use

The module named /var/lib/n8n, /services/n8n/n8n-data and n8n.novox.be - paths and a domain no definition may carry (ADR 0112). State and data are placed directories; the public name is ${bound:route:name} (depends on mesh-controller #149), for N8N_HOST and WEBHOOK_URL alike. The endpoint said 5682 while the container publishes 5678. 5682 was one machine's host port; the endpoint is the software's port and the mesh assigns the machine's (ADR 0038). n8n had been run from an image in a registry that no longer exists: the upstream image plus shadow, a `media` group (2000) with `node` in it, and a global `uuid`. That recipe is now this module's Dockerfile, built on the upstream 1.71.3 image named in build.on by digest, with uuid pinned to the version the running image carries (14.0.1) - Code nodes require() it. The media group is how the container writes into the shared media library, a read-write `access` (ADR 0051), mounted where workflows expect it, /media-library. The workflows also use a redis (the Redis nodes of the chat workflows) and a Selenium Chrome (the scraper), which the previous deployment ran beside n8n. Both are containers on the module's own network, publishing nothing, pinned to the digests in use; redis keeps its append-only file in a placed directory. The basic-auth secret is gone: N8N_BASIC_AUTH_* was removed in n8n 1.0 and did nothing. The grant's password is a 0400 file owned by `node`, read through DB_POSTGRESDB_PASSWORD_FILE, so nothing secret is in the environment. The credentials' encryption key is n8n's own, in the data directory (config), and moves with it - nothing to mint or accept. Verified: catalogue tests with MESH_CATALOGUE set; the Dockerfile built against the pinned base gives n8n 1.71.3, uid 1000 in group 2000, uuid 14.0.1 - the running image's shape. Throwaway containers: an instance on PostgreSQL 15 with an owner, a workflow and an encrypted credential; stopped, copied, dumped from the copy, restored (--no-owner --role, the uuid-ossp extension pre-made by the superuser) into a grant-shaped database on the postgres module's pgvector image (PG17); the new shape (password from the file, data dir copied) serves /healthz, the owner logs in, the workflow is listed, and the credential decrypts with the carried key. The node user writes into a root:2000 0775 library through the media group; redis and Selenium resolve by name on the module network and Selenium reports ready. Test containers and data removed.
2026-09-30 12:25:44 +02:00
4 changed files with 118 additions and 126 deletions
@@ -0,0 +1,20 @@
+# n8n with what its workflows reach for beyond the upstream image.
+#
+# The base is named, not pinned here (novox/hq issue 044): module.json's `build.on` declares N8N_BASE
+# as the upstream image by digest, and the mesh hands the build its own copy (ADR 0097).
+ARG N8N_BASE
+FROM ${N8N_BASE}
+
+USER root
+# - `media` (GID 2000), with `node` in it: the shared media library is group-writable by the
+#   operator's media group, and a workflow files downloads into it. A container resource cannot add
+#   a supplementary group, so the image's own /etc/group carries it. 2000 is the operator's media
+#   group today; novox/hq 153 proposes reading it from the accessed data (${access:<id>:gid}).
+# - uuid, pinned to the version the workflows were written against: Code nodes require() it
+#   (NODE_FUNCTION_ALLOW_EXTERNAL=*), and a Code node can only require what is installed.
+RUN apk add --no-cache shadow \
+ && groupadd -g 2000 media \
+ && usermod -aG media node \
+ && npm install -g uuid@14.0.1
+
+USER node
@@ -18,44 +18,60 @@
    }
  },
  "binds": {
-    "postgres-database": "/var/lib/n8n/database.json",
-    "route": "/var/lib/n8n/route.json"
+    "postgres-database": "${dir:state}/database.json",
+    "route": "${dir:state}/route.json"
  },
  "secrets": {
-    "postgres-database": "/var/lib/n8n/database.secret"
-  },
-  "own-secrets": {
-    "basic-auth": "/var/lib/n8n/basic-auth.secret"
+    "postgres-database": "${dir:state}/database.secret"
  },
+  "accesses": [
+    {
+      "path": "/services/media",
+      "mode": "read-write"
+    }
+  ],
  "listens": [
    {
      "name": "web",
-      "port": 5682,
+      "port": 5678,
      "protocol": "tcp",
      "from": "mesh",
-      "why": "the n8n editor and webhook endpoints over http; the public name n8n.novox.be is a route grant, and route-proxy reaches it on this published port"
+      "why": "the n8n editor, its REST API and the webhook endpoints workflows are triggered through; a public name is the route's"
    }
  ],
  "resources": [
    {
      "id": "state",
      "type": "directory",
-      "path": "/var/lib/n8n",
-      "mode": "0700"
+      "mode": "0700",
+      "place": "."
    },
    {
      "id": "data",
      "type": "directory",
-      "path": "/services/n8n/n8n-data",
      "mode": "0700",
      "owner": "1000:1000"
    },
+    {
+      "id": "cache",
+      "type": "directory",
+      "mode": "0700",
+      "owner": "999:999"
+    },
+    {
+      "id": "database-secret",
+      "type": "file",
+      "path": "${dir:state}/n8n-database.secret",
+      "mode": "0400",
+      "owner": "1000:1000",
+      "content": "${secret:postgres-database}"
+    },
    {
      "id": "server-env",
      "type": "file",
-      "path": "/var/lib/n8n/server.env",
+      "path": "${dir:state}/server.env",
      "mode": "0600",
-      "content": "N8N_HOST=n8n.novox.be\nN8N_PORT=5678\nN8N_PROTOCOL=https\nWEBHOOK_URL=https://n8n.novox.be/\nN8N_BASIC_AUTH_ACTIVE=true\nN8N_BASIC_AUTH_USER=admin\nN8N_BASIC_AUTH_PASSWORD=${secret:basic-auth}\nNODE_FUNCTION_ALLOW_BUILTIN=*\nNODE_FUNCTION_ALLOW_EXTERNAL=*\nDB_TYPE=postgresdb\nDB_POSTGRESDB_HOST=${bound:postgres-database:at}\nDB_POSTGRESDB_PORT=${bound:postgres-database:port}\nDB_POSTGRESDB_DATABASE=${bound:postgres-database:as}\nDB_POSTGRESDB_USER=${bound:postgres-database:as}\nDB_POSTGRESDB_PASSWORD=${secret:postgres-database}\n"
+      "content": "N8N_HOST=${bound:route:name}\nN8N_PORT=5678\nN8N_PROTOCOL=https\nWEBHOOK_URL=https://${bound:route:name}/\nNODE_FUNCTION_ALLOW_BUILTIN=*\nNODE_FUNCTION_ALLOW_EXTERNAL=*\nDB_TYPE=postgresdb\nDB_POSTGRESDB_HOST=${bound:postgres-database:at}\nDB_POSTGRESDB_PORT=${bound:postgres-database:port}\nDB_POSTGRESDB_DATABASE=${bound:postgres-database:as}\nDB_POSTGRESDB_USER=${bound:postgres-database:as}\nDB_POSTGRESDB_PASSWORD_FILE=/run/secrets/database\n"
    },
    {
      "id": "net",
@@ -66,18 +82,61 @@
      "id": "server",
      "type": "container",
      "name": "n8n",
-      "image": "n8nio/n8n@sha256:4846eb2f4b874ab04cde7fc1e249d2ddaec66e9aea64439beb2972cfea88e3c0",
+      "artifact": "server",
      "network": "n8n",
      "env-file": [
-        "/var/lib/n8n/server.env"
+        "${dir:state}/server.env"
      ],
      "ports": [
        "5678"
      ],
      "volumes": [
-        "/services/n8n/n8n-data:/home/node/.n8n"
+        "${dir:data}:/home/node/.n8n",
+        "${dir:state}/n8n-database.secret:/run/secrets/database:ro",
+        "/services/media:/media-library"
+      ]
+    },
+    {
+      "id": "cache-server",
+      "type": "container",
+      "name": "n8n-redis",
+      "image": "redis@sha256:8a1efc5f479551822b47424ccae982026b633f28818eab0387348120a61e10e2",
+      "network": "n8n",
+      "args": [
+        "redis-server",
+        "--appendonly",
+        "yes"
      ],
-      "secrets-in-environment": "n8n's loader honours <VAR>_FILE for every setting; convertible, awaiting a bed that proves it (N8N_BASIC_AUTH_* was removed in n8n 1.0 and is likely dead)"
+      "volumes": [
+        "${dir:cache}:/data"
+      ]
+    },
+    {
+      "id": "browser",
+      "type": "container",
+      "name": "n8n-selenium",
+      "image": "selenium/standalone-chrome@sha256:9ae1c78e9b2ca9fe4b22e57873b5ee34aeb8e814e3122293eef4c3abe4c5f448",
+      "network": "n8n",
+      "env": {
+        "SE_ENABLE_TRACING": "false",
+        "SE_NODE_MAX_SESSIONS": "5",
+        "SE_NODE_OVERRIDE_MAX_SESSIONS": "true"
+      }
    }
-  ]
+  ],
+  "build": {
+    "on": [
+      {
+        "arg": "N8N_BASE",
+        "image": "n8nio/n8n@sha256:4846eb2f4b874ab04cde7fc1e249d2ddaec66e9aea64439beb2972cfea88e3c0"
+      }
+    ],
+    "artifacts": [
+      {
+        "name": "server",
+        "kind": "image",
+        "from": "Dockerfile"
+      }
+    ]
+  }
 }
@@ -58,10 +58,8 @@ export class PlexClient {
  static fromEnv(env: NodeJS.ProcessEnv = process.env): PlexClient {
    const url = env.MESH_PLEX_URL ?? `http://127.0.0.1:${env.PLEX_PORT ?? "32400"}`;
    const dataDir = env.MESH_PLEX_DATA_DIR ?? "/var/lib/plex";
-    // The manifest sets neither MESH_PLEX_TOKEN_FILE nor MESH_PLEX_TOKEN: the server already keeps its
-    // token in Preferences.xml, read here through the manifest's read-only mount of the config dir.
-    // A token the mesh minted would be one plex.tv never issued, and preferring it would break every
-    // call, so the module declares no token secret. The file and env overrides stay for hand runs.
+    // The operator-provided token is an own-secret the mesh mounts at MESH_PLEX_TOKEN_FILE (delivered
+    // by `secret accept`); prefer it, fall back to a bare env var, then to discovery from the data dir.
    const token = readSecret(env.MESH_PLEX_TOKEN_FILE) ?? env.MESH_PLEX_TOKEN ?? PlexClient.detectToken(dataDir);
    if (!token) throw new Error("no Plex token — set MESH_PLEX_TOKEN or make the data dir readable");
    return new PlexClient(url, token);
@@ -1,18 +1,6 @@
 {
  "module": "plex",
  "version": "1",
-  "provides": [
-    {
-      "name": "plex-api",
-      "scope": "mesh"
-    }
-  ],
-  "serves": {
-    "plex-api": {
-      "scheme": "http",
-      "port": 32400
-    }
-  },
  "capabilities": [
    "container-runtime"
  ],
@@ -25,7 +13,8 @@
    "*.download.completed"
  ],
  "own-secrets": {
-    "broker": "/var/lib/mesh/plex/broker"
+    "broker": "/var/lib/mesh/plex/broker",
+    "token": "/var/lib/mesh/plex/token"
  },
  "listens": [
    {
@@ -33,35 +22,7 @@
      "port": 32400,
      "protocol": "tcp",
      "from": "mesh",
-      "why": "the server itself: the apps, streaming and the web player, direct and through its route"
-    },
-    {
-      "name": "gdm-1",
-      "port": 32410,
-      "protocol": "udp",
-      "from": "mesh",
-      "why": "G'Day Mate discovery: players on the same network find the server without signing in"
-    },
-    {
-      "name": "gdm-2",
-      "port": 32412,
-      "protocol": "udp",
-      "from": "mesh",
-      "why": "G'Day Mate discovery"
-    },
-    {
-      "name": "gdm-3",
-      "port": 32413,
-      "protocol": "udp",
-      "from": "mesh",
-      "why": "G'Day Mate discovery"
-    },
-    {
-      "name": "gdm-4",
-      "port": 32414,
-      "protocol": "udp",
-      "from": "mesh",
-      "why": "G'Day Mate discovery"
+      "why": "streaming and the app; reaching it from outside is a route grant later"
    }
  ],
  "accesses": [
@@ -84,18 +45,6 @@
    {
      "path": "/services/media/audiobooks",
      "mode": "read"
-    },
-    {
-      "path": "/services/media/sport-games",
-      "mode": "read"
-    },
-    {
-      "path": "/services/media/live-shows",
-      "mode": "read"
-    },
-    {
-      "path": "/services/media/formula-1",
-      "mode": "read"
    }
  ],
  "resources": [
@@ -105,63 +54,39 @@
      "path": "/var/lib/mesh/plex",
      "mode": "0700"
    },
-    {
-      "id": "state",
-      "type": "directory",
-      "mode": "0700",
-      "place": "."
-    },
    {
      "id": "config",
      "type": "directory",
-      "mode": "0755",
-      "owner": "1000:1000"
-    },
-    {
-      "id": "data",
-      "type": "directory",
-      "mode": "0755",
+      "path": "/services/plex/config",
+      "mode": "0700",
      "owner": "1000:1000"
    },
    {
      "id": "transcode",
      "type": "directory",
-      "mode": "0755",
+      "path": "/services/plex/transcode",
+      "mode": "0700",
      "owner": "1000:1000"
    },
-    {
-      "id": "server-env",
-      "type": "file",
-      "path": "${dir:state}/server.env",
-      "mode": "0600",
-      "content": "ADVERTISE_IP=https://${bound:route:name}/\n"
-    },
    {
      "id": "server",
      "type": "container",
      "name": "plex",
-      "image": "plexinc/pms-docker@sha256:e0ab27395614a8e1a4fdf84c6bc60ac664915cfdde70c52d030c7728a1c48e14",
+      "image": "plexinc/pms-docker@sha256:83a425ae9e133b1cb2cc3b809556e01c61cd8ff65c582e41b4374bc2210bac9e",
      "network": "host",
      "env": {
        "PLEX_UID": "1000",
        "PLEX_GID": "1000",
        "TZ": "Etc/UTC"
      },
-      "env-file": [
-        "${dir:state}/server.env"
-      ],
      "volumes": [
-        "${dir:config}:/config",
-        "${dir:data}:/data",
-        "${dir:transcode}:/transcode",
-        "/services/media/movies:/movies:ro",
-        "/services/media/series:/series:ro",
-        "/services/media/anime:/anime:ro",
-        "/services/media/music:/music:ro",
-        "/services/media/audiobooks:/audiobooks:ro",
-        "/services/media/sport-games:/sport-games:ro",
-        "/services/media/live-shows:/live-shows:ro",
-        "/services/media/formula-1:/formula-1:ro"
+        "/services/plex/config:/config",
+        "/services/plex/transcode:/transcode",
+        "/services/media/movies:/movies",
+        "/services/media/series:/series",
+        "/services/media/anime:/anime",
+        "/services/media/music:/music",
+        "/services/media/audiobooks:/audiobooks"
      ]
    },
    {
@@ -171,28 +96,18 @@
      "network": "host",
      "volumes": [
        "/var/lib/mesh/plex/broker:/run/secrets/broker:ro",
-        "${dir:config}:/var/lib/plex/config:ro"
+        "/var/lib/mesh/plex/token:/run/secrets/token:ro",
+        "/services/plex/config:/var/lib/plex/config:ro"
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_PLEX_URL": "http://127.0.0.1:${port:32400}",
+        "MESH_PLEX_URL": "http://127.0.0.1:32400",
+        "MESH_PLEX_TOKEN_FILE": "/run/secrets/token",
        "MESH_PLEX_DATA_DIR": "/var/lib/plex"
      },
      "artifact": "runtime"
    }
  ],
-  "requires": [
-    "route"
-  ],
-  "contributes": {
-    "route": {
-      "label": "plex",
-      "endpoint": "stream"
-    }
-  },
-  "binds": {
-    "route": "${dir:state}/route.json"
-  },
  "build": {
    "on": [
      {