baserow: placed directories, the database password from a file, and the build ace runs

The module named /var/lib/baserow and /services/baserow/data, a layout no definition may carry (ADR 0112). State and data are now placed directories; bindings and the grant's secret live in the placed state. The all-in-one image's entrypoint honours DATABASE_PASSWORD_FILE (file_env in /baserow.sh), so the grant's password is mounted rather than put in an env-file, and "secrets-in-environment" is gone (ADR 0086). SECRET_KEY is no longer minted: the image keeps it, and its JWT signing key, in the data directory (.secret, .jwt_signing_key) and imports them on start, so a moved data directory carries the keys its sessions and tokens were made with. DISABLE_EMBEDDED_PSQL makes a missing grant fail loudly instead of starting an empty embedded database. BASEROW_PUBLIC_URL was http://localhost. Baserow answers only the host of that URL - any other Host is looked up as a published builder site and gets 404, /api/_health/ included - so it is now https://${bound:route:name} (depends on mesh-controller #149). The runtime's tools could never have worked: its config was "{}", and the client's Host override was silently dropped by Node's fetch, so calls by container name would 404 even with credentials. The client now uses node:http (which sends the Host it is given, with a Content-Length - Baserow reads a chunked body as empty) and re-authenticates once when a cached JWT is refused (access tokens last minutes, the runtime weeks). The password is the accepted `admin` secret; the email is an assignment setting merged into the same file, the host is the route's name. Image pinned to the develop-latest build ace runs today (Baserow 2.3.4, built 2026-09-18). The old pin (built 2026-09-04) is older than ace's data. Verified: catalogue tests with MESH_CATALOGUE set; tsc -p tsconfig.json in the mesh-tools build image. In throwaway containers of the pinned image: a fresh embedded-PG instance with a user, workspace and 5-row table; stopped, copied, dumped from the copy (start-only-db); restored with --no-owner --role into a grant-shaped database on the pgvector image the postgres module pins (PG17); started with this shape (root 0600 password file, embedded PSQL disabled, copied data dir without postgres/): health 200, the user logs in, the 5 rows are there, SECRET_KEY and the JWT key are imported from the data dir. The patched client lists applications and rows through the container name with the public Host, and recovers from a refused token. Test containers and data removed.
2026-09-30 12:16:04 +02:00
11 changed files with 106 additions and 144 deletions
@@ -2,12 +2,15 @@
 // module's tools and anything else baserow-specific import it; nothing outside baserow does.
 //
 // Baserow authenticates a person with email + password, exchanged for a JWT at /api/user/token-auth/.
-// Those credentials are the mesh's own: a person signs up in Baserow (the standard image creates no
-// admin from env), and the credential is placed in the runtime config file the mesh mounts. Until
-// that happens fromEnv throws and the module simply exposes no tools — the same dormant-until-
-// configured shape gitea uses for its token.
+// The standard image creates no admin from env, so the account is one a person made in Baserow: its
+// password is the module's `admin` secret, accepted from the operator, and its email and the public
+// host Baserow answers to reach the runtime config file the mesh mounts (the email from the
+// assignment's settings). Until both are there fromEnv throws and the module exposes no tools — the
+// same dormant-until-configured shape gitea uses for its token.

 import { readFileSync } from "node:fs";
+import { request as httpRequest } from "node:http";
+import { request as httpsRequest } from "node:https";

 export interface BaserowApplication {
  id: number;
@@ -68,35 +71,63 @@ export class BaserowClient {
    return h;
  }

-  /** Exchange email + password for a JWT, caching it for the client's lifetime. Handles both the
+  /**
+   * One HTTP exchange. Not `fetch`: Node's fetch drops a caller's Host header and sends the URL's
+   * own, and Baserow answers only the host of its BASEROW_PUBLIC_URL — any other Host is looked up
+   * as a published builder site and gets 404, `/api/_health/` included. A co-located caller reaching
+   * it by container name must present the public host, so the request is made with node:http, which
+   * sends the Host it is given.
+   */
+  private send(path: string, method: string, headers: Record<string, string>, body?: string): Promise<{ status: number; text: string }> {
+    const url = new URL(`${this.baseUrl}${path}`);
+    const request = url.protocol === "https:" ? httpsRequest : httpRequest;
+    // A length, never chunked: Baserow's server reads a chunked body as empty.
+    const sent = body === undefined ? headers : { ...headers, "Content-Length": String(Buffer.byteLength(body)) };
+    return new Promise((resolve, reject) => {
+      const req = request(url, { method, headers: sent }, (res) => {
+        let text = "";
+        res.setEncoding("utf8");
+        res.on("data", (chunk: string) => (text += chunk));
+        res.on("end", () => resolve({ status: res.statusCode ?? 0, text }));
+        res.on("error", reject);
+      });
+      req.on("error", reject);
+      if (body !== undefined) req.write(body);
+      req.end();
+    });
+  }
+
+  /** Exchange email + password for a JWT, caching it until Baserow refuses it. Handles both the
   *  older `{ token }` and the newer `{ access_token }` response shapes. */
  async authenticate(): Promise<string> {
    if (this.token) return this.token;
-    const res = await fetch(`${this.baseUrl}/api/user/token-auth/`, {
-      method: "POST",
-      headers: this.headers(),
-      body: JSON.stringify({ email: this.email, password: this.password }),
-    });
-    if (!res.ok) throw new Error(`baserow auth failed: ${res.status} ${await res.text()}`);
-    const data = (await res.json()) as { token?: string; access_token?: string };
+    const res = await this.send(
+      "/api/user/token-auth/",
+      "POST",
+      this.headers(),
+      JSON.stringify({ email: this.email, password: this.password }),
+    );
+    if (res.status < 200 || res.status >= 300) throw new Error(`baserow auth failed: ${res.status} ${res.text}`);
+    const data = JSON.parse(res.text) as { token?: string; access_token?: string };
    const token = data.access_token ?? data.token;
    if (!token) throw new Error("baserow auth returned no token");
    this.token = token;
    return token;
  }

-  private async authed<T>(path: string, options: RequestInit = {}): Promise<T> {
-    const token = await this.authenticate();
-    const res = await fetch(`${this.baseUrl}${path}`, {
-      ...options,
-      headers: this.headers({
-        Authorization: `JWT ${token}`,
-        ...(options.headers as Record<string, string> | undefined),
-      }),
-    });
-    if (!res.ok) throw new Error(`baserow ${path}: ${res.status} ${await res.text()}`);
-    const text = await res.text();
-    return (text ? JSON.parse(text) : null) as T;
+  /** An authenticated GET. A refused token is dropped and the call made once more with a fresh one:
+   *  Baserow's access tokens expire after minutes, and the runtime lives for weeks. */
+  private async authed<T>(path: string): Promise<T> {
+    for (let attempt = 0; ; attempt++) {
+      const token = await this.authenticate();
+      const res = await this.send(path, "GET", this.headers({ Authorization: `JWT ${token}` }));
+      if (res.status === 401 && attempt === 0) {
+        this.token = null;
+        continue;
+      }
+      if (res.status < 200 || res.status >= 300) throw new Error(`baserow ${path}: ${res.status} ${res.text}`);
+      return (res.text ? JSON.parse(res.text) : null) as T;
+    }
  }

  /** The applications (databases) the account can see, across all its workspaces. */
@@ -18,14 +18,14 @@
    }
  },
  "binds": {
-    "postgres-database": "/var/lib/baserow/database.json",
-    "route": "/var/lib/baserow/route.json"
+    "postgres-database": "${dir:state}/database.json",
+    "route": "${dir:state}/route.json"
  },
  "secrets": {
-    "postgres-database": "/var/lib/baserow/database.secret"
+    "postgres-database": "${dir:state}/database.secret"
  },
  "own-secrets": {
-    "secret-key": "/var/lib/baserow/secret-key.secret",
+    "admin": "${dir:state}/admin.secret",
    "broker": "/var/lib/mesh/baserow/broker"
  },
  "listens": [
@@ -34,7 +34,7 @@
      "port": 80,
      "protocol": "tcp",
      "from": "mesh",
-      "why": "the Baserow web UI and REST API; a public name is a route grant later"
+      "why": "the Baserow web UI and REST API, served by the image's own Caddy; a public name is the route's"
    }
  ],
  "resources": [
@@ -47,22 +47,21 @@
    {
      "id": "state",
      "type": "directory",
-      "path": "/var/lib/baserow",
-      "mode": "0700"
+      "mode": "0700",
+      "place": "."
    },
    {
      "id": "data",
      "type": "directory",
-      "path": "/services/baserow/data",
      "mode": "0755",
      "owner": "9999:9999"
    },
    {
      "id": "server-env",
      "type": "file",
-      "path": "/var/lib/baserow/server.env",
+      "path": "${dir:state}/server.env",
      "mode": "0600",
-      "content": "DATABASE_HOST=${bound:postgres-database:at}\nDATABASE_PORT=${bound:postgres-database:port}\nDATABASE_NAME=${bound:postgres-database:as}\nDATABASE_USER=${bound:postgres-database:as}\nDATABASE_PASSWORD=${secret:postgres-database}\nSECRET_KEY=${secret:secret-key}\nBASEROW_PUBLIC_URL=http://localhost\n"
+      "content": "DATABASE_HOST=${bound:postgres-database:at}\nDATABASE_PORT=${bound:postgres-database:port}\nDATABASE_NAME=${bound:postgres-database:as}\nDATABASE_USER=${bound:postgres-database:as}\nDATABASE_PASSWORD_FILE=/run/secrets/database\nDISABLE_EMBEDDED_PSQL=true\nBASEROW_PUBLIC_URL=https://${bound:route:name}\n"
    },
    {
      "id": "net",
@@ -73,25 +72,25 @@
      "id": "server",
      "type": "container",
      "name": "baserow",
-      "image": "baserow/baserow@sha256:834424a10413798567f76428f255dc259445b7f8dcec56598c05b4073bb2a124",
+      "image": "baserow/baserow@sha256:263ea6c4b72c9eccabcd975ffe9fdebf23913a293a514bec6a3897a5e0a5a080",
      "network": "baserow",
      "env-file": [
-        "/var/lib/baserow/server.env"
+        "${dir:state}/server.env"
      ],
      "ports": [
        "80"
      ],
      "volumes": [
-        "/services/baserow/data:/baserow/data"
-      ],
-      "secrets-in-environment": "baserow reads DATABASE_PASSWORD and SECRET_KEY with os.getenv and has no _FILE twin (settings/base.py); not convertible"
+        "${dir:data}:/baserow/data",
+        "${dir:state}/database.secret:/run/secrets/database:ro"
+      ]
    },
    {
      "id": "runtime-config",
      "type": "file",
      "path": "/var/lib/mesh/baserow/config.json",
      "mode": "0600",
-      "content": "{}\n",
+      "content": "{\n  \"password\": \"${secret:admin}\",\n  \"host\": \"${bound:route:name}\"\n}\n",
      "merge": "json"
    },
    {
@@ -94,7 +94,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_BAZARR_URL": "http://127.0.0.1:${port:6767}",
+        "MESH_BAZARR_URL": "http://127.0.0.1:6767",
        "MESH_BAZARR_API_KEY_FILE": "/run/secrets/api-key",
        "MESH_BAZARR_CONFIG_FILE": "/run/config/config.json",
        "MESH_BAZARR_CONFIG_DIR": "/var/lib/bazarr/config"
@@ -76,7 +76,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_BOOKSHELF_URL": "http://127.0.0.1:${port:8787}",
+        "MESH_BOOKSHELF_URL": "http://127.0.0.1:8787",
        "MESH_BOOKSHELF_CONFIG_DIR": "/var/lib/bookshelf/config"
      },
      "artifact": "runtime"
@@ -75,7 +75,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_LIDARR_URL": "http://127.0.0.1:${port:8686}",
+        "MESH_LIDARR_URL": "http://127.0.0.1:8686",
        "MESH_LIDARR_CONFIG_DIR": "/var/lib/lidarr/config"
      },
      "artifact": "runtime"
@@ -81,7 +81,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_NZBGET_URL": "http://127.0.0.1:${port:6789}",
+        "MESH_NZBGET_URL": "http://127.0.0.1:6789",
        "MESH_NZBGET_PASSWORD_FILE": "/run/secrets/password",
        "MESH_NZBGET_CONFIG_FILE": "/run/config/config.json",
        "MESH_NZBGET_CONFIG_DIR": "/var/lib/nzbget/config"
@@ -3,10 +3,8 @@
 // qbittorrent. Both this module's tools and its events entrypoint import it, and nothing outside
 // qbittorrent does.
 //
-// The WebUI authenticates with a session cookie obtained by POSTing credentials, and guards
-// against CSRF by checking the Referer header. The cookie was `SID` before qBittorrent 5.2 and is
-// `QBT_SID_<port>` since, and a successful login answers 200 "Ok." before and 204 with no body
-// since — both are accepted. Node's fetch keeps no cookie jar, so the cookie is
+// The WebUI authenticates with a session cookie (SID) obtained by POSTing credentials, and guards
+// against CSRF by checking the Referer header. Node's fetch keeps no cookie jar, so the SID is
 // captured on login and carried by hand on every later call, with a single re-login on expiry.

 import { readFileSync } from "node:fs";
@@ -41,21 +39,6 @@ function meshConfig(file?: string): Record<string, string> {
  catch { return {}; }
 }

-/** The WebUI username from qBittorrent's own qBittorrent.conf, in the config directory the mesh
- *  mounts read-only (MESH_QBITTORRENT_CONFIG_DIR, which is the container's /config). The software's
- *  file is the truth about who may log in, so the tools ask it rather than a setting that could
- *  disagree. Absent, unreadable or unset yields undefined. */
-function confUsername(dir: string | undefined): string | undefined {
-  if (!dir) return undefined;
-  try {
-    const line = readFileSync(`${dir.replace(/\/$/, "")}/qBittorrent/qBittorrent.conf`, "utf8")
-      .split(/\r?\n/)
-      .find((l) => l.startsWith("WebUI\\Username="));
-    const value = line?.slice("WebUI\\Username=".length).trim();
-    return value ? value : undefined;
-  } catch { return undefined; }
-}
-
 /** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
 *  absent or unreadable yields undefined so callers fall back rather than crash. */
 function readSecret(file?: string): string | undefined {
@@ -66,8 +49,7 @@ function readSecret(file?: string): string | undefined {

 export class QbittorrentClient {
  readonly baseUrl: string;
-  /** The session cookie as `name=value`, sent back exactly as it was set. */
-  private session: string | null = null;
+  private sid: string | null = null;

  constructor(
    baseUrl: string,
@@ -81,9 +63,7 @@ export class QbittorrentClient {
   * Build from the module's resolved environment. URL and password are read from
   * MESH_QBITTORRENT_URL and MESH_QBITTORRENT_PASSWORD; both must be present — an unconfigured
   * qBittorrent throws rather than pretend to be reachable, so the tools/events simply do not load
-   * (the harness treats the throw as "exposes nothing"). The user is read from qBittorrent.conf,
-   * falling back to "admin", the image's default. The password cannot be read there — qBittorrent
-   * keeps only a PBKDF2 hash — so it is the own-secret the operator accepts.
+   * (the harness treats the throw as "exposes nothing"). The user defaults to "admin".
   */
  static fromEnv(env: NodeJS.ProcessEnv = process.env): QbittorrentClient {
    const cfg = meshConfig(env.MESH_QBITTORRENT_CONFIG_FILE);
@@ -92,9 +72,7 @@ export class QbittorrentClient {
    if (!url || !password) {
      throw new Error("qBittorrent not configured — set MESH_QBITTORRENT_URL and MESH_QBITTORRENT_PASSWORD");
    }
-    // The WebUI username: a setting or the environment if one says so, else whatever
-    // qBittorrent.conf holds (an adopted machine keeps its own), else the image's "admin".
-    const user = cfg.user ?? env.MESH_QBITTORRENT_USER ?? confUsername(env.MESH_QBITTORRENT_CONFIG_DIR) ?? "admin";
+    const user = cfg.user ?? env.MESH_QBITTORRENT_USER ?? "admin";
    return new QbittorrentClient(url, user, password);
  }

@@ -104,22 +82,19 @@ export class QbittorrentClient {
      headers: { "Content-Type": "application/x-www-form-urlencoded", Referer: this.baseUrl },
      body: new URLSearchParams({ username: this.user, password: this.password }),
    });
-    if (res.status === 401) throw new Error("qBittorrent login rejected — check credentials");
    if (!res.ok) throw new Error(`qBittorrent login: ${res.status} ${await res.text()}`);
-    // 4.x/5.0/5.1 answer 200 "Ok." or 200 "Fails."; 5.2 answers 204 with no body, or 401.
-    const body = (await res.text()).trim();
-    if (res.status !== 204 && body !== "Ok.") {
+    if ((await res.text()).trim() !== "Ok.") {
      throw new Error("qBittorrent login rejected — check credentials");
    }
-    const match = res.headers.get("set-cookie")?.match(/((?:QBT_)?SID(?:_\d+)?)=([^;]+)/);
-    if (!match) throw new Error("qBittorrent login returned no session cookie");
-    this.session = `${match[1]}=${match[2]}`;
+    const match = res.headers.get("set-cookie")?.match(/SID=([^;]+)/);
+    if (!match) throw new Error("qBittorrent login returned no SID cookie");
+    this.sid = match[1];
  }

  private async call(method: "GET" | "POST", path: string, form?: Record<string, string>): Promise<Response> {
-    if (!this.session) await this.login();
+    if (!this.sid) await this.login();
    const doFetch = (): Promise<Response> => {
-      const headers: Record<string, string> = { Referer: this.baseUrl, Cookie: this.session ?? "" };
+      const headers: Record<string, string> = { Referer: this.baseUrl, Cookie: `SID=${this.sid}` };
      const init: RequestInit = { method, headers };
      if (form) {
        headers["Content-Type"] = "application/x-www-form-urlencoded";
@@ -128,8 +103,8 @@ export class QbittorrentClient {
      return fetch(`${this.baseUrl}/api/v2/${path}`, init);
    };
    let res = await doFetch();
-    if (res.status === 403 || res.status === 401) {
-      // The session expired — re-authenticate once and retry, rather than fail a routine call.
+    if (res.status === 403) {
+      // The SID expired — re-authenticate once and retry, rather than fail a routine call.
      await this.login();
      res = await doFetch();
    }
@@ -17,24 +17,10 @@
  "listens": [
    {
      "name": "web",
-      "port": 8112,
+      "port": 8080,
      "protocol": "tcp",
      "from": "mesh",
-      "why": "the download client's pages, and the WebUI API its consumers and its own tools call. qBittorrent refuses a request whose Host names a port other than the one it listens on, so it listens on the machine port itself (host network, WEBUI_PORT=${port:8112}) and the two cannot differ on any machine"
-    },
-    {
-      "name": "peers",
-      "port": 6881,
-      "protocol": "tcp",
-      "from": "mesh",
-      "why": "incoming BitTorrent peer connections; announced to trackers and peers, so qBittorrent listens on the machine port itself (TORRENTING_PORT=${port:6881})"
-    },
-    {
-      "name": "peers-udp",
-      "port": 6881,
-      "protocol": "udp",
-      "from": "mesh",
-      "why": "DHT and uTP on the same number as the peer port"
+      "why": "the download client's pages"
    }
  ],
  "accesses": [
@@ -50,15 +36,10 @@
      "path": "/var/lib/mesh/qbittorrent",
      "mode": "0700"
    },
-    {
-      "id": "state",
-      "type": "directory",
-      "mode": "0700",
-      "place": "."
-    },
    {
      "id": "config",
      "type": "directory",
+      "path": "/services/qbittorrent/config",
      "mode": "0700",
      "owner": "1000:1000"
    },
@@ -66,24 +47,24 @@
      "id": "server",
      "type": "container",
      "name": "qbittorrent",
-      "image": "lscr.io/linuxserver/qbittorrent@sha256:457e4eec2ee3f5e4ef59f237ad51f6143deba9f7445ab48bb5204a98888ef9aa",
+      "image": "lscr.io/linuxserver/qbittorrent@sha256:a00b6a597a3832a1814cde0ef60abc55c94644f3f80902c3432f6af6de8d4a96",
      "env": {
        "PUID": "1000",
        "PGID": "1000",
-        "TZ": "Etc/UTC",
-        "WEBUI_PORT": "${port:8112}",
-        "TORRENTING_PORT": "${port:6881}"
+        "TZ": "Etc/UTC"
      },
-      "volumes": [
-        "${dir:config}:/config",
-        "/services/media/downloads:/downloads"
+      "ports": [
+        "8080"
      ],
-      "network": "host"
+      "volumes": [
+        "/services/qbittorrent/config:/config",
+        "/services/media/downloads:/downloads"
+      ]
    },
    {
      "id": "runtime-config",
      "type": "file",
-      "path": "${dir:state}/config.json",
+      "path": "/var/lib/mesh/qbittorrent/config.json",
      "mode": "0600",
      "content": "{}\n",
      "merge": "json"
@@ -96,46 +77,22 @@
      "volumes": [
        "/var/lib/mesh/qbittorrent/broker:/run/secrets/broker:ro",
        "/var/lib/mesh/qbittorrent/password:/run/secrets/password:ro",
-        "${dir:state}/config.json:/run/config/config.json:ro",
-        "${dir:config}:/var/lib/qbittorrent/config:ro"
+        "/var/lib/mesh/qbittorrent/config.json:/run/config/config.json:ro",
+        "/services/qbittorrent/config:/var/lib/qbittorrent/config:ro"
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_QBITTORRENT_URL": "http://127.0.0.1:${port:8112}",
+        "MESH_QBITTORRENT_URL": "http://127.0.0.1:8080",
        "MESH_QBITTORRENT_PASSWORD_FILE": "/run/secrets/password",
        "MESH_QBITTORRENT_CONFIG_FILE": "/run/config/config.json",
        "MESH_QBITTORRENT_CONFIG_DIR": "/var/lib/qbittorrent/config"
      },
      "restart-on": [
-        "runtime-config",
-        "needs-password"
+        "runtime-config"
      ],
      "artifact": "runtime"
    }
  ],
-  "provides": [
-    "qbittorrent-api"
-  ],
-  "serves": {
-    "qbittorrent-api": {
-      "scheme": "http",
-      "port": 8112,
-      "url-base": "",
-      "username": "admin"
-    }
-  },
-  "requires": [
-    "route"
-  ],
-  "contributes": {
-    "route": {
-      "label": "qbittorrent",
-      "endpoint": "web"
-    }
-  },
-  "binds": {
-    "route": "${dir:state}/route.json"
-  },
  "build": {
    "on": [
      {
@@ -75,7 +75,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_RADARR_URL": "http://127.0.0.1:${port:7878}",
+        "MESH_RADARR_URL": "http://127.0.0.1:7878",
        "MESH_RADARR_CONFIG_DIR": "/var/lib/radarr/config"
      },
      "artifact": "runtime"
@@ -100,7 +100,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_SEARXNG_URL": "http://127.0.0.1:${port:8080}",
+        "MESH_SEARXNG_URL": "http://127.0.0.1:8080",
        "MESH_SEARXNG_CONFIG_FILE": "/run/config/config.json"
      },
      "restart-on": [
@@ -80,7 +80,7 @@
      ],
      "env": {
        "MESH_BROKER_FILE": "/run/secrets/broker",
-        "MESH_SONARR_URL": "http://127.0.0.1:${port:8989}",
+        "MESH_SONARR_URL": "http://127.0.0.1:8989",
        "MESH_SONARR_CONFIG_DIR": "/var/lib/sonarr/config"
      },
      "artifact": "runtime"