Compare commits

..
Author SHA1 Message Date
jschoubben 991e33f749 tautulli: reach plex through the mesh, written before Tautulli starts
Tautulli reached plex at 172.18.0.1, the gateway of a HAL network that goes
away with HAL, and the plan was to retype it by hand in the window. Tautulli
now requires plex-api, and where plex is comes from the binding.

Tautulli keeps the connection only in config.ini, reads it at start and
writes its whole config back on every shutdown; its API cannot set it and
its settings form needs an admin login. So a step after start would be
overwritten the moment the container is recreated. The write is made where
nothing can overwrite it: the linuxserver image's custom-init runs
plex/mesh-plex.py as root before Tautulli starts, and the server restarts on
its binding and credential, so a moved plex or an accepted token lands.

It writes only [PMS] keys, only when they differ, every other line byte for
byte: pms_ip, pms_port, pms_ssl and pms_url from the binding; pms_identifier
from plex's /identity; pms_token only when plex takes it. A minted value -
before the operator accepts the server's X-Plex-Token for this pair - is
never written, while the address still is, so Tautulli's own working token
keeps working at plex's new address.

A failure in custom-init is a log line nobody reads, so a run-once `plex`
step, declared last so it gates nothing (ADR 0136), checks what the mesh can
report: plex takes the credential (else it names the secret accept), Tautulli
holds the bound URL, and Tautulli says it is connected. It writes nothing.

The script is kept as plex/mesh-plex.py and plex/50-mesh-plex; module.json
carries copies, and a test fails when they differ. Tests run the script with
python3 against a fake plex (skipped where there is none) and the step
against fakes; `npm test` builds first.
2026-09-30 13:09:43 +02:00
jschoubben 3e378170df tautulli: its config is placed, it runs the build in use, and its runtime finds its own key
The module stated /services/tautulli/config and /var/lib/mesh/tautulli/route.json,
novox's layout, which no definition may carry (ADR 0112). Tautulli's config dir is
now a placed directory (${dir:config}) and the route binds into the placed state
(${dir:state}), as searxng and mosquitto do.

The image was pinned to v2.18.1-ls242; ace runs ls244 (2026-09-11), and Tautulli
migrates its own database schema, so the pin moves to the digest ace runs.

The runtime called http://127.0.0.1:8181, which is the software's port, not the
machine port the mesh assigns; it now asks with ${port:8181}, as gitea does.

The runtime mounted Tautulli's config dir read-only but never read it: its API key
could only come from MESH_TAUTULLI_APIKEY or the settings-merged config.json, i.e.
a secret in settings. Tautulli mints and owns that key in its config.ini, so the
runtime now reads it from there. Nothing for the mesh to mint or accept.

Verified: catalogue tests with MESH_CATALOGUE pointing here (not skipped); the
pinned image started in a throwaway container on a dir owned 1001:2000 with
PUID/PGID 1000 answers /status 200 and re-owns /config to 1000:1000 on start;
client.ts, run under node, read the key from that instance's config.ini and got
success from get_activity and get_history; without a config.ini it throws, which
the tools and events entrypoints already treat as "not configured".
2026-09-29 23:41:47 +02:00
31 changed files with 892 additions and 2066 deletions
+1 -1
View File
@@ -94,7 +94,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BAZARR_URL": "http://127.0.0.1:${port:6767}",
"MESH_BAZARR_URL": "http://127.0.0.1:6767",
"MESH_BAZARR_API_KEY_FILE": "/run/secrets/api-key",
"MESH_BAZARR_CONFIG_FILE": "/run/config/config.json",
"MESH_BAZARR_CONFIG_DIR": "/var/lib/bazarr/config"
+1 -1
View File
@@ -76,7 +76,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BOOKSHELF_URL": "http://127.0.0.1:${port:8787}",
"MESH_BOOKSHELF_URL": "http://127.0.0.1:8787",
"MESH_BOOKSHELF_CONFIG_DIR": "/var/lib/bookshelf/config"
},
"artifact": "runtime"
+1 -4
View File
@@ -13,7 +13,7 @@ ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/lidarr
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts downloads/settings.ts downloads/index.ts \
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
@@ -22,6 +22,3 @@ COPY --from=build /app/modules/lidarr/dist /app/modules/lidarr/dist
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/lidarr/dist/index.js,/app/modules/lidarr/dist/tools/index.js
# NOT dist/downloads/index.js: that is a step the host runs to completion, named by the `downloads`
# container's args as `mesh-tools run …` (novox/hq ADR 0052). Listed here it would run inside the
# serving sidecar too, and exit it.
-188
View File
@@ -1,188 +0,0 @@
// The downloads step — run once by the host after the app's server starts, and again whenever a
// binding, a pair credential or the step's settings change (the container's `restart-on`,
// novox/hq ADR 0099). Byte-identical in sonarr, radarr, lidarr and bookshelf; see settings.ts.
//
// **A step, not a loop**: everything it does is a function of files the mesh writes, and the host
// already knows when they change. It connects to no broker.
//
// Exits non-zero when anything could not be put right — a refused credential, an unreachable
// provider, an entry the app would not save — so the node reports the step failed and the host
// runs it again on the next apply. Declared last in the manifest, so its failing gates nothing
// else of the module's (novox/hq ADR 0136).
//
// Reads, in MESH_DOWNLOADS_DIR, `<provision>.json` (the binding), `<provision>.secret` (the pair
// credential) and the settings file; remembers in MESH_DOWNLOADS_MEMORY the jackett hosts it has
// pointed feeds at, so a jackett that moves takes its feeds with it. Never prints a credential.
import { mkdir, rename, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import {
CLIENTS,
JACKETT,
acceptRemedy,
appConfig,
appReady,
listEntries,
providerTakes,
readIfThere,
readJson,
reconcileClient,
reconcileIndexers,
scrub,
stepSettings,
wanted,
type App,
type Binding,
type Http,
type Entry,
type Outcome,
type Took,
} from "./settings.js";
const module = process.env.MESH_DOWNLOADS_APP ?? "app";
const dir = process.env.MESH_DOWNLOADS_DIR ?? "/run/downloads";
const settingsFile = process.env.MESH_DOWNLOADS_SETTINGS ?? join(dir, "downloads.json");
const memoryFile = process.env.MESH_DOWNLOADS_MEMORY ?? "/var/lib/downloads/memory.json";
const waitSeconds = Number(process.env.MESH_DOWNLOADS_WAIT_SECONDS ?? "180");
const tag = `[${module}-downloads]`;
const http: Http = { fetch: (u, init) => fetch(u, init) };
const secrets: string[] = [];
const say = (line: string) => console.log(scrub(`${tag} ${line}`, secrets));
const fail = (line: string) => console.error(scrub(`${tag} ${line}`, secrets));
const config = appConfig((await readIfThere(process.env.MESH_DOWNLOADS_APP_CONFIG)) ?? "");
if (!config.apiKey) {
fail(`no API key in ${module}'s config.xml yet — ${module} writes it on its first start; the step runs again on the next apply`);
process.exit(1);
}
secrets.push(config.apiKey);
const app: App = {
module,
url: `${(process.env.MESH_DOWNLOADS_APP_URL ?? "http://127.0.0.1").replace(/\/$/, "")}${config.urlBase}`,
apiKey: config.apiKey,
api: process.env.MESH_DOWNLOADS_API ?? "v3",
};
const settings = stepSettings(await readJson(settingsFile));
if (!(await appReady(http, app, waitSeconds * 1000))) {
fail(`${module} did not answer at ${app.url} within ${waitSeconds}s`);
process.exit(1);
}
const outcomes: Outcome[] = [];
// The download clients.
let clients: Entry[];
try {
clients = await listEntries(http, app, "downloadclient");
} catch (err) {
fail(err instanceof Error ? err.message : String(err));
process.exit(1);
}
for (const kind of CLIENTS) {
const credential = await readIfThere(join(dir, `${kind.provision}.secret`));
if (credential) secrets.push(credential.trim());
const w = wanted(kind.provision, (await readJson(join(dir, `${kind.provision}.json`))) as Binding | undefined, credential, true);
// `in`, not `!w.ok`: the Dockerfile compiles without strict, where a boolean discriminant does not
// narrow.
if ("problem" in w) {
outcomes.push({ what: kind.provision, result: "refused", problem: w.problem });
continue;
}
const ep = w.endpoint;
try {
if (!(await providerTakes(http, kind.provision, ep)).took) {
outcomes.push({
what: kind.provision,
result: "refused",
problem: acceptRemedy(app, settings.node, kind.provision, kind.provider, kind.secretName, ep.from),
});
continue;
}
} catch (err) {
outcomes.push({
what: kind.provision,
result: "refused",
problem: `${kind.provider} could not be asked whether it takes the credential at ${ep.host}:${ep.port}: ${err instanceof Error ? err.message : String(err)}`,
});
continue;
}
outcomes.push(...(await reconcileClient(http, app, kind, settings.names[kind.provision], ep, clients)));
}
// The indexers, through jackett.
{
const credential = await readIfThere(join(dir, `${JACKETT}.secret`));
if (credential) secrets.push(credential.trim());
const w = wanted(JACKETT, (await readJson(join(dir, `${JACKETT}.json`))) as Binding | undefined, credential, false);
if ("problem" in w) {
outcomes.push({ what: JACKETT, result: "refused", problem: w.problem });
} else {
const ep = w.endpoint;
let took: Took | undefined;
try {
took = await providerTakes(http, JACKETT, ep);
} catch (err) {
outcomes.push({
what: JACKETT,
result: "refused",
problem: `jackett could not be asked whether it takes the key at ${ep.host}:${ep.port}: ${err instanceof Error ? err.message : String(err)}`,
});
}
if (took && !took.took) {
outcomes.push({ what: JACKETT, result: "refused", problem: acceptRemedy(app, settings.node, JACKETT, "jackett", "API key", ep.from) });
} else if (took) {
const memory = ((await readJson(memoryFile)) ?? {}) as Record<string, { hosts?: string[] } | undefined>;
const remembered = Array.isArray(memory[JACKETT]?.hosts) ? (memory[JACKETT]?.hosts as string[]) : [];
try {
const indexers = await listEntries(http, app, "indexer");
outcomes.push(
...(await reconcileIndexers(http, app, ep, took.configured ?? new Map(), settings.indexers, remembered, indexers)),
);
} catch (err) {
outcomes.push({ what: JACKETT, result: "refused", problem: err instanceof Error ? err.message : String(err) });
}
// Remember where the feeds now point, so they are still recognised as the mesh's if jackett
// moves. Written whole and renamed, so a crash leaves the old memory, never half of one.
const hosts = [...new Set([...remembered, ep.host.toLowerCase()])].sort();
if (hosts.join() !== [...remembered].sort().join()) {
try {
await mkdir(dirname(memoryFile), { recursive: true });
await writeFile(`${memoryFile}.tmp`, JSON.stringify({ ...memory, [JACKETT]: { hosts } }, null, 2) + "\n", { mode: 0o600 });
await rename(`${memoryFile}.tmp`, memoryFile);
} catch (err) {
outcomes.push({
what: JACKETT,
result: "notice",
note: `could not remember ${ep.host} as a jackett host (${err instanceof Error ? err.message : String(err)}); if jackett moves, list it in downloads.jackett-api.adopt-hosts`,
});
}
}
}
}
}
let failed = 0;
for (const o of outcomes) {
switch (o.result) {
case "unchanged":
say(`${o.what}: already as the mesh says${o.untested ? "" : "; connection tested"}`);
break;
case "written":
say(`${o.what}: wrote ${o.fields.join(", ")}${o.untested ? "" : "; connection tested"}`);
break;
case "created":
say(`${o.what}: registered; connection tested`);
break;
case "notice":
say(`${o.what}: ${o.note}`);
break;
case "refused":
failed++;
fail(`${o.what}: ${o.problem}`);
break;
}
}
process.exitCode = failed > 0 ? 1 : 0;
-778
View File
@@ -1,778 +0,0 @@
// Where a Servarr app reaches its download clients and its indexer proxy — decided by the mesh,
// written into the app by the app's own API.
//
// **One file, four copies.** sonarr, radarr, lidarr and bookshelf (a Readarr fork) keep their
// download clients and indexers behind the same API — `/api/v3/…` for the first two, `/api/v1/…`
// for the others — so this step is the same code in each. Each module builds from its own
// directory (novox/hq ADR 0069), so each carries a byte-identical copy under `downloads/`;
// `test/downloads.test.ts` checks the copies agree wherever the siblings are checked out beside
// it. Change all four together.
//
// **Why this exists.** The app keeps its connection to nzbget, qBittorrent and jackett in its own
// database, not in a file, so the mesh has nowhere to write `${bound:nzbget-api:at}` for it. The
// module requires `nzbget-api`, `qbittorrent-api` and `jackett-api`; the mesh delivers, for each,
// a binding (where the provider is — `at` — and what it serves: `port`, `scheme`, `url-base`, and
// for a download client the `username`) and a pair credential (nzbget's ControlPassword,
// qBittorrent's WebUI password, jackett's API key — each the provider's one and only, accepted by
// the operator per pair; the mesh cannot mint them). This step reads those files and makes the
// app's entries say the same thing.
//
// **Which entries are the mesh's.** Never a guess from what an entry looks like:
// - a download client is the mesh's when its name is the one the module's settings give for
// that provision (`downloads.<provision>.name`) and its kind is that provider's. On a fresh
// machine the step registers it under that name; on a migrated one the assignment names the
// entry the migration adopts ("NZBGet" on ace).
// - a Torznab indexer is the mesh's when it reads a jackett feed
// (`…/api/v2.0/indexers/<id>/results/torznab`) on a host the mesh put there — the bound `at`,
// one it bound before (remembered by the step), or one the settings adopt
// (`downloads.jackett-api.adopt-hosts`, which is how a migration names the entries that pointed
// at the old container). `downloads.jackett-api.indexers` lists the jackett indexers the app
// should have; one missing is registered.
// Everything else — every entry a person made, an nzbget elsewhere, a seedbox's jackett, a Newznab
// indexer — is left exactly as it is. **Nothing is ever deleted.**
//
// **Only the connection, and only when it differs.** Host, port, TLS, base path, user name and
// the credential. Categories, priorities, "enabled", seed criteria and every other choice made in
// the app are left alone. The stored credential cannot be read back (the app masks it), so the
// step asks the app to test the entry with the mesh's host and port and the credential it already
// holds: passing, the credential is already the provider's one and only; failing, the delivered
// credential is written.
//
// **A credential the provider refuses is never written.** Until the operator accepts the
// provider's secret for this pair, the mesh delivers a value it minted itself, which no provider
// will ever accept (novox/hq ADR 0092). Writing it would replace a working password with a dead
// one. So it is tried against the provider first; refused, nothing of that provision is written
// and the step fails naming the `secret accept` that fixes it.
//
// Pure logic and a small HTTP seam, so it is tested against fake servers (test/downloads.test.ts).
// Never prints a credential, an API key, or a URL carrying one.
import { readFile } from "node:fs/promises";
/** A download-client provision and the shape of its entry in the app. */
export interface ClientKind {
provision: "nzbget-api" | "qbittorrent-api";
/** The app's `implementation` for it. */
implementation: "Nzbget" | "QBittorrent";
/** The provider, as a person calls it. */
provider: string;
/** What the pair credential is, in the provider's words. */
secretName: string;
}
export const CLIENTS: readonly ClientKind[] = [
{ provision: "nzbget-api", implementation: "Nzbget", provider: "nzbget", secretName: "ControlPassword" },
{ provision: "qbittorrent-api", implementation: "QBittorrent", provider: "qBittorrent", secretName: "WebUI password" },
];
export const JACKETT = "jackett-api";
/** What the mesh wrote at `binds.<provision>`: the binding document. */
export interface Binding {
provision?: string;
from?: string;
at?: string;
as?: string;
serves?: Record<string, unknown>;
}
/** Where a provider is, as the mesh bound it, and the credential for it. */
export interface Endpoint {
scheme: "http" | "https";
host: string;
port: number;
/** "" at the root, otherwise "/base" — one leading slash, none trailing. */
urlBase: string;
/** The user a download client logs in as; "" for jackett, which takes a key. */
username: string;
credential: string;
/** The provider's node, for the `secret accept` remedy. */
from: string;
}
export type Wanted = { ok: true; endpoint: Endpoint } | { ok: false; problem: string };
/**
* The endpoint the mesh says to use, from a binding and its pair credential.
*
* Refused rather than guessed when the binding cannot be dialled from the app's own container: a
* loopback `at` — what the mesh hands a machine that is not on the private network — is the app's
* container itself.
*/
export function wanted(
provision: string,
binding: Binding | undefined,
credential: string | undefined,
needsUser: boolean,
): Wanted {
if (!binding) {
return { ok: false, problem: `no binding for ${provision} was delivered — the mesh writes it before this step runs` };
}
const host = typeof binding.at === "string" ? binding.at.trim() : "";
const serves = binding.serves ?? {};
const port = Number(serves.port);
if (!host) return { ok: false, problem: `the ${provision} binding names no host (at)` };
if (isLoopback(host)) {
return {
ok: false,
problem:
`the ${provision} binding says the provider is at ${host}, which from the app's own container is the ` +
`app itself. The mesh hands loopback to a machine that is not on the private network; put it on the ` +
`private network so the provider has an address the app can dial`,
};
}
if (!Number.isInteger(port) || port <= 0 || port > 65535) {
return { ok: false, problem: `the ${provision} binding serves no usable port (${String(serves.port)})` };
}
const scheme = typeof serves.scheme === "string" && serves.scheme ? serves.scheme : "http";
if (scheme !== "http" && scheme !== "https") {
return { ok: false, problem: `the ${provision} binding serves scheme ${scheme}, which the app cannot dial` };
}
const username = typeof serves.username === "string" ? serves.username.trim() : "";
if (needsUser && !username) {
return { ok: false, problem: `the ${provision} binding serves no username for the app to log in as` };
}
const key = (credential ?? "").trim();
if (!key) return { ok: false, problem: `the ${provision} credential is empty or was not delivered` };
return {
ok: true,
endpoint: {
scheme,
host,
port,
urlBase: normBase(serves["url-base"]),
username,
credential: key,
from: typeof binding.from === "string" ? binding.from : "",
},
};
}
/** A URL base as "" or "/x/y": slashes trimmed, one put back in front. */
export function normBase(v: unknown): string {
const s = typeof v === "string" ? v.trim().replace(/^\/+|\/+$/g, "") : "";
return s === "" ? "" : `/${s}`;
}
function isLoopback(host: string): boolean {
const h = host.toLowerCase();
return h === "localhost" || h === "::1" || h === "[::1]" || /^127\./.test(h);
}
function hostForUrl(host: string): string {
return host.includes(":") && !host.startsWith("[") ? `[${host}]` : host;
}
/** The provider's base URL, as the step dials it and as the app is given it. */
export function baseUrl(ep: Endpoint): string {
return `${ep.scheme}://${hostForUrl(ep.host)}:${ep.port}${ep.urlBase}`;
}
// ---------------------------------------------------------------------------------------------
// HTTP
/** The HTTP the step needs, so a test can stand fakes in for the app and the providers. */
export interface Http {
fetch(
url: string,
init?: { method?: string; headers?: Record<string, string>; body?: string },
): Promise<{ status: number; text(): Promise<string> }>;
}
/** The app, as the step reaches it from the host. */
export interface App {
/** The module, for messages and the remedy: sonarr, radarr, lidarr, bookshelf. */
module: string;
/** Its own URL, base path included. */
url: string;
apiKey: string;
/** v3 (sonarr, radarr) or v1 (lidarr, bookshelf). */
api: string;
}
interface Answer {
status: number;
body: unknown;
}
async function appCall(http: Http, app: App, method: string, path: string, body?: unknown): Promise<Answer> {
const res = await http.fetch(`${app.url.replace(/\/$/, "")}/api/${app.api}${path}`, {
method,
headers: {
"X-Api-Key": app.apiKey,
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
const text = await res.text();
let parsed: unknown = undefined;
if (text) {
try {
parsed = JSON.parse(text) as unknown;
} catch {
parsed = text;
}
}
return { status: res.status, body: parsed };
}
async function appGet(http: Http, app: App, path: string): Promise<unknown> {
const a = await appCall(http, app, "GET", path);
if (a.status < 200 || a.status >= 300) throw new Error(`${app.module} GET ${path} answered ${a.status}`);
return a.body;
}
/** One of the app's validation failures: the only parts of an answer the step ever prints. */
interface Failure {
property: string;
message: string;
warning: boolean;
}
function failuresOf(body: unknown): Failure[] {
const list = Array.isArray(body) ? body : [];
return list.map((f) => {
const o = (f ?? {}) as Record<string, unknown>;
return {
property: String(o.propertyName ?? ""),
message: String(o.errorMessage ?? ""),
warning: o.isWarning === true,
};
});
}
/**
* One shape rather than a union on `ok`: the Dockerfile compiles without strict, where a boolean
* discriminant does not narrow.
*/
interface Verdict {
ok: boolean;
/** Warnings the app raised on a pass; errors (and any warnings) on a failure. */
failures: Failure[];
status: number;
saved?: Entry;
}
function verdict(a: Answer): Verdict {
if (a.status >= 200 && a.status < 300) return { ok: true, failures: [], status: a.status };
const failures = failuresOf(a.body);
if (a.status === 400 && failures.length > 0 && failures.every((f) => f.warning)) return { ok: true, failures, status: a.status };
return { ok: false, failures, status: a.status };
}
/** The app's own test of an entry, from the app's own container. */
async function appTest(http: Http, app: App, resource: string, entry: Entry): Promise<Verdict> {
return verdict(await appCall(http, app, "POST", `/${resource}/test`, entry));
}
/**
* Save an entry. The app tests an enabled entry before saving it; a test that passes with only
* warnings ("a category is recommended") is saved with `forceSave`, the same as pressing "save
* anyway" in its screen. An error is never forced.
*/
async function appSave(http: Http, app: App, resource: string, entry: Entry, create: boolean): Promise<Verdict> {
const path = create ? `/${resource}` : `/${resource}/${entry.id}`;
const method = create ? "POST" : "PUT";
let a = await appCall(http, app, method, path, entry);
let v = verdict(a);
if (v.ok && (a.status < 200 || a.status >= 300)) {
a = await appCall(http, app, method, `${path}?forceSave=true`, entry);
v = verdict(a);
if (v.ok && (a.status < 200 || a.status >= 300)) {
return { ok: false, failures: failuresOf(a.body), status: a.status };
}
}
if (!v.ok) return v;
return { ...v, saved: (a.body ?? undefined) as Entry | undefined };
}
// ---------------------------------------------------------------------------------------------
// Entries
/** An entry as the app's API gives it: a download client or an indexer. */
export interface Entry {
id?: number;
name?: string;
implementation?: string;
enable?: boolean;
enableRss?: boolean;
enableAutomaticSearch?: boolean;
enableInteractiveSearch?: boolean;
supportsRss?: boolean;
supportsSearch?: boolean;
fields?: { name: string; value?: unknown; [k: string]: unknown }[];
[k: string]: unknown;
}
export function field(e: Entry, name: string): unknown {
return e.fields?.find((f) => f.name === name)?.value;
}
/** The entry with these fields' values replaced — added when the app's shape lacks one. */
export function withFields(e: Entry, values: Record<string, unknown>): Entry {
const fields = (e.fields ?? []).map((f) => (f.name in values ? { ...f, value: values[f.name] } : { ...f }));
for (const [name, value] of Object.entries(values)) {
if (!fields.some((f) => f.name === name)) fields.push({ name, value });
}
return { ...e, fields };
}
function enabled(e: Entry): boolean {
return e.enable === true || e.enableRss === true || e.enableAutomaticSearch === true || e.enableInteractiveSearch === true;
}
/** What one provision, or one entry of it, came to. */
export type Outcome =
| { what: string; result: "unchanged"; untested?: boolean }
| { what: string; result: "written"; fields: string[]; untested?: boolean }
| { what: string; result: "created"; fields: string[] }
| { what: string; result: "notice"; note: string }
| { what: string; result: "refused"; problem: string };
function said(fs: Failure[]): string {
return fs.length === 0 ? "no reason given" : fs.map((f) => (f.property ? `${f.property}: ${f.message}` : f.message)).join("; ");
}
/**
* Bring one of the mesh's entries in line: `connection` is the non-secret fields as the mesh says
* them, `differs` which of them the entry does not already say, `secretField` where the credential
* goes. Never throws: every failure is an outcome with a reason.
*/
export async function reconcileEntry(
http: Http,
app: App,
resource: "downloadclient" | "indexer",
what: string,
current: Entry,
connection: Record<string, unknown>,
differs: string[],
secretField: string,
credential: string,
testable = true,
): Promise<Outcome> {
try {
const base = withFields(current, connection); // the credential as the app holds it — masked
if (!testable) {
// Nothing to test against (jackett no longer has the indexer it names): the connection is
// still the mesh's to state, and the key goes with it when the address changes.
if (differs.length === 0) return { what, result: "unchanged", untested: true };
const s = await appSave(http, app, resource, withFields(base, { [secretField]: credential }), false);
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not save it: ${said(s.failures)}` };
return { what, result: "written", fields: [...differs, secretField], untested: true };
}
const first = await appTest(http, app, resource, base);
if (first.ok) {
// The credential the app holds already works against the provider at the mesh's address —
// and the provider has exactly one, so it is the delivered one.
if (differs.length === 0) return { what, result: "unchanged" };
const s = await appSave(http, app, resource, base, false);
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not save it: ${said(s.failures)}` };
return { what, result: "written", fields: differs };
}
// The app's test failed with what it holds. The delivered credential was already checked
// against the provider, so writing it is safe; the app tests an enabled entry again on save.
const s = await appSave(http, app, resource, withFields(base, { [secretField]: credential }), false);
if (!s.ok) {
return {
what,
result: "refused",
problem: `${app.module} tested it at the mesh's address and would not save it (nothing was written): ${said(s.failures)}`,
};
}
const stored = (await appGet(http, app, `/${resource}/${current.id}`)) as Entry;
const second = await appTest(http, app, resource, stored);
const fields = [...differs, secretField];
if (second.ok) return { what, result: "written", fields };
const why = `written (${fields.join(", ")}), and ${app.module}'s own test still fails: ${said(second.failures)}`;
return enabled(stored) ? { what, result: "refused", problem: why } : { what, result: "notice", note: `${why} — it is disabled, so nothing uses it` };
} catch (err) {
return { what, result: "refused", problem: message(err) };
}
}
/**
* Register a new entry from the app's own template for that kind: its defaults, the name, the
* connection and the credential. A category the provider does not have (nzbget refuses one it was
* not told about) is left empty rather than failing the registration, and said.
*/
export async function createEntry(
http: Http,
app: App,
resource: "downloadclient" | "indexer",
what: string,
implementation: string,
name: string,
values: Record<string, unknown>,
): Promise<Outcome> {
try {
const schema = (await appGet(http, app, `/${resource}/schema`)) as Entry[];
const template = (schema ?? []).find((s) => s.implementation === implementation);
if (!template) return { what, result: "refused", problem: `${app.module} has no ${implementation} to register` };
let entry: Entry = withFields({ ...template, id: undefined, name }, values);
if (resource === "downloadclient") entry.enable = true;
else {
entry.enableRss = template.supportsRss !== false;
entry.enableAutomaticSearch = template.supportsSearch !== false;
entry.enableInteractiveSearch = template.supportsSearch !== false;
}
let s = await appSave(http, app, resource, entry, true);
let note = "";
if (!s.ok && s.failures.length > 0 && s.failures.every((f) => /category/i.test(f.property))) {
const emptied: Record<string, unknown> = {};
for (const f of entry.fields ?? []) if (/category$/i.test(f.name) && !/imported/i.test(f.name)) emptied[f.name] = "";
entry = withFields(entry, emptied);
s = await appSave(http, app, resource, entry, true);
note = " with its category left empty — the provider has none by the app's default name; set one in both";
}
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not register it: ${said(s.failures)}` };
const created = s.saved;
if (created?.id !== undefined) {
const t = await appTest(http, app, resource, (await appGet(http, app, `/${resource}/${created.id}`)) as Entry);
if (!t.ok) {
return { what, result: "refused", problem: `registered as "${name}"${note}, and ${app.module}'s own test fails: ${said(t.failures)}` };
}
}
return note
? { what, result: "notice", note: `registered as "${name}"${note}` }
: { what, result: "created", fields: Object.keys(values) };
} catch (err) {
return { what, result: "refused", problem: message(err) };
}
}
// ---------------------------------------------------------------------------------------------
// Providers: does it take the credential?
/**
* Does the provider take this credential? `took` it does (for jackett, with the ids of the indexers
* it has configured); `refused` it said no; a thrown error when it could not be asked.
*/
export interface Took {
took: boolean;
configured?: Map<string, string>;
}
export async function providerTakes(http: Http, provision: string, ep: Endpoint): Promise<Took> {
const base = baseUrl(ep);
if (provision === "nzbget-api") {
const auth = Buffer.from(`${ep.username}:${ep.credential}`).toString("base64");
const res = await http.fetch(`${base}/jsonrpc/version`, { method: "GET", headers: { Authorization: `Basic ${auth}` } });
await res.text();
if (res.status === 401 || res.status === 403) return { took: false };
if (res.status >= 200 && res.status < 300) return { took: true };
throw new Error(`nzbget answered ${res.status}`);
}
if (provision === "qbittorrent-api") {
const form = `username=${encodeURIComponent(ep.username)}&password=${encodeURIComponent(ep.credential)}`;
const res = await http.fetch(`${base}/api/v2/auth/login`, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: form,
});
const text = (await res.text()).trim();
if (res.status === 401 || /^fails\.?$/i.test(text)) return { took: false };
if (res.status === 403) {
throw new Error(
"qBittorrent answered 403: it has banned this address after failed logins (WebUI\\BanDuration); it lifts by itself",
);
}
if (res.status >= 200 && res.status < 300) return { took: true };
throw new Error(`qBittorrent answered ${res.status}`);
}
if (provision === JACKETT) {
const url = `${base}/api/v2.0/indexers/all/results/torznab/api?t=indexers&configured=true&apikey=${encodeURIComponent(ep.credential)}`;
const res = await http.fetch(url, { method: "GET" });
const text = await res.text();
if (res.status === 401 || res.status === 403) return { took: false };
if (res.status < 200 || res.status >= 300) throw new Error(`jackett answered ${res.status}`);
// jackett answers a wrong key 200 with an error document: code 100, "Invalid API Key".
if (/<error\b[^>]*\bcode="100"/i.test(text)) return { took: false };
if (/<error\b/i.test(text)) throw new Error("jackett answered with an error document");
return { took: true, configured: configuredIndexers(text) };
}
throw new Error(`no check for ${provision}`);
}
/** jackett's configured indexers, id → title, from its `t=indexers` feed. */
export function configuredIndexers(xml: string): Map<string, string> {
const out = new Map<string, string>();
const re = /<indexer\b[^>]*\bid="([^"]+)"[^>]*>([\s\S]*?)<\/indexer>/g;
for (let m = re.exec(xml); m; m = re.exec(xml)) {
const title = /<title>([\s\S]*?)<\/title>/.exec(m[2])?.[1]?.trim() ?? m[1];
out.set(m[1], decodeXml(title));
}
return out;
}
function decodeXml(s: string): string {
return s.replace(/&lt;/g, "<").replace(/&gt;/g, ">").replace(/&quot;/g, '"').replace(/&apos;/g, "'").replace(/&amp;/g, "&");
}
/** The remedy for a refused credential, in the controller's own words (ADR 0092). */
export function acceptRemedy(app: App, node: string, provision: string, provider: string, secretName: string, from: string): string {
return (
`${provider} refuses the ${provision} credential the mesh delivered, so nothing of ${provision} was written ` +
`into ${app.module}. ${provider} has one ${secretName} and the mesh cannot make it: accept it for this pair — ` +
`\`secret accept ${node || "<this node>"} ${app.module} ${provision} --provider ${from || "<its node>"} ` +
`--from <file holding ${provider}'s ${secretName}>\``
);
}
// ---------------------------------------------------------------------------------------------
// Download clients
/** The non-secret fields of a download client as the mesh says them. */
export function clientConnection(ep: Endpoint): Record<string, unknown> {
return { host: ep.host, port: ep.port, useSsl: ep.scheme === "https", urlBase: ep.urlBase === "" ? null : ep.urlBase, username: ep.username };
}
/** Which of them the entry does not already say. Names only. */
export function clientDiffers(e: Entry, ep: Endpoint): string[] {
const out: string[] = [];
if (String(field(e, "host") ?? "").toLowerCase() !== ep.host.toLowerCase()) out.push("host");
if (Number(field(e, "port") ?? 0) !== ep.port) out.push("port");
if (Boolean(field(e, "useSsl")) !== (ep.scheme === "https")) out.push("useSsl");
if (normBase(field(e, "urlBase")) !== ep.urlBase) out.push("urlBase");
if (String(field(e, "username") ?? "") !== ep.username) out.push("username");
return out;
}
/**
* Bring the app's download client for one provision in line: the entry by that name and kind is
* the mesh's and is updated; none, it is registered. An entry by that name of another kind, or by
* that name in another case, is somebody else's and is refused rather than touched.
*/
export async function reconcileClient(
http: Http,
app: App,
kind: ClientKind,
name: string,
ep: Endpoint,
entries: Entry[],
): Promise<Outcome[]> {
const what = `${kind.provision} ("${name}")`;
const exact = entries.filter((e) => e.name === name);
const alike = entries.filter((e) => e.name !== name && String(e.name ?? "").toLowerCase() === name.toLowerCase());
if (exact.length > 0 && exact[0].implementation !== kind.implementation) {
return [{
what,
result: "refused",
problem: `the download client named "${name}" is a ${exact[0].implementation}, not ${kind.implementation}; it is not the mesh's and was left alone. Name the mesh's entry otherwise in downloads.${kind.provision}.name`,
}];
}
if (exact.length === 0 && alike.length > 0) {
return [{
what,
result: "refused",
problem: `${app.module} already has "${alike[0].name}", which is not the mesh's name ("${name}") and was left alone. To have the mesh manage it, set downloads.${kind.provision}.name to "${alike[0].name}"`,
}];
}
const outcomes: Outcome[] = [];
const others = entries.filter((e) => e.implementation === kind.implementation && e.name !== name);
if (others.length > 0) {
outcomes.push({
what,
result: "notice",
note: `also present and not the mesh's, left alone: ${others.map((o) => `"${o.name}"`).join(", ")}`,
});
}
if (exact.length === 0) {
outcomes.push(
await createEntry(http, app, "downloadclient", what, kind.implementation, name, {
...clientConnection(ep),
password: ep.credential,
}),
);
return outcomes;
}
outcomes.push(
await reconcileEntry(http, app, "downloadclient", what, exact[0], clientConnection(ep), clientDiffers(exact[0], ep), "password", ep.credential),
);
return outcomes;
}
// ---------------------------------------------------------------------------------------------
// Indexers
const FEED = /\/api\/v2\.0\/indexers\/([^/]+)\/results\/torznab\/?$/;
/** A Torznab entry reading a jackett feed: the host it points at and the jackett indexer's id. */
export function jackettFeed(e: Entry): { host: string; id: string } | undefined {
if (e.implementation !== "Torznab") return undefined;
const raw = String(field(e, "baseUrl") ?? "").trim();
if (!raw) return undefined;
let u: URL;
try {
u = new URL(raw);
} catch {
return undefined;
}
const path = `${u.pathname.replace(/\/+$/, "")}/${String(field(e, "apiPath") ?? "").replace(/^\/+/, "")}`;
const m = FEED.exec(path);
if (!m) return undefined;
return { host: u.hostname.replace(/^\[|\]$/g, "").toLowerCase(), id: decodeURIComponent(m[1]) };
}
/** The feed's two fields as the mesh says them. */
export function feedConnection(ep: Endpoint, id: string): Record<string, unknown> {
return { baseUrl: baseUrl(ep), apiPath: `/api/v2.0/indexers/${id}/results/torznab/` };
}
function sameUrl(a: string, b: string): boolean {
try {
const x = new URL(a);
const y = new URL(b);
const port = (u: URL) => u.port || (u.protocol === "https:" ? "443" : "80");
return (
x.protocol === y.protocol &&
x.hostname.toLowerCase() === y.hostname.toLowerCase() &&
port(x) === port(y) &&
x.pathname.replace(/\/+$/, "") === y.pathname.replace(/\/+$/, "")
);
} catch {
return false;
}
}
export function feedDiffers(e: Entry, ep: Endpoint, id: string): string[] {
const want = feedConnection(ep, id);
const out: string[] = [];
if (!sameUrl(String(field(e, "baseUrl") ?? ""), String(want.baseUrl))) out.push("baseUrl");
const path = (v: unknown) => `/${String(v ?? "").replace(/^\/+|\/+$/g, "")}`;
if (path(field(e, "apiPath")) !== path(want.apiPath)) out.push("apiPath");
return out;
}
export interface IndexerSettings {
/** Hosts whose jackett feeds the mesh takes over — a migration's old names. */
adoptHosts: string[];
/** jackett indexer ids the app should have. */
indexers: string[];
}
/**
* Bring the app's jackett feeds in line: every one the mesh manages is pointed at the bound jackett
* with its key, and every listed jackett indexer the app lacks is registered.
*/
export async function reconcileIndexers(
http: Http,
app: App,
ep: Endpoint,
configured: Map<string, string>,
settings: IndexerSettings,
remembered: string[],
entries: Entry[],
): Promise<Outcome[]> {
const ours = new Set([ep.host, ...settings.adoptHosts, ...remembered].map((h) => h.trim().toLowerCase()).filter(Boolean));
const outcomes: Outcome[] = [];
const managed = new Set<string>();
for (const e of entries) {
const feed = jackettFeed(e);
if (!feed || !ours.has(feed.host)) continue;
managed.add(feed.id);
const what = `jackett-api ("${e.name}", jackett indexer ${feed.id})`;
const known = feed.id === "all" || configured.has(feed.id);
const outcome = await reconcileEntry(
http, app, "indexer", what, e, feedConnection(ep, feed.id), feedDiffers(e, ep, feed.id), "apiKey", ep.credential, known,
);
outcomes.push(outcome);
if (!known && outcome.result !== "refused") {
outcomes.push({ what, result: "notice", note: `jackett has no indexer "${feed.id}" configured, so it was not tested; configure it in jackett or remove the entry in ${app.module}` });
}
}
for (const id of settings.indexers) {
if (managed.has(id)) continue;
const what = `jackett-api (jackett indexer ${id})`;
if (id !== "all" && !configured.has(id)) {
outcomes.push({ what, result: "refused", problem: `downloads.jackett-api.indexers lists "${id}", and jackett has no indexer by that id configured` });
continue;
}
const name = `Jackett - ${id === "all" ? "all" : configured.get(id)}`;
outcomes.push(await createEntry(http, app, "indexer", what, "Torznab", name, { ...feedConnection(ep, id), apiKey: ep.credential }));
}
if (outcomes.length === 0) {
outcomes.push({ what: "jackett-api", result: "notice", note: `${app.module} has no jackett feed the mesh manages, and downloads.jackett-api.indexers lists none to register` });
}
return outcomes;
}
// ---------------------------------------------------------------------------------------------
// Files
/** The module's settings for this step, from its merged `downloads.json`. */
export interface StepSettings {
node: string;
names: Record<string, string>;
indexers: IndexerSettings;
}
export function stepSettings(raw: unknown): StepSettings {
const doc = (raw ?? {}) as Record<string, unknown>;
const d = (doc.downloads ?? {}) as Record<string, Record<string, unknown> | undefined>;
const names: Record<string, string> = {};
for (const k of CLIENTS) {
const n = d[k.provision]?.name;
names[k.provision] = typeof n === "string" && n.trim() ? n.trim() : k.provision.replace(/-api$/, "");
}
const j = d[JACKETT] ?? {};
const strings = (v: unknown) => (Array.isArray(v) ? v.filter((x): x is string => typeof x === "string" && x.trim() !== "").map((x) => x.trim()) : []);
return {
node: typeof doc.node === "string" ? doc.node : "",
names,
indexers: { adoptHosts: strings(j["adopt-hosts"]), indexers: strings(j.indexers) },
};
}
/** The app's own key and base path, from its config.xml. */
export function appConfig(xml: string): { apiKey: string; urlBase: string } {
const tag = (t: string) => new RegExp(`<${t}>([^<]*)</${t}>`).exec(xml)?.[1]?.trim() ?? "";
return { apiKey: tag("ApiKey"), urlBase: normBase(tag("UrlBase")) };
}
/** A file the mesh wrote, or undefined when it is not there. */
export async function readIfThere(path: string | undefined): Promise<string | undefined> {
if (!path) return undefined;
return readFile(path, "utf8").catch(() => undefined);
}
/** A JSON file parsed, or undefined when absent or not JSON. */
export async function readJson(path: string | undefined): Promise<unknown> {
const raw = await readIfThere(path);
if (raw === undefined) return undefined;
try {
return JSON.parse(raw) as unknown;
} catch {
return undefined;
}
}
/** Wait for the app to answer, because the step runs right after its container starts. */
export async function appReady(http: Http, app: App, waitMs: number, pauseMs = 2000): Promise<boolean> {
const until = Date.now() + waitMs;
for (;;) {
try {
const a = await appCall(http, app, "GET", "/system/status");
if (a.status === 200) return true;
} catch {
// not listening yet
}
if (Date.now() >= until) return false;
await new Promise((r) => setTimeout(r, pauseMs));
}
}
export async function listEntries(http: Http, app: App, resource: "downloadclient" | "indexer"): Promise<Entry[]> {
return ((await appGet(http, app, `/${resource}`)) as Entry[]) ?? [];
}
/** Anything printed goes through this: the credentials the step holds never reach a log. */
export function scrub(text: string, secrets: string[]): string {
let out = text;
for (const s of secrets) if (s && s.length >= 4) out = out.split(s).join("<redacted>");
return out;
}
function message(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
+6 -101
View File
@@ -1,19 +1,6 @@
{
"module": "lidarr",
"version": "1",
"provides": [
{
"name": "lidarr-api",
"scope": "mesh"
}
],
"serves": {
"lidarr-api": {
"scheme": "http",
"port": 8686,
"url-base": ""
}
},
"capabilities": [
"container-runtime"
],
@@ -51,33 +38,18 @@
"path": "/var/lib/mesh/lidarr",
"mode": "0700"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "config",
"type": "directory",
"path": "/services/lidarr/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "custom-cont-init",
"type": "directory",
"mode": "0755"
},
{
"id": "custom-services",
"type": "directory",
"mode": "0755"
},
{
"id": "server",
"type": "container",
"name": "lidarr",
"image": "lscr.io/linuxserver/lidarr@sha256:8ab0fd370b604ae034d9a9c261a9d8d873bece33d9736852e7ce4f3566e4a35d",
"image": "lscr.io/linuxserver/lidarr@sha256:6b38dd330b0c653351c2e23c8b962ea51c95683dd7acace9d106c922baf85f75",
"env": {
"PUID": "1000",
"PGID": "1000",
@@ -87,9 +59,7 @@
"8686"
],
"volumes": [
"${dir:config}:/config",
"${dir:custom-cont-init}:/custom-cont-init.d",
"${dir:custom-services}:/custom-services.d",
"/services/lidarr/config:/config",
"/services/media/music:/music",
"/services/media/downloads:/downloads"
]
@@ -101,74 +71,17 @@
"network": "host",
"volumes": [
"/var/lib/mesh/lidarr/broker:/run/secrets/broker:ro",
"${dir:config}:/var/lib/lidarr/config:ro"
"/services/lidarr/config:/var/lib/lidarr/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_LIDARR_URL": "http://127.0.0.1:${port:8686}",
"MESH_LIDARR_URL": "http://127.0.0.1:8686",
"MESH_LIDARR_CONFIG_DIR": "/var/lib/lidarr/config"
},
"artifact": "runtime"
},
{
"id": "downloads-config",
"type": "file",
"path": "${dir:state}/downloads.json",
"mode": "0600",
"content": "{\n \"node\": \"${machine:name}\",\n \"downloads\": {\n \"nzbget-api\": {\n \"name\": \"nzbget\"\n },\n \"qbittorrent-api\": {\n \"name\": \"qbittorrent\"\n },\n \"jackett-api\": {\n \"adopt-hosts\": [],\n \"indexers\": []\n }\n }\n}\n",
"merge": "json"
},
{
"id": "downloads-memory",
"type": "directory",
"mode": "0700"
},
{
"id": "downloads",
"type": "container",
"name": "mesh-lidarr-downloads",
"network": "host",
"run-once": true,
"volumes": [
"${dir:state}/downloads.json:/run/downloads/downloads.json:ro",
"${dir:state}/nzbget-api.json:/run/downloads/nzbget-api.json:ro",
"${dir:state}/nzbget-api.secret:/run/downloads/nzbget-api.secret:ro",
"${dir:state}/qbittorrent-api.json:/run/downloads/qbittorrent-api.json:ro",
"${dir:state}/qbittorrent-api.secret:/run/downloads/qbittorrent-api.secret:ro",
"${dir:state}/jackett-api.json:/run/downloads/jackett-api.json:ro",
"${dir:state}/jackett-api.secret:/run/downloads/jackett-api.secret:ro",
"${dir:config}:/var/lib/lidarr/config:ro",
"${dir:downloads-memory}:/var/lib/downloads"
],
"env": {
"MESH_DOWNLOADS_APP": "lidarr",
"MESH_DOWNLOADS_API": "v1",
"MESH_DOWNLOADS_APP_URL": "http://127.0.0.1:${port:8686}",
"MESH_DOWNLOADS_APP_CONFIG": "/var/lib/lidarr/config/config.xml",
"MESH_DOWNLOADS_DIR": "/run/downloads",
"MESH_DOWNLOADS_SETTINGS": "/run/downloads/downloads.json",
"MESH_DOWNLOADS_MEMORY": "/var/lib/downloads/memory.json"
},
"args": [
"run",
"/app/modules/lidarr/dist/downloads/index.js"
],
"restart-on": [
"downloads-config",
"bound-nzbget-api",
"secret-nzbget-api",
"bound-qbittorrent-api",
"secret-qbittorrent-api",
"bound-jackett-api",
"secret-jackett-api"
],
"artifact": "runtime"
}
],
"requires": [
"jackett-api",
"nzbget-api",
"qbittorrent-api",
"route"
],
"contributes": {
@@ -178,15 +91,7 @@
}
},
"binds": {
"route": "${dir:state}/route.json",
"nzbget-api": "${dir:state}/nzbget-api.json",
"qbittorrent-api": "${dir:state}/qbittorrent-api.json",
"jackett-api": "${dir:state}/jackett-api.json"
},
"secrets": {
"nzbget-api": "${dir:state}/nzbget-api.secret",
"qbittorrent-api": "${dir:state}/qbittorrent-api.secret",
"jackett-api": "${dir:state}/jackett-api.secret"
"route": "/var/lib/mesh/lidarr/route.json"
},
"build": {
"on": [
-5
View File
@@ -4,11 +4,6 @@
"description": "lidarr — music management. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts downloads/settings.ts downloads/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
-365
View File
@@ -1,365 +0,0 @@
// What holds the downloads step (downloads/settings.ts): the app's download clients and jackett
// feeds are made to say what the mesh bound — host, port, TLS, base path, user, credential — and
// nothing else they keep is touched; only the mesh's entries are touched, and nothing is ever
// deleted; nothing is written when nothing differs; a missing one is registered; and a credential
// the provider refuses (the mesh's own minted value, before the operator accepts the provider's)
// is never written, with the `secret accept` that fixes it named.
//
// The app and the providers are fakes: the routes the step touches, answering as the real ones do
// (checked against the catalogue's pinned sonarr, radarr, lidarr, bookshelf, nzbget, qBittorrent
// and jackett): the app masks a stored password as "********", tests an enabled entry before
// saving it, refuses a warning unless forceSave, and jackett answers a wrong key 200 with an error.
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import {
CLIENTS,
acceptRemedy,
jackettFeed,
providerTakes,
reconcileClient,
reconcileIndexers,
stepSettings,
wanted,
type App,
type Binding,
type Endpoint,
type Entry,
type Http,
} from "../downloads/settings.ts";
const NZBGET = CLIENTS.find((c) => c.provision === "nzbget-api")!;
const QBIT = CLIENTS.find((c) => c.provision === "qbittorrent-api")!;
const APP: App = { module: "sonarr", url: "http://127.0.0.1:8989", apiKey: "app-key", api: "v3" };
const MASK = "********";
interface Provider {
host: string;
port: number;
user?: string;
secret: string;
configured?: Record<string, string>;
}
const nzbget: Provider = { host: "ace.internal", port: 20201, user: "luffy", secret: "nzb-real" };
const qbit: Provider = { host: "ace.internal", port: 8112, user: "luffy", secret: "qbt-real" };
const jackett: Provider = { host: "ace.internal", port: 20204, secret: "jackett-real", configured: { rutracker: "RuTracker", torrent9: "Torrent9" } };
function binding(p: Provider, provision: string, extra: Record<string, unknown> = {}): Binding {
return {
provision,
from: "ace",
at: p.host,
as: "mesh_ace_sonarr",
serves: { scheme: "http", port: p.port, "url-base": "", ...(p.user ? { username: p.user } : {}), ...extra },
};
}
function endpoint(p: Provider, provision: string, credential = p.secret): Endpoint {
const w = wanted(provision, binding(p, provision), credential, provision !== "jackett-api");
if (!w.ok) throw new Error(w.problem);
return w.endpoint;
}
const f = (name: string, value: unknown) => ({ name, value });
function client(id: number, name: string, implementation: string, host: string, port: number, password: string, extra: Record<string, unknown> = {}): Entry {
return {
id, name, implementation, enable: true, priority: 1, tags: [],
fields: [f("host", host), f("port", port), f("useSsl", false), f("urlBase", null), f("username", "luffy"), f("password", password), f("tvCategory", "Series"), ...Object.entries(extra).map(([k, v]) => f(k, v))],
};
}
function feed(id: number, name: string, baseUrl: string, jid: string, key: string, on = true): Entry {
return {
id, name, implementation: "Torznab", enableRss: on, enableAutomaticSearch: on, enableInteractiveSearch: on, priority: 25,
fields: [f("baseUrl", baseUrl), f("apiPath", `/api/v2.0/indexers/${jid}/results/torznab/`), f("apiKey", key), f("categories", [5000])],
};
}
interface Call {
method: string;
url: string;
body?: unknown;
}
/** A Servarr app and the three providers behind one fetch. */
function fakes(start: { clients?: Entry[]; indexers?: Entry[] }) {
const calls: Call[] = [];
const store: Record<string, Entry[]> = {
downloadclient: structuredClone(start.clients ?? []),
indexer: structuredClone(start.indexers ?? []),
};
let next = 100;
const secretOf: Record<string, string> = { downloadclient: "password", indexer: "apiKey" };
const masked = (e: Entry): Entry => ({ ...e, fields: e.fields!.map((x) => (x.name === "password" || x.name === "apiKey") && x.value ? { ...x, value: MASK } : { ...x }) });
const val = (e: Entry, n: string) => e.fields?.find((x) => x.name === n)?.value;
/** What the app's own test says: dialled from its container, with its stored secret for a mask. */
const appTest = (kind: string, e: Entry): { propertyName: string; errorMessage: string; isWarning: boolean }[] => {
let secret = val(e, secretOf[kind]);
if (secret === MASK) secret = val(store[kind].find((s) => s.id === e.id) ?? {}, secretOf[kind]);
if (kind === "downloadclient") {
const p = e.implementation === "Nzbget" ? nzbget : qbit;
if (val(e, "host") !== p.host || val(e, "port") !== p.port) return [{ propertyName: "Host", errorMessage: "Unable to connect", isWarning: false }];
if (secret !== p.secret || val(e, "username") !== p.user) return [{ propertyName: "Username", errorMessage: "Authentication Failure", isWarning: false }];
if (val(e, "tvCategory") === "") return [{ propertyName: "TvCategory", errorMessage: "A category is recommended", isWarning: true }];
if (e.implementation === "Nzbget" && val(e, "tvCategory") === "tv") return [{ propertyName: "TvCategory", errorMessage: "Category does not exist", isWarning: false }];
return [];
}
const u = new URL(String(val(e, "baseUrl")));
if (u.hostname !== jackett.host || Number(u.port) !== jackett.port) return [{ propertyName: "BaseUrl", errorMessage: "Unable to connect", isWarning: false }];
if (secret !== jackett.secret) return [{ propertyName: "ApiKey", errorMessage: "Invalid API Key", isWarning: false }];
const id = /indexers\/([^/]+)\//.exec(String(val(e, "apiPath")))?.[1] ?? "";
if (!(id in jackett.configured!)) return [{ propertyName: "", errorMessage: "Unknown indexer", isWarning: false }];
return [];
};
const http: Http = {
async fetch(url, init) {
const method = init?.method ?? "GET";
const body = init?.body && init.headers?.["Content-Type"] === "application/json" ? (JSON.parse(init.body) as Entry) : init?.body;
calls.push({ method, url, body });
const reply = (status: number, value?: unknown) => ({ status, text: async () => (value === undefined ? "" : typeof value === "string" ? value : JSON.stringify(value)) });
const u = new URL(url);
// Providers.
if (u.pathname === "/jsonrpc/version") {
const want = "Basic " + Buffer.from(`${nzbget.user}:${nzbget.secret}`).toString("base64");
return init?.headers?.Authorization === want ? reply(200, { result: "26.0" }) : reply(401);
}
if (u.pathname === "/api/v2/auth/login") {
const form = new URLSearchParams(String(init?.body ?? ""));
return form.get("username") === qbit.user && form.get("password") === qbit.secret ? reply(204) : reply(401, "Unauthorized");
}
if (u.pathname.endsWith("/torznab/api")) {
if (u.searchParams.get("apikey") !== jackett.secret) return reply(200, '<?xml version="1.0"?><error code="100" description="Invalid API Key" />');
const items = Object.entries(jackett.configured!).map(([id, t]) => `<indexer id="${id}" configured="true"><title>${t}</title></indexer>`).join("");
return reply(200, `<?xml version="1.0"?><indexers>${items}</indexers>`);
}
// The app.
if (init?.headers?.["X-Api-Key"] !== APP.apiKey) return reply(401);
const m = /^\/api\/v3\/(downloadclient|indexer)(?:\/(schema|test|\d+))?$/.exec(u.pathname);
if (!m) return reply(404);
const [, kind, sub] = m;
const force = u.searchParams.get("forceSave") === "true";
if (method === "GET" && !sub) return reply(200, store[kind].map(masked));
if (method === "GET" && sub === "schema") {
return reply(200, kind === "downloadclient"
? [client(0, "", "Nzbget", "localhost", 6789, MASK, {}), client(0, "", "QBittorrent", "localhost", 8080, "", {})].map((e) => ({ ...e, fields: e.fields!.map((x) => x.name === "tvCategory" ? { ...x, value: e.implementation === "Nzbget" ? "tv" : "tv-sonarr" } : x.name === "username" ? { ...x, value: null } : x) }))
: [{ ...feed(0, "", "", "x", "", false), supportsRss: true, supportsSearch: true, fields: [f("baseUrl", null), f("apiPath", "/api"), f("apiKey", null), f("categories", [5030, 5040])] }]);
}
if (method === "GET") {
const e = store[kind].find((s) => s.id === Number(sub));
return e ? reply(200, masked(e)) : reply(404);
}
if (method === "POST" && sub === "test") {
const fails = appTest(kind, body as Entry);
return fails.length ? reply(400, fails) : reply(200);
}
// Save: tested when enabled; a mask keeps what is stored.
const e = body as Entry;
const on = e.enable === true || e.enableRss === true || e.enableAutomaticSearch === true;
if (on) {
const fails = appTest(kind, e);
if (fails.some((x) => !x.isWarning) || (fails.length && !force)) return reply(400, fails);
}
if (method === "POST" && !sub) {
if (store[kind].some((s) => String(s.name).toLowerCase() === String(e.name).toLowerCase())) {
return reply(400, [{ propertyName: "Name", errorMessage: "Should be unique", isWarning: false }]);
}
const created = { ...e, id: next++ };
store[kind].push(created);
return reply(201, masked(created));
}
if (method === "PUT") {
const i = store[kind].findIndex((s) => s.id === Number(sub));
if (i < 0) return reply(404);
const was = store[kind][i];
store[kind][i] = { ...e, fields: e.fields!.map((x) => x.value === MASK ? { ...x, value: val(was, x.name) } : x) };
return reply(202, masked(store[kind][i]));
}
return reply(405);
},
};
const saves = () => calls.filter((c) => (c.method === "PUT" || (c.method === "POST" && !c.url.endsWith("/test"))) && c.url.includes("/api/v3/"));
return { http, calls, store, saves };
}
const aceClients = () => [
client(1, "NZBGet", "Nzbget", "nzbget", 6789, "nzb-real"),
client(2, "qBitTorrent", "QBittorrent", "qbittorrent", 8112, "qbt-real"),
];
test("the migration's download client is repointed, its category and everything else kept", async () => {
const f = fakes({ clients: aceClients() });
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(nzbget, "nzbget-api"), f.store.downloadclient.map((e) => ({ ...e })));
assert.deepEqual(out, [{ what: 'nzbget-api ("NZBGet")', result: "written", fields: ["host", "port"] }]);
const saved = f.store.downloadclient.find((e) => e.id === 1)!;
const v = (n: string) => saved.fields!.find((x) => x.name === n)?.value;
assert.equal(v("host"), "ace.internal");
assert.equal(v("port"), 20201);
assert.equal(v("tvCategory"), "Series");
assert.equal(v("password"), "nzb-real", "the password it held works, so it was kept, not rewritten");
assert.equal(saved.priority, 1);
});
test("rerun: already as the mesh says, nothing saved", async () => {
const f = fakes({ clients: [client(1, "NZBGet", "Nzbget", "ace.internal", 20201, "nzb-real")] });
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(nzbget, "nzbget-api"), f.store.downloadclient);
assert.deepEqual(out, [{ what: 'nzbget-api ("NZBGet")', result: "unchanged" }]);
assert.equal(f.saves().length, 0);
});
test("a stale password is replaced by the delivered one, which the provider took first", async () => {
const f = fakes({ clients: [client(2, "qBitTorrent", "QBittorrent", "ace.internal", 8112, "old-pass")] });
const out = await reconcileClient(f.http, APP, QBIT, "qBitTorrent", endpoint(qbit, "qbittorrent-api"), f.store.downloadclient);
assert.deepEqual(out, [{ what: 'qbittorrent-api ("qBitTorrent")', result: "written", fields: ["password"] }]);
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "password")?.value, "qbt-real");
});
test("a minted credential is refused by the provider: nothing written, the accept named", async () => {
const f = fakes({ clients: aceClients() });
const ep = endpoint(nzbget, "nzbget-api", "a-value-the-mesh-minted");
assert.deepEqual(await providerTakes(f.http, "nzbget-api", ep), { took: false });
const remedy = acceptRemedy(APP, "ace", "nzbget-api", "nzbget", "ControlPassword", ep.from);
assert.match(remedy, /`secret accept ace sonarr nzbget-api --provider ace --from <file holding nzbget's ControlPassword>`/);
assert.doesNotMatch(remedy, /minted/);
assert.equal(f.calls.some((c) => c.url.includes("/api/v3/")), false, "the app was not even asked");
const q = endpoint(qbit, "qbittorrent-api", "minted");
assert.deepEqual(await providerTakes(f.http, "qbittorrent-api", q), { took: false });
const j = endpoint(jackett, "jackett-api", "minted");
assert.deepEqual(await providerTakes(f.http, "jackett-api", j), { took: false });
assert.equal(f.calls.find((c) => c.url.includes("apikey="))?.url.includes("jackett-real"), false);
});
test("a fresh app gets the mesh's client registered, under the mesh's name", async () => {
const f = fakes({});
const out = await reconcileClient(f.http, APP, QBIT, "qbittorrent", endpoint(qbit, "qbittorrent-api"), []);
assert.equal(out[0].result, "created");
const e = f.store.downloadclient[0];
assert.equal(e.name, "qbittorrent");
assert.equal(e.enable, true);
assert.equal(e.fields!.find((x) => x.name === "tvCategory")?.value, "tv-sonarr", "the app's own default category");
});
test("nzbget without the app's default category: registered with none, and said", async () => {
const f = fakes({});
const out = await reconcileClient(f.http, APP, NZBGET, "nzbget", endpoint(nzbget, "nzbget-api"), []);
assert.equal(out[0].result, "notice");
assert.match((out[0] as { note: string }).note, /category left empty/);
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "tvCategory")?.value, "");
});
test("somebody else's entries are never touched: another name of the same kind, and a clash of names", async () => {
const mine = client(7, "My seedbox", "QBittorrent", "seedbox.example", 443, "theirs");
const f = fakes({ clients: [mine] });
const out = await reconcileClient(f.http, APP, QBIT, "qbittorrent", endpoint(qbit, "qbittorrent-api"), f.store.downloadclient);
assert.equal(out[0].result, "notice");
assert.deepEqual(f.store.downloadclient.find((e) => e.id === 7), mine);
// Same name in another case: refused, and the setting that adopts it named.
const g = fakes({ clients: aceClients() });
const clash = await reconcileClient(g.http, APP, NZBGET, "nzbget", endpoint(nzbget, "nzbget-api"), g.store.downloadclient);
assert.equal(clash[0].result, "refused");
assert.match((clash[0] as { problem: string }).problem, /downloads\.nzbget-api\.name to "NZBGet"/);
assert.equal(g.saves().length, 0);
});
test("jackett feeds: the migration's are repointed, a person's is left, a listed one is registered", async () => {
const personal = feed(9, "Seedbox jackett", "https://jackett.seedbox.example", "rutracker", "their-key");
const f = fakes({
indexers: [
feed(5, "Torznab - RuTracker", "https://indexers.zurag.be", "rutracker-ru", "jackett-real", false),
feed(6, "Torznab - Torrent9", "https://indexers.zurag.be", "torrent9", "jackett-real", false),
feed(4, "Jackett - RARBG", "http://jackett:9117", "therarbg", "old", false),
personal,
],
});
const configured = new Map(Object.entries(jackett.configured!));
const out = await reconcileIndexers(
f.http, APP, endpoint(jackett, "jackett-api"), configured,
{ adoptHosts: ["indexers.zurag.be", "jackett"], indexers: ["rutracker"] }, [], f.store.indexer,
);
const byWhat = Object.fromEntries(out.map((o) => [o.what + ":" + o.result, o]));
// torrent9: jackett has it; repointed, key already right, tested.
assert.ok(byWhat['jackett-api ("Torznab - Torrent9", jackett indexer torrent9):written']);
// rutracker-ru and therarbg: jackett has neither — repointed with the key, untested, and said.
assert.ok(byWhat['jackett-api ("Jackett - RARBG", jackett indexer therarbg):written']);
assert.ok(byWhat['jackett-api ("Jackett - RARBG", jackett indexer therarbg):notice']);
// rutracker is listed and nothing of the mesh's reads it: registered.
assert.ok(byWhat["jackett-api (jackett indexer rutracker):created"]);
assert.deepEqual(f.store.indexer.find((e) => e.id === 9), personal, "a person's jackett is not the mesh's");
const t9 = f.store.indexer.find((e) => e.id === 6)!;
assert.equal(t9.fields!.find((x) => x.name === "baseUrl")?.value, "http://ace.internal:20204");
assert.equal(t9.enableRss, false, "disabled stays disabled");
const created = f.store.indexer.find((e) => e.name === "Jackett - RuTracker")!;
assert.equal(created.enableRss, true);
assert.equal(f.calls.some((c) => c.method === "DELETE"), false, "nothing is ever deleted");
});
test("jackett feeds rerun: nothing saved, and a remembered host keeps a moved jackett's feeds", async () => {
const f = fakes({ indexers: [feed(6, "Torznab - Torrent9", "http://ace.internal:20204", "torrent9", "jackett-real")] });
const configured = new Map(Object.entries(jackett.configured!));
const out = await reconcileIndexers(f.http, APP, endpoint(jackett, "jackett-api"), configured, { adoptHosts: [], indexers: ["torrent9"] }, [], f.store.indexer);
assert.deepEqual(out.map((o) => o.result), ["unchanged"]);
assert.equal(f.saves().length, 0);
// jackett moved to novox: the feed on ace.internal is still the mesh's because it was remembered.
const moved = { ...jackett, host: "novox.internal" };
const g = fakes({ indexers: [feed(6, "Torznab - Torrent9", "http://ace.internal:20204", "torrent9", "jackett-real")] });
const saved = jackett.host;
jackett.host = moved.host;
try {
const again = await reconcileIndexers(g.http, APP, endpoint(moved, "jackett-api"), configured, { adoptHosts: [], indexers: [] }, ["ace.internal"], g.store.indexer);
assert.equal(again[0].result, "written");
assert.equal(g.store.indexer[0].fields!.find((x) => x.name === "baseUrl")?.value, "http://novox.internal:20204");
} finally {
jackett.host = saved;
}
});
test("an enabled entry the app cannot test at the mesh's address is not saved", async () => {
const f = fakes({ clients: [client(1, "NZBGet", "Nzbget", "nzbget", 6789, "nzb-real")] });
const elsewhere = { ...nzbget, port: 1 };
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(elsewhere, "nzbget-api"), f.store.downloadclient);
assert.equal(out[0].result, "refused");
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "host")?.value, "nzbget", "left as it was");
});
test("bindings: loopback, no user, no credential are refused; the base path is normalised", () => {
assert.equal(wanted("nzbget-api", { ...binding(nzbget, "nzbget-api"), at: "127.0.0.1" }, "x", true).ok, false);
assert.equal(wanted("nzbget-api", binding({ ...nzbget, user: undefined }, "nzbget-api"), "x", true).ok, false);
assert.equal(wanted("nzbget-api", binding(nzbget, "nzbget-api"), " \n", true).ok, false);
const w = wanted("jackett-api", binding(jackett, "jackett-api", { "url-base": "jackett/" }), "k", false);
assert.equal(w.ok && w.endpoint.urlBase, "/jackett");
});
test("a feed is read whole: base path in the base URL or in the API path", () => {
const e = feed(1, "x", "http://ace.internal:9117/jackett", "rutracker", "k");
assert.deepEqual(jackettFeed(e), { host: "ace.internal", id: "rutracker" });
assert.equal(jackettFeed({ ...e, implementation: "Newznab" }), undefined);
assert.equal(jackettFeed({ ...e, fields: [f("baseUrl", "https://api.nzbgeek.info"), f("apiPath", "/api")] }), undefined);
});
test("settings: names default to the provider's, adoption and registration read from the merged file", () => {
assert.deepEqual(stepSettings({ node: "ace" }), { node: "ace", names: { "nzbget-api": "nzbget", "qbittorrent-api": "qbittorrent" }, indexers: { adoptHosts: [], indexers: [] } });
const s = stepSettings({ node: "ace", downloads: { "nzbget-api": { name: "NZBGet" }, "jackett-api": { "adopt-hosts": ["jackett"], indexers: ["torrent9", 3] } } });
assert.equal(s.names["nzbget-api"], "NZBGet");
assert.deepEqual(s.indexers, { adoptHosts: ["jackett"], indexers: ["torrent9"] });
});
// The four copies are one step. Where the siblings are checked out beside this module, they must
// be byte-identical — a fix made in one and not the others is a bug in three apps.
test("the step is the same in sonarr, radarr, lidarr and bookshelf", () => {
const here = dirname(dirname(fileURLToPath(import.meta.url)));
const modules = dirname(here);
for (const file of ["downloads/settings.ts", "downloads/index.ts", "test/downloads.test.ts"]) {
const mine = readFileSync(join(here, file), "utf8");
for (const sibling of ["sonarr", "radarr", "lidarr", "bookshelf"]) {
const theirs = join(modules, sibling, file);
if (existsSync(theirs)) assert.equal(readFileSync(theirs, "utf8"), mine, `${sibling}/${file} differs`);
}
}
});
+1 -1
View File
@@ -8,5 +8,5 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts", "downloads/settings.ts", "downloads/index.ts"]
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
+1 -1
View File
@@ -81,7 +81,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_NZBGET_URL": "http://127.0.0.1:${port:6789}",
"MESH_NZBGET_URL": "http://127.0.0.1:6789",
"MESH_NZBGET_PASSWORD_FILE": "/run/secrets/password",
"MESH_NZBGET_CONFIG_FILE": "/run/config/config.json",
"MESH_NZBGET_CONFIG_DIR": "/var/lib/nzbget/config"
+1 -4
View File
@@ -13,7 +13,7 @@ ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/ombi
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts servarr/settings.ts servarr/index.ts \
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
@@ -22,6 +22,3 @@ COPY --from=build /app/modules/ombi/dist /app/modules/ombi/dist
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/ombi/dist/index.js,/app/modules/ombi/dist/tools/index.js
# NOT dist/servarr/index.js: that is a step the host runs to completion, named by the `servarr`
# container's args as `mesh-tools run …` (novox/hq ADR 0052). Listed here it would run inside the
# serving sidecar too, and exit it.
+10 -58
View File
@@ -28,15 +28,10 @@
"path": "/var/lib/mesh/ombi",
"mode": "0700"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "config",
"type": "directory",
"path": "/services/ombi/config",
"mode": "0700",
"owner": "1000:1000"
},
@@ -44,7 +39,7 @@
"id": "server",
"type": "container",
"name": "ombi",
"image": "lscr.io/linuxserver/ombi@sha256:22d6ebadbaaa728571353e74dc2173719e0fb02d4eaec551a7e9d2ee99ef68ac",
"image": "lscr.io/linuxserver/ombi@sha256:a6f76ac521ba01eee2e9f0c23a3fed22e56630d97a04d5eeaeaa36c1e681640d",
"env": {
"PUID": "1000",
"PGID": "1000",
@@ -54,7 +49,7 @@
"3579"
],
"volumes": [
"${dir:config}:/config"
"/services/ombi/config:/config"
]
},
{
@@ -73,59 +68,24 @@
"volumes": [
"/var/lib/mesh/ombi/broker:/run/secrets/broker:ro",
"/var/lib/mesh/ombi/api-key:/run/secrets/api-key:ro",
"/var/lib/mesh/ombi/config.json:/run/config/config.json:ro"
"/var/lib/mesh/ombi/config.json:/run/config/config.json:ro",
"/services/ombi/config:/var/lib/ombi/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_OMBI_URL": "http://127.0.0.1:${port:3579}",
"MESH_OMBI_URL": "http://127.0.0.1:3579",
"MESH_OMBI_API_KEY_FILE": "/run/secrets/api-key",
"MESH_OMBI_CONFIG_FILE": "/run/config/config.json"
"MESH_OMBI_CONFIG_FILE": "/run/config/config.json",
"MESH_OMBI_CONFIG_DIR": "/var/lib/ombi/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
},
{
"id": "servarr",
"type": "container",
"name": "mesh-ombi-servarr",
"network": "host",
"run-once": true,
"volumes": [
"/var/lib/mesh/ombi/api-key:/run/secrets/api-key:ro",
"${dir:state}/sonarr-api.json:/run/servarr/sonarr-api.json:ro",
"${dir:state}/sonarr-api.secret:/run/servarr/sonarr-api.secret:ro",
"${dir:state}/radarr-api.json:/run/servarr/radarr-api.json:ro",
"${dir:state}/radarr-api.secret:/run/servarr/radarr-api.secret:ro",
"${dir:state}/lidarr-api.json:/run/servarr/lidarr-api.json:ro",
"${dir:state}/lidarr-api.secret:/run/servarr/lidarr-api.secret:ro"
],
"env": {
"MESH_OMBI_URL": "http://127.0.0.1:${port:3579}",
"MESH_OMBI_API_KEY_FILE": "/run/secrets/api-key",
"MESH_SERVARR_DIR": "/run/servarr"
},
"args": [
"run",
"/app/modules/ombi/dist/servarr/index.js"
],
"restart-on": [
"bound-sonarr-api",
"secret-sonarr-api",
"bound-radarr-api",
"secret-radarr-api",
"bound-lidarr-api",
"secret-lidarr-api"
],
"artifact": "runtime"
}
],
"requires": [
"lidarr-api",
"radarr-api",
"route",
"sonarr-api"
"route"
],
"contributes": {
"route": {
@@ -134,15 +94,7 @@
}
},
"binds": {
"route": "${dir:state}/route.json",
"sonarr-api": "${dir:state}/sonarr-api.json",
"radarr-api": "${dir:state}/radarr-api.json",
"lidarr-api": "${dir:state}/lidarr-api.json"
},
"secrets": {
"sonarr-api": "${dir:state}/sonarr-api.secret",
"radarr-api": "${dir:state}/radarr-api.secret",
"lidarr-api": "${dir:state}/lidarr-api.secret"
"route": "/var/lib/mesh/ombi/route.json"
},
"build": {
"on": [
-5
View File
@@ -4,11 +4,6 @@
"description": "ombi — media requests. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts servarr/settings.ts servarr/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
-59
View File
@@ -1,59 +0,0 @@
// ombi's Servarr step — run once by the host after ombi's server starts, and run again whenever a
// binding or pair credential it reads changes (the container's `restart-on`, novox/hq ADR 0099).
//
// **A step, not a loop**, for the reason route-adapter gives: everything it does is a function of
// files the mesh writes, and the host already knows when they change. It connects to no broker.
//
// Exits non-zero when any app could not be put right — a refused credential, an unreachable app, an
// ombi that cannot reach it — so the node reports the step failed and the host runs it again on the
// next apply. It is declared last in the manifest, so its failing gates nothing else of ombi's
// (novox/hq ADR 0136).
//
// Reads, per app, `<dir>/<provision>.json` (the binding) and `<dir>/<provision>.secret` (the pair
// credential), where <dir> is MESH_SERVARR_DIR. Never prints a key.
import { join } from "node:path";
import { APPS, ombiReady, readBinding, readIfThere, reconcileApp, type Http } from "./settings.js";
const dir = process.env.MESH_SERVARR_DIR ?? "/run/servarr";
const url = process.env.MESH_OMBI_URL ?? "http://127.0.0.1:3579";
const apiKey = (await readIfThere(process.env.MESH_OMBI_API_KEY_FILE))?.trim() ?? process.env.MESH_OMBI_API_KEY ?? "";
const waitSeconds = Number(process.env.MESH_OMBI_WAIT_SECONDS ?? "180");
const http: Http = { fetch: (u, init) => fetch(u, init) };
if (!apiKey) {
console.error("[ombi-servarr] no ombi API key — ombi's own `api-key` secret has not been accepted");
process.exit(1);
}
const ombi = { url, apiKey };
if (!(await ombiReady(http, ombi, waitSeconds * 1000))) {
console.error(`[ombi-servarr] ombi did not answer at ${url} within ${waitSeconds}s`);
process.exit(1);
}
let failed = 0;
for (const spec of APPS) {
const outcome = await reconcileApp(
http,
ombi,
spec,
await readBinding(join(dir, `${spec.provision}.json`)),
await readIfThere(join(dir, `${spec.provision}.secret`)),
);
switch (outcome.result) {
case "unchanged":
console.log(`[ombi-servarr] ${outcome.app}: already as the mesh says; connection tested`);
break;
case "written":
console.log(`[ombi-servarr] ${outcome.app}: wrote ${outcome.fields.join(", ")}; connection tested`);
break;
case "refused":
failed++;
console.error(`[ombi-servarr] ${outcome.app}: ${outcome.problem}`);
break;
}
}
process.exitCode = failed > 0 ? 1 : 0;
-304
View File
@@ -1,304 +0,0 @@
// Where ombi reaches Sonarr, Radarr and Lidarr — decided by the mesh, written into ombi by ombi's
// own API.
//
// **Why this exists.** ombi keeps its connection to each Servarr app in its own database
// (OmbiSettings.db), not in a file, so the mesh has nowhere to write `${bound:sonarr-api:at}` for it.
// ombi requires `sonarr-api`, `radarr-api` and `lidarr-api`; the mesh delivers, for each, a binding
// (where the app is: `at`, and what it serves: `port`, `scheme`, `url-base`) and a pair credential
// (the app's API key, accepted by the operator — a Servarr app has exactly one key and the mesh
// cannot mint it). This step reads those files and makes ombi's settings say the same thing.
//
// **Only the connection, and only when it differs.** Host, port, TLS, base path and API key. The
// quality profile, root folder, language profile, tags, "enabled" and every other choice an operator
// made in ombi's settings screen are left exactly as they are: the mesh knows where the app is, not
// what ombi should do with it. Radarr's 4K instance is a different Radarr and is not touched.
//
// **A credential the app refuses is never written.** Until the operator accepts the app's API key
// for this pair, the mesh delivers a value it minted itself, which no Servarr app will ever accept
// (novox/hq ADR 0092). Writing it would replace a working key in ombi with a dead one. So the key is
// tried against the app first; refused, nothing for that app is written and the step fails naming
// the `secret accept` that fixes it.
//
// Pure logic and a small HTTP seam, so it is tested against fake servers (test/servarr.test.ts).
import { readFile } from "node:fs/promises";
/** One Servarr app ombi connects to, and the shape of that connection in ombi's API. */
export interface ServarrApp {
/** The app, as ombi's API names it: /Settings/<app>, /Tester/<app>. */
app: "sonarr" | "radarr" | "lidarr";
/** The provision it is required as — the manifest's `requires`, `binds` and `secrets` key. */
provision: string;
/** The app's own status endpoint, which answers 401 to a wrong key. */
statusPath: string;
/**
* Where the one connection sits in ombi's settings document. Radarr's is `{radarr, radarr4K}`
* (two Radarr instances); only `radarr` is this provision's.
*/
within?: string;
}
export const APPS: readonly ServarrApp[] = [
{ app: "sonarr", provision: "sonarr-api", statusPath: "/api/v3/system/status" },
{ app: "radarr", provision: "radarr-api", statusPath: "/api/v3/system/status", within: "radarr" },
{ app: "lidarr", provision: "lidarr-api", statusPath: "/api/v1/system/status" },
];
/** The connection fields ombi keeps for an app — the only ones this step ever writes. */
export interface Connection {
ip: string;
port: number;
ssl: boolean;
/** ombi's name for the app's URL base; null when the app is served at the root. */
subDir: string | null;
apiKey: string;
}
/** What the mesh wrote at `binds.<provision>`: the binding document (controller's boundFile). */
export interface Binding {
provision?: string;
from?: string;
at?: string;
as?: string;
serves?: Record<string, unknown>;
}
export type Wanted = { ok: true; connection: Connection; from: string } | { ok: false; problem: string };
/**
* The connection the mesh says ombi should use, from the binding and the pair credential.
*
* Refused rather than guessed when the binding cannot be dialled from ombi's own container: a
* loopback `at` — what the mesh hands a machine that is not on the private network — is ombi's
* container itself, not the app.
*/
export function wanted(spec: ServarrApp, binding: Binding | undefined, credential: string | undefined): Wanted {
if (!binding) {
return { ok: false, problem: `no binding for ${spec.provision} was delivered — the mesh writes it before this step runs` };
}
const at = typeof binding.at === "string" ? binding.at.trim() : "";
const serves = binding.serves ?? {};
const port = Number(serves.port);
if (!at) {
return { ok: false, problem: `the ${spec.provision} binding names no host (at)` };
}
if (isLoopback(at)) {
return {
ok: false,
problem:
`the ${spec.provision} binding says ${spec.app} is at ${at}, which from ombi's own container is ` +
`ombi itself. The mesh hands loopback to a machine that is not on the private network; put it ` +
`on the private network so ${spec.app} has an address ombi can dial`,
};
}
if (!Number.isInteger(port) || port <= 0 || port > 65535) {
return { ok: false, problem: `the ${spec.provision} binding serves no usable port (${String(serves.port)})` };
}
const scheme = typeof serves.scheme === "string" && serves.scheme ? serves.scheme : "http";
if (scheme !== "http" && scheme !== "https") {
return { ok: false, problem: `the ${spec.provision} binding serves scheme ${scheme}, which ombi cannot dial` };
}
const key = (credential ?? "").trim();
if (!key) {
return { ok: false, problem: `the ${spec.provision} credential is empty or was not delivered` };
}
return {
ok: true,
from: typeof binding.from === "string" ? binding.from : "",
connection: { ip: at, port, ssl: scheme === "https", subDir: subDirOf(serves["url-base"]), apiKey: key },
};
}
/** ombi's `subDir`: the URL base with its slashes trimmed, null when there is none. */
export function subDirOf(urlBase: unknown): string | null {
const trimmed = typeof urlBase === "string" ? urlBase.trim().replace(/^\/+|\/+$/g, "") : "";
return trimmed === "" ? null : trimmed;
}
function isLoopback(host: string): boolean {
const h = host.toLowerCase();
return h === "localhost" || h === "::1" || h === "[::1]" || /^127\./.test(h);
}
/** Which connection fields differ between what ombi holds and what the mesh says. Names only. */
export function differing(current: Record<string, unknown> | undefined, want: Connection): (keyof Connection)[] {
const now = current ?? {};
const out: (keyof Connection)[] = [];
if (String(now.ip ?? "") !== want.ip) out.push("ip");
if (Number(now.port ?? 0) !== want.port) out.push("port");
if (Boolean(now.ssl) !== want.ssl) out.push("ssl");
if (subDirOf(now.subDir) !== want.subDir) out.push("subDir");
if (String(now.apiKey ?? "") !== want.apiKey) out.push("apiKey");
return out;
}
/** ombi's settings for the app with the connection laid over them and nothing else changed. */
export function withConnection(current: Record<string, unknown> | undefined, want: Connection): Record<string, unknown> {
return { ...(current ?? {}), ip: want.ip, port: want.port, ssl: want.ssl, subDir: want.subDir, apiKey: want.apiKey };
}
/** The app's base URL as the step dials it — the same host and port ombi will be given. */
export function appUrl(want: Connection): string {
const scheme = want.ssl ? "https" : "http";
const host = want.ip.includes(":") && !want.ip.startsWith("[") ? `[${want.ip}]` : want.ip;
return `${scheme}://${host}:${want.port}${want.subDir ? `/${want.subDir}` : ""}`;
}
/** How one app came out. */
export type Outcome =
| { app: string; result: "unchanged" }
| { app: string; result: "written"; fields: string[] }
| { app: string; result: "refused"; problem: string };
/** The HTTP the step needs, so a test can stand fakes in for ombi and the apps. */
export interface Http {
fetch(url: string, init?: { method?: string; headers?: Record<string, string>; body?: string }): Promise<{
status: number;
text(): Promise<string>;
}>;
}
export interface Ombi {
url: string;
apiKey: string;
}
async function ombiCall(http: Http, ombi: Ombi, method: string, path: string, body?: unknown): Promise<unknown> {
const res = await http.fetch(`${ombi.url.replace(/\/$/, "")}/api/v1${path}`, {
method,
headers: {
ApiKey: ombi.apiKey,
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (res.status < 200 || res.status >= 300) {
// The body is ombi's error, never a request echo, so it carries no key.
throw new Error(`ombi ${method} ${path} answered ${res.status}${text ? `: ${text.slice(0, 200)}` : ""}`);
}
return text ? (JSON.parse(text) as unknown) : undefined;
}
/**
* Does the app take this key? `true` it does, `false` it refused it (401/403), and a thrown error
* when it could not be asked — unreachable, or answering something that is neither.
*/
export async function appTakes(http: Http, spec: ServarrApp, want: Connection): Promise<boolean> {
const res = await http.fetch(`${appUrl(want)}${spec.statusPath}`, {
method: "GET",
headers: { "X-Api-Key": want.apiKey, Accept: "application/json" },
});
if (res.status === 401 || res.status === 403) return false;
if (res.status >= 200 && res.status < 300) return true;
throw new Error(`${spec.app} answered ${res.status} at ${spec.statusPath}`);
}
/** The remedy for a refused key, in the controller's own words (ADR 0092). */
export function acceptRemedy(spec: ServarrApp, from: string): string {
return (
`${spec.app} refuses the ${spec.provision} credential the mesh delivered, so it was not written ` +
`into ombi. A Servarr app has one API key and the mesh cannot make it: accept ${spec.app}'s own ` +
`key for this pair — \`secret accept <this node> ombi ${spec.provision} --provider ${from || "<its node>"} ` +
`--from <file holding ${spec.app}'s ApiKey>\``
);
}
/**
* Bring ombi's connection to one app in line with the mesh: check the key against the app, compare,
* write only the connection fields when they differ, then have ombi test the connection from its own
* container. Never throws: every failure is an outcome with a reason.
*/
export async function reconcileApp(
http: Http,
ombi: Ombi,
spec: ServarrApp,
binding: Binding | undefined,
credential: string | undefined,
): Promise<Outcome> {
const w = wanted(spec, binding, credential);
// `in`, not `!w.ok`: the Dockerfile compiles without strict, where a boolean discriminant does not
// narrow.
if ("problem" in w) return { app: spec.app, result: "refused", problem: w.problem };
const want = w.connection;
try {
if (!(await appTakes(http, spec, want))) {
return { app: spec.app, result: "refused", problem: acceptRemedy(spec, w.from) };
}
} catch (err) {
return {
app: spec.app,
result: "refused",
problem: `${spec.app} could not be asked whether it takes the key at ${want.ip}:${want.port}: ${message(err)}`,
};
}
try {
const document = (await ombiCall(http, ombi, "GET", `/Settings/${spec.app}`)) as Record<string, unknown> | undefined;
const current = spec.within ? (document?.[spec.within] as Record<string, unknown> | undefined) : document;
const fields = differing(current, want);
if (fields.length > 0) {
const next = withConnection(current, want);
const body = spec.within ? { ...(document ?? {}), [spec.within]: next } : next;
const saved = await ombiCall(http, ombi, "POST", `/Settings/${spec.app}`, body);
if (saved === false) {
return { app: spec.app, result: "refused", problem: `ombi declined to save its ${spec.app} settings` };
}
}
// ombi's own test, from ombi's own container — the path the step's check above did not take.
const tested = (await ombiCall(http, ombi, "POST", `/Tester/${spec.app}`, withConnection(current, want))) as
| { isValid?: boolean; expectedSubDir?: string | null }
| undefined;
if (!tested?.isValid) {
const hint = tested?.expectedSubDir ? ` (ombi expected the base path ${tested.expectedSubDir})` : "";
return {
app: spec.app,
result: "refused",
problem:
`ombi cannot reach ${spec.app} at ${want.ip}:${want.port} from its own container${hint}` +
(fields.length > 0 ? `; its settings were written (${fields.join(", ")})` : ""),
};
}
return fields.length > 0 ? { app: spec.app, result: "written", fields } : { app: spec.app, result: "unchanged" };
} catch (err) {
return { app: spec.app, result: "refused", problem: message(err) };
}
}
/** Wait for ombi to answer, because the step runs right after its container starts. */
export async function ombiReady(http: Http, ombi: Ombi, waitMs: number, pauseMs = 2000): Promise<boolean> {
const until = Date.now() + waitMs;
for (;;) {
try {
const res = await http.fetch(`${ombi.url.replace(/\/$/, "")}/api/v1/Status`, { method: "GET" });
if (res.status === 200) return true;
} catch {
// not listening yet
}
if (Date.now() >= until) return false;
await new Promise((r) => setTimeout(r, pauseMs));
}
}
/** A file the mesh wrote, or undefined when it is not there. */
export async function readIfThere(path: string | undefined): Promise<string | undefined> {
if (!path) return undefined;
return readFile(path, "utf8").catch(() => undefined);
}
/** A binding file parsed, or undefined when absent or not JSON. */
export async function readBinding(path: string | undefined): Promise<Binding | undefined> {
const raw = await readIfThere(path);
if (raw === undefined) return undefined;
try {
return JSON.parse(raw) as Binding;
} catch {
return undefined;
}
}
function message(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
-147
View File
@@ -1,147 +0,0 @@
// What holds ombi's Servarr step (servarr/settings.ts): the connection ombi keeps for each app is
// made to say what the mesh bound — host, port, TLS, base path, key — and nothing else it keeps is
// touched; nothing is written when nothing differs; Radarr's 4K instance is left alone; and a key the
// app refuses (the mesh's own minted value, before the operator accepts the app's key) is never
// written, with the `secret accept` that fixes it named.
//
// ombi and the apps are fakes: the routes the step touches, answering as the real ones do (checked
// against lscr.io/linuxserver/ombi 4.53.10 and the catalogue's pinned sonarr/radarr/lidarr).
import { test } from "node:test";
import assert from "node:assert/strict";
import { APPS, differing, reconcileApp, subDirOf, wanted, type Binding, type Http, type ServarrApp } from "../servarr/settings.ts";
const SONARR = APPS.find((a) => a.app === "sonarr") as ServarrApp;
const RADARR = APPS.find((a) => a.app === "radarr") as ServarrApp;
const LIDARR = APPS.find((a) => a.app === "lidarr") as ServarrApp;
const THE_KEY = "the-apps-own-key";
function binding(provision: string, port: number, at = "ace.internal"): Binding {
return { binding: 1, provision, from: "ace", at, as: "mesh_ace_ombi", serves: { scheme: "http", port, "url-base": "" } } as Binding;
}
interface Call {
method: string;
url: string;
body?: unknown;
}
/** ombi's settings store and the apps' key check, behind one fetch. */
function fakes(settings: Record<string, unknown>, opts: { appKey?: string; reachable?: boolean } = {}) {
const calls: Call[] = [];
const appKey = opts.appKey ?? THE_KEY;
const http: Http = {
async fetch(url, init) {
const method = init?.method ?? "GET";
const body = init?.body ? (JSON.parse(init.body) as unknown) : undefined;
calls.push({ method, url, body });
const reply = (status: number, value?: unknown) => ({
status,
text: async () => (value === undefined ? "" : JSON.stringify(value)),
});
const u = new URL(url);
if (u.pathname.endsWith("/system/status")) {
if (opts.reachable === false) throw new Error("connect ECONNREFUSED");
return init?.headers?.["X-Api-Key"] === appKey ? reply(200, { version: "4" }) : reply(401);
}
if (init?.headers?.ApiKey !== "ombi-key") return reply(401);
const m = u.pathname.match(/^\/api\/v1\/(Settings|Tester)\/(\w+)$/);
if (!m) return reply(404);
const [, kind, app] = m;
if (kind === "Settings" && method === "GET") return reply(200, settings[app]);
if (kind === "Settings" && method === "POST") {
settings[app] = body;
return reply(200, true);
}
const tried = body as { apiKey?: string };
return reply(200, { isValid: tried.apiKey === appKey, expectedSubDir: null });
},
};
return { http, calls, settings };
}
const OMBI = { url: "http://127.0.0.1:3579", apiKey: "ombi-key" };
const operatorSonarr = () => ({
enabled: true, apiKey: "old-key", qualityProfile: "3", seasonFolders: true, rootPath: "10",
qualityProfileAnime: "7", rootPathAnime: "9", languageProfile: 1, ssl: false, subDir: null,
ip: "sonarr", port: 8989, id: 5,
});
test("it writes the connection the mesh bound, and keeps every other setting ombi had", async () => {
const f = fakes({ sonarr: operatorSonarr() });
const out = await reconcileApp(f.http, OMBI, SONARR, binding("sonarr-api", 20101), `${THE_KEY}\n`);
assert.deepEqual(out, { app: "sonarr", result: "written", fields: ["ip", "port", "apiKey"] });
assert.deepEqual(f.settings.sonarr, {
...operatorSonarr(), ip: "ace.internal", port: 20101, apiKey: THE_KEY, ssl: false, subDir: null,
});
// Checked against the app itself, at the bound address, before anything was written.
assert.equal(f.calls[0].url, "http://ace.internal:20101/api/v3/system/status");
});
test("nothing is written when ombi already says what the mesh says", async () => {
const f = fakes({ sonarr: { ...operatorSonarr(), ip: "ace.internal", port: 20101, apiKey: THE_KEY } });
const out = await reconcileApp(f.http, OMBI, SONARR, binding("sonarr-api", 20101), THE_KEY);
assert.deepEqual(out, { app: "sonarr", result: "unchanged" });
assert.equal(f.calls.filter((c) => c.method === "POST" && c.url.includes("/Settings/")).length, 0);
});
test("a key the app refuses is never written, and the accept that fixes it is named", async () => {
const f = fakes({ sonarr: operatorSonarr() });
const out = await reconcileApp(f.http, OMBI, SONARR, binding("sonarr-api", 20101), "a-value-the-mesh-minted");
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /secret accept <this node> ombi sonarr-api --provider ace/);
assert.doesNotMatch((out as { problem: string }).problem, /a-value-the-mesh-minted/);
assert.deepEqual(f.settings.sonarr, operatorSonarr(), "ombi's working settings were left alone");
assert.equal(f.calls.some((c) => c.url.includes("/api/v1/")), false, "ombi was not even asked");
});
test("an app it cannot reach is reported, and ombi is left alone", async () => {
const f = fakes({ sonarr: operatorSonarr() }, { reachable: false });
const out = await reconcileApp(f.http, OMBI, SONARR, binding("sonarr-api", 20101), THE_KEY);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /could not be asked.*ECONNREFUSED/);
assert.deepEqual(f.settings.sonarr, operatorSonarr());
});
test("radarr's connection is written inside its combined document, and the 4K instance is untouched", async () => {
const fourK = { enabled: true, apiKey: "4k-key", ip: "radarr4k", port: 7879, defaultQualityProfile: "9", id: 7 };
const f = fakes({ radarr: { radarr: { enabled: true, apiKey: "old", ip: "radarr", port: 7878, defaultRootPath: "/movies", id: 6 }, radarr4K: fourK } });
const out = await reconcileApp(f.http, OMBI, RADARR, binding("radarr-api", 20102), THE_KEY);
assert.equal(out.result, "written");
const doc = f.settings.radarr as { radarr: Record<string, unknown>; radarr4K: unknown };
assert.deepEqual(doc.radarr4K, fourK);
assert.equal(doc.radarr.ip, "ace.internal");
assert.equal(doc.radarr.port, 20102);
assert.equal(doc.radarr.defaultRootPath, "/movies");
});
test("lidarr is checked on its own API version", async () => {
const f = fakes({ lidarr: { enabled: true, apiKey: null, ip: null, port: 0, id: 0 } });
const out = await reconcileApp(f.http, OMBI, LIDARR, binding("lidarr-api", 20103), THE_KEY);
assert.equal(out.result, "written");
assert.equal(f.calls[0].url, "http://ace.internal:20103/api/v1/system/status");
});
test("a loopback binding is refused: from ombi's container it is ombi itself", () => {
const w = wanted(SONARR, binding("sonarr-api", 20101, "127.0.0.1"), THE_KEY);
assert.equal(w.ok, false);
assert.match((w as { problem: string }).problem, /private network/);
});
test("the base path is ombi's subDir, slashes trimmed; empty is none", () => {
assert.equal(subDirOf(""), null);
assert.equal(subDirOf("/sonarr/"), "sonarr");
assert.deepEqual(
differing({ ip: "h", port: 1, ssl: false, subDir: "", apiKey: "k" }, { ip: "h", port: 1, ssl: false, subDir: null, apiKey: "k" }),
[],
);
});
test("an https binding sets ombi's ssl flag", () => {
const b = binding("sonarr-api", 443);
(b.serves as Record<string, unknown>).scheme = "https";
const w = wanted(SONARR, b, THE_KEY);
assert.equal(w.ok && w.connection.ssl, true);
});
+1 -1
View File
@@ -8,5 +8,5 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts", "servarr/settings.ts", "servarr/index.ts"]
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
+1 -1
View File
@@ -82,7 +82,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_QBITTORRENT_URL": "http://127.0.0.1:${port:8080}",
"MESH_QBITTORRENT_URL": "http://127.0.0.1:8080",
"MESH_QBITTORRENT_PASSWORD_FILE": "/run/secrets/password",
"MESH_QBITTORRENT_CONFIG_FILE": "/run/config/config.json",
"MESH_QBITTORRENT_CONFIG_DIR": "/var/lib/qbittorrent/config"
+1 -14
View File
@@ -1,19 +1,6 @@
{
"module": "radarr",
"version": "1",
"provides": [
{
"name": "radarr-api",
"scope": "mesh"
}
],
"serves": {
"radarr-api": {
"scheme": "http",
"port": 7878,
"url-base": ""
}
},
"capabilities": [
"container-runtime"
],
@@ -88,7 +75,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_RADARR_URL": "http://127.0.0.1:${port:7878}",
"MESH_RADARR_URL": "http://127.0.0.1:7878",
"MESH_RADARR_CONFIG_DIR": "/var/lib/radarr/config"
},
"artifact": "runtime"
+1 -1
View File
@@ -100,7 +100,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_SEARXNG_URL": "http://127.0.0.1:${port:8080}",
"MESH_SEARXNG_URL": "http://127.0.0.1:8080",
"MESH_SEARXNG_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
+1 -14
View File
@@ -1,19 +1,6 @@
{
"module": "sonarr",
"version": "1",
"provides": [
{
"name": "sonarr-api",
"scope": "mesh"
}
],
"serves": {
"sonarr-api": {
"scheme": "http",
"port": 8989,
"url-base": ""
}
},
"capabilities": [
"container-runtime"
],
@@ -93,7 +80,7 @@
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_SONARR_URL": "http://127.0.0.1:${port:8989}",
"MESH_SONARR_URL": "http://127.0.0.1:8989",
"MESH_SONARR_CONFIG_DIR": "/var/lib/sonarr/config"
},
"artifact": "runtime"
+4 -1
View File
@@ -13,7 +13,7 @@ ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/tautulli
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts plex/check.ts plex/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
@@ -22,3 +22,6 @@ COPY --from=build /app/modules/tautulli/dist /app/modules/tautulli/dist
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/tautulli/dist/index.js,/app/modules/tautulli/dist/tools/index.js
# NOT dist/plex/index.js: that is a step the host runs to completion, named by the `plex`
# container's args as `mesh-tools run …` (novox/hq ADR 0052). Listed here it would run inside the
# serving sidecar too, and exit it.
+31 -4
View File
@@ -41,6 +41,32 @@ function meshConfig(file?: string): Record<string, string> {
catch { return {}; }
}
/**
* The API key Tautulli minted for itself, read from its own config.ini (mounted read-only).
*
* Tautulli owns this key: it writes it on first run and every client of its API — this runtime
* included — must present the same one. So the mesh does not mint or hold it; the one place it
* lives is the file Tautulli keeps, and a key regenerated in Tautulli's settings is simply read
* again on the next start. Undefined when there is no file or no key yet (a fresh install whose
* setup wizard has not run).
*/
export function keyOfTautulli(dir?: string): string | undefined {
if (!dir) return undefined;
let ini: string;
try { ini = readFileSync(`${dir.replace(/\/$/, "")}/config.ini`, "utf8"); }
catch { return undefined; }
let section = "";
for (const raw of ini.split(/\r?\n/)) {
const line = raw.trim();
const header = /^\[(.+)\]$/.exec(line);
if (header) { section = header[1]; continue; }
if (section !== "General") continue;
const kv = /^api_key\s*=\s*"?([^"]*)"?$/.exec(line);
if (kv && kv[1]) return kv[1];
}
return undefined;
}
export class TautulliClient {
readonly baseUrl: string;
@@ -52,15 +78,16 @@ export class TautulliClient {
}
/**
* Build from the module's resolved environment. The API key is read from MESH_TAUTULLI_APIKEY
* (Tautulli mints it in Settings → Web Interface); the base URL defaults to the local container.
* Build from the module's resolved environment. The API key is the one Tautulli minted for
* itself (Settings → Web Interface), read from its config.ini under MESH_TAUTULLI_CONFIG_DIR;
* MESH_TAUTULLI_APIKEY still wins where it is set. The base URL defaults to the local container.
* Throws when no key is configured — the module then contributes nothing rather than failing.
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): TautulliClient {
const cfg = meshConfig(env.MESH_TAUTULLI_CONFIG_FILE);
const url = cfg.url ?? (env.MESH_TAUTULLI_URL ?? `http://127.0.0.1:${env.TAUTULLI_PORT ?? "8181"}`);
const apiKey = cfg.apiKey ?? env.MESH_TAUTULLI_APIKEY;
if (!apiKey) throw new Error("no Tautulli API key — set MESH_TAUTULLI_APIKEY");
const apiKey = cfg.apiKey ?? env.MESH_TAUTULLI_APIKEY ?? keyOfTautulli(env.MESH_TAUTULLI_CONFIG_DIR);
if (!apiKey) throw new Error("no Tautulli API key — Tautulli's config.ini has none yet (finish its setup and enable the API)");
return new TautulliClient(url, apiKey);
}
File diff suppressed because one or more lines are too long
+5
View File
@@ -4,6 +4,11 @@
"description": "tautulli — Plex watch statistics. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts plex/check.ts plex/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
+8
View File
@@ -0,0 +1,8 @@
#!/bin/bash
# Run by the linuxserver image's custom-init each time Tautulli's container starts, as root, before
# Tautulli: puts where the mesh says plex is into Tautulli's config.ini (mesh-plex.py says why).
#
# Written by the mesh from the tautulli module's manifest. Editing it here lasts until the next apply.
PY=/lsiopy/bin/python3
[ -x "$PY" ] || PY=python3
exec "$PY" /run/mesh/mesh-plex.py
+175
View File
@@ -0,0 +1,175 @@
// Whether Tautulli reaches Plex as the mesh says — the half of tautulli's plex-api consumer the
// mesh can see fail.
//
// **The write is not here.** Tautulli keeps its Plex connection only in config.ini, rewrites that
// file from memory on every shutdown, and has no API command that sets it; so the write happens in
// the server container itself, before Tautulli starts (plex/mesh-plex.py, run by the image's
// custom-init). A failure there is a line in Tautulli's log that nobody reads. This step runs after
// the server, as a run-once container declared last, and fails the node's report when:
//
// - the pair credential is one plex refuses — the mesh's own minted value, before the operator
// accepts the server's X-Plex-Token for this pair — naming the `secret accept` that fixes it;
// - Tautulli is not pointed where the binding says (the start-time write did not land);
// - Tautulli is pointed there and still not connected to plex (its own connection state).
//
// It writes nothing, to Tautulli or to plex. Pure logic and a small HTTP seam, tested against fakes
// (test/plex.test.ts).
/** What the mesh wrote at `binds.plex-api`: the binding document (controller's boundFile). */
export interface Binding {
provision?: string;
from?: string;
at?: string;
as?: string;
serves?: Record<string, unknown>;
}
export const PROVISION = "plex-api";
export interface Wanted {
url: string;
token: string;
from: string;
}
/** The HTTP the step needs, so a test can stand fakes in for Tautulli and plex. */
export interface Http {
fetch(url: string, init?: { method?: string; headers?: Record<string, string> }): Promise<{
status: number;
text(): Promise<string>;
}>;
}
export type Outcome = { result: "connected"; url: string } | { result: "refused"; problem: string };
function isLoopback(host: string): boolean {
const h = host.toLowerCase();
return h === "localhost" || h === "::1" || h === "[::1]" || /^127\./.test(h);
}
/** The URL Tautulli should hold as pms_url — the same one mesh-plex.py writes. */
export function wanted(binding: Binding | undefined, credential: string | undefined): Wanted | { problem: string } {
if (!binding) return { problem: `no binding for ${PROVISION} was delivered — the mesh writes it before this step runs` };
const at = typeof binding.at === "string" ? binding.at.trim() : "";
const serves = binding.serves ?? {};
const port = Number(serves.port);
const scheme = typeof serves.scheme === "string" && serves.scheme ? serves.scheme : "http";
if (!at) return { problem: `the ${PROVISION} binding names no host (at)` };
if (isLoopback(at)) {
return {
problem:
`the ${PROVISION} binding says plex is at ${at}, which from Tautulli's container is Tautulli itself; ` +
`the mesh hands loopback to a machine that is not on the private network`,
};
}
if (!Number.isInteger(port) || port <= 0 || port > 65535) {
return { problem: `the ${PROVISION} binding serves no usable port (${String(serves.port)})` };
}
if (scheme !== "http" && scheme !== "https") return { problem: `the ${PROVISION} binding serves scheme ${scheme}, which Tautulli cannot dial` };
const token = (credential ?? "").trim();
if (!token) return { problem: `the ${PROVISION} credential is empty or was not delivered` };
const host = at.includes(":") && !at.startsWith("[") ? `[${at}]` : at;
return { url: `${scheme}://${host}:${port}`, token, from: typeof binding.from === "string" ? binding.from : "" };
}
/** The remedy for a refused token, in the controller's own words (ADR 0092). */
export function acceptRemedy(from: string): string {
return (
`plex refuses the ${PROVISION} credential the mesh delivered, so Tautulli was not given it. plex's ` +
`token is issued by plex.tv and the mesh cannot make it: accept the server's own token for this pair — ` +
`\`secret accept <this node> tautulli ${PROVISION} --provider ${from || "<its node>"} ` +
`--from <file holding the server's X-Plex-Token>\``
);
}
/** Does plex take the token? 401/403, or 400 on a network plex trusts, is a refusal. */
export async function plexTakes(http: Http, want: Wanted): Promise<boolean> {
const res = await http.fetch(`${want.url}/`, { method: "GET", headers: { "X-Plex-Token": want.token, Accept: "application/json" } });
if (res.status === 400 || res.status === 401 || res.status === 403) return false;
if (res.status >= 200 && res.status < 300) return true;
throw new Error(`plex answered ${res.status} at /`);
}
export interface Tautulli {
url: string;
apiKey: string;
}
/** One Tautulli API command's `data`, or a thrown error. The error never carries the key. */
export async function tautulliCmd(http: Http, t: Tautulli, cmd: string): Promise<any> {
const q = new URLSearchParams({ apikey: t.apiKey, cmd });
const res = await http.fetch(`${t.url.replace(/\/$/, "")}/api/v2?${q.toString()}`, { method: "GET" });
const text = await res.text();
if (res.status !== 200) throw new Error(`Tautulli ${cmd} answered ${res.status}`);
const body = (JSON.parse(text) as { response?: { result?: string; message?: string; data?: unknown } }).response ?? {};
if (body.result !== "success") throw new Error(`Tautulli ${cmd}: ${body.message ?? "error"}`);
return body.data;
}
/** Wait for Tautulli to answer, because the step runs right after its container starts. */
export async function tautulliReady(http: Http, t: Tautulli, waitMs: number, pauseMs = 2000): Promise<boolean> {
const until = Date.now() + waitMs;
for (;;) {
try {
const res = await http.fetch(`${t.url.replace(/\/$/, "")}/status`, { method: "GET" });
if (res.status === 200) return true;
} catch {
// not listening yet
}
if (Date.now() >= until) return false;
await new Promise((r) => setTimeout(r, pauseMs));
}
}
/**
* Check Tautulli against the mesh: the token plex takes, the URL Tautulli holds, and Tautulli's own
* connection state, polled for `connectMs` because Tautulli connects to plex a moment after start.
* Never throws.
*/
export async function check(
http: Http,
t: Tautulli,
binding: Binding | undefined,
credential: string | undefined,
connectMs = 60_000,
pauseMs = 3000,
): Promise<Outcome> {
const w = wanted(binding, credential);
if ("problem" in w) return { result: "refused", problem: w.problem };
try {
if (!(await plexTakes(http, w))) return { result: "refused", problem: acceptRemedy(w.from) };
} catch (err) {
return { result: "refused", problem: `plex could not be asked whether it takes the token at ${w.url}: ${message(err)}` };
}
try {
const info = (await tautulliCmd(http, t, "get_server_info")) as { pms_url?: unknown } | undefined;
const holds = typeof info?.pms_url === "string" ? info.pms_url : "";
if (holds.replace(/\/$/, "") !== w.url) {
return {
result: "refused",
problem:
`Tautulli reaches plex at ${holds || "nothing"}, not ${w.url} as the mesh says. Its container writes ` +
`this into config.ini as it starts (custom-init 50-mesh-plex); its log says why it did not`,
};
}
const until = Date.now() + connectMs;
for (;;) {
// server_status answers {connected}, not wrapped in `data` on every version: accept both.
const status = (await tautulliCmd(http, t, "server_status")) as { connected?: unknown } | undefined;
if (status?.connected === true) return { result: "connected", url: w.url };
if (Date.now() >= until) {
return {
result: "refused",
problem: `Tautulli holds ${w.url} and is not connected to plex there — its log says why (a token plex no longer takes, or plex down)`,
};
}
await new Promise((r) => setTimeout(r, pauseMs));
}
} catch (err) {
return { result: "refused", problem: message(err) };
}
}
function message(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
+47
View File
@@ -0,0 +1,47 @@
// tautulli's plex step — run once by the host after Tautulli's server container, and again whenever
// its binding, its pair credential or the server changed (the container's `restart-on`, novox/hq
// ADR 0099). It checks; it writes nothing (plex/check.ts says why, and where the write is).
//
// Exits non-zero when Tautulli does not reach plex as the mesh says, so the node reports the step
// failed. Declared last in the manifest, so its failing gates nothing else of tautulli's (novox/hq
// ADR 0136). Never prints a key or a token.
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { keyOfTautulli } from "../client.js";
import { check, tautulliReady, PROVISION, type Binding, type Http } from "./check.js";
const dir = process.env.MESH_PLEX_DIR ?? "/run/plex";
const url = process.env.MESH_TAUTULLI_URL ?? "http://127.0.0.1:8181";
const waitSeconds = Number(process.env.MESH_TAUTULLI_WAIT_SECONDS ?? "180");
const http: Http = { fetch: (u, init) => fetch(u, init) };
const read = (path: string) => readFile(path, "utf8").catch(() => undefined);
const apiKey = keyOfTautulli(process.env.MESH_TAUTULLI_CONFIG_DIR);
if (!apiKey) {
console.error("[tautulli-plex] Tautulli's config.ini holds no API key yet — it writes one on its first start");
process.exit(1);
}
const tautulli = { url, apiKey };
if (!(await tautulliReady(http, tautulli, waitSeconds * 1000))) {
console.error(`[tautulli-plex] Tautulli did not answer at ${url} within ${waitSeconds}s`);
process.exit(1);
}
let binding: Binding | undefined;
try {
const raw = await read(join(dir, `${PROVISION}.json`));
binding = raw === undefined ? undefined : (JSON.parse(raw) as Binding);
} catch {
binding = undefined;
}
const outcome = await check(http, tautulli, binding, await read(join(dir, `${PROVISION}.secret`)));
if (outcome.result === "connected") {
console.log(`[tautulli-plex] Tautulli reaches plex at ${outcome.url} as the mesh says; connected`);
} else {
console.error(`[tautulli-plex] ${outcome.problem}`);
process.exitCode = 1;
}
+260
View File
@@ -0,0 +1,260 @@
# Where Tautulli reaches Plex — decided by the mesh, written into Tautulli's config.ini before
# Tautulli starts.
#
# Written by the mesh from the tautulli module's manifest, and run by the linuxserver image's
# custom-init (50-mesh-plex) each time the `server` container starts, as root, before Tautulli.
# Editing it here lasts until the next apply.
#
# WHY BEFORE START, AND NOT A STEP AFTER IT. Tautulli keeps its Plex connection only in config.ini
# ([PMS]), reads that file at start, and writes its whole in-memory config back on every shutdown.
# A step editing the file while Tautulli runs is overwritten the moment the container is recreated;
# its API has no command that sets the connection, and its settings form needs an admin login. The
# one moment the file is Tautulli's to read and nobody's to overwrite is here: after the old
# container stopped (and wrote), before the new one reads. The container restarts on its binding
# and credential (`restart-on`), so a plex that moves or a token that is accepted lands here.
#
# WHAT IT WRITES, AND WHEN. Only [PMS] keys, and only when they differ from what the mesh says:
# pms_ip, pms_port, pms_ssl, pms_url - from the binding; always, when the binding is usable
# pms_identifier - plex's own machineIdentifier, when plex answers /identity
# pms_token - the pair credential, ONLY when plex takes it
# Every other key and section, comment and ordering stays as it was, byte for byte.
#
# A TOKEN PLEX REFUSES IS NEVER WRITTEN. Until the operator accepts the server's X-Plex-Token for
# this pair, the mesh delivers a value it minted, which plex answers with 401 (400 on a network it
# trusts). Writing it would replace a working token with a dead one. The address is still written:
# it is right whatever the token, and Tautulli's existing token keeps working at the new address.
# The refusal is loud here and in the `plex` step, which fails naming the `secret accept`.
#
# Never prints the token. Exits 0 even on a refusal: custom-init ignores the code, and Tautulli
# starting on what it had is better than not starting. The step after the server is what fails.
import json
import os
import re
import sys
import tempfile
import time
import urllib.error
import urllib.request
BINDING = os.environ.get("MESH_PLEX_BINDING", "/run/mesh/plex-api.json")
SECRET = os.environ.get("MESH_PLEX_SECRET", "/run/mesh/plex-api.secret")
CONFIG = os.environ.get("MESH_TAUTULLI_CONFIG", "/config/config.ini")
WAIT = float(os.environ.get("MESH_PLEX_WAIT_SECONDS", "60"))
PROVISION = "plex-api"
def say(message):
print("[mesh-plex] " + message, flush=True)
def is_loopback(host):
h = host.lower()
return h in ("localhost", "::1", "[::1]") or h.startswith("127.")
def wanted_address(binding):
"""The [PMS] address keys the binding says, or a reason it cannot say them."""
if not isinstance(binding, dict):
return None, "no binding for %s was delivered" % PROVISION
at = binding.get("at")
at = at.strip() if isinstance(at, str) else ""
serves = binding.get("serves") if isinstance(binding.get("serves"), dict) else {}
try:
port = int(serves.get("port"))
except (TypeError, ValueError):
port = 0
scheme = serves.get("scheme") or "http"
if not at:
return None, "the %s binding names no host (at)" % PROVISION
if is_loopback(at):
return None, (
"the %s binding says plex is at %s, which from Tautulli's container is Tautulli itself; "
"the mesh hands loopback to a machine that is not on the private network" % (PROVISION, at))
if not 0 < port < 65536:
return None, "the %s binding serves no usable port (%r)" % (PROVISION, serves.get("port"))
if scheme not in ("http", "https"):
return None, "the %s binding serves scheme %s, which Tautulli cannot dial" % (PROVISION, scheme)
host = "[%s]" % at if ":" in at and not at.startswith("[") else at
url = "%s://%s:%d" % (scheme, host, port)
return {"pms_ip": at, "pms_port": str(port), "pms_ssl": "1" if scheme == "https" else "0", "pms_url": url}, None
def plex_get(url, path, token=None):
"""(status, parsed JSON or None); raises OSError when plex cannot be asked."""
headers = {"Accept": "application/json"}
if token is not None:
headers["X-Plex-Token"] = token
request = urllib.request.Request(url + path, headers=headers)
try:
with urllib.request.urlopen(request, timeout=10) as response:
body = response.read()
try:
return response.status, json.loads(body)
except ValueError:
return response.status, None
except urllib.error.HTTPError as err:
return err.code, None
def ask_plex(url, token):
"""(takes: True/False/None, machineIdentifier or None). None: plex could not be asked in time."""
deadline = time.monotonic() + WAIT
while True:
try:
status, _ = plex_get(url, "/", token)
if status in (400, 401, 403):
takes = False
elif 200 <= status < 300:
takes = True
else:
raise OSError("plex answered %d at /" % status)
identifier = None
status, body = plex_get(url, "/identity")
if status == 200 and isinstance(body, dict):
value = (body.get("MediaContainer") or {}).get("machineIdentifier")
identifier = value if isinstance(value, str) and value else None
return takes, identifier
except OSError as err:
if time.monotonic() >= deadline:
say("plex could not be asked at %s: %s" % (url, err))
return None, None
time.sleep(3)
SECTION = re.compile(r"^\s*\[([^\]]+)\]\s*$")
KEY = re.compile(r"^(\s*)([A-Za-z0-9_]+)(\s*=\s*)(.*?)\s*$")
def unquoted(value):
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
return value[1:-1]
return value
def plain(value):
"""ConfigObj reads a value unquoted unless it holds one of these; none of ours should."""
return not re.search(r"[#,\"'\r\n]", value) and value == value.strip()
def laid_over(text, wanted):
"""config.ini's text with [PMS] saying `wanted`, and the names of the keys that changed."""
lines = text.splitlines(True)
if lines and not lines[-1].endswith("\n"):
lines[-1] += "\n"
changed = []
start = end = None
for i, line in enumerate(lines):
m = SECTION.match(line)
if m:
if start is not None:
end = i
break
if m.group(1).strip() == "PMS":
start = i
if start is None:
if lines and lines[-1].strip():
lines.append("\n")
lines.append("[PMS]\n")
start, end = len(lines) - 1, len(lines)
elif end is None:
end = len(lines)
seen = set()
for i in range(start + 1, end):
m = KEY.match(lines[i])
if not m or m.group(2) not in wanted:
continue
key = m.group(2)
seen.add(key)
if unquoted(m.group(4)) != wanted[key]:
lines[i] = "%s%s%s%s\n" % (m.group(1), key, m.group(3), wanted[key])
changed.append(key)
missing = [k for k in wanted if k not in seen]
# Insert after the section's last key, not after the blank lines that separate it from the next.
at = end
while at > start + 1 and not lines[at - 1].strip():
at -= 1
for key in missing:
lines.insert(at, "%s = %s\n" % (key, wanted[key]))
at += 1
changed.append(key)
return "".join(lines), changed
def write_config(text):
"""Replace config.ini whole, keeping its owner and mode; a new one takes the directory's owner."""
directory = os.path.dirname(CONFIG) or "."
try:
st = os.stat(CONFIG)
uid, gid, mode = st.st_uid, st.st_gid, st.st_mode & 0o7777
except FileNotFoundError:
st = os.stat(directory)
uid, gid, mode = st.st_uid, st.st_gid, 0o644
fd, tmp = tempfile.mkstemp(prefix=".config.ini.", dir=directory)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(text)
os.chmod(tmp, mode)
try:
os.chown(tmp, uid, gid)
except PermissionError:
pass
os.replace(tmp, CONFIG)
except BaseException:
if os.path.exists(tmp):
os.unlink(tmp)
raise
def read(path):
try:
with open(path, encoding="utf-8") as f:
return f.read()
except FileNotFoundError:
return None
def main():
raw = read(BINDING)
try:
binding = json.loads(raw) if raw is not None else None
except ValueError:
binding = None
address, problem = wanted_address(binding)
if problem:
say("left Tautulli's Plex connection as it was: " + problem)
return 0
wanted = dict(address)
token = (read(SECRET) or "").strip()
takes, identifier = ask_plex(address["pms_url"], token) if token else (False, None)
if identifier:
wanted["pms_identifier"] = identifier
frm = binding.get("from") or "<its node>"
if not token:
say("no %s credential was delivered; only the address was written" % PROVISION)
elif takes and plain(token):
wanted["pms_token"] = token
elif takes is False:
say("plex refuses the %s credential the mesh delivered, so it was not written; the address was. "
"plex's token is issued by plex.tv and the mesh cannot make it: accept the server's own token "
"for this pair - `secret accept <this node> tautulli %s --provider %s --from <file holding the "
"server's X-Plex-Token>`" % (PROVISION, PROVISION, frm))
elif takes:
say("the %s credential holds characters config.ini cannot carry unquoted; it was not written" % PROVISION)
else:
say("plex could not be asked whether it takes the %s credential; only the address was written" % PROVISION)
if not all(plain(v) for v in wanted.values()):
say("the %s binding holds characters config.ini cannot carry unquoted; nothing was written" % PROVISION)
return 0
before = read(CONFIG)
after, changed = laid_over(before or "", wanted)
if not changed:
say("Tautulli's Plex connection is already as the mesh says (%s)" % address["pms_url"])
return 0
write_config(after)
say("wrote %s into Tautulli's [PMS] (%s)" % (", ".join(changed), address["pms_url"]))
return 0
if __name__ == "__main__":
sys.exit(main())
+266
View File
@@ -0,0 +1,266 @@
// What holds tautulli's plex-api consumer — both halves.
//
// The write (plex/mesh-plex.py, run in the server container before Tautulli starts): [PMS] is made
// to say what the mesh bound and every other line of config.ini stays byte for byte; nothing is
// written when nothing differs; a token plex refuses is never written, while the address still is;
// a config.ini that does not exist yet is started with [PMS] alone. Run with the machine's python3
// against a fake plex; skipped where there is no python3.
//
// The check (plex/check.ts, the step declared last): a refused token fails naming the `secret
// accept`; a Tautulli pointed elsewhere, or not connected, fails; one pointed where the binding says
// and connected passes.
//
// And the manifest carries exactly the files in plex/ — they are the source, module.json the copy.
//
// Fakes answer as the real ones do (checked against lscr.io/linuxserver/tautulli 2.18.1-ls244 and
// plexinc/pms-docker 1.43.4: plex answers 401 to an unknown token from another network, 400 on one
// it trusts). Imports the compiled step, as keycloak's tests do.
import { test } from "node:test";
import assert from "node:assert/strict";
import { execFile, spawnSync } from "node:child_process";
import { createServer, type Server } from "node:http";
import { mkdtempSync, readFileSync, statSync, writeFileSync, existsSync } from "node:fs";
import { networkInterfaces, tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { check, type Binding, type Http } from "../dist/plex/check.js";
const here = fileURLToPath(new URL("..", import.meta.url));
const TOKEN = "the-servers-own-token";
const MACHINE = "5c47d9a165d10b622995d55b3ae1f168242f33bd";
// ---- the manifest carries the files ------------------------------------------------------------
test("module.json carries plex/mesh-plex.py and plex/50-mesh-plex exactly", () => {
const m = JSON.parse(readFileSync(join(here, "module.json"), "utf8")) as { resources: { id: string; content?: string }[] };
const byId = (id: string) => m.resources.find((r) => r.id === id)?.content;
assert.equal(byId("plex-init-code"), readFileSync(join(here, "plex/mesh-plex.py"), "utf8"));
assert.equal(byId("plex-init"), readFileSync(join(here, "plex/50-mesh-plex"), "utf8"));
// Nothing in them the mesh would read as a placeholder.
assert.doesNotMatch(byId("plex-init-code") ?? "", /\$\{/);
assert.doesNotMatch(byId("plex-init") ?? "", /\$\{/);
});
// ---- the write: mesh-plex.py -------------------------------------------------------------------
const python = spawnSync("python3", ["--version"]).status === 0 ? "python3" : undefined;
/** An address of this machine that is not loopback, which the script refuses as plex's. */
function outwardAddress(): string | undefined {
for (const list of Object.values(networkInterfaces())) {
for (const a of list ?? []) if (a.family === "IPv4" && !a.internal) return a.address;
}
return undefined;
}
const outward = outwardAddress();
function fakePlex(opts: { trusted?: boolean } = {}): Promise<{ server: Server; port: number }> {
const server = createServer((req, res) => {
if (req.url === "/identity") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ MediaContainer: { machineIdentifier: MACHINE } }));
return;
}
if (req.headers["x-plex-token"] !== TOKEN) {
res.writeHead(opts.trusted ? 400 : 401);
res.end();
return;
}
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ MediaContainer: { friendlyName: "ace" } }));
});
return new Promise((resolve) => server.listen(0, "0.0.0.0", () => resolve({ server, port: (server.address() as { port: number }).port })));
}
// An operator's config.ini, shaped as Tautulli writes it: plex at HAL's network gateway.
const OPERATOR_INI = [
"[General]",
"first_run_complete = 1",
"api_key = 0123456789abcdef0123456789abcdef",
"",
"[PMS]",
"pms_identifier = " + MACHINE,
"pms_ip = 172.18.0.1",
"pms_is_remote = 0",
"pms_name = ace",
"pms_port = 32400",
'pms_token = "' + TOKEN + '"',
"pms_ssl = 0",
"pms_url = http://172.18.0.1:32400",
"pms_url_manual = 0",
"",
"[Monitoring]",
"monitor_pms_updates = 0",
"",
].join("\n");
function runScript(dir: string, at: string, port: number, credential: string, wait = "5") {
writeFileSync(join(dir, "plex-api.json"), JSON.stringify({ binding: 1, provision: "plex-api", from: "ace", at, serves: { scheme: "http", port } }));
writeFileSync(join(dir, "plex-api.secret"), credential + "\n");
// Asynchronously: the fake plex answers from this same process, so a blocking spawn would starve it.
return new Promise<{ status: number; out: string }>((resolve) => {
execFile(python as string, [join(here, "plex/mesh-plex.py")], {
env: {
...process.env,
MESH_PLEX_BINDING: join(dir, "plex-api.json"),
MESH_PLEX_SECRET: join(dir, "plex-api.secret"),
MESH_TAUTULLI_CONFIG: join(dir, "config.ini"),
MESH_PLEX_WAIT_SECONDS: wait,
},
encoding: "utf8",
}, (err, stdout, stderr) => resolve({ status: err ? Number((err as { code?: unknown }).code ?? 1) : 0, out: `${stdout}${stderr}` }));
});
}
const skip = !python ? "no python3 here" : !outward ? "no non-loopback address to serve a fake plex on" : false;
test("the write: [PMS] says what the mesh bound, and every other line stays", { skip }, async () => {
const { server, port } = await fakePlex();
try {
const dir = mkdtempSync(join(tmpdir(), "mesh-plex-"));
writeFileSync(join(dir, "config.ini"), OPERATOR_INI);
const r = await runScript(dir, outward as string, port, TOKEN);
assert.equal(r.status, 0);
// The token was already the server's (quoted, as ConfigObj may write it): not rewritten.
assert.match(r.out, /wrote pms_ip, pms_port, pms_url into Tautulli's \[PMS\]/);
const url = `http://${outward}:${port}`;
const expected = OPERATOR_INI
.replace("pms_ip = 172.18.0.1", `pms_ip = ${outward}`)
.replace("pms_port = 32400", `pms_port = ${port}`)
.replace("pms_url = http://172.18.0.1:32400", `pms_url = ${url}`);
assert.equal(readFileSync(join(dir, "config.ini"), "utf8"), expected);
assert.doesNotMatch(r.out, new RegExp(TOKEN));
// Again: nothing differs, nothing is written.
const before = statSync(join(dir, "config.ini")).mtimeMs;
const again = await runScript(dir, outward as string, port, TOKEN);
assert.match(again.out, /already as the mesh says/);
assert.equal(statSync(join(dir, "config.ini")).mtimeMs, before);
} finally {
server.close();
}
});
test("the write: a token plex refuses is never written; the address still is", { skip }, async () => {
for (const trusted of [false, true]) {
const { server, port } = await fakePlex({ trusted });
try {
const dir = mkdtempSync(join(tmpdir(), "mesh-plex-"));
writeFileSync(join(dir, "config.ini"), OPERATOR_INI);
const r = await runScript(dir, outward as string, port, "a-value-the-mesh-minted");
assert.equal(r.status, 0, "custom-init ignores the code; Tautulli starts on what it had");
assert.match(r.out, /secret accept <this node> tautulli plex-api --provider ace/);
assert.doesNotMatch(r.out, /a-value-the-mesh-minted/);
const ini = readFileSync(join(dir, "config.ini"), "utf8");
assert.match(ini, new RegExp(`pms_token = "${TOKEN}"`), "the working token stays");
assert.match(ini, new RegExp(`pms_ip = ${outward!.replace(/\./g, "\\.")}\n`));
} finally {
server.close();
}
}
});
test("the write: a Tautulli with no config.ini yet is started with [PMS] alone", { skip }, async () => {
const { server, port } = await fakePlex();
try {
const dir = mkdtempSync(join(tmpdir(), "mesh-plex-"));
await runScript(dir, outward as string, port, TOKEN);
assert.equal(
readFileSync(join(dir, "config.ini"), "utf8"),
`[PMS]\npms_ip = ${outward}\npms_port = ${port}\npms_ssl = 0\npms_url = http://${outward}:${port}\n` +
`pms_identifier = ${MACHINE}\npms_token = ${TOKEN}\n`,
);
} finally {
server.close();
}
});
test("the write: a plex that cannot be asked gets its address written and no token", { skip }, async () => {
const { server, port } = await fakePlex();
await new Promise((r) => server.close(r)); // nothing listens there now
const dir = mkdtempSync(join(tmpdir(), "mesh-plex-"));
writeFileSync(join(dir, "config.ini"), OPERATOR_INI);
const r = await runScript(dir, outward as string, port, TOKEN, "0");
assert.match(r.out, /could not be asked/);
const ini = readFileSync(join(dir, "config.ini"), "utf8");
assert.match(ini, new RegExp(`pms_url = http://${outward!.replace(/\./g, "\\.")}:${port}\n`));
assert.match(ini, new RegExp(`pms_token = "${TOKEN}"`), "left exactly as it was");
});
test("the write: a loopback binding writes nothing", { skip: !python ? "no python3 here" : false }, async () => {
const dir = mkdtempSync(join(tmpdir(), "mesh-plex-"));
writeFileSync(join(dir, "config.ini"), OPERATOR_INI);
const r = await runScript(dir, "127.0.0.1", 32400, TOKEN, "0");
assert.match(r.out, /private network/);
assert.equal(readFileSync(join(dir, "config.ini"), "utf8"), OPERATOR_INI);
assert.equal(existsSync(join(dir, "config.ini")), true);
});
// ---- the check: plex/check.ts ------------------------------------------------------------------
function binding(at = "ace.internal", port = 32400): Binding {
return { provision: "plex-api", from: "ace", at, serves: { scheme: "http", port } };
}
function fakes(opts: { holds?: string; connected?: boolean; trusted?: boolean } = {}) {
const calls: string[] = [];
const http: Http = {
async fetch(url, init) {
calls.push(url);
const reply = (status: number, value?: unknown) => ({ status, text: async () => (value === undefined ? "" : JSON.stringify(value)) });
const u = new URL(url);
if (u.hostname === "ace.internal") {
return init?.headers?.["X-Plex-Token"] === TOKEN ? reply(200, {}) : reply(opts.trusted ? 400 : 401);
}
if (u.searchParams.get("apikey") !== "tautulli-key") return reply(401);
const cmd = u.searchParams.get("cmd");
if (cmd === "get_server_info") {
return reply(200, { response: { result: "success", data: { pms_url: opts.holds ?? "http://ace.internal:32400", pms_ip: "ace.internal" } } });
}
if (cmd === "server_status") {
return reply(200, { response: { result: "success", data: { result: "success", connected: opts.connected ?? true } } });
}
return reply(404);
},
};
return { http, calls };
}
const TAUTULLI = { url: "http://127.0.0.1:8181", apiKey: "tautulli-key" };
test("the check: pointed where the binding says and connected passes", async () => {
const f = fakes();
assert.deepEqual(await check(f.http, TAUTULLI, binding(), TOKEN, 0, 0), { result: "connected", url: "http://ace.internal:32400" });
});
test("the check: a token plex refuses fails naming the accept, and never prints it", async () => {
for (const trusted of [false, true]) {
const f = fakes({ trusted });
const out = await check(f.http, TAUTULLI, binding(), "a-value-the-mesh-minted", 0, 0);
assert.equal(out.result, "refused");
const problem = (out as { problem: string }).problem;
assert.match(problem, /secret accept <this node> tautulli plex-api --provider ace/);
assert.doesNotMatch(problem, /a-value-the-mesh-minted/);
assert.equal(f.calls.some((c) => c.includes("/api/v2")), false, "Tautulli was not even asked");
}
});
test("the check: a Tautulli pointed elsewhere fails, naming where it points", async () => {
const out = await check(fakes({ holds: "http://172.18.0.1:32400" }).http, TAUTULLI, binding(), TOKEN, 0, 0);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /at http:\/\/172\.18\.0\.1:32400, not http:\/\/ace\.internal:32400/);
});
test("the check: pointed right but not connected fails", async () => {
const out = await check(fakes({ connected: false }).http, TAUTULLI, binding(), TOKEN, 0, 0);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /not connected/);
});
test("the check: a loopback binding is refused", async () => {
const out = await check(fakes().http, TAUTULLI, binding("127.0.0.1"), TOKEN, 0, 0);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /private network/);
});
+1 -1
View File
@@ -8,5 +8,5 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
"include": ["client.ts", "index.ts", "tools/index.ts", "plex/check.ts", "plex/index.ts"]
}