Compare commits

..
Author SHA1 Message Date
jschoubben 5e6d29747e nzbget: placed config, the build ace runs, its password a file, its API provided
The manifest named /services/nzbget/config and /var/lib/mesh/nzbget/config.json,
host paths ADR 0112 takes out of definitions. The config dir is now pathless
(${dir:config}); the runtime's config and route binding live in a placed state dir.

The image is pinned to v26.0-ls233, the digest ace runs. The old pin (v26.3-ls261)
is newer but unproven against ace's queue; moving up is a later, separate step.

The password own-secret reached the tools and nothing else, so a fresh machine ran
nzbget's well-known default while the tools held a minted value that matched
nothing. The server now reads it too, through the image's FILE__NZBGET_PASS (a
path in the environment, the value from a 0600 root file - ADR 0086), and both
restart on it. An adopted machine accepts its existing ControlPassword.

The tools assumed the control user is "nzbget"; they now read ControlUsername
from nzbget.conf on the read-only config mount (ace's is not "nzbget").

sonarr, radarr, lidarr and bookshelf reached nzbget by container name on HAL's
shared network. nzbget now provides nzbget-api (node scope: a download client
must share the consumer's download spool) and serves scheme, port, url-base and
username; the password is the operator-accepted pair credential, as for #156.
The web endpoint is routed (label nzbget). The runtime dials ${port:6789}, the
same line as #154.

Verified: catalogue tests with MESH_CATALOGUE set (not skipped); rendered for ace
with a pinned port and username setting; a throwaway of the pinned image on a
fresh 0700 dir with the secret as a 0600 root file answered the secret (200),
refused a wrong and the default password (401); the compiled client read the
username from nzbget.conf and reached version/status/queue/history; strict
typecheck and the module's Dockerfile build pass.
2026-09-30 12:00:31 +02:00
mesh-admin 8064e5da8f Merge pull request 'searxng: its settings are a file the mesh writes, not the image's defaults' (#143) from feat/searxng-settings-as-a-file into main 2026-09-29 20:45:13 +00:00
jschoubben 63a255c5cb searxng: bind its route where its state now lives
The route binding still named /var/lib/searxng-module, the directory the
previous commit placed elsewhere — the host would have written it into a
directory nothing declares. Same shape as gitea and nextcloud.
2026-09-29 22:41:56 +02:00
jschoubben 7ad1fbd5c6 searxng: its settings are a file the mesh writes, not the image's defaults
The module ran searxng on the image's built-in settings, which serve html
only — so the module's own search tool (format=json) was refused by the
software it fronts. And there was no way to configure it per machine: the
only file settings reach was the sidecar's.

settings.yml is now the module's one mergeable file (JSON is YAML): generic
defaults in the manifest (json format on, limiter and image proxy off,
valkey wired), and whatever differs per machine — base_url, method,
autocomplete, suspended times — set as the assignment's settings. The
secret key is filled on the machine through ${secret:secret}, so the
secrets-in-environment exception and the env file go. Directories are
placed. Image pinned to 2026.9.20, what ace runs today (the old pin was
older, 2026.9.1).

The sidecar's config.json is no longer mergeable: settings merge into every
mergeable file of a module, and the sidecar would have received searxng's
keys. It only ever read an optional url, which its env already carries.

Verified on ace: the pinned image serves html and json from a read-only,
root-owned 0600 JSON settings.yml.
2026-09-29 22:33:38 +02:00
jschoubben 67f5f4cffd Merge pull request 'ca-trust: a machine trusts the mesh's authority because a module put its root there' (#142) from feat/ca-trust into main 2026-09-29 14:06:36 +00:00
jschoubben 8797335fbc ca-trust: a machine trusts the mesh's authority because a module put its root there
novox/hq ADR 0147, issue 129. Every internal HTTPS name fails verification
on every machine: the certificates are genuine and nothing on a machine has
ever been told what issued them. The proxy's fetch answers for the proxy and
for nothing else — a browser, git over HTTPS and every module calling another
by an internal name read the machine's own trust store.

The module requires internal-acme-ca, fetches the root over the mesh's own
network (no prior trust to have; that is what this establishes), installs it
among the machine's anchors and refreshes the extracted bundles. Being
unassigned stops the unit, and stopping it takes the anchor away and
refreshes them again.

Arch's layout is named out loud: a machine that keeps anchors elsewhere fails
visibly rather than writing a file nothing reads.
2026-09-29 15:07:40 +02:00
mesh-admin 53dc108603 Merge pull request 'Remove the network-checker module: it does not do what was decided' (#141) from chore/remove-the-network-checker-module into main 2026-09-29 12:43:34 +00:00
jschoubben ebf5ba2d4c Remove the network-checker module: it does not do what was decided
What was in the catalogue was the first thing I built, not the thing ADR 0146
describes. It dialled raw ports on machine addresses from one hosting form and
emitted nothing, so findings would have sat in a file on the machine — the exact
thing issue 145 is about. It was never registered, never assigned, and never ran.

0146 says names per hosting form, fetched over TLS with the certificate verified,
and machines discovered over the bus. That shares nothing with this but the word
checker, so it goes rather than being bent into shape. Recorded as work to be
analysed and built deliberately.

Connectivity is checked by hand in the meantime, against the services the mesh
already runs.
2026-09-29 14:43:25 +02:00
mesh-admin bbac08a7d2 Merge pull request 'A network-checker module: dial what the mesh claims, from where the callers are' (#140) from feat/a-network-checker-module into main 2026-09-29 11:44:37 +00:00
jschoubben 784a5a6514 A network-checker module: dial what the mesh claims, from where the callers are
The mesh asserts three things are callable (ADR 0144) — what runs on the same
machine, another machine's service exposed to the private network, and another
machine's service exposed publicly — and has never checked any of them. The first
was broken for eleven hours while the mesh reported every machine healthy.

This runs on every machine, on the cadence the mesh already has, in its own
container: the same position every other module calls from. Not the host and not
the control plane, both of which reach these addresses by paths no ordinary caller
uses and would have passed throughout that outage.

**Its probe is its own endpoint, and that is the point.** Declared reachable over
the private network like any other service, so it is admitted by exactly the rule
that governs every internally-exposed service and fails when that rule is wrong.
The tempting target is a service every machine has, and those are the ones never
closed — ssh above all — which would have passed while the thing that actually
broke was a service exposed to the private network.

It resolves before it dials and says which failed, because a name that does not
resolve and a port that does not answer have different owners. One failure is not
a fault: a machine rebooting is ordinary, so a path is broken after consecutive
runs and the count travels with the result. It reports and repairs nothing.

novox/hq ADR 0145. Eight tests; the consecutive-failure logic proved by reverting
it once. Not yet registered or assigned.
2026-09-29 13:44:16 +02:00
mesh-admin 0c31499fb0 Merge pull request 'Every module names its endpoints, and every route names the one it serves' (#139) from feat/modules-name-their-endpoints into main 2026-09-29 09:51:56 +00:00
4 changed files with 142 additions and 33 deletions
+56
View File
@@ -0,0 +1,56 @@
{
"module": "ca-trust",
"version": "1",
"slug": "catrust",
"capabilities": [
"service-manager"
],
"requires": [
"internal-acme-ca"
],
"seats": [
{
"name": "the-mesh-trust-anchor",
"scope": "node"
}
],
"claims": [
{
"name": "the-mesh-trust-anchor",
"scope": "node"
}
],
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "anchor",
"type": "file",
"path": "${dir:state}/anchor",
"mode": "0755",
"content": "#!/bin/sh\n# The mesh's internal certificate authority, trusted by this machine.\n#\n# Written by the mesh from the ca-trust module's manifest (novox/hq ADR 0147).\n# Editing it here lasts until the next apply.\n#\n# There is no prior trust to verify the fetch against \u2014 this is the thing that\n# establishes it \u2014 so it is made over the mesh's own private network, which is\n# what authenticates it (novox/hq ADR 0098, the same reasoning that lets the\n# route proxy fetch this root for itself). What comes back is checked here: a\n# body that is not a certificate is refused now, rather than believed and then\n# failed by whatever reads the trust store next.\nset -eu\n\nROOTS='https://${bound:internal-acme-ca:at}:${bound:internal-acme-ca:port}${bound:internal-acme-ca:roots}'\nANCHORS=/etc/ca-certificates/trust-source/anchors\nANCHOR=\"$ANCHORS/mesh-internal-ca.crt\"\n\n# Arch's layout, said out loud rather than assumed: a machine that keeps its\n# anchors elsewhere fails here, visibly, instead of writing a file nothing\n# reads. That failure is the signal that this belongs in the host, where one\n# operating system's difference lives (novox/hq ADR 0147, option 2).\n[ -d \"$ANCHORS\" ] || {\n\techo \"this machine keeps no trust anchors in $ANCHORS; ca-trust is written for that layout\" >&2\n\texit 1\n}\n\ncase \"${1:-}\" in\ninstall)\n\ttmp=$(mktemp)\n\ttrap 'rm -f \"$tmp\"' EXIT\n\t# The authority may still be starting, or this machine may have come up\n\t# before it: two minutes of asking, then an honest failure.\n\tn=0\n\twhile [ \"$n\" -lt 60 ]; do\n\t\tif curl --fail --silent --show-error --insecure --max-time 10 \\\n\t\t\t--output \"$tmp\" \"$ROOTS\" &&\n\t\t\tgrep -q 'BEGIN CERTIFICATE' \"$tmp\"; then\n\t\t\tinstall -m 0644 \"$tmp\" \"$ANCHOR\"\n\t\t\tupdate-ca-trust\n\t\t\texit 0\n\t\tfi\n\t\tn=$((n + 1))\n\t\tsleep 2\n\tdone\n\techo \"the authority at $ROOTS did not serve a certificate within two minutes\" >&2\n\texit 1\n\t;;\nremove)\n\t# What stopping the unit does, and therefore what being unassigned does.\n\trm -f \"$ANCHOR\"\n\tupdate-ca-trust\n\t;;\n*)\n\techo \"usage: $(basename \"$0\") install|remove\" >&2\n\texit 2\n\t;;\nesac\n"
},
{
"id": "unit",
"type": "file",
"path": "/etc/systemd/system/mesh-ca-trust.service",
"mode": "0644",
"content": "[Unit]\nDescription=The mesh's internal certificate authority, trusted by this machine\n# novox/hq ADR 0147. Starting this unit places the mesh's root among this\n# machine's trust anchors; stopping it takes the root away again, which is what\n# the host does when the module is no longer assigned here.\nWants=network-online.target\nAfter=network-online.target\n\n[Service]\nType=oneshot\nRemainAfterExit=yes\nExecStart=${dir:state}/anchor install\nExecStop=${dir:state}/anchor remove\n\n[Install]\nWantedBy=multi-user.target\n"
},
{
"id": "trust",
"type": "service",
"unit": "mesh-ca-trust.service",
"state": "running",
"boot": "enabled",
"restart-on": [
"anchor",
"unit"
]
}
]
}
+20 -2
View File
@@ -48,6 +48,20 @@ function meshConfig(file?: string): Record<string, string> {
catch { return {}; } catch { return {}; }
} }
/** One key of nzbget's own nzbget.conf, from the config directory the mesh mounts read-only
* (MESH_NZBGET_CONFIG_DIR). The software's file is the truth about who may log in, so the tools
* ask it rather than a setting that could disagree. Absent, unreadable or unset yields undefined. */
function confValue(dir: string | undefined, key: string): string | undefined {
if (!dir) return undefined;
try {
const line = readFileSync(`${dir.replace(/\/$/, "")}/nzbget.conf`, "utf8")
.split("\n")
.find((l) => l.startsWith(`${key}=`));
const value = line?.slice(key.length + 1).trim();
return value ? value : undefined;
} catch { return undefined; }
}
/** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`); /** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
* absent or unreadable yields undefined so callers fall back rather than crash. */ * absent or unreadable yields undefined so callers fall back rather than crash. */
function readSecret(file?: string): string | undefined { function readSecret(file?: string): string | undefined {
@@ -69,7 +83,9 @@ export class NzbgetClient {
* Build from the module's resolved environment. URL and password are read from MESH_NZBGET_URL * Build from the module's resolved environment. URL and password are read from MESH_NZBGET_URL
* and MESH_NZBGET_PASSWORD; both must be present — an unconfigured NZBGet throws rather than * and MESH_NZBGET_PASSWORD; both must be present — an unconfigured NZBGet throws rather than
* pretend to be reachable, so the tools/events simply do not load (the harness treats the throw * pretend to be reachable, so the tools/events simply do not load (the harness treats the throw
* as "exposes nothing"). The control username defaults to "nzbget", NZBGet's own default. * as "exposes nothing"). The password file is the same own-secret the server container is started
* with (FILE__NZBGET_PASS), so the two cannot disagree. The control username is read from
* nzbget.conf, falling back to "nzbget", NZBGet's own default.
*/ */
static fromEnv(env: NodeJS.ProcessEnv = process.env): NzbgetClient { static fromEnv(env: NodeJS.ProcessEnv = process.env): NzbgetClient {
const cfg = meshConfig(env.MESH_NZBGET_CONFIG_FILE); const cfg = meshConfig(env.MESH_NZBGET_CONFIG_FILE);
@@ -78,7 +94,9 @@ export class NzbgetClient {
if (!url || !password) { if (!url || !password) {
throw new Error("NZBGet not configured — set MESH_NZBGET_URL and MESH_NZBGET_PASSWORD"); throw new Error("NZBGet not configured — set MESH_NZBGET_URL and MESH_NZBGET_PASSWORD");
} }
const user = cfg.user ?? env.MESH_NZBGET_USER ?? "nzbget"; // The control username: a setting or the environment if one says so, else whatever
// nzbget.conf holds (an adopted machine keeps its own, e.g. not "nzbget"), else NZBGet's default.
const user = cfg.user ?? env.MESH_NZBGET_USER ?? confValue(env.MESH_NZBGET_CONFIG_DIR, "ControlUsername") ?? "nzbget";
return new NzbgetClient(url, user, password); return new NzbgetClient(url, user, password);
} }
+45 -11
View File
@@ -19,7 +19,7 @@
"port": 6789, "port": 6789,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "the download client's pages" "why": "the download client's pages, and the JSON-RPC API its consumers and its own tools call"
} }
], ],
"accesses": [ "accesses": [
@@ -35,10 +35,15 @@
"path": "/var/lib/mesh/nzbget", "path": "/var/lib/mesh/nzbget",
"mode": "0700" "mode": "0700"
}, },
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{ {
"id": "config", "id": "config",
"type": "directory", "type": "directory",
"path": "/services/nzbget/config",
"mode": "0700", "mode": "0700",
"owner": "1000:1000" "owner": "1000:1000"
}, },
@@ -46,24 +51,29 @@
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "nzbget", "name": "nzbget",
"image": "lscr.io/linuxserver/nzbget@sha256:5f3d3fa71029004156eff2cbf4ef4455ce4ce59517cf13fa7d1d7c8a4cd2c8a4", "image": "lscr.io/linuxserver/nzbget@sha256:ec3ef0ae7dc410084086a7fdd447deda4747531154aa1cc5c7c48ac60794e277",
"env": { "env": {
"PUID": "1000", "PUID": "1000",
"PGID": "1000", "PGID": "1000",
"TZ": "Etc/UTC" "TZ": "Etc/UTC",
"FILE__NZBGET_PASS": "/run/secrets/password"
}, },
"ports": [ "ports": [
"6789" "6789"
], ],
"volumes": [ "volumes": [
"/services/nzbget/config:/config", "${dir:config}:/config",
"/services/media/downloads:/downloads" "/services/media/downloads:/downloads",
"/var/lib/mesh/nzbget/password:/run/secrets/password:ro"
],
"restart-on": [
"needs-password"
] ]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "/var/lib/mesh/nzbget/config.json", "path": "${dir:state}/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n",
"merge": "json" "merge": "json"
@@ -76,22 +86,46 @@
"volumes": [ "volumes": [
"/var/lib/mesh/nzbget/broker:/run/secrets/broker:ro", "/var/lib/mesh/nzbget/broker:/run/secrets/broker:ro",
"/var/lib/mesh/nzbget/password:/run/secrets/password:ro", "/var/lib/mesh/nzbget/password:/run/secrets/password:ro",
"/var/lib/mesh/nzbget/config.json:/run/config/config.json:ro", "${dir:state}/config.json:/run/config/config.json:ro",
"/services/nzbget/config:/var/lib/nzbget/config:ro" "${dir:config}:/var/lib/nzbget/config:ro"
], ],
"env": { "env": {
"MESH_BROKER_FILE": "/run/secrets/broker", "MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_NZBGET_URL": "http://127.0.0.1:6789", "MESH_NZBGET_URL": "http://127.0.0.1:${port:6789}",
"MESH_NZBGET_PASSWORD_FILE": "/run/secrets/password", "MESH_NZBGET_PASSWORD_FILE": "/run/secrets/password",
"MESH_NZBGET_CONFIG_FILE": "/run/config/config.json", "MESH_NZBGET_CONFIG_FILE": "/run/config/config.json",
"MESH_NZBGET_CONFIG_DIR": "/var/lib/nzbget/config" "MESH_NZBGET_CONFIG_DIR": "/var/lib/nzbget/config"
}, },
"restart-on": [ "restart-on": [
"runtime-config" "runtime-config",
"needs-password"
], ],
"artifact": "runtime" "artifact": "runtime"
} }
], ],
"provides": [
"nzbget-api"
],
"serves": {
"nzbget-api": {
"scheme": "http",
"port": 6789,
"url-base": "",
"username": "nzbget"
}
},
"requires": [
"route"
],
"contributes": {
"route": {
"label": "nzbget",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:state}/route.json"
},
"build": { "build": {
"on": [ "on": [
{ {
+21 -20
View File
@@ -5,7 +5,7 @@
"container-runtime" "container-runtime"
], ],
"own-secrets": { "own-secrets": {
"secret": "/var/lib/searxng-module/secret.secret", "secret": "/var/lib/mesh/searxng/secret",
"broker": "/var/lib/mesh/searxng/broker" "broker": "/var/lib/mesh/searxng/broker"
}, },
"listens": [ "listens": [
@@ -27,22 +27,14 @@
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"path": "/var/lib/searxng-module", "mode": "0700",
"mode": "0700" "place": "."
}, },
{ {
"id": "valkey-data", "id": "valkey-data",
"type": "directory", "type": "directory",
"path": "/var/lib/searxng-module/valkey-data",
"mode": "0700" "mode": "0700"
}, },
{
"id": "server-env",
"type": "file",
"path": "/var/lib/searxng-module/server.env",
"mode": "0600",
"content": "SEARXNG_SECRET=${secret:secret}\nSEARXNG_VALKEY_URL=valkey://valkey:6379/0\n"
},
{ {
"id": "net", "id": "net",
"type": "network", "type": "network",
@@ -63,30 +55,39 @@
"warning" "warning"
], ],
"volumes": [ "volumes": [
"/var/lib/searxng-module/valkey-data:/data" "${dir:valkey-data}:/data"
] ]
}, },
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.yml",
"mode": "0600",
"merge": "json",
"content": "{\n \"use_default_settings\": true,\n \"server\": {\n \"secret_key\": \"${secret:secret}\",\n \"base_url\": false,\n \"limiter\": false,\n \"image_proxy\": false,\n \"public_instance\": false\n },\n \"search\": {\n \"formats\": [\"html\", \"json\"]\n },\n \"valkey\": {\n \"url\": \"valkey://valkey:6379/0\"\n }\n}\n"
},
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "searxng", "name": "searxng",
"image": "searxng/searxng@sha256:c7cc75852051bf6254afda6ed1b920dd1677d8efe4ab141bf558f02e582f4371", "image": "searxng/searxng@sha256:cd8812607ab73730a0b1a0dc4990223fe1b9e383f6f35947114d0bef7f8bb441",
"network": "searxng", "network": "searxng",
"env-file": [
"/var/lib/searxng-module/server.env"
],
"ports": [ "ports": [
"8080" "8080"
], ],
"secrets-in-environment": "SEARXNG_SECRET is env-only, but settings.yml carries server.secret_key; convertible by mounting a generated settings.yml, not yet done" "volumes": [
"${dir:state}/settings.yml:/etc/searxng/settings.yml:ro"
],
"restart-on": [
"settings"
]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "/var/lib/mesh/searxng/config.json", "path": "/var/lib/mesh/searxng/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n"
"merge": "json"
}, },
{ {
"id": "runtime", "id": "runtime",
@@ -118,7 +119,7 @@
} }
}, },
"binds": { "binds": {
"route": "/var/lib/searxng-module/route.json" "route": "${dir:state}/route.json"
}, },
"build": { "build": {
"on": [ "on": [