Compare commits

..
Author SHA1 Message Date
jschoubben bd0c6b3dc9 bookshelf: reach nzbget, qbittorrent and jackett through the mesh
bookshelf reached its download clients as nzbget:6789 and qbittorrent:8112 and its indexers
through jackett:9117 or indexers.zurag.be - HAL container names and a public route,
typed into its database by hand. Nothing on the mesh answers those names.

It now requires nzbget-api, qbittorrent-api and jackett-api. A run-once step
(downloads/, declared last, restart-on its bindings, credentials and settings) writes
host, port, TLS, base path, user and credential into bookshelf through bookshelf's own API:

- A download client is the mesh's when its name is downloads.<provision>.name and its
  kind the provider's; it is registered when missing. A jackett feed is the mesh's when
  its host is the bound one, one the step bound before, or one in
  downloads.jackett-api.adopt-hosts; the jackett indexers in
  downloads.jackett-api.indexers are registered when missing. Every other entry is left
  alone, and nothing is ever deleted.
- Only connection fields, only when they differ. The stored password is masked, so the
  app tests the entry with the credential it holds; only if that fails is the delivered
  one written. Categories, priorities and "enabled" are never touched.
- Each credential is tried against its provider first. A minted value (nothing accepted
  yet) is never written; the step exits 1 naming the exact secret accept.

The step is byte-identical in sonarr, radarr, lidarr and bookshelf (the same Servarr
API, v3 or v1): each module builds from its own directory, so each carries a copy, and
test/downloads.test.ts fails if a sibling's copy differs.

Verified: strict typecheck, the Dockerfile build, 14 unit tests; and the compiled step
against fresh pinned sonarr/radarr/lidarr/bookshelf with throwaway nzbget, qBittorrent
and jackett - minted credentials refused with nothing written, accepted ones registered
and tested by the app, a migration-shaped radarr repointed, reruns unchanged.
2026-09-30 13:07:10 +02:00
jschoubben 287e5e6d3f bookshelf: its config dir is placed, its image is the one ace runs
The config directory was the stated path /services/bookshelf/config; it
is now a pathless directory the mesh places (0700, 1000:1000), and the
route binding lives in a placed state dir, as in jackett and searxng.
/var/lib/mesh/bookshelf stays for the broker secret.

The image is pinned to the digest ace runs (hardcover, 0.4.21.182,
ls36). The old pin was a February build: older than the database it
would open, which Readarr-family apps migrate forward only.

The sidecar dials ${port:8787}, the port the mesh assigned, not the
software's own (the same one-line change as #154).

Not carried from HAL: its install hook seeded config.xml with an API key
(the image writes its own on first start, and the data keeps it) and
registered nzbget, qbittorrent and jackett through the API with wrong
hosts and ports, so ace's bookshelf has no download client and no
indexer today. Wiring them is provisioning work that depends on the
download-client and jackett providers, and on where ace's books and
downloads live (hq 153).

Verified: catalogue tests with MESH_CATALOGUE; a scratch resolution with
ace's assignment composes books.zurag.be and keeps the port to the
mesh; a throwaway container at the pinned digest on a fresh 0700
1000:1000 dir answered /ping and /api/v1/system/status (401 on a wrong
key); client.ts typechecks strict, found the key in config.xml and read
the queue.
2026-09-30 11:58:37 +02:00
409 changed files with 9536 additions and 50599 deletions
+22
View File
@@ -0,0 +1,22 @@
# anthropic-consumer's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/anthropic-consumer
COPY . .
RUN node /app/node_modules/typescript/bin/tsc apply/index.ts usage/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/anthropic-consumer/dist /app/modules/anthropic-consumer/dist
# No serve-time entrypoints: every container of this module names its command (`run` on a
# schedule), so nothing here serves — deliberately no MESH_TOOL_MODULES.
+67 -30
View File
@@ -9,20 +9,29 @@
"model-access" "model-access"
], ],
"binds": { "binds": {
"model-access": "${dir:state}/model.json" "model-access": "/var/lib/anthropic-consumer/model.json"
}, },
"secrets": { "secrets": {
"model-access": "${dir:state}/access-token" "model-access": "/var/lib/anthropic-consumer/access-token"
},
"own-secrets": {
"broker": "/var/lib/mesh/anthropic-consumer/broker"
}, },
"emits": [ "emits": [
"usage.session" "usage.session"
], ],
"resources": [ "resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/anthropic-consumer",
"mode": "0700"
},
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/anthropic-consumer",
"place": "." "mode": "0700"
}, },
{ {
"id": "claude-home", "id": "claude-home",
@@ -33,43 +42,71 @@
{ {
"id": "out", "id": "out",
"type": "directory", "type": "directory",
"path": "/var/lib/anthropic-consumer/out",
"mode": "0700" "mode": "0700"
}, },
{ {
"id": "apply", "id": "apply",
"type": "process", "type": "container",
"name": "anthropic-consumer-apply", "name": "mesh-anthropic-consumer-apply",
"artifact": "code", "network": "host",
"run": [
"node",
"apply/index.js"
],
"schedule": "*/5 * * * *", "schedule": "*/5 * * * *",
"args": [
"run",
"/app/modules/anthropic-consumer/dist/apply/index.js"
],
"volumes": [
"/var/lib/anthropic-consumer:/run/state"
],
"env": { "env": {
"MESH_MODEL_ACCESS_SECRET_FILE": "${dir:state}/access-token", "MESH_MODEL_ACCESS_SECRET_FILE": "/run/state/access-token",
"MESH_MODEL_ACCESS_BIND_FILE": "${dir:state}/model.json", "MESH_MODEL_ACCESS_BIND_FILE": "/run/state/model.json",
"MESH_CLAUDE_CREDENTIALS_FILE": "${dir:state}/claude/.credentials.json", "MESH_CLAUDE_CREDENTIALS_FILE": "/run/state/claude/.credentials.json",
"MESH_CLAUDE_IDENTITY_FILE": "${dir:state}/claude/.claude.json" "MESH_CLAUDE_IDENTITY_FILE": "/run/state/claude/.claude.json"
} },
"artifact": "runtime"
},
{
"id": "usage",
"type": "container",
"name": "mesh-anthropic-consumer-usage",
"network": "host",
"schedule": "*/5 * * * *",
"args": [
"run",
"/app/modules/anthropic-consumer/dist/usage/index.js"
],
"volumes": [
"/var/lib/mesh/anthropic-consumer/broker:/run/secrets/broker:ro",
"/var/lib/anthropic-consumer:/run/state"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_CLAUDE_PROJECTS_DIR": "/run/state/claude/projects",
"MESH_ANTHROPIC_USAGE_OUT": "/run/state/out/session-usage.json",
"MESH_TOOLS_MAIN": "/app/dist/main.js"
},
"artifact": "runtime"
} }
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"apply/index.js",
"usage/index.js"
],
"loads": [
"usage/index.js"
],
"env": {
"MESH_CLAUDE_PROJECTS_DIR": "${dir:state}/claude/projects",
"MESH_ANTHROPIC_USAGE_OUT": "${dir:state}/out/session-usage.json"
}
} }
] ]
} }
+18 -19
View File
@@ -3,15 +3,12 @@
// per session. The consumer IS the (node,module) session's fixed binding, so no per-message account // per session. The consumer IS the (node,module) session's fixed binding, so no per-message account
// attribution is done — just the totals (port map "don't-map" #3). // attribution is done — just the totals (port map "don't-map" #3).
// //
// Runs in the node's runtime (novox/hq ADR 0198), every five minutes, so events are emitted through // Runs as `mesh-tools run` (no broker), so events are emitted best-effort via the sibling mesh-tools
// the runtime as this module; the totals are also written to a file so the reading is observable // `emit` primitive; the totals are also written to a file so the reading is observable without one.
// without one.
import { readdirSync, statSync, readFileSync, writeFileSync, renameSync, mkdirSync } from "node:fs"; import { readdirSync, statSync, readFileSync, writeFileSync, renameSync, mkdirSync } from "node:fs";
import { join, dirname } from "node:path"; import { join, dirname } from "node:path";
import { emit } from "@novox/mesh-sdk/events";
import { readSessionFile, type SessionUsage } from "../transcript.js"; import { readSessionFile, type SessionUsage } from "../transcript.js";
/** The vendor-neutral usage row ADR 0054 fixes — the shape the model-usage store upserts. Kept local /** The vendor-neutral usage row ADR 0054 fixes — the shape the model-usage store upserts. Kept local
@@ -119,20 +116,22 @@ function atomicWrite(path: string, content: string): void {
renameSync(tmp, path); renameSync(tmp, path);
} }
/** Emit best-effort through the runtime: a reading that could not be announced is still in the file. */ /** Emit best-effort via the sibling mesh-tools `emit`, which wires a broker a run step has none. */
async function emitUsage(body: Record<string, unknown>): Promise<void> { async function emitUsage(body: Record<string, unknown>): Promise<void> {
try { const main = process.env.MESH_TOOLS_MAIN ?? "/app/dist/main.js";
await emit("usage.session", body); const { spawn } = await import("node:child_process");
} catch (err) { await new Promise<void>((resolve) => {
console.error(`[anthropic-consumer] could not emit usage: ${err}`); const child = spawn(
} process.execPath,
[main, "emit", "usage.session", JSON.stringify(body)],
{ stdio: "inherit" },
);
child.on("exit", () => resolve());
child.on("error", (err) => {
console.error(`[anthropic-consumer] could not emit usage: ${err}`);
resolve();
});
});
} }
// The cadence the scheduled container had: once at start, then every five minutes. Not awaited, so the await main();
// runtime's handshake is answered while a long first reading is still under way.
const EVERY_MS = 5 * 60 * 1000;
const tick = (): void => {
void main().catch((err) => console.error(`[anthropic-consumer] usage reading failed: ${err}`));
};
tick();
setInterval(tick, EVERY_MS);
+8 -8
View File
@@ -9,13 +9,13 @@
"model-access" "model-access"
], ],
"binds": { "binds": {
"model-access": "${dir:mesh-state}/model.json" "model-access": "/var/lib/mesh/anthropic-manager/model.json"
}, },
"secrets": { "secrets": {
"model-access": "${dir:mesh-state}/refresh-token" "model-access": "/var/lib/mesh/anthropic-manager/refresh-token"
}, },
"own-secrets": { "own-secrets": {
"broker": "${dir:mesh-state}/broker" "broker": "/var/lib/mesh/anthropic-manager/broker"
}, },
"emits": [ "emits": [
"usage.read" "usage.read"
@@ -24,13 +24,13 @@
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/anthropic-manager",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "out", "id": "out",
"type": "directory", "type": "directory",
"path": "${dir:mesh-state}/out", "path": "/var/lib/mesh/anthropic-manager/out",
"mode": "0700" "mode": "0700"
}, },
{ {
@@ -45,8 +45,8 @@
"/app/modules/anthropic-manager/dist/refresh/index.js" "/app/modules/anthropic-manager/dist/refresh/index.js"
], ],
"volumes": [ "volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro", "/var/lib/mesh/anthropic-manager/broker:/run/secrets/broker:ro",
"${dir:mesh-state}:/run/state" "/var/lib/mesh/anthropic-manager:/run/state"
], ],
"env": { "env": {
"MESH_BROKER_FILE": "/run/secrets/broker", "MESH_BROKER_FILE": "/run/secrets/broker",
+33
View File
@@ -0,0 +1,33 @@
# audit-logger's runtime: the shared runtime image, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The toolkit is in the base image, so
# nothing is copied out of a neighbouring checkout — which is what lets the mesh build this from a
# repository and a path (novox/hq ADR 0069) rather than only on a workstation that happens to have
# the siblings laid out beside it.
# Two bases, named rather than pinned: the image this is COMPILED in, and the image it RUNS in.
# They are different images on purpose — the first carries a compiler and the second must not, or
# every running container would carry one it never invokes. The mesh answers both with the copies it
# holds, because a fingerprint written here would name one particular copy and no other mesh has it
# (novox/hq issue 044). Declared in module.json's `build.on`; deliberately no defaults, so a build
# nobody told stops here and says which module to build first.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the toolkit it will run against.
WORKDIR /app/modules/audit-logger
COPY . .
# The compiler is invoked by its real path rather than through node_modules/.bin, whose entries are
# symlinks to a launcher that requires its library relatively — resolved away when the base image
# was assembled.
RUN node /app/node_modules/typescript/bin/tsc audit.ts index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/audit-logger/dist /app/modules/audit-logger/dist
# **Served, not run.** This subscribes on import, and the serve mode binds the broker before it
# imports anything — `run` exists for a step that works offline and exits, and would leave this
# with nothing to subscribe to.
ENV MESH_TOOL_MODULES=/app/modules/audit-logger/dist/index.js
+36 -14
View File
@@ -5,21 +5,27 @@
"consumes": [ "consumes": [
"**" "**"
], ],
"own-secrets": {
"broker": "/var/lib/audit-logger/broker"
},
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"index.js"
],
"loads": [
"index.js"
],
"env": {
"AUDIT_LOG": "${dir:trail}/audit.log"
}
} }
] ]
}, },
@@ -27,13 +33,29 @@
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/audit-logger",
"place": "." "mode": "0700"
}, },
{ {
"id": "trail", "id": "trail",
"type": "directory", "type": "directory",
"path": "/var/lib/audit-logger/trail",
"mode": "0700" "mode": "0700"
},
{
"id": "run",
"type": "container",
"name": "mesh-audit-logger",
"network": "host",
"volumes": [
"/var/lib/audit-logger/broker:/run/secrets/broker:ro",
"/var/lib/audit-logger/trail:/trail"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"AUDIT_LOG": "/trail/audit.log"
},
"artifact": "runtime"
} }
], ],
"capabilities": [ "capabilities": [
+2 -4
View File
@@ -15,7 +15,7 @@ test("audit-logger records every event to the trail as one line each", async ()
const path = join(dir, "audit.log"); const path = join(dir, "audit.log");
// The audit-logger's whole behaviour: consume everything, record it. // The audit-logger's whole behaviour: consume everything, record it.
await on("#", async (event) => record(event, path)); // the pattern index.ts subscribes await on("**", async (event) => record(event, path));
process.env.MESH_MODULE = "umami"; process.env.MESH_MODULE = "umami";
process.env.MESH_NODE = "anchor"; process.env.MESH_NODE = "anchor";
@@ -24,9 +24,7 @@ test("audit-logger records every event to the trail as one line each", async ()
const lines = (await readFile(path, "utf8")).trim().split("\n").map((l) => JSON.parse(l)); const lines = (await readFile(path, "utf8")).trim().split("\n").map((l) => JSON.parse(l));
assert.equal(lines.length, 2); assert.equal(lines.length, 2);
// A module names its events locally (design 29); the module is the `source`, which together with assert.deepEqual(lines.map((l) => l.type), ["umami.site.created", "node.anchor.joined"]);
// the type says whose event it was. This broker does no namespacing, so the type is as emitted.
assert.deepEqual(lines.map((l) => l.type), ["site.created", "node.anchor.joined"]);
assert.equal(lines[0].source, "umami"); assert.equal(lines[0].source, "umami");
assert.equal(lines[0].node, "anchor"); assert.equal(lines[0].node, "anchor");
assert.equal(lines[0].body.domain, "my-app"); assert.equal(lines[0].body.domain, "my-app");
+24
View File
@@ -0,0 +1,24 @@
# baserow's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/baserow
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/baserow/dist /app/modules/baserow/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/baserow/dist/tools/index.js
+24 -55
View File
@@ -2,15 +2,12 @@
// module's tools and anything else baserow-specific import it; nothing outside baserow does. // module's tools and anything else baserow-specific import it; nothing outside baserow does.
// //
// Baserow authenticates a person with email + password, exchanged for a JWT at /api/user/token-auth/. // Baserow authenticates a person with email + password, exchanged for a JWT at /api/user/token-auth/.
// The standard image creates no admin from env, so the account is one a person made in Baserow: its // Those credentials are the mesh's own: a person signs up in Baserow (the standard image creates no
// password is the module's `admin` secret, accepted from the operator, and its email and the public // admin from env), and the credential is placed in the runtime config file the mesh mounts. Until
// host Baserow answers to reach the runtime config file the mesh mounts (the email from the // that happens fromEnv throws and the module simply exposes no tools — the same dormant-until-
// assignment's settings). Until both are there fromEnv throws and the module exposes no tools — the // configured shape gitea uses for its token.
// same dormant-until-configured shape gitea uses for its token.
import { readFileSync } from "node:fs"; import { readFileSync } from "node:fs";
import { request as httpRequest } from "node:http";
import { request as httpsRequest } from "node:https";
export interface BaserowApplication { export interface BaserowApplication {
id: number; id: number;
@@ -71,63 +68,35 @@ export class BaserowClient {
return h; return h;
} }
/** /** Exchange email + password for a JWT, caching it for the client's lifetime. Handles both the
* One HTTP exchange. Not `fetch`: Node's fetch drops a caller's Host header and sends the URL's
* own, and Baserow answers only the host of its BASEROW_PUBLIC_URL — any other Host is looked up
* as a published builder site and gets 404, `/api/_health/` included. A co-located caller reaching
* it by container name must present the public host, so the request is made with node:http, which
* sends the Host it is given.
*/
private send(path: string, method: string, headers: Record<string, string>, body?: string): Promise<{ status: number; text: string }> {
const url = new URL(`${this.baseUrl}${path}`);
const request = url.protocol === "https:" ? httpsRequest : httpRequest;
// A length, never chunked: Baserow's server reads a chunked body as empty.
const sent = body === undefined ? headers : { ...headers, "Content-Length": String(Buffer.byteLength(body)) };
return new Promise((resolve, reject) => {
const req = request(url, { method, headers: sent }, (res) => {
let text = "";
res.setEncoding("utf8");
res.on("data", (chunk: string) => (text += chunk));
res.on("end", () => resolve({ status: res.statusCode ?? 0, text }));
res.on("error", reject);
});
req.on("error", reject);
if (body !== undefined) req.write(body);
req.end();
});
}
/** Exchange email + password for a JWT, caching it until Baserow refuses it. Handles both the
* older `{ token }` and the newer `{ access_token }` response shapes. */ * older `{ token }` and the newer `{ access_token }` response shapes. */
async authenticate(): Promise<string> { async authenticate(): Promise<string> {
if (this.token) return this.token; if (this.token) return this.token;
const res = await this.send( const res = await fetch(`${this.baseUrl}/api/user/token-auth/`, {
"/api/user/token-auth/", method: "POST",
"POST", headers: this.headers(),
this.headers(), body: JSON.stringify({ email: this.email, password: this.password }),
JSON.stringify({ email: this.email, password: this.password }), });
); if (!res.ok) throw new Error(`baserow auth failed: ${res.status} ${await res.text()}`);
if (res.status < 200 || res.status >= 300) throw new Error(`baserow auth failed: ${res.status} ${res.text}`); const data = (await res.json()) as { token?: string; access_token?: string };
const data = JSON.parse(res.text) as { token?: string; access_token?: string };
const token = data.access_token ?? data.token; const token = data.access_token ?? data.token;
if (!token) throw new Error("baserow auth returned no token"); if (!token) throw new Error("baserow auth returned no token");
this.token = token; this.token = token;
return token; return token;
} }
/** An authenticated GET. A refused token is dropped and the call made once more with a fresh one: private async authed<T>(path: string, options: RequestInit = {}): Promise<T> {
* Baserow's access tokens expire after minutes, and the runtime lives for weeks. */ const token = await this.authenticate();
private async authed<T>(path: string): Promise<T> { const res = await fetch(`${this.baseUrl}${path}`, {
for (let attempt = 0; ; attempt++) { ...options,
const token = await this.authenticate(); headers: this.headers({
const res = await this.send(path, "GET", this.headers({ Authorization: `JWT ${token}` })); Authorization: `JWT ${token}`,
if (res.status === 401 && attempt === 0) { ...(options.headers as Record<string, string> | undefined),
this.token = null; }),
continue; });
} if (!res.ok) throw new Error(`baserow ${path}: ${res.status} ${await res.text()}`);
if (res.status < 200 || res.status >= 300) throw new Error(`baserow ${path}: ${res.status} ${res.text}`); const text = await res.text();
return (res.text ? JSON.parse(res.text) : null) as T; return (text ? JSON.parse(text) : null) as T;
}
} }
/** The applications (databases) the account can see, across all its workspaces. */ /** The applications (databases) the account can see, across all its workspaces. */
+54 -31
View File
@@ -18,14 +18,15 @@
} }
}, },
"binds": { "binds": {
"postgres-database": "${dir:state}/database.json", "postgres-database": "/var/lib/baserow/database.json",
"route": "${dir:state}/route.json" "route": "/var/lib/baserow/route.json"
}, },
"secrets": { "secrets": {
"postgres-database": "${dir:state}/database.secret" "postgres-database": "/var/lib/baserow/database.secret"
}, },
"own-secrets": { "own-secrets": {
"admin": "${dir:state}/admin.secret" "secret-key": "/var/lib/baserow/secret-key.secret",
"broker": "/var/lib/mesh/baserow/broker"
}, },
"listens": [ "listens": [
{ {
@@ -33,34 +34,35 @@
"port": 80, "port": 80,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "the Baserow web UI and REST API, served by the image's own Caddy; a public name is the route's" "why": "the Baserow web UI and REST API; a public name is a route grant later"
} }
], ],
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/baserow",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/baserow",
"place": "." "mode": "0700"
}, },
{ {
"id": "data", "id": "data",
"type": "directory", "type": "directory",
"path": "/services/baserow/data",
"mode": "0755", "mode": "0755",
"owner": "9999:9999" "owner": "9999:9999"
}, },
{ {
"id": "server-env", "id": "server-env",
"type": "file", "type": "file",
"path": "${dir:state}/server.env", "path": "/var/lib/baserow/server.env",
"mode": "0600", "mode": "0600",
"content": "DATABASE_HOST=${bound:postgres-database:at}\nDATABASE_PORT=${bound:postgres-database:port}\nDATABASE_NAME=${bound:postgres-database:as}\nDATABASE_USER=${bound:postgres-database:as}\nDATABASE_PASSWORD_FILE=/run/secrets/database\nDISABLE_EMBEDDED_PSQL=true\nBASEROW_PUBLIC_URL=https://${bound:route:name}\n" "content": "DATABASE_HOST=${bound:postgres-database:at}\nDATABASE_PORT=${bound:postgres-database:port}\nDATABASE_NAME=${bound:postgres-database:as}\nDATABASE_USER=${bound:postgres-database:as}\nDATABASE_PASSWORD=${secret:postgres-database}\nSECRET_KEY=${secret:secret-key}\nBASEROW_PUBLIC_URL=http://localhost\n"
}, },
{ {
"id": "net", "id": "net",
@@ -71,44 +73,65 @@
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "baserow", "name": "baserow",
"image": "baserow/baserow@sha256:263ea6c4b72c9eccabcd975ffe9fdebf23913a293a514bec6a3897a5e0a5a080", "image": "baserow/baserow@sha256:834424a10413798567f76428f255dc259445b7f8dcec56598c05b4073bb2a124",
"network": "baserow", "network": "baserow",
"env-file": [ "env-file": [
"${dir:state}/server.env" "/var/lib/baserow/server.env"
], ],
"ports": [ "ports": [
"80" "80"
], ],
"volumes": [ "volumes": [
"${dir:data}:/baserow/data", "/services/baserow/data:/baserow/data"
"${dir:state}/database.secret:/run/secrets/database:ro" ],
] "secrets-in-environment": "baserow reads DATABASE_PASSWORD and SECRET_KEY with os.getenv and has no _FILE twin (settings/base.py); not convertible"
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/baserow/config.json",
"mode": "0600", "mode": "0600",
"content": "{\n \"password\": \"${secret:admin}\",\n \"host\": \"${bound:route:name}\"\n}\n", "content": "{}\n",
"merge": "json" "merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-baserow",
"network": "baserow",
"volumes": [
"/var/lib/mesh/baserow/broker:/run/secrets/broker:ro",
"/var/lib/mesh/baserow/config.json:/run/config/config.json:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BASEROW_URL": "http://baserow:80",
"MESH_BASEROW_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
} }
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "tools", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_BASEROW_URL": "http://127.0.0.1:${port:80}",
"MESH_BASEROW_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
} }
] ]
} }
+24
View File
@@ -0,0 +1,24 @@
# bazarr's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/bazarr
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/bazarr/dist /app/modules/bazarr/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/bazarr/dist/index.js,/app/modules/bazarr/dist/tools/index.js
+173
View File
@@ -0,0 +1,173 @@
// The Bazarr API client — bazarr's own code, living in the module (novox/hq ADR 0039). Bazarr
// manages subtitles for a Sonarr/Radarr library: it tracks which episodes and movies are still
// missing subtitles, searches providers for them, and records what it downloaded. This client
// talks its /api surface (keyed by an X-API-KEY header); bazarr's tools and events import it.
import { readFileSync } from "node:fs";
export interface WantedSubtitle {
kind: "episode" | "movie";
title: string; // series + episode, or movie title
path?: string;
seriesId?: number; // sonarr series id (episodes)
episodeId?: number; // sonarr episode id (episodes)
radarrId?: number; // radarr movie id (movies)
missing: string[]; // language names still missing
}
export interface ProviderSubtitle {
provider: string;
language: string;
hearingImpaired: boolean;
forced: boolean;
score?: number;
release?: string;
subtitle: string; // the opaque token Bazarr uses to download this exact result
}
export interface HistoryEntry {
kind: "episode" | "movie";
id: string; // stable dedup key across polls
title: string;
language?: string;
provider?: string;
path?: string;
timestamp?: string;
description?: string;
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> {
if (!file) return {};
try { return JSON.parse(readFileSync(file, "utf8")) as Record<string, string>; }
catch { return {}; }
}
/** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
* absent or unreadable yields undefined so callers fall back rather than crash. */
function readSecret(file?: string): string | undefined {
if (!file) return undefined;
try { return readFileSync(file, "utf8").trim(); }
catch { return undefined; }
}
export class BazarrClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/** Build from the module's resolved environment. Bazarr's API is keyed; without URL and key
* there is nothing to talk to, so this throws rather than run half-configured. */
static fromEnv(env: NodeJS.ProcessEnv = process.env): BazarrClient {
const cfg = meshConfig(env.MESH_BAZARR_CONFIG_FILE);
const url = cfg.url ?? env.MESH_BAZARR_URL;
const apiKey = cfg.apiKey ?? readSecret(env.MESH_BAZARR_API_KEY_FILE) ?? env.MESH_BAZARR_API_KEY;
if (!url) throw new Error("no Bazarr URL — set MESH_BAZARR_URL");
if (!apiKey) throw new Error("no Bazarr API key — set MESH_BAZARR_API_KEY");
return new BazarrClient(url, apiKey);
}
private async request(method: string, path: string, params: Record<string, string> = {}): Promise<any> {
const url = new URL(`${this.baseUrl}/api${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url.toString(), { method, headers: { "X-API-KEY": this.apiKey, Accept: "application/json" } });
if (!res.ok) throw new Error(`Bazarr API ${method} ${path}: ${res.status} ${await res.text()}`);
// Downloads/patches return an empty body; only GETs carry JSON.
const text = await res.text();
return text ? JSON.parse(text) : {};
}
private get(path: string, params?: Record<string, string>): Promise<any> {
return this.request("GET", path, params);
}
private languageNames(missing: any[]): string[] {
return (missing ?? []).map((m: any) => m?.name ?? m?.code2 ?? m?.code3).filter(Boolean);
}
/** Episodes and movies still missing subtitles — Bazarr's core "what's left to do" list. */
async getWanted(limit = 50): Promise<WantedSubtitle[]> {
const [eps, movies] = await Promise.all([
this.get("/episodes/wanted", { start: "0", length: String(limit) }),
this.get("/movies/wanted", { start: "0", length: String(limit) }),
]);
const episodes: WantedSubtitle[] = (eps?.data ?? []).map((e: any) => ({
kind: "episode" as const,
title: `${e.seriesTitle ?? e.series ?? "Unknown"} — ${e.episodeTitle ?? e.episode_title ?? ""}`.trim(),
path: e.path,
seriesId: e.sonarrSeriesId,
episodeId: e.sonarrEpisodeId,
missing: this.languageNames(e.missing_subtitles),
}));
const films: WantedSubtitle[] = (movies?.data ?? []).map((m: any) => ({
kind: "movie" as const,
title: m.title ?? "Unknown",
path: m.path,
radarrId: m.radarrId,
missing: this.languageNames(m.missing_subtitles),
}));
return [...episodes, ...films];
}
/** Ask providers what subtitles are available for one wanted episode — a manual search. */
async searchEpisode(episodeId: number): Promise<ProviderSubtitle[]> {
const raw = await this.get("/providers/episodes", { episodeid: String(episodeId) });
return this.mapProviderResults(raw);
}
/** Ask providers what subtitles are available for one movie — a manual search. */
async searchMovie(radarrId: number): Promise<ProviderSubtitle[]> {
const raw = await this.get("/providers/movies", { radarrid: String(radarrId) });
return this.mapProviderResults(raw);
}
private mapProviderResults(raw: any): ProviderSubtitle[] {
const list = Array.isArray(raw) ? raw : (raw?.data ?? []);
return list.map((r: any) => ({
provider: r.provider,
language: r.language?.name ?? r.language ?? "unknown",
hearingImpaired: Boolean(r.hearing_impaired ?? r.hi),
forced: Boolean(r.forced),
score: r.score,
release: r.release_info?.[0] ?? r.release_info,
subtitle: r.subtitle,
}));
}
/** Recent subtitle-download history, episodes and movies together, newest first. Each entry
* carries a stable id so the events poller can tell a fresh download from one already seen. */
async getHistory(limit = 40): Promise<HistoryEntry[]> {
const [eps, movies] = await Promise.all([
this.get("/episodes/history", { start: "0", length: String(limit) }),
this.get("/movies/history", { start: "0", length: String(limit) }),
]);
const key = (kind: string, r: any): string =>
`${kind}:${r.timestamp ?? r.parsed_timestamp ?? ""}:${r.subtitles_path ?? r.path ?? ""}:${r.language?.code3 ?? r.language ?? ""}`;
const episodes: HistoryEntry[] = (eps?.data ?? []).map((r: any) => ({
kind: "episode" as const,
id: key("episode", r),
title: `${r.seriesTitle ?? "Unknown"} — ${r.episodeTitle ?? ""}`.trim(),
language: r.language?.name ?? r.language,
provider: r.provider,
path: r.subtitles_path,
timestamp: r.timestamp,
description: r.description,
}));
const films: HistoryEntry[] = (movies?.data ?? []).map((r: any) => ({
kind: "movie" as const,
id: key("movie", r),
title: r.title ?? "Unknown",
language: r.language?.name ?? r.language,
provider: r.provider,
path: r.subtitles_path,
timestamp: r.timestamp,
description: r.description,
}));
return [...episodes, ...films];
}
}
+47
View File
@@ -0,0 +1,47 @@
// bazarr's events. The tool runtime imports this once the broker is bound. Bazarr's one genuinely
// observable thing is a subtitle arriving: it works away in the background, searching providers for
// the missing-subtitle list, and when it succeeds a subtitle appears in its history. That is worth
// announcing to the mesh.
//
// Emits (novox/hq ADR 0041/0042):
// module.bazarr.subtitle.downloaded — a subtitle was fetched for an episode or movie
//
// Bazarr has nothing on the mesh it usefully reacts to (a download completing is Sonarr/Radarr's
// business, and they trigger Bazarr directly), so it consumes nothing — a pure emitter.
//
// The event is observation-based: poll history and diff. Primed silently on the first look, or a
// restart would re-announce the whole recent history as freshly downloaded.
import { emit } from "@novox/mesh-sdk/events";
import { BazarrClient } from "./client.js";
const bazarr = BazarrClient.fromEnv();
const seen = new Set<string>();
let primed = false;
async function pollHistory(): Promise<void> {
const entries = await bazarr.getHistory(40);
for (const entry of entries) {
if (seen.has(entry.id)) continue;
if (primed) {
await emit("subtitle.downloaded", {
kind: entry.kind,
title: entry.title,
language: entry.language,
provider: entry.provider,
path: entry.path,
});
}
seen.add(entry.id);
}
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[bazarr] ${err}`));
setInterval(run, everyMs);
run();
};
tick(pollHistory, 60_000);
console.log("[bazarr] watching subtitle-download history");
+141
View File
@@ -0,0 +1,141 @@
{
"module": "bazarr",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"subtitle.downloaded"
],
"own-secrets": {
"broker": "/var/lib/mesh/bazarr/broker",
"api-key": "/var/lib/mesh/bazarr/api-key"
},
"listens": [
{
"name": "web",
"port": 6767,
"protocol": "tcp",
"from": "mesh",
"why": "managing subtitles"
}
],
"accesses": [
{
"path": "/services/media/movies",
"mode": "read-write"
},
{
"path": "/services/media/series",
"mode": "read-write"
},
{
"path": "/services/media/anime",
"mode": "read-write"
},
{
"path": "/services/media/downloads",
"mode": "read"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/bazarr",
"mode": "0700"
},
{
"id": "config",
"type": "directory",
"path": "/services/bazarr/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "bazarr",
"image": "lscr.io/linuxserver/bazarr@sha256:3a820372f19fcb2981ea19fe4b5382934d67414afaba974bce831ddda0a64a02",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"6767"
],
"volumes": [
"/services/bazarr/config:/config",
"/services/media/movies:/movies",
"/services/media/series:/series",
"/services/media/anime:/anime",
"/services/media/downloads:/downloads"
]
},
{
"id": "runtime-config",
"type": "file",
"path": "/var/lib/mesh/bazarr/config.json",
"mode": "0600",
"content": "{}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-bazarr",
"network": "host",
"volumes": [
"/var/lib/mesh/bazarr/broker:/run/secrets/broker:ro",
"/var/lib/mesh/bazarr/api-key:/run/secrets/api-key:ro",
"/var/lib/mesh/bazarr/config.json:/run/config/config.json:ro",
"/services/bazarr/config:/var/lib/bazarr/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BAZARR_URL": "http://127.0.0.1:6767",
"MESH_BAZARR_API_KEY_FILE": "/run/secrets/api-key",
"MESH_BAZARR_CONFIG_FILE": "/run/config/config.json",
"MESH_BAZARR_CONFIG_DIR": "/var/lib/bazarr/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "subs",
"endpoint": "web"
}
},
"binds": {
"route": "/var/lib/mesh/bazarr/route.json"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
+14
View File
@@ -0,0 +1,14 @@
{
"name": "@novox/module-bazarr",
"version": "0.1.0",
"description": "bazarr — subtitle management. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
+57
View File
@@ -0,0 +1,57 @@
// bazarr's tools — its own code (novox/hq ADR 0039), importing bazarr's client. They return
// structured data; the mesh serves them through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { BazarrClient } from "../client.js";
export function getBazarrTools(bazarr: BazarrClient): ToolDefinition[] {
return [
{
name: "bazarr_wanted",
description: "Episodes and movies still missing subtitles, with the languages each still needs.",
input: { limit: { type: "number", description: "max items per kind (default 50)" } },
run: async (args) => {
const wanted = await bazarr.getWanted(args.limit ? Number(args.limit) : 50);
return { count: wanted.length, wanted };
},
},
{
name: "bazarr_search_subtitles",
description: "Manually search subtitle providers for one wanted item — pass an episodeId or a radarrId.",
input: {
episodeId: { type: "number", description: "a Sonarr episode id (from bazarr_wanted)" },
radarrId: { type: "number", description: "a Radarr movie id (from bazarr_wanted)" },
},
run: async (args) => {
if (args.episodeId !== undefined) {
const results = await bazarr.searchEpisode(Number(args.episodeId));
return { kind: "episode", episodeId: Number(args.episodeId), count: results.length, results };
}
if (args.radarrId !== undefined) {
const results = await bazarr.searchMovie(Number(args.radarrId));
return { kind: "movie", radarrId: Number(args.radarrId), count: results.length, results };
}
throw new Error("pass either episodeId or radarrId");
},
},
{
name: "bazarr_history",
description: "Recent subtitle-download history — what was downloaded, for which title, from which provider.",
input: { limit: { type: "number", description: "max entries per kind (default 40)" } },
run: async (args) => {
const history = await bazarr.getHistory(args.limit ? Number(args.limit) : 40);
return { count: history.length, history };
},
},
];
}
// Exposed only when Bazarr is configured; otherwise bazarr contributes no tools rather than
// failing the whole runtime.
registerModuleTools("bazarr", (env) => {
try {
return getBazarrTools(BazarrClient.fromEnv(env));
} catch {
return [];
}
});
@@ -8,8 +8,5 @@
"skipLibCheck": true, "skipLibCheck": true,
"noEmit": true "noEmit": true
}, },
"include": [ "include": ["client.ts", "index.ts", "tools/index.ts"]
"tools/index.ts",
"client.ts"
]
} }
+27
View File
@@ -0,0 +1,27 @@
# bookshelf's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/bookshelf
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts downloads/settings.ts downloads/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/bookshelf/dist /app/modules/bookshelf/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/bookshelf/dist/index.js,/app/modules/bookshelf/dist/tools/index.js
# NOT dist/downloads/index.js: that is a step the host runs to completion, named by the `downloads`
# container's args as `mesh-tools run …` (novox/hq ADR 0052). Listed here it would run inside the
# serving sidecar too, and exit it.
+136
View File
@@ -0,0 +1,136 @@
// The Bookshelf API client — bookshelf's own code, living in the module (novox/hq ADR 0039).
// Ported from the shared hal `arr` client, but self-contained: in nox each Servarr app owns its own
// copy, so a change to Bookshelf's API rebuilds only bookshelf and nothing else. Both this module's
// tools and its events entrypoint import it, and nothing outside bookshelf does.
//
// Bookshelf is a Readarr fork (ghcr.io/pennydreadful/bookshelf). It speaks the Servarr v1 API; its
// content is "book". Unlike Sonarr/Radarr it exposes no calendar endpoint, so there is no calendar
// tool here — matching hal, which excluded bookshelf from its calendar-capable apps.
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
// Bookshelf speaks the v1 API; its content is "book".
const API_VERSION = "v1";
const CONTENT_ENDPOINT = "book";
const APP_NAME = "Bookshelf";
export interface BookshelfQueueItem {
/** The queue record id — stable while the item is in the queue, so events can diff on it. */
id: number;
title: string;
status: string;
size: string;
sizeleft: string;
timeleft?: string;
}
export interface BookshelfContentItem {
title: string;
author?: string;
year?: number;
status?: string;
monitored: boolean;
}
export class BookshelfClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/**
* Build from the module's resolved environment. The URL defaults to the server on this node (the
* runtime shares its network), and the API key is read from MESH_BOOKSHELF_API_KEY or, failing
* that, discovered from the server's own config.xml under MESH_BOOKSHELF_CONFIG_DIR — the same
* file Bookshelf writes it to, so a running server needs nothing configured by hand. Throws when
* no key can be found, so the tools/events simply do not load (the harness treats the throw as
* "exposes nothing").
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): BookshelfClient {
const url = env.MESH_BOOKSHELF_URL ?? `http://127.0.0.1:${env.MESH_BOOKSHELF_PORT ?? "8787"}`;
const configDir = env.MESH_BOOKSHELF_CONFIG_DIR ?? "/config";
const apiKey = env.MESH_BOOKSHELF_API_KEY ?? BookshelfClient.detectApiKey(configDir);
if (!apiKey) {
throw new Error("Bookshelf not configured — set MESH_BOOKSHELF_API_KEY or make the config dir readable");
}
return new BookshelfClient(url, apiKey);
}
/** Discover the API key from the server's config.xml, falling back to null. Every Servarr app
* writes <ApiKey> into config.xml at the root of its config directory. */
static detectApiKey(configDir: string): string | null {
const config = join(configDir, "config.xml");
if (existsSync(config)) {
const match = readFileSync(config, "utf8").match(/<ApiKey>([^<]+)<\/ApiKey>/);
if (match) return match[1];
}
return null;
}
private async get(endpoint: string, params?: Record<string, string>): Promise<unknown> {
const url = new URL(`${this.baseUrl}/api/${API_VERSION}/${endpoint}`);
if (params) {
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
}
const res = await fetch(url.toString(), { headers: { "X-Api-Key": this.apiKey } });
if (!res.ok) throw new Error(`${APP_NAME} API /${endpoint}: ${res.status} ${await res.text()}`);
return res.json();
}
async getStatus(): Promise<{ appName: string; version: string }> {
const data = (await this.get("system/status")) as { appName?: string; version?: string };
return { appName: data.appName || APP_NAME, version: data.version ?? "unknown" };
}
async getContent(limit?: number): Promise<BookshelfContentItem[]> {
const data = await this.get(CONTENT_ENDPOINT);
const items: any[] = Array.isArray(data) ? data : ((data as any)?.records ?? []);
const mapped = items.map((item) => ({
title: item.title ?? "Unknown",
author: item.author?.authorName ?? item.authorName,
year: item.releaseDate ? new Date(item.releaseDate).getFullYear() : item.year,
status: item.status,
monitored: item.monitored ?? true,
}));
return limit ? mapped.slice(0, limit) : mapped;
}
/** Library search is a filter over existing content, not an indexer lookup — same as hal's. */
async searchContent(term: string): Promise<BookshelfContentItem[]> {
const all = await this.getContent();
const lower = term.toLowerCase();
return all.filter(
(item) =>
item.title.toLowerCase().includes(lower) ||
(item.author?.toLowerCase().includes(lower) ?? false),
);
}
async getQueue(): Promise<{ totalRecords: number; items: BookshelfQueueItem[] }> {
const data = (await this.get("queue", { pageSize: "50" })) as { totalRecords?: number; records?: any[] };
const records = data.records ?? [];
return {
totalRecords: data.totalRecords ?? records.length,
items: records.map((r) => ({
id: r.id,
title: r.title ?? r.book?.title ?? r.author?.authorName ?? "Unknown",
status: r.status ?? "unknown",
size: formatBytes(r.size ?? 0),
sizeleft: formatBytes(r.sizeleft ?? 0),
timeleft: r.timeleft,
})),
};
}
}
function formatBytes(bytes: number): string {
if (bytes === 0) return "0 B";
const units = ["B", "KB", "MB", "GB", "TB"];
const i = Math.floor(Math.log(bytes) / Math.log(1024));
return `${(bytes / Math.pow(1024, i)).toFixed(1)} ${units[i]}`;
}
+188
View File
@@ -0,0 +1,188 @@
// The downloads step — run once by the host after the app's server starts, and again whenever a
// binding, a pair credential or the step's settings change (the container's `restart-on`,
// novox/hq ADR 0099). Byte-identical in sonarr, radarr, lidarr and bookshelf; see settings.ts.
//
// **A step, not a loop**: everything it does is a function of files the mesh writes, and the host
// already knows when they change. It connects to no broker.
//
// Exits non-zero when anything could not be put right — a refused credential, an unreachable
// provider, an entry the app would not save — so the node reports the step failed and the host
// runs it again on the next apply. Declared last in the manifest, so its failing gates nothing
// else of the module's (novox/hq ADR 0136).
//
// Reads, in MESH_DOWNLOADS_DIR, `<provision>.json` (the binding), `<provision>.secret` (the pair
// credential) and the settings file; remembers in MESH_DOWNLOADS_MEMORY the jackett hosts it has
// pointed feeds at, so a jackett that moves takes its feeds with it. Never prints a credential.
import { mkdir, rename, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import {
CLIENTS,
JACKETT,
acceptRemedy,
appConfig,
appReady,
listEntries,
providerTakes,
readIfThere,
readJson,
reconcileClient,
reconcileIndexers,
scrub,
stepSettings,
wanted,
type App,
type Binding,
type Http,
type Entry,
type Outcome,
type Took,
} from "./settings.js";
const module = process.env.MESH_DOWNLOADS_APP ?? "app";
const dir = process.env.MESH_DOWNLOADS_DIR ?? "/run/downloads";
const settingsFile = process.env.MESH_DOWNLOADS_SETTINGS ?? join(dir, "downloads.json");
const memoryFile = process.env.MESH_DOWNLOADS_MEMORY ?? "/var/lib/downloads/memory.json";
const waitSeconds = Number(process.env.MESH_DOWNLOADS_WAIT_SECONDS ?? "180");
const tag = `[${module}-downloads]`;
const http: Http = { fetch: (u, init) => fetch(u, init) };
const secrets: string[] = [];
const say = (line: string) => console.log(scrub(`${tag} ${line}`, secrets));
const fail = (line: string) => console.error(scrub(`${tag} ${line}`, secrets));
const config = appConfig((await readIfThere(process.env.MESH_DOWNLOADS_APP_CONFIG)) ?? "");
if (!config.apiKey) {
fail(`no API key in ${module}'s config.xml yet — ${module} writes it on its first start; the step runs again on the next apply`);
process.exit(1);
}
secrets.push(config.apiKey);
const app: App = {
module,
url: `${(process.env.MESH_DOWNLOADS_APP_URL ?? "http://127.0.0.1").replace(/\/$/, "")}${config.urlBase}`,
apiKey: config.apiKey,
api: process.env.MESH_DOWNLOADS_API ?? "v3",
};
const settings = stepSettings(await readJson(settingsFile));
if (!(await appReady(http, app, waitSeconds * 1000))) {
fail(`${module} did not answer at ${app.url} within ${waitSeconds}s`);
process.exit(1);
}
const outcomes: Outcome[] = [];
// The download clients.
let clients: Entry[];
try {
clients = await listEntries(http, app, "downloadclient");
} catch (err) {
fail(err instanceof Error ? err.message : String(err));
process.exit(1);
}
for (const kind of CLIENTS) {
const credential = await readIfThere(join(dir, `${kind.provision}.secret`));
if (credential) secrets.push(credential.trim());
const w = wanted(kind.provision, (await readJson(join(dir, `${kind.provision}.json`))) as Binding | undefined, credential, true);
// `in`, not `!w.ok`: the Dockerfile compiles without strict, where a boolean discriminant does not
// narrow.
if ("problem" in w) {
outcomes.push({ what: kind.provision, result: "refused", problem: w.problem });
continue;
}
const ep = w.endpoint;
try {
if (!(await providerTakes(http, kind.provision, ep)).took) {
outcomes.push({
what: kind.provision,
result: "refused",
problem: acceptRemedy(app, settings.node, kind.provision, kind.provider, kind.secretName, ep.from),
});
continue;
}
} catch (err) {
outcomes.push({
what: kind.provision,
result: "refused",
problem: `${kind.provider} could not be asked whether it takes the credential at ${ep.host}:${ep.port}: ${err instanceof Error ? err.message : String(err)}`,
});
continue;
}
outcomes.push(...(await reconcileClient(http, app, kind, settings.names[kind.provision], ep, clients)));
}
// The indexers, through jackett.
{
const credential = await readIfThere(join(dir, `${JACKETT}.secret`));
if (credential) secrets.push(credential.trim());
const w = wanted(JACKETT, (await readJson(join(dir, `${JACKETT}.json`))) as Binding | undefined, credential, false);
if ("problem" in w) {
outcomes.push({ what: JACKETT, result: "refused", problem: w.problem });
} else {
const ep = w.endpoint;
let took: Took | undefined;
try {
took = await providerTakes(http, JACKETT, ep);
} catch (err) {
outcomes.push({
what: JACKETT,
result: "refused",
problem: `jackett could not be asked whether it takes the key at ${ep.host}:${ep.port}: ${err instanceof Error ? err.message : String(err)}`,
});
}
if (took && !took.took) {
outcomes.push({ what: JACKETT, result: "refused", problem: acceptRemedy(app, settings.node, JACKETT, "jackett", "API key", ep.from) });
} else if (took) {
const memory = ((await readJson(memoryFile)) ?? {}) as Record<string, { hosts?: string[] } | undefined>;
const remembered = Array.isArray(memory[JACKETT]?.hosts) ? (memory[JACKETT]?.hosts as string[]) : [];
try {
const indexers = await listEntries(http, app, "indexer");
outcomes.push(
...(await reconcileIndexers(http, app, ep, took.configured ?? new Map(), settings.indexers, remembered, indexers)),
);
} catch (err) {
outcomes.push({ what: JACKETT, result: "refused", problem: err instanceof Error ? err.message : String(err) });
}
// Remember where the feeds now point, so they are still recognised as the mesh's if jackett
// moves. Written whole and renamed, so a crash leaves the old memory, never half of one.
const hosts = [...new Set([...remembered, ep.host.toLowerCase()])].sort();
if (hosts.join() !== [...remembered].sort().join()) {
try {
await mkdir(dirname(memoryFile), { recursive: true });
await writeFile(`${memoryFile}.tmp`, JSON.stringify({ ...memory, [JACKETT]: { hosts } }, null, 2) + "\n", { mode: 0o600 });
await rename(`${memoryFile}.tmp`, memoryFile);
} catch (err) {
outcomes.push({
what: JACKETT,
result: "notice",
note: `could not remember ${ep.host} as a jackett host (${err instanceof Error ? err.message : String(err)}); if jackett moves, list it in downloads.jackett-api.adopt-hosts`,
});
}
}
}
}
}
let failed = 0;
for (const o of outcomes) {
switch (o.result) {
case "unchanged":
say(`${o.what}: already as the mesh says${o.untested ? "" : "; connection tested"}`);
break;
case "written":
say(`${o.what}: wrote ${o.fields.join(", ")}${o.untested ? "" : "; connection tested"}`);
break;
case "created":
say(`${o.what}: registered; connection tested`);
break;
case "notice":
say(`${o.what}: ${o.note}`);
break;
case "refused":
failed++;
fail(`${o.what}: ${o.problem}`);
break;
}
}
process.exitCode = failed > 0 ? 1 : 0;
+778
View File
@@ -0,0 +1,778 @@
// Where a Servarr app reaches its download clients and its indexer proxy — decided by the mesh,
// written into the app by the app's own API.
//
// **One file, four copies.** sonarr, radarr, lidarr and bookshelf (a Readarr fork) keep their
// download clients and indexers behind the same API — `/api/v3/…` for the first two, `/api/v1/…`
// for the others — so this step is the same code in each. Each module builds from its own
// directory (novox/hq ADR 0069), so each carries a byte-identical copy under `downloads/`;
// `test/downloads.test.ts` checks the copies agree wherever the siblings are checked out beside
// it. Change all four together.
//
// **Why this exists.** The app keeps its connection to nzbget, qBittorrent and jackett in its own
// database, not in a file, so the mesh has nowhere to write `${bound:nzbget-api:at}` for it. The
// module requires `nzbget-api`, `qbittorrent-api` and `jackett-api`; the mesh delivers, for each,
// a binding (where the provider is — `at` — and what it serves: `port`, `scheme`, `url-base`, and
// for a download client the `username`) and a pair credential (nzbget's ControlPassword,
// qBittorrent's WebUI password, jackett's API key — each the provider's one and only, accepted by
// the operator per pair; the mesh cannot mint them). This step reads those files and makes the
// app's entries say the same thing.
//
// **Which entries are the mesh's.** Never a guess from what an entry looks like:
// - a download client is the mesh's when its name is the one the module's settings give for
// that provision (`downloads.<provision>.name`) and its kind is that provider's. On a fresh
// machine the step registers it under that name; on a migrated one the assignment names the
// entry the migration adopts ("NZBGet" on ace).
// - a Torznab indexer is the mesh's when it reads a jackett feed
// (`…/api/v2.0/indexers/<id>/results/torznab`) on a host the mesh put there — the bound `at`,
// one it bound before (remembered by the step), or one the settings adopt
// (`downloads.jackett-api.adopt-hosts`, which is how a migration names the entries that pointed
// at the old container). `downloads.jackett-api.indexers` lists the jackett indexers the app
// should have; one missing is registered.
// Everything else — every entry a person made, an nzbget elsewhere, a seedbox's jackett, a Newznab
// indexer — is left exactly as it is. **Nothing is ever deleted.**
//
// **Only the connection, and only when it differs.** Host, port, TLS, base path, user name and
// the credential. Categories, priorities, "enabled", seed criteria and every other choice made in
// the app are left alone. The stored credential cannot be read back (the app masks it), so the
// step asks the app to test the entry with the mesh's host and port and the credential it already
// holds: passing, the credential is already the provider's one and only; failing, the delivered
// credential is written.
//
// **A credential the provider refuses is never written.** Until the operator accepts the
// provider's secret for this pair, the mesh delivers a value it minted itself, which no provider
// will ever accept (novox/hq ADR 0092). Writing it would replace a working password with a dead
// one. So it is tried against the provider first; refused, nothing of that provision is written
// and the step fails naming the `secret accept` that fixes it.
//
// Pure logic and a small HTTP seam, so it is tested against fake servers (test/downloads.test.ts).
// Never prints a credential, an API key, or a URL carrying one.
import { readFile } from "node:fs/promises";
/** A download-client provision and the shape of its entry in the app. */
export interface ClientKind {
provision: "nzbget-api" | "qbittorrent-api";
/** The app's `implementation` for it. */
implementation: "Nzbget" | "QBittorrent";
/** The provider, as a person calls it. */
provider: string;
/** What the pair credential is, in the provider's words. */
secretName: string;
}
export const CLIENTS: readonly ClientKind[] = [
{ provision: "nzbget-api", implementation: "Nzbget", provider: "nzbget", secretName: "ControlPassword" },
{ provision: "qbittorrent-api", implementation: "QBittorrent", provider: "qBittorrent", secretName: "WebUI password" },
];
export const JACKETT = "jackett-api";
/** What the mesh wrote at `binds.<provision>`: the binding document. */
export interface Binding {
provision?: string;
from?: string;
at?: string;
as?: string;
serves?: Record<string, unknown>;
}
/** Where a provider is, as the mesh bound it, and the credential for it. */
export interface Endpoint {
scheme: "http" | "https";
host: string;
port: number;
/** "" at the root, otherwise "/base" — one leading slash, none trailing. */
urlBase: string;
/** The user a download client logs in as; "" for jackett, which takes a key. */
username: string;
credential: string;
/** The provider's node, for the `secret accept` remedy. */
from: string;
}
export type Wanted = { ok: true; endpoint: Endpoint } | { ok: false; problem: string };
/**
* The endpoint the mesh says to use, from a binding and its pair credential.
*
* Refused rather than guessed when the binding cannot be dialled from the app's own container: a
* loopback `at` — what the mesh hands a machine that is not on the private network — is the app's
* container itself.
*/
export function wanted(
provision: string,
binding: Binding | undefined,
credential: string | undefined,
needsUser: boolean,
): Wanted {
if (!binding) {
return { ok: false, problem: `no binding for ${provision} was delivered — the mesh writes it before this step runs` };
}
const host = typeof binding.at === "string" ? binding.at.trim() : "";
const serves = binding.serves ?? {};
const port = Number(serves.port);
if (!host) return { ok: false, problem: `the ${provision} binding names no host (at)` };
if (isLoopback(host)) {
return {
ok: false,
problem:
`the ${provision} binding says the provider is at ${host}, which from the app's own container is the ` +
`app itself. The mesh hands loopback to a machine that is not on the private network; put it on the ` +
`private network so the provider has an address the app can dial`,
};
}
if (!Number.isInteger(port) || port <= 0 || port > 65535) {
return { ok: false, problem: `the ${provision} binding serves no usable port (${String(serves.port)})` };
}
const scheme = typeof serves.scheme === "string" && serves.scheme ? serves.scheme : "http";
if (scheme !== "http" && scheme !== "https") {
return { ok: false, problem: `the ${provision} binding serves scheme ${scheme}, which the app cannot dial` };
}
const username = typeof serves.username === "string" ? serves.username.trim() : "";
if (needsUser && !username) {
return { ok: false, problem: `the ${provision} binding serves no username for the app to log in as` };
}
const key = (credential ?? "").trim();
if (!key) return { ok: false, problem: `the ${provision} credential is empty or was not delivered` };
return {
ok: true,
endpoint: {
scheme,
host,
port,
urlBase: normBase(serves["url-base"]),
username,
credential: key,
from: typeof binding.from === "string" ? binding.from : "",
},
};
}
/** A URL base as "" or "/x/y": slashes trimmed, one put back in front. */
export function normBase(v: unknown): string {
const s = typeof v === "string" ? v.trim().replace(/^\/+|\/+$/g, "") : "";
return s === "" ? "" : `/${s}`;
}
function isLoopback(host: string): boolean {
const h = host.toLowerCase();
return h === "localhost" || h === "::1" || h === "[::1]" || /^127\./.test(h);
}
function hostForUrl(host: string): string {
return host.includes(":") && !host.startsWith("[") ? `[${host}]` : host;
}
/** The provider's base URL, as the step dials it and as the app is given it. */
export function baseUrl(ep: Endpoint): string {
return `${ep.scheme}://${hostForUrl(ep.host)}:${ep.port}${ep.urlBase}`;
}
// ---------------------------------------------------------------------------------------------
// HTTP
/** The HTTP the step needs, so a test can stand fakes in for the app and the providers. */
export interface Http {
fetch(
url: string,
init?: { method?: string; headers?: Record<string, string>; body?: string },
): Promise<{ status: number; text(): Promise<string> }>;
}
/** The app, as the step reaches it from the host. */
export interface App {
/** The module, for messages and the remedy: sonarr, radarr, lidarr, bookshelf. */
module: string;
/** Its own URL, base path included. */
url: string;
apiKey: string;
/** v3 (sonarr, radarr) or v1 (lidarr, bookshelf). */
api: string;
}
interface Answer {
status: number;
body: unknown;
}
async function appCall(http: Http, app: App, method: string, path: string, body?: unknown): Promise<Answer> {
const res = await http.fetch(`${app.url.replace(/\/$/, "")}/api/${app.api}${path}`, {
method,
headers: {
"X-Api-Key": app.apiKey,
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
const text = await res.text();
let parsed: unknown = undefined;
if (text) {
try {
parsed = JSON.parse(text) as unknown;
} catch {
parsed = text;
}
}
return { status: res.status, body: parsed };
}
async function appGet(http: Http, app: App, path: string): Promise<unknown> {
const a = await appCall(http, app, "GET", path);
if (a.status < 200 || a.status >= 300) throw new Error(`${app.module} GET ${path} answered ${a.status}`);
return a.body;
}
/** One of the app's validation failures: the only parts of an answer the step ever prints. */
interface Failure {
property: string;
message: string;
warning: boolean;
}
function failuresOf(body: unknown): Failure[] {
const list = Array.isArray(body) ? body : [];
return list.map((f) => {
const o = (f ?? {}) as Record<string, unknown>;
return {
property: String(o.propertyName ?? ""),
message: String(o.errorMessage ?? ""),
warning: o.isWarning === true,
};
});
}
/**
* One shape rather than a union on `ok`: the Dockerfile compiles without strict, where a boolean
* discriminant does not narrow.
*/
interface Verdict {
ok: boolean;
/** Warnings the app raised on a pass; errors (and any warnings) on a failure. */
failures: Failure[];
status: number;
saved?: Entry;
}
function verdict(a: Answer): Verdict {
if (a.status >= 200 && a.status < 300) return { ok: true, failures: [], status: a.status };
const failures = failuresOf(a.body);
if (a.status === 400 && failures.length > 0 && failures.every((f) => f.warning)) return { ok: true, failures, status: a.status };
return { ok: false, failures, status: a.status };
}
/** The app's own test of an entry, from the app's own container. */
async function appTest(http: Http, app: App, resource: string, entry: Entry): Promise<Verdict> {
return verdict(await appCall(http, app, "POST", `/${resource}/test`, entry));
}
/**
* Save an entry. The app tests an enabled entry before saving it; a test that passes with only
* warnings ("a category is recommended") is saved with `forceSave`, the same as pressing "save
* anyway" in its screen. An error is never forced.
*/
async function appSave(http: Http, app: App, resource: string, entry: Entry, create: boolean): Promise<Verdict> {
const path = create ? `/${resource}` : `/${resource}/${entry.id}`;
const method = create ? "POST" : "PUT";
let a = await appCall(http, app, method, path, entry);
let v = verdict(a);
if (v.ok && (a.status < 200 || a.status >= 300)) {
a = await appCall(http, app, method, `${path}?forceSave=true`, entry);
v = verdict(a);
if (v.ok && (a.status < 200 || a.status >= 300)) {
return { ok: false, failures: failuresOf(a.body), status: a.status };
}
}
if (!v.ok) return v;
return { ...v, saved: (a.body ?? undefined) as Entry | undefined };
}
// ---------------------------------------------------------------------------------------------
// Entries
/** An entry as the app's API gives it: a download client or an indexer. */
export interface Entry {
id?: number;
name?: string;
implementation?: string;
enable?: boolean;
enableRss?: boolean;
enableAutomaticSearch?: boolean;
enableInteractiveSearch?: boolean;
supportsRss?: boolean;
supportsSearch?: boolean;
fields?: { name: string; value?: unknown; [k: string]: unknown }[];
[k: string]: unknown;
}
export function field(e: Entry, name: string): unknown {
return e.fields?.find((f) => f.name === name)?.value;
}
/** The entry with these fields' values replaced — added when the app's shape lacks one. */
export function withFields(e: Entry, values: Record<string, unknown>): Entry {
const fields = (e.fields ?? []).map((f) => (f.name in values ? { ...f, value: values[f.name] } : { ...f }));
for (const [name, value] of Object.entries(values)) {
if (!fields.some((f) => f.name === name)) fields.push({ name, value });
}
return { ...e, fields };
}
function enabled(e: Entry): boolean {
return e.enable === true || e.enableRss === true || e.enableAutomaticSearch === true || e.enableInteractiveSearch === true;
}
/** What one provision, or one entry of it, came to. */
export type Outcome =
| { what: string; result: "unchanged"; untested?: boolean }
| { what: string; result: "written"; fields: string[]; untested?: boolean }
| { what: string; result: "created"; fields: string[] }
| { what: string; result: "notice"; note: string }
| { what: string; result: "refused"; problem: string };
function said(fs: Failure[]): string {
return fs.length === 0 ? "no reason given" : fs.map((f) => (f.property ? `${f.property}: ${f.message}` : f.message)).join("; ");
}
/**
* Bring one of the mesh's entries in line: `connection` is the non-secret fields as the mesh says
* them, `differs` which of them the entry does not already say, `secretField` where the credential
* goes. Never throws: every failure is an outcome with a reason.
*/
export async function reconcileEntry(
http: Http,
app: App,
resource: "downloadclient" | "indexer",
what: string,
current: Entry,
connection: Record<string, unknown>,
differs: string[],
secretField: string,
credential: string,
testable = true,
): Promise<Outcome> {
try {
const base = withFields(current, connection); // the credential as the app holds it — masked
if (!testable) {
// Nothing to test against (jackett no longer has the indexer it names): the connection is
// still the mesh's to state, and the key goes with it when the address changes.
if (differs.length === 0) return { what, result: "unchanged", untested: true };
const s = await appSave(http, app, resource, withFields(base, { [secretField]: credential }), false);
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not save it: ${said(s.failures)}` };
return { what, result: "written", fields: [...differs, secretField], untested: true };
}
const first = await appTest(http, app, resource, base);
if (first.ok) {
// The credential the app holds already works against the provider at the mesh's address —
// and the provider has exactly one, so it is the delivered one.
if (differs.length === 0) return { what, result: "unchanged" };
const s = await appSave(http, app, resource, base, false);
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not save it: ${said(s.failures)}` };
return { what, result: "written", fields: differs };
}
// The app's test failed with what it holds. The delivered credential was already checked
// against the provider, so writing it is safe; the app tests an enabled entry again on save.
const s = await appSave(http, app, resource, withFields(base, { [secretField]: credential }), false);
if (!s.ok) {
return {
what,
result: "refused",
problem: `${app.module} tested it at the mesh's address and would not save it (nothing was written): ${said(s.failures)}`,
};
}
const stored = (await appGet(http, app, `/${resource}/${current.id}`)) as Entry;
const second = await appTest(http, app, resource, stored);
const fields = [...differs, secretField];
if (second.ok) return { what, result: "written", fields };
const why = `written (${fields.join(", ")}), and ${app.module}'s own test still fails: ${said(second.failures)}`;
return enabled(stored) ? { what, result: "refused", problem: why } : { what, result: "notice", note: `${why} — it is disabled, so nothing uses it` };
} catch (err) {
return { what, result: "refused", problem: message(err) };
}
}
/**
* Register a new entry from the app's own template for that kind: its defaults, the name, the
* connection and the credential. A category the provider does not have (nzbget refuses one it was
* not told about) is left empty rather than failing the registration, and said.
*/
export async function createEntry(
http: Http,
app: App,
resource: "downloadclient" | "indexer",
what: string,
implementation: string,
name: string,
values: Record<string, unknown>,
): Promise<Outcome> {
try {
const schema = (await appGet(http, app, `/${resource}/schema`)) as Entry[];
const template = (schema ?? []).find((s) => s.implementation === implementation);
if (!template) return { what, result: "refused", problem: `${app.module} has no ${implementation} to register` };
let entry: Entry = withFields({ ...template, id: undefined, name }, values);
if (resource === "downloadclient") entry.enable = true;
else {
entry.enableRss = template.supportsRss !== false;
entry.enableAutomaticSearch = template.supportsSearch !== false;
entry.enableInteractiveSearch = template.supportsSearch !== false;
}
let s = await appSave(http, app, resource, entry, true);
let note = "";
if (!s.ok && s.failures.length > 0 && s.failures.every((f) => /category/i.test(f.property))) {
const emptied: Record<string, unknown> = {};
for (const f of entry.fields ?? []) if (/category$/i.test(f.name) && !/imported/i.test(f.name)) emptied[f.name] = "";
entry = withFields(entry, emptied);
s = await appSave(http, app, resource, entry, true);
note = " with its category left empty — the provider has none by the app's default name; set one in both";
}
if (!s.ok) return { what, result: "refused", problem: `${app.module} would not register it: ${said(s.failures)}` };
const created = s.saved;
if (created?.id !== undefined) {
const t = await appTest(http, app, resource, (await appGet(http, app, `/${resource}/${created.id}`)) as Entry);
if (!t.ok) {
return { what, result: "refused", problem: `registered as "${name}"${note}, and ${app.module}'s own test fails: ${said(t.failures)}` };
}
}
return note
? { what, result: "notice", note: `registered as "${name}"${note}` }
: { what, result: "created", fields: Object.keys(values) };
} catch (err) {
return { what, result: "refused", problem: message(err) };
}
}
// ---------------------------------------------------------------------------------------------
// Providers: does it take the credential?
/**
* Does the provider take this credential? `took` it does (for jackett, with the ids of the indexers
* it has configured); `refused` it said no; a thrown error when it could not be asked.
*/
export interface Took {
took: boolean;
configured?: Map<string, string>;
}
export async function providerTakes(http: Http, provision: string, ep: Endpoint): Promise<Took> {
const base = baseUrl(ep);
if (provision === "nzbget-api") {
const auth = Buffer.from(`${ep.username}:${ep.credential}`).toString("base64");
const res = await http.fetch(`${base}/jsonrpc/version`, { method: "GET", headers: { Authorization: `Basic ${auth}` } });
await res.text();
if (res.status === 401 || res.status === 403) return { took: false };
if (res.status >= 200 && res.status < 300) return { took: true };
throw new Error(`nzbget answered ${res.status}`);
}
if (provision === "qbittorrent-api") {
const form = `username=${encodeURIComponent(ep.username)}&password=${encodeURIComponent(ep.credential)}`;
const res = await http.fetch(`${base}/api/v2/auth/login`, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: form,
});
const text = (await res.text()).trim();
if (res.status === 401 || /^fails\.?$/i.test(text)) return { took: false };
if (res.status === 403) {
throw new Error(
"qBittorrent answered 403: it has banned this address after failed logins (WebUI\\BanDuration); it lifts by itself",
);
}
if (res.status >= 200 && res.status < 300) return { took: true };
throw new Error(`qBittorrent answered ${res.status}`);
}
if (provision === JACKETT) {
const url = `${base}/api/v2.0/indexers/all/results/torznab/api?t=indexers&configured=true&apikey=${encodeURIComponent(ep.credential)}`;
const res = await http.fetch(url, { method: "GET" });
const text = await res.text();
if (res.status === 401 || res.status === 403) return { took: false };
if (res.status < 200 || res.status >= 300) throw new Error(`jackett answered ${res.status}`);
// jackett answers a wrong key 200 with an error document: code 100, "Invalid API Key".
if (/<error\b[^>]*\bcode="100"/i.test(text)) return { took: false };
if (/<error\b/i.test(text)) throw new Error("jackett answered with an error document");
return { took: true, configured: configuredIndexers(text) };
}
throw new Error(`no check for ${provision}`);
}
/** jackett's configured indexers, id → title, from its `t=indexers` feed. */
export function configuredIndexers(xml: string): Map<string, string> {
const out = new Map<string, string>();
const re = /<indexer\b[^>]*\bid="([^"]+)"[^>]*>([\s\S]*?)<\/indexer>/g;
for (let m = re.exec(xml); m; m = re.exec(xml)) {
const title = /<title>([\s\S]*?)<\/title>/.exec(m[2])?.[1]?.trim() ?? m[1];
out.set(m[1], decodeXml(title));
}
return out;
}
function decodeXml(s: string): string {
return s.replace(/&lt;/g, "<").replace(/&gt;/g, ">").replace(/&quot;/g, '"').replace(/&apos;/g, "'").replace(/&amp;/g, "&");
}
/** The remedy for a refused credential, in the controller's own words (ADR 0092). */
export function acceptRemedy(app: App, node: string, provision: string, provider: string, secretName: string, from: string): string {
return (
`${provider} refuses the ${provision} credential the mesh delivered, so nothing of ${provision} was written ` +
`into ${app.module}. ${provider} has one ${secretName} and the mesh cannot make it: accept it for this pair — ` +
`\`secret accept ${node || "<this node>"} ${app.module} ${provision} --provider ${from || "<its node>"} ` +
`--from <file holding ${provider}'s ${secretName}>\``
);
}
// ---------------------------------------------------------------------------------------------
// Download clients
/** The non-secret fields of a download client as the mesh says them. */
export function clientConnection(ep: Endpoint): Record<string, unknown> {
return { host: ep.host, port: ep.port, useSsl: ep.scheme === "https", urlBase: ep.urlBase === "" ? null : ep.urlBase, username: ep.username };
}
/** Which of them the entry does not already say. Names only. */
export function clientDiffers(e: Entry, ep: Endpoint): string[] {
const out: string[] = [];
if (String(field(e, "host") ?? "").toLowerCase() !== ep.host.toLowerCase()) out.push("host");
if (Number(field(e, "port") ?? 0) !== ep.port) out.push("port");
if (Boolean(field(e, "useSsl")) !== (ep.scheme === "https")) out.push("useSsl");
if (normBase(field(e, "urlBase")) !== ep.urlBase) out.push("urlBase");
if (String(field(e, "username") ?? "") !== ep.username) out.push("username");
return out;
}
/**
* Bring the app's download client for one provision in line: the entry by that name and kind is
* the mesh's and is updated; none, it is registered. An entry by that name of another kind, or by
* that name in another case, is somebody else's and is refused rather than touched.
*/
export async function reconcileClient(
http: Http,
app: App,
kind: ClientKind,
name: string,
ep: Endpoint,
entries: Entry[],
): Promise<Outcome[]> {
const what = `${kind.provision} ("${name}")`;
const exact = entries.filter((e) => e.name === name);
const alike = entries.filter((e) => e.name !== name && String(e.name ?? "").toLowerCase() === name.toLowerCase());
if (exact.length > 0 && exact[0].implementation !== kind.implementation) {
return [{
what,
result: "refused",
problem: `the download client named "${name}" is a ${exact[0].implementation}, not ${kind.implementation}; it is not the mesh's and was left alone. Name the mesh's entry otherwise in downloads.${kind.provision}.name`,
}];
}
if (exact.length === 0 && alike.length > 0) {
return [{
what,
result: "refused",
problem: `${app.module} already has "${alike[0].name}", which is not the mesh's name ("${name}") and was left alone. To have the mesh manage it, set downloads.${kind.provision}.name to "${alike[0].name}"`,
}];
}
const outcomes: Outcome[] = [];
const others = entries.filter((e) => e.implementation === kind.implementation && e.name !== name);
if (others.length > 0) {
outcomes.push({
what,
result: "notice",
note: `also present and not the mesh's, left alone: ${others.map((o) => `"${o.name}"`).join(", ")}`,
});
}
if (exact.length === 0) {
outcomes.push(
await createEntry(http, app, "downloadclient", what, kind.implementation, name, {
...clientConnection(ep),
password: ep.credential,
}),
);
return outcomes;
}
outcomes.push(
await reconcileEntry(http, app, "downloadclient", what, exact[0], clientConnection(ep), clientDiffers(exact[0], ep), "password", ep.credential),
);
return outcomes;
}
// ---------------------------------------------------------------------------------------------
// Indexers
const FEED = /\/api\/v2\.0\/indexers\/([^/]+)\/results\/torznab\/?$/;
/** A Torznab entry reading a jackett feed: the host it points at and the jackett indexer's id. */
export function jackettFeed(e: Entry): { host: string; id: string } | undefined {
if (e.implementation !== "Torznab") return undefined;
const raw = String(field(e, "baseUrl") ?? "").trim();
if (!raw) return undefined;
let u: URL;
try {
u = new URL(raw);
} catch {
return undefined;
}
const path = `${u.pathname.replace(/\/+$/, "")}/${String(field(e, "apiPath") ?? "").replace(/^\/+/, "")}`;
const m = FEED.exec(path);
if (!m) return undefined;
return { host: u.hostname.replace(/^\[|\]$/g, "").toLowerCase(), id: decodeURIComponent(m[1]) };
}
/** The feed's two fields as the mesh says them. */
export function feedConnection(ep: Endpoint, id: string): Record<string, unknown> {
return { baseUrl: baseUrl(ep), apiPath: `/api/v2.0/indexers/${id}/results/torznab/` };
}
function sameUrl(a: string, b: string): boolean {
try {
const x = new URL(a);
const y = new URL(b);
const port = (u: URL) => u.port || (u.protocol === "https:" ? "443" : "80");
return (
x.protocol === y.protocol &&
x.hostname.toLowerCase() === y.hostname.toLowerCase() &&
port(x) === port(y) &&
x.pathname.replace(/\/+$/, "") === y.pathname.replace(/\/+$/, "")
);
} catch {
return false;
}
}
export function feedDiffers(e: Entry, ep: Endpoint, id: string): string[] {
const want = feedConnection(ep, id);
const out: string[] = [];
if (!sameUrl(String(field(e, "baseUrl") ?? ""), String(want.baseUrl))) out.push("baseUrl");
const path = (v: unknown) => `/${String(v ?? "").replace(/^\/+|\/+$/g, "")}`;
if (path(field(e, "apiPath")) !== path(want.apiPath)) out.push("apiPath");
return out;
}
export interface IndexerSettings {
/** Hosts whose jackett feeds the mesh takes over — a migration's old names. */
adoptHosts: string[];
/** jackett indexer ids the app should have. */
indexers: string[];
}
/**
* Bring the app's jackett feeds in line: every one the mesh manages is pointed at the bound jackett
* with its key, and every listed jackett indexer the app lacks is registered.
*/
export async function reconcileIndexers(
http: Http,
app: App,
ep: Endpoint,
configured: Map<string, string>,
settings: IndexerSettings,
remembered: string[],
entries: Entry[],
): Promise<Outcome[]> {
const ours = new Set([ep.host, ...settings.adoptHosts, ...remembered].map((h) => h.trim().toLowerCase()).filter(Boolean));
const outcomes: Outcome[] = [];
const managed = new Set<string>();
for (const e of entries) {
const feed = jackettFeed(e);
if (!feed || !ours.has(feed.host)) continue;
managed.add(feed.id);
const what = `jackett-api ("${e.name}", jackett indexer ${feed.id})`;
const known = feed.id === "all" || configured.has(feed.id);
const outcome = await reconcileEntry(
http, app, "indexer", what, e, feedConnection(ep, feed.id), feedDiffers(e, ep, feed.id), "apiKey", ep.credential, known,
);
outcomes.push(outcome);
if (!known && outcome.result !== "refused") {
outcomes.push({ what, result: "notice", note: `jackett has no indexer "${feed.id}" configured, so it was not tested; configure it in jackett or remove the entry in ${app.module}` });
}
}
for (const id of settings.indexers) {
if (managed.has(id)) continue;
const what = `jackett-api (jackett indexer ${id})`;
if (id !== "all" && !configured.has(id)) {
outcomes.push({ what, result: "refused", problem: `downloads.jackett-api.indexers lists "${id}", and jackett has no indexer by that id configured` });
continue;
}
const name = `Jackett - ${id === "all" ? "all" : configured.get(id)}`;
outcomes.push(await createEntry(http, app, "indexer", what, "Torznab", name, { ...feedConnection(ep, id), apiKey: ep.credential }));
}
if (outcomes.length === 0) {
outcomes.push({ what: "jackett-api", result: "notice", note: `${app.module} has no jackett feed the mesh manages, and downloads.jackett-api.indexers lists none to register` });
}
return outcomes;
}
// ---------------------------------------------------------------------------------------------
// Files
/** The module's settings for this step, from its merged `downloads.json`. */
export interface StepSettings {
node: string;
names: Record<string, string>;
indexers: IndexerSettings;
}
export function stepSettings(raw: unknown): StepSettings {
const doc = (raw ?? {}) as Record<string, unknown>;
const d = (doc.downloads ?? {}) as Record<string, Record<string, unknown> | undefined>;
const names: Record<string, string> = {};
for (const k of CLIENTS) {
const n = d[k.provision]?.name;
names[k.provision] = typeof n === "string" && n.trim() ? n.trim() : k.provision.replace(/-api$/, "");
}
const j = d[JACKETT] ?? {};
const strings = (v: unknown) => (Array.isArray(v) ? v.filter((x): x is string => typeof x === "string" && x.trim() !== "").map((x) => x.trim()) : []);
return {
node: typeof doc.node === "string" ? doc.node : "",
names,
indexers: { adoptHosts: strings(j["adopt-hosts"]), indexers: strings(j.indexers) },
};
}
/** The app's own key and base path, from its config.xml. */
export function appConfig(xml: string): { apiKey: string; urlBase: string } {
const tag = (t: string) => new RegExp(`<${t}>([^<]*)</${t}>`).exec(xml)?.[1]?.trim() ?? "";
return { apiKey: tag("ApiKey"), urlBase: normBase(tag("UrlBase")) };
}
/** A file the mesh wrote, or undefined when it is not there. */
export async function readIfThere(path: string | undefined): Promise<string | undefined> {
if (!path) return undefined;
return readFile(path, "utf8").catch(() => undefined);
}
/** A JSON file parsed, or undefined when absent or not JSON. */
export async function readJson(path: string | undefined): Promise<unknown> {
const raw = await readIfThere(path);
if (raw === undefined) return undefined;
try {
return JSON.parse(raw) as unknown;
} catch {
return undefined;
}
}
/** Wait for the app to answer, because the step runs right after its container starts. */
export async function appReady(http: Http, app: App, waitMs: number, pauseMs = 2000): Promise<boolean> {
const until = Date.now() + waitMs;
for (;;) {
try {
const a = await appCall(http, app, "GET", "/system/status");
if (a.status === 200) return true;
} catch {
// not listening yet
}
if (Date.now() >= until) return false;
await new Promise((r) => setTimeout(r, pauseMs));
}
}
export async function listEntries(http: Http, app: App, resource: "downloadclient" | "indexer"): Promise<Entry[]> {
return ((await appGet(http, app, `/${resource}`)) as Entry[]) ?? [];
}
/** Anything printed goes through this: the credentials the step holds never reach a log. */
export function scrub(text: string, secrets: string[]): string {
let out = text;
for (const s of secrets) if (s && s.length >= 4) out = out.split(s).join("<redacted>");
return out;
}
function message(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
+74
View File
@@ -0,0 +1,74 @@
// bookshelf's events. The tool runtime imports this once the broker is bound. It watches the
// download queue and turns its comings and goings into mesh events — the same mechanism radarr uses,
// applied to a Servarr book manager.
//
// Emits (novox/hq ADR 0041/0042):
// module.bookshelf.book.grabbed — a release entered the queue (Bookshelf grabbed it)
// module.bookshelf.download.completed — a release left the queue, imported. This routing key is
// what the plex module consumes (module.*.download.completed)
// to rescan, so a new audiobook becomes a visible item.
// Consumes: none.
//
// NOTE: the hal bookshelf module emitted no events (its hooks only did install-time provisioning).
// This queue watcher is new in nox, modelled exactly on radarr's — bookshelf is a Servarr app with
// the same queue semantics, so the diff-and-emit pattern carries over unchanged.
//
// The queue is polled and diffed, primed silently on the first look (like plex's index.ts) so a
// restart mid-download does not re-announce everything already in flight as freshly grabbed.
import { emit } from "@novox/mesh-sdk/events";
import { BookshelfClient, type BookshelfQueueItem } from "./client.js";
// Building the client throws when Bookshelf has no URL/key yet. Like the tools (see tools/index.ts),
// the events entrypoint must not crash the runtime for that — it stays idle until configured.
function buildClient(): BookshelfClient | null {
try {
return BookshelfClient.fromEnv();
} catch {
return null;
}
}
const bookshelf = buildClient();
// Bookshelf removes an item from the queue once it has been imported; a "warning"/"failed" status is
// how a stuck or broken grab shows itself, so we do not call those a completion when they vanish.
const FAILED_STATUSES = new Set(["failed", "warning"]);
const inQueue = new Map<number, BookshelfQueueItem>();
let primed = false;
async function pollQueue(bookshelf: BookshelfClient): Promise<void> {
const { items } = await bookshelf.getQueue();
const now = new Map(items.map((i) => [i.id, i]));
if (primed) {
// Entered the queue since last look — Bookshelf grabbed a release.
for (const [id, item] of now) {
if (!inQueue.has(id)) await emit("book.grabbed", { title: item.title, status: item.status });
}
// Left the queue — imported and done, unless it was last seen failing.
for (const [id, item] of inQueue) {
if (!now.has(id) && !FAILED_STATUSES.has(item.status)) {
await emit("download.completed", { title: item.title });
}
}
}
inQueue.clear();
for (const [id, item] of now) inQueue.set(id, item);
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[bookshelf] ${err}`));
setInterval(run, everyMs);
run();
};
if (bookshelf) {
tick(() => pollQueue(bookshelf), 30_000);
console.log("[bookshelf] watching the download queue, emitting grabs and completions");
} else {
console.log("[bookshelf] not configured — events idle until an API key is available");
}
+188
View File
@@ -0,0 +1,188 @@
{
"module": "bookshelf",
"version": "1",
"slug": "books",
"capabilities": [
"container-runtime"
],
"emits": [
"book.grabbed",
"download.completed"
],
"consumes": [],
"own-secrets": {
"broker": "/var/lib/mesh/bookshelf/broker"
},
"listens": [
{
"name": "web",
"port": 8787,
"protocol": "tcp",
"from": "mesh",
"why": "managing the ebook/audiobook library"
}
],
"accesses": [
{
"path": "/services/media/books",
"mode": "read-write"
},
{
"path": "/services/media/downloads",
"mode": "read-write"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/bookshelf",
"mode": "0700"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "config",
"type": "directory",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "bookshelf",
"image": "ghcr.io/pennydreadful/bookshelf@sha256:67498dd5ece516867d72ee642abd6c1a66b36a135c8f7da0127109564372beb1",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"8787"
],
"volumes": [
"${dir:config}:/config",
"/services/media/books:/books",
"/services/media/downloads:/downloads"
]
},
{
"id": "runtime",
"type": "container",
"name": "mesh-bookshelf",
"network": "host",
"volumes": [
"/var/lib/mesh/bookshelf/broker:/run/secrets/broker:ro",
"${dir:config}:/var/lib/bookshelf/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BOOKSHELF_URL": "http://127.0.0.1:${port:8787}",
"MESH_BOOKSHELF_CONFIG_DIR": "/var/lib/bookshelf/config"
},
"artifact": "runtime"
},
{
"id": "downloads-config",
"type": "file",
"path": "${dir:state}/downloads.json",
"mode": "0600",
"content": "{\n \"node\": \"${machine:name}\",\n \"downloads\": {\n \"nzbget-api\": {\n \"name\": \"nzbget\"\n },\n \"qbittorrent-api\": {\n \"name\": \"qbittorrent\"\n },\n \"jackett-api\": {\n \"adopt-hosts\": [],\n \"indexers\": []\n }\n }\n}\n",
"merge": "json"
},
{
"id": "downloads-memory",
"type": "directory",
"mode": "0700"
},
{
"id": "downloads",
"type": "container",
"name": "mesh-bookshelf-downloads",
"network": "host",
"run-once": true,
"volumes": [
"${dir:state}/downloads.json:/run/downloads/downloads.json:ro",
"${dir:state}/nzbget-api.json:/run/downloads/nzbget-api.json:ro",
"${dir:state}/nzbget-api.secret:/run/downloads/nzbget-api.secret:ro",
"${dir:state}/qbittorrent-api.json:/run/downloads/qbittorrent-api.json:ro",
"${dir:state}/qbittorrent-api.secret:/run/downloads/qbittorrent-api.secret:ro",
"${dir:state}/jackett-api.json:/run/downloads/jackett-api.json:ro",
"${dir:state}/jackett-api.secret:/run/downloads/jackett-api.secret:ro",
"${dir:config}:/var/lib/bookshelf/config:ro",
"${dir:downloads-memory}:/var/lib/downloads"
],
"env": {
"MESH_DOWNLOADS_APP": "bookshelf",
"MESH_DOWNLOADS_API": "v1",
"MESH_DOWNLOADS_APP_URL": "http://127.0.0.1:${port:8787}",
"MESH_DOWNLOADS_APP_CONFIG": "/var/lib/bookshelf/config/config.xml",
"MESH_DOWNLOADS_DIR": "/run/downloads",
"MESH_DOWNLOADS_SETTINGS": "/run/downloads/downloads.json",
"MESH_DOWNLOADS_MEMORY": "/var/lib/downloads/memory.json"
},
"args": [
"run",
"/app/modules/bookshelf/dist/downloads/index.js"
],
"restart-on": [
"downloads-config",
"bound-nzbget-api",
"secret-nzbget-api",
"bound-qbittorrent-api",
"secret-qbittorrent-api",
"bound-jackett-api",
"secret-jackett-api"
],
"artifact": "runtime"
}
],
"requires": [
"jackett-api",
"nzbget-api",
"qbittorrent-api",
"route"
],
"contributes": {
"route": {
"label": "books",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:state}/route.json",
"nzbget-api": "${dir:state}/nzbget-api.json",
"qbittorrent-api": "${dir:state}/qbittorrent-api.json",
"jackett-api": "${dir:state}/jackett-api.json"
},
"secrets": {
"nzbget-api": "${dir:state}/nzbget-api.secret",
"qbittorrent-api": "${dir:state}/qbittorrent-api.secret",
"jackett-api": "${dir:state}/jackett-api.secret"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
+19
View File
@@ -0,0 +1,19 @@
{
"name": "@novox/module-bookshelf",
"version": "0.1.0",
"description": "bookshelf — ebook/audiobook management (Readarr fork). Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts downloads/settings.ts downloads/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
+365
View File
@@ -0,0 +1,365 @@
// What holds the downloads step (downloads/settings.ts): the app's download clients and jackett
// feeds are made to say what the mesh bound — host, port, TLS, base path, user, credential — and
// nothing else they keep is touched; only the mesh's entries are touched, and nothing is ever
// deleted; nothing is written when nothing differs; a missing one is registered; and a credential
// the provider refuses (the mesh's own minted value, before the operator accepts the provider's)
// is never written, with the `secret accept` that fixes it named.
//
// The app and the providers are fakes: the routes the step touches, answering as the real ones do
// (checked against the catalogue's pinned sonarr, radarr, lidarr, bookshelf, nzbget, qBittorrent
// and jackett): the app masks a stored password as "********", tests an enabled entry before
// saving it, refuses a warning unless forceSave, and jackett answers a wrong key 200 with an error.
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import {
CLIENTS,
acceptRemedy,
jackettFeed,
providerTakes,
reconcileClient,
reconcileIndexers,
stepSettings,
wanted,
type App,
type Binding,
type Endpoint,
type Entry,
type Http,
} from "../downloads/settings.ts";
const NZBGET = CLIENTS.find((c) => c.provision === "nzbget-api")!;
const QBIT = CLIENTS.find((c) => c.provision === "qbittorrent-api")!;
const APP: App = { module: "sonarr", url: "http://127.0.0.1:8989", apiKey: "app-key", api: "v3" };
const MASK = "********";
interface Provider {
host: string;
port: number;
user?: string;
secret: string;
configured?: Record<string, string>;
}
const nzbget: Provider = { host: "ace.internal", port: 20201, user: "luffy", secret: "nzb-real" };
const qbit: Provider = { host: "ace.internal", port: 8112, user: "luffy", secret: "qbt-real" };
const jackett: Provider = { host: "ace.internal", port: 20204, secret: "jackett-real", configured: { rutracker: "RuTracker", torrent9: "Torrent9" } };
function binding(p: Provider, provision: string, extra: Record<string, unknown> = {}): Binding {
return {
provision,
from: "ace",
at: p.host,
as: "mesh_ace_sonarr",
serves: { scheme: "http", port: p.port, "url-base": "", ...(p.user ? { username: p.user } : {}), ...extra },
};
}
function endpoint(p: Provider, provision: string, credential = p.secret): Endpoint {
const w = wanted(provision, binding(p, provision), credential, provision !== "jackett-api");
if (!w.ok) throw new Error(w.problem);
return w.endpoint;
}
const f = (name: string, value: unknown) => ({ name, value });
function client(id: number, name: string, implementation: string, host: string, port: number, password: string, extra: Record<string, unknown> = {}): Entry {
return {
id, name, implementation, enable: true, priority: 1, tags: [],
fields: [f("host", host), f("port", port), f("useSsl", false), f("urlBase", null), f("username", "luffy"), f("password", password), f("tvCategory", "Series"), ...Object.entries(extra).map(([k, v]) => f(k, v))],
};
}
function feed(id: number, name: string, baseUrl: string, jid: string, key: string, on = true): Entry {
return {
id, name, implementation: "Torznab", enableRss: on, enableAutomaticSearch: on, enableInteractiveSearch: on, priority: 25,
fields: [f("baseUrl", baseUrl), f("apiPath", `/api/v2.0/indexers/${jid}/results/torznab/`), f("apiKey", key), f("categories", [5000])],
};
}
interface Call {
method: string;
url: string;
body?: unknown;
}
/** A Servarr app and the three providers behind one fetch. */
function fakes(start: { clients?: Entry[]; indexers?: Entry[] }) {
const calls: Call[] = [];
const store: Record<string, Entry[]> = {
downloadclient: structuredClone(start.clients ?? []),
indexer: structuredClone(start.indexers ?? []),
};
let next = 100;
const secretOf: Record<string, string> = { downloadclient: "password", indexer: "apiKey" };
const masked = (e: Entry): Entry => ({ ...e, fields: e.fields!.map((x) => (x.name === "password" || x.name === "apiKey") && x.value ? { ...x, value: MASK } : { ...x }) });
const val = (e: Entry, n: string) => e.fields?.find((x) => x.name === n)?.value;
/** What the app's own test says: dialled from its container, with its stored secret for a mask. */
const appTest = (kind: string, e: Entry): { propertyName: string; errorMessage: string; isWarning: boolean }[] => {
let secret = val(e, secretOf[kind]);
if (secret === MASK) secret = val(store[kind].find((s) => s.id === e.id) ?? {}, secretOf[kind]);
if (kind === "downloadclient") {
const p = e.implementation === "Nzbget" ? nzbget : qbit;
if (val(e, "host") !== p.host || val(e, "port") !== p.port) return [{ propertyName: "Host", errorMessage: "Unable to connect", isWarning: false }];
if (secret !== p.secret || val(e, "username") !== p.user) return [{ propertyName: "Username", errorMessage: "Authentication Failure", isWarning: false }];
if (val(e, "tvCategory") === "") return [{ propertyName: "TvCategory", errorMessage: "A category is recommended", isWarning: true }];
if (e.implementation === "Nzbget" && val(e, "tvCategory") === "tv") return [{ propertyName: "TvCategory", errorMessage: "Category does not exist", isWarning: false }];
return [];
}
const u = new URL(String(val(e, "baseUrl")));
if (u.hostname !== jackett.host || Number(u.port) !== jackett.port) return [{ propertyName: "BaseUrl", errorMessage: "Unable to connect", isWarning: false }];
if (secret !== jackett.secret) return [{ propertyName: "ApiKey", errorMessage: "Invalid API Key", isWarning: false }];
const id = /indexers\/([^/]+)\//.exec(String(val(e, "apiPath")))?.[1] ?? "";
if (!(id in jackett.configured!)) return [{ propertyName: "", errorMessage: "Unknown indexer", isWarning: false }];
return [];
};
const http: Http = {
async fetch(url, init) {
const method = init?.method ?? "GET";
const body = init?.body && init.headers?.["Content-Type"] === "application/json" ? (JSON.parse(init.body) as Entry) : init?.body;
calls.push({ method, url, body });
const reply = (status: number, value?: unknown) => ({ status, text: async () => (value === undefined ? "" : typeof value === "string" ? value : JSON.stringify(value)) });
const u = new URL(url);
// Providers.
if (u.pathname === "/jsonrpc/version") {
const want = "Basic " + Buffer.from(`${nzbget.user}:${nzbget.secret}`).toString("base64");
return init?.headers?.Authorization === want ? reply(200, { result: "26.0" }) : reply(401);
}
if (u.pathname === "/api/v2/auth/login") {
const form = new URLSearchParams(String(init?.body ?? ""));
return form.get("username") === qbit.user && form.get("password") === qbit.secret ? reply(204) : reply(401, "Unauthorized");
}
if (u.pathname.endsWith("/torznab/api")) {
if (u.searchParams.get("apikey") !== jackett.secret) return reply(200, '<?xml version="1.0"?><error code="100" description="Invalid API Key" />');
const items = Object.entries(jackett.configured!).map(([id, t]) => `<indexer id="${id}" configured="true"><title>${t}</title></indexer>`).join("");
return reply(200, `<?xml version="1.0"?><indexers>${items}</indexers>`);
}
// The app.
if (init?.headers?.["X-Api-Key"] !== APP.apiKey) return reply(401);
const m = /^\/api\/v3\/(downloadclient|indexer)(?:\/(schema|test|\d+))?$/.exec(u.pathname);
if (!m) return reply(404);
const [, kind, sub] = m;
const force = u.searchParams.get("forceSave") === "true";
if (method === "GET" && !sub) return reply(200, store[kind].map(masked));
if (method === "GET" && sub === "schema") {
return reply(200, kind === "downloadclient"
? [client(0, "", "Nzbget", "localhost", 6789, MASK, {}), client(0, "", "QBittorrent", "localhost", 8080, "", {})].map((e) => ({ ...e, fields: e.fields!.map((x) => x.name === "tvCategory" ? { ...x, value: e.implementation === "Nzbget" ? "tv" : "tv-sonarr" } : x.name === "username" ? { ...x, value: null } : x) }))
: [{ ...feed(0, "", "", "x", "", false), supportsRss: true, supportsSearch: true, fields: [f("baseUrl", null), f("apiPath", "/api"), f("apiKey", null), f("categories", [5030, 5040])] }]);
}
if (method === "GET") {
const e = store[kind].find((s) => s.id === Number(sub));
return e ? reply(200, masked(e)) : reply(404);
}
if (method === "POST" && sub === "test") {
const fails = appTest(kind, body as Entry);
return fails.length ? reply(400, fails) : reply(200);
}
// Save: tested when enabled; a mask keeps what is stored.
const e = body as Entry;
const on = e.enable === true || e.enableRss === true || e.enableAutomaticSearch === true;
if (on) {
const fails = appTest(kind, e);
if (fails.some((x) => !x.isWarning) || (fails.length && !force)) return reply(400, fails);
}
if (method === "POST" && !sub) {
if (store[kind].some((s) => String(s.name).toLowerCase() === String(e.name).toLowerCase())) {
return reply(400, [{ propertyName: "Name", errorMessage: "Should be unique", isWarning: false }]);
}
const created = { ...e, id: next++ };
store[kind].push(created);
return reply(201, masked(created));
}
if (method === "PUT") {
const i = store[kind].findIndex((s) => s.id === Number(sub));
if (i < 0) return reply(404);
const was = store[kind][i];
store[kind][i] = { ...e, fields: e.fields!.map((x) => x.value === MASK ? { ...x, value: val(was, x.name) } : x) };
return reply(202, masked(store[kind][i]));
}
return reply(405);
},
};
const saves = () => calls.filter((c) => (c.method === "PUT" || (c.method === "POST" && !c.url.endsWith("/test"))) && c.url.includes("/api/v3/"));
return { http, calls, store, saves };
}
const aceClients = () => [
client(1, "NZBGet", "Nzbget", "nzbget", 6789, "nzb-real"),
client(2, "qBitTorrent", "QBittorrent", "qbittorrent", 8112, "qbt-real"),
];
test("the migration's download client is repointed, its category and everything else kept", async () => {
const f = fakes({ clients: aceClients() });
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(nzbget, "nzbget-api"), f.store.downloadclient.map((e) => ({ ...e })));
assert.deepEqual(out, [{ what: 'nzbget-api ("NZBGet")', result: "written", fields: ["host", "port"] }]);
const saved = f.store.downloadclient.find((e) => e.id === 1)!;
const v = (n: string) => saved.fields!.find((x) => x.name === n)?.value;
assert.equal(v("host"), "ace.internal");
assert.equal(v("port"), 20201);
assert.equal(v("tvCategory"), "Series");
assert.equal(v("password"), "nzb-real", "the password it held works, so it was kept, not rewritten");
assert.equal(saved.priority, 1);
});
test("rerun: already as the mesh says, nothing saved", async () => {
const f = fakes({ clients: [client(1, "NZBGet", "Nzbget", "ace.internal", 20201, "nzb-real")] });
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(nzbget, "nzbget-api"), f.store.downloadclient);
assert.deepEqual(out, [{ what: 'nzbget-api ("NZBGet")', result: "unchanged" }]);
assert.equal(f.saves().length, 0);
});
test("a stale password is replaced by the delivered one, which the provider took first", async () => {
const f = fakes({ clients: [client(2, "qBitTorrent", "QBittorrent", "ace.internal", 8112, "old-pass")] });
const out = await reconcileClient(f.http, APP, QBIT, "qBitTorrent", endpoint(qbit, "qbittorrent-api"), f.store.downloadclient);
assert.deepEqual(out, [{ what: 'qbittorrent-api ("qBitTorrent")', result: "written", fields: ["password"] }]);
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "password")?.value, "qbt-real");
});
test("a minted credential is refused by the provider: nothing written, the accept named", async () => {
const f = fakes({ clients: aceClients() });
const ep = endpoint(nzbget, "nzbget-api", "a-value-the-mesh-minted");
assert.deepEqual(await providerTakes(f.http, "nzbget-api", ep), { took: false });
const remedy = acceptRemedy(APP, "ace", "nzbget-api", "nzbget", "ControlPassword", ep.from);
assert.match(remedy, /`secret accept ace sonarr nzbget-api --provider ace --from <file holding nzbget's ControlPassword>`/);
assert.doesNotMatch(remedy, /minted/);
assert.equal(f.calls.some((c) => c.url.includes("/api/v3/")), false, "the app was not even asked");
const q = endpoint(qbit, "qbittorrent-api", "minted");
assert.deepEqual(await providerTakes(f.http, "qbittorrent-api", q), { took: false });
const j = endpoint(jackett, "jackett-api", "minted");
assert.deepEqual(await providerTakes(f.http, "jackett-api", j), { took: false });
assert.equal(f.calls.find((c) => c.url.includes("apikey="))?.url.includes("jackett-real"), false);
});
test("a fresh app gets the mesh's client registered, under the mesh's name", async () => {
const f = fakes({});
const out = await reconcileClient(f.http, APP, QBIT, "qbittorrent", endpoint(qbit, "qbittorrent-api"), []);
assert.equal(out[0].result, "created");
const e = f.store.downloadclient[0];
assert.equal(e.name, "qbittorrent");
assert.equal(e.enable, true);
assert.equal(e.fields!.find((x) => x.name === "tvCategory")?.value, "tv-sonarr", "the app's own default category");
});
test("nzbget without the app's default category: registered with none, and said", async () => {
const f = fakes({});
const out = await reconcileClient(f.http, APP, NZBGET, "nzbget", endpoint(nzbget, "nzbget-api"), []);
assert.equal(out[0].result, "notice");
assert.match((out[0] as { note: string }).note, /category left empty/);
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "tvCategory")?.value, "");
});
test("somebody else's entries are never touched: another name of the same kind, and a clash of names", async () => {
const mine = client(7, "My seedbox", "QBittorrent", "seedbox.example", 443, "theirs");
const f = fakes({ clients: [mine] });
const out = await reconcileClient(f.http, APP, QBIT, "qbittorrent", endpoint(qbit, "qbittorrent-api"), f.store.downloadclient);
assert.equal(out[0].result, "notice");
assert.deepEqual(f.store.downloadclient.find((e) => e.id === 7), mine);
// Same name in another case: refused, and the setting that adopts it named.
const g = fakes({ clients: aceClients() });
const clash = await reconcileClient(g.http, APP, NZBGET, "nzbget", endpoint(nzbget, "nzbget-api"), g.store.downloadclient);
assert.equal(clash[0].result, "refused");
assert.match((clash[0] as { problem: string }).problem, /downloads\.nzbget-api\.name to "NZBGet"/);
assert.equal(g.saves().length, 0);
});
test("jackett feeds: the migration's are repointed, a person's is left, a listed one is registered", async () => {
const personal = feed(9, "Seedbox jackett", "https://jackett.seedbox.example", "rutracker", "their-key");
const f = fakes({
indexers: [
feed(5, "Torznab - RuTracker", "https://indexers.zurag.be", "rutracker-ru", "jackett-real", false),
feed(6, "Torznab - Torrent9", "https://indexers.zurag.be", "torrent9", "jackett-real", false),
feed(4, "Jackett - RARBG", "http://jackett:9117", "therarbg", "old", false),
personal,
],
});
const configured = new Map(Object.entries(jackett.configured!));
const out = await reconcileIndexers(
f.http, APP, endpoint(jackett, "jackett-api"), configured,
{ adoptHosts: ["indexers.zurag.be", "jackett"], indexers: ["rutracker"] }, [], f.store.indexer,
);
const byWhat = Object.fromEntries(out.map((o) => [o.what + ":" + o.result, o]));
// torrent9: jackett has it; repointed, key already right, tested.
assert.ok(byWhat['jackett-api ("Torznab - Torrent9", jackett indexer torrent9):written']);
// rutracker-ru and therarbg: jackett has neither — repointed with the key, untested, and said.
assert.ok(byWhat['jackett-api ("Jackett - RARBG", jackett indexer therarbg):written']);
assert.ok(byWhat['jackett-api ("Jackett - RARBG", jackett indexer therarbg):notice']);
// rutracker is listed and nothing of the mesh's reads it: registered.
assert.ok(byWhat["jackett-api (jackett indexer rutracker):created"]);
assert.deepEqual(f.store.indexer.find((e) => e.id === 9), personal, "a person's jackett is not the mesh's");
const t9 = f.store.indexer.find((e) => e.id === 6)!;
assert.equal(t9.fields!.find((x) => x.name === "baseUrl")?.value, "http://ace.internal:20204");
assert.equal(t9.enableRss, false, "disabled stays disabled");
const created = f.store.indexer.find((e) => e.name === "Jackett - RuTracker")!;
assert.equal(created.enableRss, true);
assert.equal(f.calls.some((c) => c.method === "DELETE"), false, "nothing is ever deleted");
});
test("jackett feeds rerun: nothing saved, and a remembered host keeps a moved jackett's feeds", async () => {
const f = fakes({ indexers: [feed(6, "Torznab - Torrent9", "http://ace.internal:20204", "torrent9", "jackett-real")] });
const configured = new Map(Object.entries(jackett.configured!));
const out = await reconcileIndexers(f.http, APP, endpoint(jackett, "jackett-api"), configured, { adoptHosts: [], indexers: ["torrent9"] }, [], f.store.indexer);
assert.deepEqual(out.map((o) => o.result), ["unchanged"]);
assert.equal(f.saves().length, 0);
// jackett moved to novox: the feed on ace.internal is still the mesh's because it was remembered.
const moved = { ...jackett, host: "novox.internal" };
const g = fakes({ indexers: [feed(6, "Torznab - Torrent9", "http://ace.internal:20204", "torrent9", "jackett-real")] });
const saved = jackett.host;
jackett.host = moved.host;
try {
const again = await reconcileIndexers(g.http, APP, endpoint(moved, "jackett-api"), configured, { adoptHosts: [], indexers: [] }, ["ace.internal"], g.store.indexer);
assert.equal(again[0].result, "written");
assert.equal(g.store.indexer[0].fields!.find((x) => x.name === "baseUrl")?.value, "http://novox.internal:20204");
} finally {
jackett.host = saved;
}
});
test("an enabled entry the app cannot test at the mesh's address is not saved", async () => {
const f = fakes({ clients: [client(1, "NZBGet", "Nzbget", "nzbget", 6789, "nzb-real")] });
const elsewhere = { ...nzbget, port: 1 };
const out = await reconcileClient(f.http, APP, NZBGET, "NZBGet", endpoint(elsewhere, "nzbget-api"), f.store.downloadclient);
assert.equal(out[0].result, "refused");
assert.equal(f.store.downloadclient[0].fields!.find((x) => x.name === "host")?.value, "nzbget", "left as it was");
});
test("bindings: loopback, no user, no credential are refused; the base path is normalised", () => {
assert.equal(wanted("nzbget-api", { ...binding(nzbget, "nzbget-api"), at: "127.0.0.1" }, "x", true).ok, false);
assert.equal(wanted("nzbget-api", binding({ ...nzbget, user: undefined }, "nzbget-api"), "x", true).ok, false);
assert.equal(wanted("nzbget-api", binding(nzbget, "nzbget-api"), " \n", true).ok, false);
const w = wanted("jackett-api", binding(jackett, "jackett-api", { "url-base": "jackett/" }), "k", false);
assert.equal(w.ok && w.endpoint.urlBase, "/jackett");
});
test("a feed is read whole: base path in the base URL or in the API path", () => {
const e = feed(1, "x", "http://ace.internal:9117/jackett", "rutracker", "k");
assert.deepEqual(jackettFeed(e), { host: "ace.internal", id: "rutracker" });
assert.equal(jackettFeed({ ...e, implementation: "Newznab" }), undefined);
assert.equal(jackettFeed({ ...e, fields: [f("baseUrl", "https://api.nzbgeek.info"), f("apiPath", "/api")] }), undefined);
});
test("settings: names default to the provider's, adoption and registration read from the merged file", () => {
assert.deepEqual(stepSettings({ node: "ace" }), { node: "ace", names: { "nzbget-api": "nzbget", "qbittorrent-api": "qbittorrent" }, indexers: { adoptHosts: [], indexers: [] } });
const s = stepSettings({ node: "ace", downloads: { "nzbget-api": { name: "NZBGet" }, "jackett-api": { "adopt-hosts": ["jackett"], indexers: ["torrent9", 3] } } });
assert.equal(s.names["nzbget-api"], "NZBGet");
assert.deepEqual(s.indexers, { adoptHosts: ["jackett"], indexers: ["torrent9"] });
});
// The four copies are one step. Where the siblings are checked out beside this module, they must
// be byte-identical — a fix made in one and not the others is a bug in three apps.
test("the step is the same in sonarr, radarr, lidarr and bookshelf", () => {
const here = dirname(dirname(fileURLToPath(import.meta.url)));
const modules = dirname(here);
for (const file of ["downloads/settings.ts", "downloads/index.ts", "test/downloads.test.ts"]) {
const mine = readFileSync(join(here, file), "utf8");
for (const sibling of ["sonarr", "radarr", "lidarr", "bookshelf"]) {
const theirs = join(modules, sibling, file);
if (existsSync(theirs)) assert.equal(readFileSync(theirs, "utf8"), mine, `${sibling}/${file} differs`);
}
}
});
+69
View File
@@ -0,0 +1,69 @@
// bookshelf's tools — ported from the shared hal `arr` sdk (novox/hq ADR 0039), importing
// bookshelf's own client. They return structured data (not the pre-formatted text hal returned); the
// mesh serves them through the sdk's tool harness. Bookshelf has no calendar endpoint, so there is
// no calendar tool — matching hal, which excluded it from its calendar-capable apps.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { BookshelfClient } from "../client.js";
export function getBookshelfTools(bookshelf: BookshelfClient): ToolDefinition[] {
return [
{
name: "bookshelf_status",
description: "Bookshelf status overview: version, book count, monitored count, queue size.",
input: {},
run: async () => {
const [status, content, queue] = await Promise.all([
bookshelf.getStatus(),
bookshelf.getContent(),
bookshelf.getQueue(),
]);
return {
app: status.appName,
version: status.version,
books: content.length,
monitored: content.filter((c) => c.monitored).length,
queue: queue.totalRecords,
};
},
},
{
name: "bookshelf_library",
description: "List books from the Bookshelf library.",
input: { limit: { type: "number", description: "max items to return (default 50)" } },
run: async (args) => {
const items = await bookshelf.getContent(args.limit ? Number(args.limit) : 50);
return { count: items.length, books: items };
},
},
{
name: "bookshelf_search",
description:
"Search the Bookshelf library for books by title or author (filters existing content, not indexers).",
input: { query: { type: "string", description: "the search term" } },
run: async (args) => {
const query = String(args.query);
return { query, results: await bookshelf.searchContent(query) };
},
},
{
name: "bookshelf_queue",
description: "Show the Bookshelf download queue — what is downloading and how far along.",
input: {},
run: async () => {
const queue = await bookshelf.getQueue();
return { count: queue.totalRecords, items: queue.items };
},
},
];
}
// The tools exist only when Bookshelf is configured; without a URL and key, bookshelf contributes
// none rather than failing the whole runtime.
registerModuleTools("bookshelf", (env) => {
try {
return getBookshelfTools(BookshelfClient.fromEnv(env));
} catch {
return [];
}
});
@@ -8,5 +8,5 @@
"skipLibCheck": true, "skipLibCheck": true,
"noEmit": true "noEmit": true
}, },
"include": ["records.ts", "index.ts", "tools/index.ts"] "include": ["client.ts", "index.ts", "tools/index.ts", "downloads/settings.ts", "downloads/index.ts"]
} }
-15
View File
@@ -1,15 +0,0 @@
# build-agent
The mesh's build machine as a role every machine can hold (novox/hq ADR 0190). It holds the node seat
`node-build-agent`: every holder pulls one build at a time from the role's one work queue when it is
idle, so a tier of many images is built by as many machines as hold the seat and are online, and a
machine that is off builds nothing and blocks nothing. The controller asks the role, never a machine;
the outcome names the machine that built it.
What a holding machine needs is what the builder always needed, said here once: a container runtime
(the socket is mounted), the artifact store and the package registry as provisions, a workspace, and
the bus credential. The code is `cmd/mesh-builder` in the mesh-controller repository, compiled from
that repository's main (`build.artifacts[].context`); this module ships the packaging.
Assign it to every machine with a container runtime. It replaces `builder`, the one-holder form of the
same thing; retire that once this is assigned where it was.
@@ -1,6 +1,6 @@
ARG GO_BASE ARG GO_BASE
ARG ALPINE_BASE ARG ALPINE_BASE
# build-agent's image: the build machine itself, compiled into a container (novox/hq ADR 0190). # builder's own image: the build machine itself, compiled into a container.
# #
# **The source is not vendored here.** builder's actual code — cmd/mesh-builder, internal/builder, # **The source is not vendored here.** builder's actual code — cmd/mesh-builder, internal/builder,
# internal/catalogue — lives in the mesh-controller repository, the same control plane it is one # internal/catalogue — lives in the mesh-controller repository, the same control plane it is one
@@ -1,14 +1,13 @@
{ {
"module": "build-agent", "module": "builder",
"version": "1", "version": "1",
"slug": "agent",
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"claims": [ "claims": [
{ {
"name": "node-build-agent", "name": "mesh-build-machine",
"scope": "node" "scope": "mesh"
} }
], ],
"requires": [ "requires": [
@@ -16,48 +15,49 @@
"npm-package-registry" "npm-package-registry"
], ],
"binds": { "binds": {
"npm-package-registry": "${dir:mesh-state}/package-registry.json" "npm-package-registry": "/var/lib/mesh/builder/package-registry.json"
}, },
"secrets": { "secrets": {
"npm-package-registry": "${dir:mesh-state}/package-registry.secret" "npm-package-registry": "/var/lib/mesh/builder/package-registry.secret"
}, },
"own-secrets": { "own-secrets": {
"broker": "${dir:mesh-state}/broker" "broker": "/var/lib/mesh/builder/broker"
}, },
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/builder",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "workspace", "id": "workspace",
"type": "directory", "type": "directory",
"path": "/var/lib/builder/workspace",
"mode": "0700" "mode": "0700"
}, },
{ {
"id": "agent-env", "id": "builder-env",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/build-agent.env", "path": "/var/lib/mesh/builder/builder.env",
"mode": "0600", "mode": "0600",
"content": "MESH_BROKER_FILE=/run/mesh/broker\nMESH_NODE=${machine:name}\nMESH_REGISTRY=${bound:artifact-store:at}:${bound:artifact-store:port}\nMESH_PACKAGE_BINDING=/run/mesh/package-registry.json\nMESH_NPM_TOKEN_FILE=/run/mesh/package-registry.secret\nMESH_WORKSPACE=${dir:workspace}\n" "content": "MESH_BROKER_FILE=/run/mesh/broker\nMESH_NODE=${machine:name}\nMESH_REGISTRY=${bound:artifact-store:at}:${bound:artifact-store:port}\nMESH_PACKAGE_BINDING=/run/mesh/package-registry.json\nMESH_NPM_TOKEN_FILE=/run/mesh/package-registry.secret\nMESH_WORKSPACE=/var/lib/builder/workspace\n"
}, },
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "mesh-build-agent", "name": "mesh-builder",
"artifact": "server", "artifact": "server",
"env-file": [ "env-file": [
"${dir:mesh-state}/build-agent.env" "/var/lib/mesh/builder/builder.env"
], ],
"volumes": [ "volumes": [
"${dir:mesh-state}:/run/mesh:ro", "/var/lib/mesh/builder:/run/mesh:ro",
"${dir:workspace}:${dir:workspace}", "/var/lib/builder/workspace:/var/lib/builder/workspace",
"/var/run/docker.sock:/var/run/docker.sock" "/var/run/docker.sock:/var/run/docker.sock"
], ],
"restart-on": [ "restart-on": [
"agent-env" "builder-env"
], ],
"network": "host" "network": "host"
} }
@@ -69,8 +69,7 @@
"kind": "image", "kind": "image",
"from": "Dockerfile", "from": "Dockerfile",
"context": { "context": {
"seat": "git", "repository": "https://git.novox.be/novox/mesh-controller.git",
"repository": "novox/mesh-controller",
"ref": "main" "ref": "main"
} }
} }
-81
View File
@@ -1,81 +0,0 @@
# claude-code
The operator's agent on a machine (novox/hq design 36): its package, its machine-wide managed
configuration, and the consumer side of the Anthropic licence manager (design 39, ADR 0183).
## What it owns
Two directories, declared, so the mesh refuses a second module owning either:
- `/etc/claude-code`, the agent's machine-wide managed directory, root's, `0755`.
- `~/.claude` under the operator account's home, the operator's, `0700`. The module owns the directory —
that it exists, who owns it, its mode — and of what is inside only what it writes. Everything else
in it (memory, history, projects, local settings, a person's own rules and skills) is the person's
and is never read or written (hq ADR 0182). Unassigned, the module leaves the directory: the host
removes a directory only when it is empty.
## What it writes
Under the agent's managed directory, `/etc/claude-code`, owned whole by this module and rewritten
whenever the node's tool runtime collects the module's tools:
| file | holds |
|---|---|
| `managed-mcp.json` | the tool servers every session loads: the mesh's console as `mesh`, and the servers set in this module's `mcp_servers` setting. **Exclusive**: a server not listed here does not load — not one added with `claude mcp add`, not a project's `.mcp.json`, not a plugin's |
| `managed-settings.json` | the repositories' attribution convention, the claude.ai connectors kept beside the managed servers, and the key-helper while the node holds an API-key licence |
| `CLAUDE.md` | how a session on this mesh works, this node's name and role, the conventions |
Under the operator's home, only `~/.claude/.credentials.json`, and only when the licence manager hands
this node a subscription token. Nothing else under the home is read or written.
## Over NATS
Everything between this module and the rest of the mesh is NATS, in three kinds: an **event** says that
something happened and carries no secret, because a stream keeps it; a **request** carries a token,
because nothing keeps it (hq design 32 §10); and **state** is the current value of something every node
must see, a node that joins later included — kept, so it carries no secret either (hq ADR 0201).
| what | how |
|---|---|
| what this node holds | the module's `holdings` state, one key for this node — the account, the kind, fingerprints and expiries, never a token — written at start and whenever the credentials file changes (hq ADR 0206) |
| a person ran `/login` here | the credentials file gains a refresh token this module never writes; its next report shows it, and the licence manager asks `claude_code_grant` for it, giving its key — the one time a refresh token leaves the node, for the manager to adopt by refreshing it |
| what this node should hold | the licence manager's `bindings` state, this node's key; on a newer generation this module asks `anthropic-licence-manager.current` for its token, sealed to the key it sends, and writes it access-token-only — so the agent here never refreshes. A node that was off reads its key when it is back |
| an MCP server registered through this module | a key in the module's `servers` state — `all.<server>` for every node, `<node>.<server>` for one; every node watches it and renders what applies to it, a node's own entry over the one for every node. A node that joins later, or was off, reads the whole current set at start; unregistering is a delete. An entry with a secret in its `env` or `headers` is refused by the runtime |
## Tools
`claude_code_status`, `claude_code_render`, `claude_code_pull`, `claude_code_grant` (for the licence
manager), `claude_code_mcp_list`,
`claude_code_mcp_register` (this node by default; `nodes: "all"` or a list for more — called for this
node alone, its answer names the other nodes running claude-code), `claude_code_mcp_unregister`.
## Settings
Per node or for the whole mesh, through `mesh-controller.settings module=claude-code`:
- `role` — what this node is, in a few words; shown to every session.
- `mcp_servers` — extra tool servers, set by the operator for the mesh or a node, beside the ones
registered through the tools; keyed by name, in the vendor's `.mcp.json` entry shape
(`{"type":"http","url":…}` or `{"type":"stdio","command":…,"args":[…]}`). The name `mesh` is the
module's own and cannot be set. Put a person's own servers here, or they stop loading.
## On a machine that carried the predecessor
Remove these by hand, once; the mesh removes nothing it did not make (ADR 0182):
- `~/.claude/CLAUDE.md`
- `~/.claude/rules/00-hal-mesh.md`, `~/.claude/rules/conventions.md`
- `~/.claude/skills/cleanup/`, `~/.claude/skills/hal-switch-license/`
- the hand-made console entry in `~/.claude.json` under `mcpServers` — it is ignored now anyway
## Escalation
Writing `/etc/claude-code` needs root. The runtime runs as the operator account, and the module uses
that account's passwordless `sudo`; on a machine without it, `claude_code_render` says so and nothing
is written.
## Code
Go, one binary (`cmd/claude-code`) the node's runtime launches. Tested with `go test ./...`; the managed
instruction file is held to the TypeScript renderer it replaced (`testdata/rendered-by-typescript.json`),
and the sealed box is the licence manager's own format.
@@ -1,364 +0,0 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
var now = time.Now().UnixMilli()
func node(t *testing.T, name string) (Paths, map[string]string) {
t.Helper()
root := t.TempDir()
p := Paths{State: filepath.Join(root, "state"), Facts: filepath.Join(root, "state", "facts.json"),
Settings: filepath.Join(root, "state", "settings.json"), Home: filepath.Join(root, "home"), Node: name}
_ = os.MkdirAll(p.State, 0o700)
_ = os.MkdirAll(filepath.Join(p.Home, ".claude"), 0o700)
_ = os.WriteFile(p.Facts, []byte(`{"node":"`+name+`","console":"http://127.0.0.1:4270/mcp"}`), 0o600)
_ = os.WriteFile(p.Settings, []byte(`{"role":"","mcp_servers":{}}`), 0o600)
return p, map[string]string{}
}
func writer(w map[string]string) WriteManaged {
return func(name, content string) (string, error) { w[name] = content; return name + ": written", nil }
}
func writeFile(t *testing.T, path, content string) {
t.Helper()
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
func creds(t *testing.T, p Paths) map[string]any {
t.Helper()
var c map[string]any
raw, _ := os.ReadFile(p.credentials())
if err := json.Unmarshal(raw, &c); err != nil {
t.Fatal(err)
}
return c["claudeAiOauth"].(map[string]any)
}
// ---- the renderer, held to the TypeScript it replaced -------------------------------------------------
func TestTheRendererWritesWhatTheTypeScriptOneWrote(t *testing.T) {
raw, err := os.ReadFile("testdata/rendered-by-typescript.json")
if err != nil {
t.Fatal(err)
}
var f struct {
Facts Facts `json:"facts"`
Settings Settings `json:"settings"`
Registered Servers `json:"registered"`
WithKey map[string]string `json:"withKey"`
Plain map[string]string `json:"plain"`
}
if err := json.Unmarshal(raw, &f); err != nil {
t.Fatal(err)
}
same := func(label string, got, want map[string]string) {
if got["CLAUDE.md"] != want["CLAUDE.md"] {
t.Errorf("%s: CLAUDE.md differs from the TypeScript's:\n--- go\n%s\n--- typescript\n%s", label, got["CLAUDE.md"], want["CLAUDE.md"])
}
for _, file := range []string{"managed-mcp.json", "managed-settings.json"} {
var a, b any
_ = json.Unmarshal([]byte(got[file]), &a)
_ = json.Unmarshal([]byte(want[file]), &b)
if !reflect.DeepEqual(a, b) {
t.Errorf("%s: %s means something else:\n--- go\n%s\n--- typescript\n%s", label, file, got[file], want[file])
}
}
}
same("with an API key", Render(f.Facts, f.Settings, &Binding{Licence: "api", Kind: "api-key"}, "/state/api-key-helper", f.Registered), f.WithKey)
same("plain", Render(f.Facts, Settings{}, nil, "/h", Servers{}), f.Plain)
}
func TestASettingCannotReplaceTheMeshsOwnEntryAndABadNameIsLeftOut(t *testing.T) {
out := Render(Facts{Node: "w", Console: "http://127.0.0.1:4270/mcp"},
Settings{MCPServers: map[string]map[string]any{"mesh": {"type": "http", "url": "http://evil"}, "bad name": {}}}, nil, "/h", nil)
var mcp struct {
MCPServers map[string]map[string]any `json:"mcpServers"`
}
_ = json.Unmarshal([]byte(out["managed-mcp.json"]), &mcp)
if mcp.MCPServers["mesh"]["url"] != "http://127.0.0.1:4270/mcp" || mcp.MCPServers["bad name"] != nil {
t.Fatalf("%v", mcp.MCPServers)
}
if !reflect.DeepEqual(Render(Facts{Console: "x"}, Settings{}, nil, "/h", nil), Render(Facts{Console: "x"}, Settings{}, nil, "/h", nil)) {
t.Fatal("rendering is not deterministic")
}
}
// ---- the credentials file -----------------------------------------------------------------------------
func i64(v int64) *int64 { return &v }
func TestTheLineageRules(t *testing.T) {
const hour = 3_600_000
g := func(at string, exp int64, rtExp int64) Grant {
return Grant{AccessToken: at, ExpiresAt: exp, RefreshTokenExpiresAt: i64(rtExp)}
}
month := now + 30*24*hour
if d := DecideApply(&Grant{AccessToken: "A", ExpiresAt: now + hour, RefreshTokenExpiresAt: i64(month)}, g("B", now+2*hour, month), false); !d.Apply {
t.Fatal("a newer rotation was refused")
}
if d := DecideApply(&Grant{AccessToken: "new", ExpiresAt: now + 2*hour, RefreshTokenExpiresAt: i64(month)}, g("old", now+hour, month), false); d.Apply || d.Reason != "not-newer" {
t.Fatalf("a late older rotation: %+v", d)
}
if d := DecideApply(&Grant{AccessToken: "A", ExpiresAt: now + 8*hour, RefreshTokenExpiresAt: i64(month)}, g("re", now+hour, now+5*24*hour), false); !d.Apply || !d.Reissued {
t.Fatalf("a re-issued grant: %+v", d)
}
if d := DecideApply(&Grant{AccessToken: "A", ExpiresAt: now + 8*hour}, g("other", now+hour, month), true); !d.Apply {
t.Fatal("a switch was refused")
}
if d := DecideApply(&Grant{AccessToken: "A"}, Grant{AccessToken: "A"}, true); d.Apply || d.Reason != "already-current" {
t.Fatalf("the same token: %+v", d)
}
}
func TestALoginIsSeenAndStrippedWhenTheNodesOwnGrantIsWritten(t *testing.T) {
p, _ := node(t, "laptop")
writeFile(t, p.credentials(), `{"claudeAiOauth":{"accessToken":"at-login","refreshToken":"rt-login","expiresAt":1700000000000},"other":1}`)
login := ReadCredentials(p.credentials())
if !HoldsLogin(login) {
t.Fatal("a login was not seen")
}
if err := WriteCredentials(p.credentials(), WithGrant(login, Grant{AccessToken: "at-mesh", ExpiresAt: 1, Scopes: []string{"user:inference"}})); err != nil {
t.Fatal(err)
}
back := ReadCredentials(p.credentials())
raw, _ := os.ReadFile(p.credentials())
info, _ := os.Stat(p.credentials())
if HoldsLogin(back) || GrantOf(back).AccessToken != "at-mesh" || back["other"] == nil || strings.Contains(string(raw), "rt-login") || info.Mode().Perm() != 0o600 {
t.Fatalf("written %s (mode %v)", raw, info.Mode())
}
}
func TestTheAccountIsReadFromTheAgentsStateFileAndNeverGuessed(t *testing.T) {
p, _ := node(t, "laptop")
writeFile(t, p.account(), `{"oauthAccount":{"accountUuid":"u-1","emailAddress":"a@example.org"},"other":2}`)
if id := ReadIdentity(p.account()); id == nil || id.AccountUUID != "u-1" || id.EmailAddress != "a@example.org" {
t.Fatalf("%+v", id)
}
if ReadIdentity("/nonexistent/.claude.json") != nil {
t.Fatal("an identity from nothing")
}
writeFile(t, p.account(), `{}`)
if ReadIdentity(p.account()) != nil {
t.Fatal("an identity from an empty file")
}
}
// ---- the licence, ADR 0206 ----------------------------------------------------------------------------
func TestWhatANodeHoldsIsReportedWithFingerprintsAndItsAccountNeverAToken(t *testing.T) {
p, _ := node(t, "laptop")
writeFile(t, p.credentials(), `{"claudeAiOauth":{"accessToken":"at-secret","refreshToken":"rt-secret","expiresAt":2000,"refreshTokenExpiresAt":9000}}`)
writeFile(t, p.account(), `{"oauthAccount":{"accountUuid":"u-1","emailAddress":"a@example.org"}}`)
h := HoldingsOf(p)
if h.Node != "laptop" || h.Identity.AccountUUID != "u-1" || *h.Kind != "subscription" || !h.Refresh.Present ||
!strings.HasPrefix(*h.Refresh.Fingerprint, "sha256:") || h.Access.ExpiresAt != 2000 || h.ChangedAt == nil {
t.Fatalf("%+v", h)
}
raw, _ := json.Marshal(h)
if strings.Contains(string(raw), "at-secret") || strings.Contains(string(raw), "rt-secret") {
t.Fatalf("a token is in the report: %s", raw)
}
var keys map[string]any
_ = json.Unmarshal(raw, &keys)
for k := range keys {
if strings.Contains(strings.ToLower(k), "token") || strings.Contains(strings.ToLower(k), "secret") {
t.Fatalf("a field the runtime would refuse: %s", k)
}
}
// The manager reads exactly this shape.
if _, err := time.Parse(time.RFC3339Nano, *h.ChangedAt); err != nil {
t.Fatalf("the manager cannot read the report's time: %v", err)
}
}
func TestTheGrantAnswersOnlyAWaitingLoginSealedToTheManagersKey(t *testing.T) {
p, _ := node(t, "laptop")
manager, _ := GenerateKeyPair()
if a, _ := GrantFor(p, manager.PublicKey); a.Sealed != nil || a.Waiting == nil || *a.Waiting {
t.Fatalf("%+v", a)
}
writeFile(t, p.credentials(), `{"claudeAiOauth":{"accessToken":"at","refreshToken":"rt-login","expiresAt":1}}`)
writeFile(t, p.account(), `{"oauthAccount":{"accountUuid":"u-9"}}`)
a, err := GrantFor(p, manager.PublicKey)
if err != nil || a.Identity.AccountUUID != "u-9" {
t.Fatalf("%+v %v", a, err)
}
plain, _ := Open(*a.Sealed, manager.PrivateKey)
if !strings.Contains(plain, `"refreshToken":"rt-login"`) {
t.Fatalf("opened %s", plain)
}
raw, _ := json.Marshal(a)
if strings.Contains(string(raw), "rt-login") {
t.Fatal("the refresh token crossed in the clear")
}
}
// seat answers `current` as the manager does: the grant sealed to the key the node sent.
func seat(t *testing.T, licence, token string, gen int64, asked *[]string) Ask {
return func(address string, args any) (json.RawMessage, error) {
*asked = append(*asked, address)
key := args.(map[string]any)["public_key"].(string)
g, _ := json.Marshal(Grant{AccessToken: token, ExpiresAt: now + 3_600_000})
box, err := Seal(string(g), key)
if err != nil {
t.Fatal(err)
}
return json.Marshal(Current{Licence: licence, Kind: "subscription", Generation: gen, Sealed: &box})
}
}
func TestANewerGenerationFetchesTheTokenOnceByTheSeatsVerb(t *testing.T) {
p, w := node(t, "laptop")
var asked []string
ask := seat(t, "personal", "at-1", 3, &asked)
if _, err := OnBinding(p, &BindingState{Licence: "personal", Kind: "subscription", Generation: 3}, ask, writer(w)); err != nil {
t.Fatal(err)
}
if len(asked) != 1 || asked[0] != "seat:anthropic-licence-manager.current" || creds(t, p)["accessToken"] != "at-1" {
t.Fatalf("asked %v, credentials %v", asked, creds(t, p))
}
if done, _ := OnBinding(p, &BindingState{Licence: "personal", Kind: "subscription", Generation: 3}, ask, writer(w)); done != "" || len(asked) != 1 {
t.Fatal("an equal generation asked again")
}
if HoldingsOf(p).Generation != 3 || w["managed-mcp.json"] == "" {
t.Fatal("the generation or the managed files were not written")
}
}
func TestTheTokenANodeIsHandedReplacesALoginsGrantAndLeavesNoRefreshToken(t *testing.T) {
p, w := node(t, "laptop")
writeFile(t, p.credentials(), `{"claudeAiOauth":{"accessToken":"at-old","refreshToken":"rt-spent","expiresAt":`+
strings.TrimSpace(string(mustJSON(now+7_200_000)))+`}}`)
var asked []string
out, err := Pull(p, seat(t, "personal", "at-new", 1, &asked), writer(w))
if err != nil || out["applied"] != true {
t.Fatalf("%v %v", out, err)
}
c := creds(t, p)
if c["accessToken"] != "at-new" || c["refreshToken"] != nil || HoldingsOf(p).Refresh.Present {
t.Fatalf("%v", c)
}
}
func mustJSON(v any) []byte { b, _ := json.Marshal(v); return b }
// ---- MCP servers in state, ADR 0201 -------------------------------------------------------------------
// bus is the `servers` state as every node in a test shares it, with each node's watch.
type bus struct {
kept map[string]map[string]any
watchers []func(ServerChange)
}
func (b *bus) Put(key string, value any) error {
v := value.(map[string]any)
b.kept[key] = v
for _, w := range b.watchers {
w(ServerChange{Key: key, Op: "put", Value: v})
}
return nil
}
func (b *bus) Delete(key string) error {
delete(b.kept, key)
for _, w := range b.watchers {
w(ServerChange{Key: key, Op: "delete"})
}
return nil
}
func (b *bus) Keys() ([]string, error) {
var out []string
for k := range b.kept {
out = append(out, k)
}
return out, nil
}
// join is a node joining: its view takes the current state, then every change.
func (b *bus) join(p Paths, w map[string]string) *ServerView {
v := NewServerView(p)
for k, val := range b.kept {
_, _ = OnServerChange(v, ServerChange{Key: k, Op: "put", Value: val}, p, writer(w))
}
b.watchers = append(b.watchers, func(c ServerChange) { _, _ = OnServerChange(v, c, p, writer(w)) })
return v
}
func noOthers() ([]string, error) { return nil, nil }
func TestRegisteringHerePutsItUnderThisNodesKeyAndAsksAboutTheOthers(t *testing.T) {
p, w := node(t, "laptop")
b := &bus{kept: map[string]map[string]any{}}
v := b.join(p, w)
r, err := RegisterServer(p, Registration{Name: "search", Entry: map[string]any{"type": "http", "url": "https://s.example/mcp"}}, b, v, writer(w),
func() ([]string, error) { return []string{"laptop", "server", "desktop"}, nil })
if err != nil || r["here"] != "changed" || !strings.Contains(r["also"].(string), "server, desktop") || b.kept["laptop.search"] == nil {
t.Fatalf("%v %v %v", r, err, b.kept)
}
if !strings.Contains(w["managed-mcp.json"], `"search"`) {
t.Fatal("not rendered")
}
}
func TestEveryNodeRegistrationReachesTheOthersAndALateNodeReadsIt(t *testing.T) {
a, wa := node(t, "laptop")
s, ws := node(t, "server")
b := &bus{kept: map[string]map[string]any{}}
va := b.join(a, wa)
b.join(s, ws)
_, _ = RegisterServer(a, Registration{Name: "docs", Entry: map[string]any{"type": "stdio", "command": "docs-mcp"}, Nodes: []string{"all"}}, b, va, writer(wa), noOthers)
if Registered(s)["docs"] == nil || !strings.Contains(ws["managed-mcp.json"], "docs-mcp") {
t.Fatalf("the other node did not take it: %v", Registered(s))
}
late, wl := node(t, "desktop")
b.join(late, wl)
if Registered(late)["docs"] == nil {
t.Fatal("a node joining later did not read the current set")
}
_, _ = RegisterServer(a, Registration{Name: "docs", Nodes: []string{"all"}}, b, va, writer(wa), noOthers)
if Registered(s)["docs"] != nil || Registered(late)["docs"] != nil {
t.Fatal("an unregistration did not reach every node")
}
}
func TestANodesOwnRegistrationOverridesTheOneForEveryNode(t *testing.T) {
a, wa := node(t, "laptop")
s, ws := node(t, "server")
b := &bus{kept: map[string]map[string]any{}}
va := b.join(a, wa)
b.join(s, ws)
_, _ = RegisterServer(a, Registration{Name: "x", Entry: map[string]any{"type": "http", "url": "https://all"}, Nodes: []string{"all"}}, b, va, writer(wa), noOthers)
_, _ = RegisterServer(a, Registration{Name: "x", Entry: map[string]any{"type": "http", "url": "https://laptop"}}, b, va, writer(wa), noOthers)
if Registered(a)["x"]["url"] != "https://laptop" || Registered(s)["x"]["url"] != "https://all" {
t.Fatalf("%v %v", Registered(a), Registered(s))
}
r, _ := RegisterServer(a, Registration{Name: "x"}, b, va, writer(wa), noOthers)
if !strings.Contains(r["still"].(string), "still applies here") || Registered(a)["x"]["url"] != "https://all" {
t.Fatalf("%v", r)
}
}
func TestABadEntryIsRefusedBeforeAnythingIsPut(t *testing.T) {
p, w := node(t, "laptop")
b := &bus{kept: map[string]map[string]any{}}
v := b.join(p, w)
r, _ := RegisterServer(p, Registration{Name: "mesh", Entry: map[string]any{"type": "http", "url": "https://x"}}, b, v, writer(w), noOthers)
if r["registered"] != false || len(b.kept) != 0 {
t.Fatalf("%v %v", r, b.kept)
}
if done, _ := OnServerChange(v, ServerChange{Key: "server.b", Op: "put", Value: map[string]any{"type": "http", "url": "https://b"}}, p, writer(w)); done != "" {
t.Fatal("another node's key changed this one")
}
}
@@ -1,198 +0,0 @@
package main
// The agent's credentials file, and whether an offered grant may replace what it holds (novox/hq ADR 0183,
// ADR 0206, design 36 §5). Pure where it decides, so the rules are tested without a file.
//
// The file is the vendor's: `{ claudeAiOauth: { accessToken, expiresAt, refreshTokenExpiresAt?, scopes?,
// subscriptionType?, rateLimitTier? }, ... }`. A node bound to a licence never holds a refresh token, so
// the one this module writes never carries one; a refresh token found there is a person's login.
//
// The lineage rule is the predecessor's, with the incidents that earned it: a rotation of the same licence
// is applied only if newer; a grant re-issued by a login is adopted whatever its expiry; a switch to another
// licence is applied regardless, because across licences the expiries are unrelated numbers.
import (
"bytes"
"encoding/json"
"math"
"os"
"path/filepath"
)
// Grant is what the manager hands a node: an access token and its expiries, never a refresh token.
type Grant struct {
AccessToken string `json:"accessToken"`
ExpiresAt int64 `json:"expiresAt"`
RefreshTokenExpiresAt *int64 `json:"refreshTokenExpiresAt,omitempty"`
Scopes []string `json:"scopes,omitempty"`
SubscriptionType string `json:"subscriptionType,omitempty"`
RateLimitTier string `json:"rateLimitTier,omitempty"`
}
// Decision is whether a handed grant is applied, and why not.
type Decision struct {
Apply bool
Reissued bool
Reason string // already-current | not-newer
}
// generationTolerance: two refresh-token expiries within a day are one lineage; a login starts a fresh
// window weeks away.
const generationTolerance = 24 * 60 * 60 * 1000
func sameGeneration(a, b *int64) bool {
if a == nil || b == nil {
return true
}
return math.Abs(float64(*a-*b)) <= generationTolerance
}
// DecideApply says whether an offered grant replaces the one held; switch is a move to another licence.
func DecideApply(local *Grant, offered Grant, switching bool) Decision {
if local == nil || local.AccessToken == "" {
return Decision{Apply: true}
}
if local.AccessToken == offered.AccessToken {
return Decision{Reason: "already-current"}
}
reissued := !sameGeneration(local.RefreshTokenExpiresAt, offered.RefreshTokenExpiresAt)
if !switching && !reissued && local.ExpiresAt >= offered.ExpiresAt {
return Decision{Reason: "not-newer"}
}
return Decision{Apply: true, Reissued: reissued}
}
// Credentials is the file as found, every key kept — the vendor's other keys are not this module's.
type Credentials map[string]any
func (c Credentials) oauth() map[string]any {
o, _ := c["claudeAiOauth"].(map[string]any)
return o
}
// ReadCredentials reads the file, keeping numbers as written; nil when there is none.
func ReadCredentials(path string) Credentials {
raw, err := os.ReadFile(path)
if err != nil {
return nil
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var c Credentials
if dec.Decode(&c) != nil {
return nil
}
return c
}
func number(v any) (int64, bool) {
switch n := v.(type) {
case json.Number:
i, err := n.Int64()
if err != nil {
f, err := n.Float64()
return int64(f), err == nil
}
return i, true
case float64:
return int64(n), true
case int64:
return n, true
}
return 0, false
}
// GrantOf is the grant the file holds, or nil.
func GrantOf(c Credentials) *Grant {
o := c.oauth()
at, _ := o["accessToken"].(string)
if at == "" {
return nil
}
g := &Grant{AccessToken: at}
g.ExpiresAt, _ = number(o["expiresAt"])
if v, ok := number(o["refreshTokenExpiresAt"]); ok {
g.RefreshTokenExpiresAt = &v
}
return g
}
// HoldsLogin says the file holds a refresh token — which this module never writes, so a person's login.
func HoldsLogin(c Credentials) bool {
rt, _ := c.oauth()["refreshToken"].(string)
return rt != ""
}
// RefreshTokenOf is the refresh token a login left, or "".
func RefreshTokenOf(c Credentials) string {
rt, _ := c.oauth()["refreshToken"].(string)
return rt
}
// WithGrant lays the handed grant over what is there, and deletes any refresh token.
func WithGrant(local Credentials, g Grant) Credentials {
next := Credentials{}
for k, v := range local {
next[k] = v
}
oauth := map[string]any{}
for k, v := range local.oauth() {
oauth[k] = v
}
oauth["accessToken"] = g.AccessToken
oauth["expiresAt"] = g.ExpiresAt
if g.RefreshTokenExpiresAt != nil {
oauth["refreshTokenExpiresAt"] = *g.RefreshTokenExpiresAt
}
if len(g.Scopes) > 0 {
oauth["scopes"] = g.Scopes
}
if g.SubscriptionType != "" {
oauth["subscriptionType"] = g.SubscriptionType
}
if g.RateLimitTier != "" {
oauth["rateLimitTier"] = g.RateLimitTier
}
delete(oauth, "refreshToken")
next["claudeAiOauth"] = oauth
return next
}
// ReplacedBy is the handed grant in place of the old licence's, whole — scopes and subscription included;
// only keys outside the grant stay. No refresh token survives.
func ReplacedBy(local Credentials, g Grant) Credentials {
next := Credentials{}
for k, v := range local {
if k != "claudeAiOauth" {
next[k] = v
}
}
return WithGrant(next, g)
}
// WriteCredentials writes atomically at 0600: a partial credentials file must never be read as a whole one.
func WriteCredentials(path string, c Credentials) error {
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return err
}
raw, err := indented(c)
if err != nil {
return err
}
tmp := path + ".mesh-tmp"
if err := os.WriteFile(tmp, raw, 0o600); err != nil {
return err
}
return os.Rename(tmp, path)
}
// indented is JSON as the agent's own files are written: two-space indent, a trailing newline, nothing
// escaped that need not be.
func indented(v any) ([]byte, error) {
var b bytes.Buffer
enc := json.NewEncoder(&b)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
err := enc.Encode(v)
return b.Bytes(), err
}
@@ -1,84 +0,0 @@
package main
// Which account the agent is logged in as (novox/hq ADR 0183): not in the token, but in the agent's own
// state file beside the home, `~/.claude.json` → `oauthAccount`. Read to report and attribute a login;
// written, three keys and nothing else, when a licence is switched, so the account Claude Code shows is
// the one whose token it now holds.
import (
"bytes"
"encoding/json"
"os"
)
// Identity is an account as the agent's state file names it.
type Identity struct {
AccountUUID string `json:"accountUuid"`
EmailAddress string `json:"emailAddress,omitempty"`
OrganizationUUID string `json:"organizationUuid,omitempty"`
}
func readState(path string) (map[string]any, bool) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, false
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var m map[string]any
if dec.Decode(&m) != nil || m == nil {
return nil, false
}
return m, true
}
// ReadIdentity is the account the state file names, or nil — never a guess.
func ReadIdentity(path string) *Identity {
m, ok := readState(path)
if !ok {
return nil
}
a, _ := m["oauthAccount"].(map[string]any)
uuid, _ := a["accountUuid"].(string)
if uuid == "" {
return nil
}
id := &Identity{AccountUUID: uuid}
id.EmailAddress, _ = a["emailAddress"].(string)
id.OrganizationUUID, _ = a["organizationUuid"].(string)
return id
}
// WriteIdentity points the state file's account at id, keeping every other key as found; answers whether
// the file changed. A file that is there and cannot be read as an object is left alone.
func WriteIdentity(path string, id Identity) (bool, error) {
m, ok := readState(path)
if !ok {
if _, err := os.Stat(path); err == nil {
return false, nil
}
m = map[string]any{}
}
current, _ := m["oauthAccount"].(map[string]any)
if current == nil {
current = map[string]any{}
}
e, _ := current["emailAddress"].(string)
o, _ := current["organizationUuid"].(string)
if current["accountUuid"] == id.AccountUUID && e == id.EmailAddress && o == id.OrganizationUUID {
return false, nil
}
current["accountUuid"] = id.AccountUUID
current["emailAddress"] = id.EmailAddress
current["organizationUuid"] = id.OrganizationUUID
m["oauthAccount"] = current
raw, err := indented(m)
if err != nil {
return false, err
}
tmp := path + ".mesh-tmp"
if err := os.WriteFile(tmp, raw, 0o600); err != nil {
return false, err
}
return true, os.Rename(tmp, path)
}
@@ -1,12 +0,0 @@
package main
// instructionsText is the managed instruction file, generated from the TypeScript renderer it replaced so
// the file under the agent's managed directory did not change by a byte when the module moved to Go;
// a test holds it to that renderer's own output (testdata/rendered-by-typescript.json).
func instructionsText(node, role string) string {
return "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `" +
node +
"`\n- **Role:** " +
role +
"\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
}
-365
View File
@@ -1,365 +0,0 @@
// claude-code's bundle (novox/hq design 36, ADR 0183, ADR 0206): a binary the node's runtime launches over
// stdio as the operator account (ADR 0193) and is the bus for (ADR 0198). It is given its state directory
// and two files the mesh renders into it (ADR 0192), beside the runtime's own words.
//
// At start it renders the agent's managed directory, reports what this node holds as the module's
// `holdings` state and again whenever the credentials file changes, watches the licence manager's
// `bindings` state for this node and fetches the token when it says so, and watches the module's
// `servers` state — every node's MCP server registrations (ADR 0201). node.go holds the logic.
//
// stdout is the MCP channel; everything this module says, it says on stderr.
package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
func say(format string, args ...any) {
fmt.Fprintf(os.Stderr, "[claude-code] "+format+"\n", args...)
}
// writeManaged writes one managed file as root, only when its content changed. From a staged file, never
// /dev/stdin: a child's input may be a socket, which /dev/stdin cannot open (found on the first assignment).
func writeManaged(name, content string) (string, error) {
path := filepath.Join(ManagedDir, name)
if was, err := os.ReadFile(path); err == nil && string(was) == content {
return name + ": unchanged", nil
}
staged, err := os.MkdirTemp("", "claude-code-")
if err != nil {
return "", err
}
defer os.RemoveAll(staged)
source := filepath.Join(staged, name)
if err := os.WriteFile(source, []byte(content), 0o644); err != nil {
return "", err
}
args := []string{"install", "-D", "-m", "0644", source, path}
if os.Geteuid() != 0 {
args = append([]string{"sudo", "-n"}, args...)
}
if out, err := exec.Command(args[0], args[1:]...).CombinedOutput(); err != nil {
return "", fmt.Errorf("%s: could not be written to %s (%s); the module writes there through the operator account's passwordless sudo",
name, ManagedDir, strings.TrimSpace(string(out)))
}
return name + ": written", nil
}
// ask is a tool on the bus, through the runtime: its answer is the tool's value.
func ask(address string, args any) (json.RawMessage, error) { return stdio.Ask(address, args) }
// stateOf adapts the SDK's state to what node.go asks of one.
type stateOf struct{ s stdio.KeptState }
func (s stateOf) Put(key string, value any) error { _, err := s.s.Put(key, value); return err }
func (s stateOf) Delete(key string) error { return s.s.Delete(key) }
func (s stateOf) Keys() ([]string, error) { return s.s.Keys() }
// nodesRunningMe is the nodes claude-code runs on, from the controller's list of modules — for the register
// tool's question.
func nodesRunningMe() ([]string, error) {
raw, err := ask("seat:mesh-controller.modules", map[string]any{})
if err != nil {
return nil, err
}
var answer struct {
Output string `json:"output"`
}
text := string(raw)
if json.Unmarshal(raw, &answer) == nil && answer.Output != "" {
text = answer.Output
}
for _, line := range strings.Split(text, "\n") {
if !strings.HasPrefix(line, "claude-code ") {
continue
}
_, on, ok := strings.Cut(line, " on ")
if !ok || strings.TrimSpace(on) == "nothing" {
return nil, nil
}
var out []string
for _, n := range strings.Split(on, ",") {
if n = strings.TrimSpace(n); n != "" {
out = append(out, n)
}
}
return out, nil
}
return nil, nil
}
func fingerprintOfFile(path string) any {
raw, err := os.ReadFile(path)
if err != nil {
return nil
}
return Fingerprint(string(raw))
}
func status(p Paths) map[string]any {
creds := ReadCredentials(p.credentials())
var token any
if g := GrantOf(creds); g != nil {
token = map[string]any{"fingerprint": Fingerprint(g.AccessToken), "expiresAt": stamp(g.ExpiresAt), "loginWaiting": HoldsLogin(creds)}
}
var managed []map[string]any
for _, f := range []string{"managed-mcp.json", "managed-settings.json", "CLAUDE.md"} {
path := filepath.Join(ManagedDir, f)
managed = append(managed, map[string]any{"file": path, "fingerprint": fingerprintOfFile(path)})
}
var licence any
var b Binding
if readJSON(p.binding(), &b) {
licence = b
}
names := []string{}
for n := range Registered(p) {
names = append(names, n)
}
return map[string]any{"node": p.Node, "licence": licence, "token": token, "holdings": HoldingsOf(p),
"managed": managed, "registered": names}
}
func str(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
// nodesOf reads the tools' `nodes` argument: absent is this node, "all" every node, else a list.
func nodesOf(v any) []string {
s, _ := v.(string)
s = strings.TrimSpace(s)
switch s {
case "":
return nil
case "all":
return []string{"all"}
}
var out []string
for _, n := range strings.Split(s, ",") {
if n = strings.TrimSpace(n); n != "" {
out = append(out, n)
}
}
return out
}
func tools(p Paths, servers ServerState, view *ServerView) []stdio.Tool {
nodesArg := str(`more nodes: "all" for every node running claude-code, or a comma-separated list; absent is this node only`)
return []stdio.Tool{
{Name: "claude_code_status",
Description: "Claude Code on this machine as the mesh configured it: the licence it holds and when its token expires, what it reports holding, the managed files, the MCP servers registered here. Fingerprints only, never a token.",
Run: func(map[string]any) (any, error) { return status(p), nil }},
{Name: "claude_code_render",
Description: "Write Claude Code's managed directory now, from the mesh's facts, this module's settings and the servers registered here.",
Run: func(map[string]any) (any, error) {
out, err := RenderNow(p, writeManaged)
return map[string]any{"rendered": out}, err
}},
{Name: "claude_code_pull",
Description: "Ask the licence manager for this node's current token now and apply it, rather than waiting for its binding to change.",
Run: func(map[string]any) (any, error) { return Pull(p, ask, writeManaged) }},
{Name: "claude_code_grant",
Description: "For the licence manager (ADR 0206): the full grant in this node's credentials file — a login made here — sealed to the public key given, with the account it belongs to. Nothing when no login is waiting. Never answers a token in the clear.",
Input: map[string]any{"public_key": str("the manager's public key, PEM; the grant opens only with its private half")},
Run: func(a map[string]any) (any, error) {
key, _ := a["public_key"].(string)
if !strings.Contains(key, "PUBLIC KEY") {
return nil, errors.New("claude_code_grant seals to a public key, and none was given")
}
return GrantFor(p, key)
}},
{Name: "claude_code_mcp_list",
Description: "The MCP servers registered through this module: those that apply on this node (beside the console, `mesh`, and those set in the module's settings), and every registration on the mesh, by key — `all.<server>` for every node, `<node>.<server>` for one.",
Run: func(map[string]any) (any, error) {
keys, err := servers.Keys()
return map[string]any{"here": Registered(p), "everywhere": keys}, err
}},
{Name: "claude_code_mcp_register",
Description: "Register an MCP server with Claude Code on this node, every node, or a list — an http/sse server by url, or a stdio server by command. Kept on the bus, so a node that joins later takes it too. Never put a secret in env or headers: the mesh refuses one.",
Input: map[string]any{
"name": str("the server's name: letters, digits, - and _"),
"type": str("http, sse or stdio (default stdio when a command is given, http when a url is)"),
"url": str("an http or sse server's url"),
"command": str("a stdio server's program"),
"args": map[string]any{"type": "array", "description": "a stdio server's arguments"},
"env": map[string]any{"type": "object", "description": "a stdio server's environment"},
"headers": map[string]any{"type": "object", "description": "an http server's headers"},
"nodes": nodesArg,
},
Run: func(a map[string]any) (any, error) {
entry := map[string]any{}
if t, _ := a["type"].(string); t != "" {
entry["type"] = t
} else if _, hasURL := a["url"]; hasURL {
entry["type"] = "http"
} else {
entry["type"] = "stdio"
}
for _, k := range []string{"url", "command", "args", "env", "headers"} {
if v, ok := a[k]; ok {
entry[k] = v
}
}
name, _ := a["name"].(string)
return RegisterServer(p, Registration{Name: name, Entry: entry, Nodes: nodesOf(a["nodes"])}, servers, view, writeManaged, nodesRunningMe)
}},
{Name: "claude_code_mcp_unregister",
Description: "Remove an MCP server registered through this module, on this node or more.",
Input: map[string]any{"name": str("the server's name"), "nodes": nodesArg},
Run: func(a map[string]any) (any, error) {
name, _ := a["name"].(string)
return RegisterServer(p, Registration{Name: name, Nodes: nodesOf(a["nodes"])}, servers, view, writeManaged, nodesRunningMe)
}},
}
}
// persist asks the state again until it answers: its bucket or the bus's grant may arrive after the module.
func persist(what string, attempt func() error, done func(refusals int)) {
waits := []time.Duration{2 * time.Second, 5 * time.Second, 10 * time.Second, 30 * time.Second}
for n := 0; ; n++ {
err := attempt()
if err == nil {
done(n)
return
}
pause := time.Minute
if n < len(waits) {
pause = waits[n]
}
say("%s not yet (%v); asking again in %s", what, err, pause)
time.Sleep(pause)
}
}
func main() {
p, launched := PathsFrom(os.Getenv)
if !launched {
// Outside a launch — a build, a check — it serves nothing and says why.
say("not launched by the runtime with this module's words; serving no tools")
if err := stdio.Serve("", nil); err != nil {
os.Exit(1)
}
return
}
if _, err := Keypair(p); err != nil {
say("this module's key: %v", err)
}
if out, err := RenderNow(p, writeManaged); err != nil {
say("%v", err)
} else {
for _, line := range out {
if !strings.HasSuffix(line, "unchanged") {
say("%s", line)
}
}
}
servers := stateOf{stdio.State("servers")}
view := NewServerView(p)
go run(p, view)
if err := stdio.Serve("", tools(p, servers, view)); err != nil {
say("%v", err)
os.Exit(1)
}
}
// run is the module's long-running half, beside the tools (ADR 0198).
func run(p Paths, view *ServerView) {
// Every node's MCP servers: the whole current set first, then each change (ADR 0201).
go persist("watching the MCP servers", func() error {
return stdio.State("servers").Watch("", func(c stdio.StateChange) error {
var value map[string]any
_ = json.Unmarshal(c.Value, &value)
if done, err := OnServerChange(view, ServerChange{Key: c.Key, Op: c.Op, Value: value}, p, writeManaged); err != nil {
say("taking %s %s: %v", c.Op, c.Key, err) // the view took it; the next render writes it
} else if done != "" {
say("%s", done)
}
return nil
})
}, func(n int) { say("watching the MCP servers%s", refusals(n)) })
// What this node holds (ADR 0206): at start — a node already logged in is reported at once — and on
// every change of the credentials file, polled, because the file is replaced by rename and a watch on
// the old inode would go quiet. Fingerprints and expiries only.
holdings := stdio.State("holdings")
reported := ""
report := func() {
now := HoldingsOf(p)
raw, _ := json.Marshal(now)
if string(raw) == reported {
return
}
persist("reporting what this node holds", func() error { _, err := holdings.Put(p.Node, now); return err }, func(int) {
reported = string(raw)
account := "no account"
if now.Identity != nil && now.Identity.EmailAddress != "" {
account = now.Identity.EmailAddress
}
line := "reported: " + account
if now.Kind != nil {
line += ", " + *now.Kind
}
if now.Refresh.Present {
line += ", a login waiting"
}
if now.Licence != nil {
line += fmt.Sprintf(", licence %s g%d", *now.Licence, now.Generation)
}
say("%s", line)
})
}
// What this node should hold (ADR 0206): the manager's `bindings` key for this node; a newer
// generation is fetched with the seat's `current`, sealed to this module's key.
go persist("watching this node's licence binding", func() error {
return stdio.State(Manager+".bindings").Watch(p.Node, func(c stdio.StateChange) error {
if c.Key != p.Node {
return nil
}
var b *BindingState
if c.Op == "put" {
b = &BindingState{}
if err := json.Unmarshal(c.Value, b); err != nil {
return nil
}
}
if done, err := OnBinding(p, b, ask, writeManaged); err != nil {
say("fetching this node's token failed: %v", err)
} else if done != "" {
say("%s", done)
}
go report()
return nil
})
}, func(n int) { say("watching this node's licence binding%s", refusals(n)) })
report()
var last string
for range time.Tick(5 * time.Second) {
info, err := os.Stat(p.credentials())
now := "absent"
if err == nil {
now = fmt.Sprintf("%d/%d", info.ModTime().UnixNano(), info.Size())
}
if now != last {
last = now
report()
}
}
}
func refusals(n int) string {
if n == 0 {
return ""
}
return fmt.Sprintf(" (after %d refusal(s))", n)
}
-529
View File
@@ -1,529 +0,0 @@
package main
// What claude-code does on a node, written against what it is handed — a way to ask a tool on the bus, its
// own state, a way to write a managed file — so every path is tested without a bus (novox/hq design 36,
// ADR 0183, ADR 0201, ADR 0206).
//
// Over NATS, and nothing an event: what is current is state, and a secret only ever travels on a request,
// sealed to its one recipient.
// - What this node holds is the module's `holdings` state, one key per node: the account, the kind,
// fingerprints and expiries — never a token. Written at start and on every change of the credentials
// file, so the licence manager learns a login, or a node already logged in, from the state alone.
// - The grant itself leaves only when the manager asks `claude_code_grant`, sealed to the key it gives.
// - What this node should hold is the manager's `bindings` state; a newer generation for this node is
// fetched with the seat's `current` verb, sealed to this module's key, and written access-token-only.
// - An MCP server registered through this module is a key in its `servers` state — `all.<server>` for
// every node, `<node>.<server>` for one — which every node watches.
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"time"
)
// Seat is the licence manager's role, and Manager the module whose `bindings` state this one reads.
const (
Seat = "anthropic-licence-manager"
Manager = "claude-licence-manager"
)
// SeatVerb is a seat's verb as the runtime addresses it: a role, not a module.
func SeatVerb(verb string) string { return "seat:" + Seat + "." + verb }
// Fingerprint names a token without being one: the first 16 hex of its SHA-256, as the manager computes it.
func Fingerprint(s string) string {
sum := sha256.Sum256([]byte(s))
return "sha256:" + hex.EncodeToString(sum[:])[:16]
}
// Paths are where this node's files are, from the module's words (ADR 0192).
type Paths struct {
State, Facts, Settings, Home, Node string
}
// PathsFrom reads them, or answers false outside a launch.
func PathsFrom(env func(string) string) (Paths, bool) {
p := Paths{State: env("MESH_CLAUDE_CODE_STATE"), Facts: env("MESH_CLAUDE_CODE_FACTS"),
Settings: env("MESH_CLAUDE_CODE_SETTINGS"), Home: env("MESH_OPERATOR_HOME"), Node: env("MESH_NODE")}
return p, p.State != "" && p.Facts != "" && p.Settings != "" && p.Home != "" && p.Node != ""
}
func (p Paths) credentials() string { return filepath.Join(p.Home, ".claude", ".credentials.json") }
func (p Paths) account() string { return filepath.Join(p.Home, ".claude.json") }
func (p Paths) binding() string { return filepath.Join(p.State, "licence.json") }
func (p Paths) apiKey() string { return filepath.Join(p.State, "api-key") }
func (p Paths) helper() string { return filepath.Join(p.State, "api-key-helper") }
func (p Paths) registry() string { return filepath.Join(p.State, "mcp-servers.json") }
// Ask is a tool on the bus: its address and arguments in, its JSON answer out.
type Ask func(address string, args any) (json.RawMessage, error)
// WriteManaged writes one managed file and answers what happened.
type WriteManaged func(name, content string) (string, error)
func readJSON(path string, into any) bool {
raw, err := os.ReadFile(path)
return err == nil && json.Unmarshal(raw, into) == nil
}
// Keypair is this module's own, made once in its state; the TypeScript module's files are kept, so a node
// moving to this binary keeps the key it had.
func Keypair(p Paths) (KeyPair, error) {
priv, pub := filepath.Join(p.State, "key.pem"), filepath.Join(p.State, "key.pub.pem")
if _, err := os.Stat(priv); errors.Is(err, os.ErrNotExist) {
k, err := GenerateKeyPair()
if err != nil {
return KeyPair{}, err
}
if err := os.WriteFile(priv, []byte(k.PrivateKey), 0o600); err != nil {
return KeyPair{}, err
}
if err := os.WriteFile(pub, []byte(k.PublicKey), 0o644); err != nil {
return KeyPair{}, err
}
}
a, err1 := os.ReadFile(priv)
b, err2 := os.ReadFile(pub)
return KeyPair{PrivateKey: string(a), PublicKey: string(b)}, errors.Join(err1, err2)
}
// Registered is what applies here of the servers registered through this module.
func Registered(p Paths) Servers {
s := Servers{}
readJSON(p.registry(), &s)
return s
}
// RenderNow writes the managed directory from the facts, the settings, the licence held and the servers
// registered here.
func RenderNow(p Paths, write WriteManaged) ([]string, error) {
var facts Facts
if !readJSON(p.Facts, &facts) || facts.Console == "" {
return nil, fmt.Errorf("the mesh has not rendered %s yet; nothing to write", p.Facts)
}
var settings Settings
readJSON(p.Settings, &settings)
var binding *Binding
var b Binding
if readJSON(p.binding(), &b) {
binding = &b
}
files := Render(facts, settings, binding, p.helper(), Registered(p))
names := make([]string, 0, len(files))
for n := range files {
names = append(names, n)
}
sort.Strings(names)
var out []string
for _, n := range names {
line, err := write(n, files[n])
if err != nil {
return out, err
}
out = append(out, line)
}
return out, nil
}
// ---- the licence ----------------------------------------------------------------------------------
// BindingState is what the manager's `bindings` state says one consumer should hold (ADR 0206).
type BindingState struct {
Licence string `json:"licence"`
Kind string `json:"kind"`
Generation int64 `json:"generation"`
}
// Current is what the seat answers to `current`: the licence this node is bound to and its token, sealed.
type Current struct {
Licence string `json:"licence"`
Kind string `json:"kind"`
Generation int64 `json:"generation"`
Sealed *SealedBox `json:"sealed"`
Identity *Identity `json:"identity"`
}
// Holdings is what this node holds, as the `holdings` state carries it (ADR 0206): enough for the manager
// to tell a login it has not adopted from one it has, and never a token — fingerprints and expiries only.
type Holdings struct {
Node string `json:"node"`
Identity *Identity `json:"identity"`
Kind *string `json:"kind"`
Refresh struct {
Present bool `json:"present"`
Fingerprint *string `json:"fingerprint"`
ExpiresAt *int64 `json:"expiresAt"`
} `json:"refresh"`
Access *struct {
Fingerprint string `json:"fingerprint"`
ExpiresAt int64 `json:"expiresAt"`
} `json:"access"`
Licence *string `json:"licence"`
Generation int64 `json:"generation"`
ChangedAt *string `json:"changedAt"`
}
// HoldingsOf is what this node holds now.
func HoldingsOf(p Paths) Holdings {
h := Holdings{Node: p.Node, Identity: ReadIdentity(p.account())}
creds := ReadCredentials(p.credentials())
if info, err := os.Stat(p.credentials()); err == nil {
at := info.ModTime().UTC().Format("2006-01-02T15:04:05.000Z")
h.ChangedAt = &at
}
if rt := RefreshTokenOf(creds); rt != "" {
fp := Fingerprint(rt)
h.Refresh.Present, h.Refresh.Fingerprint = true, &fp
}
if v, ok := number(creds.oauth()["refreshTokenExpiresAt"]); ok {
h.Refresh.ExpiresAt = &v
}
if g := GrantOf(creds); g != nil {
h.Access = &struct {
Fingerprint string `json:"fingerprint"`
ExpiresAt int64 `json:"expiresAt"`
}{Fingerprint(g.AccessToken), g.ExpiresAt}
}
kind := ""
if _, err := os.Stat(p.apiKey()); err == nil {
kind = "api-key"
} else if h.Access != nil {
kind = "subscription"
}
if kind != "" {
h.Kind = &kind
}
var applied Binding
if readJSON(p.binding(), &applied) && applied.Licence != "" {
h.Licence, h.Generation = &applied.Licence, applied.Generation
}
return h
}
// GrantAnswer is what `claude_code_grant` answers: a login sealed to the key given, or nothing waiting.
type GrantAnswer struct {
Sealed *SealedBox `json:"sealed,omitempty"`
Identity *Identity `json:"identity,omitempty"`
Fingerprint string `json:"fingerprint,omitempty"`
Waiting *bool `json:"waiting,omitempty"`
}
// GrantFor is the full grant in the credentials file sealed to the manager's key — the one time a refresh
// token leaves this node, for the manager to adopt by refreshing it (ADR 0206). Nothing waiting when the
// file holds no refresh token.
func GrantFor(p Paths, managerPublicKey string) (GrantAnswer, error) {
creds := ReadCredentials(p.credentials())
rt := RefreshTokenOf(creds)
if rt == "" {
no := false
return GrantAnswer{Waiting: &no}, nil
}
raw, err := json.Marshal(creds.oauth())
if err != nil {
return GrantAnswer{}, err
}
box, err := Seal(string(raw), managerPublicKey)
if err != nil {
return GrantAnswer{}, err
}
return GrantAnswer{Sealed: &box, Identity: ReadIdentity(p.account()), Fingerprint: Fingerprint(rt)}, nil
}
// Pull asks the seat for this node's current token and applies it.
func Pull(p Paths, ask Ask, write WriteManaged) (map[string]any, error) {
keys, err := Keypair(p)
if err != nil {
return nil, err
}
raw, err := ask(SeatVerb("current"), map[string]any{"consumer": p.Node, "public_key": keys.PublicKey})
if err != nil {
return nil, err
}
var c Current
if err := json.Unmarshal(raw, &c); err != nil || c.Sealed == nil {
return map[string]any{"applied": false, "reason": "the seat holds no licence for this node"}, nil
}
return Apply(p, c, write)
}
// OnBinding takes a change to this node's key in the manager's `bindings` state (ADR 0206): the token is
// fetched when the generation is newer than the one applied. A released binding keeps the last token,
// which lives hours, and says so.
func OnBinding(p Paths, b *BindingState, ask Ask, write WriteManaged) (string, error) {
if b == nil {
return "this node's binding was released; it keeps its last token until it expires", nil
}
var applied Binding
if readJSON(p.binding(), &applied) && applied.Generation >= b.Generation {
return "", nil
}
out, err := Pull(p, ask, write)
if err != nil {
return "", err
}
raw, _ := json.Marshal(out)
return string(raw), nil
}
// Apply applies what the seat handed over. A switch replaces the grant whole and cleans up after the old
// licence; whatever it is, the file is written without a refresh token, so the agent here never refreshes.
func Apply(p Paths, c Current, write WriteManaged) (map[string]any, error) {
keys, err := Keypair(p)
if err != nil {
return nil, err
}
plain, err := Open(*c.Sealed, keys.PrivateKey)
if err != nil {
return nil, err
}
var previous Binding
had := readJSON(p.binding(), &previous)
switched := !had || previous.Licence != c.Licence
out := map[string]any{"applied": true, "licence": c.Licence, "kind": c.Kind, "switched": switched}
if c.Kind == "api-key" {
if err := os.WriteFile(p.apiKey(), []byte(strings.TrimSpace(plain)+"\n"), 0o600); err != nil {
return nil, err
}
if err := os.WriteFile(p.helper(), []byte("#!/bin/sh\nexec cat '"+p.apiKey()+"'\n"), 0o700); err != nil {
return nil, err
}
_ = os.Chmod(p.helper(), 0o700)
} else {
var g Grant
if err := json.Unmarshal([]byte(plain), &g); err != nil {
return nil, err
}
local := ReadCredentials(p.credentials())
// A login waiting here was handed to the manager first (ADR 0206): what comes back is its successor,
// and the refresh token in the file is the one the manager just spent.
d := DecideApply(GrantOf(local), g, switched || HoldsLogin(local))
if d.Apply {
next := WithGrant(local, g)
if switched {
next = ReplacedBy(local, g)
}
if err := WriteCredentials(p.credentials(), next); err != nil {
return nil, err
}
} else {
out = map[string]any{"applied": false, "licence": c.Licence, "reason": d.Reason}
}
// Away from the API key: it goes, with its helper.
_ = os.Remove(p.apiKey())
_ = os.Remove(p.helper())
}
if switched && c.Identity != nil && c.Identity.AccountUUID != "" {
changed, err := WriteIdentity(p.account(), *c.Identity)
if err == nil {
out["account"] = map[bool]string{true: "updated", false: "unchanged"}[changed]
}
}
gen := c.Generation
if gen == 0 {
gen = previous.Generation
}
raw, _ := json.Marshal(Binding{Licence: c.Licence, Kind: c.Kind, Generation: gen})
if err := os.WriteFile(p.binding(), append(raw, '\n'), 0o600); err != nil {
return nil, err
}
// The key-helper comes or goes with the licence's kind.
if rendered, err := RenderNow(p, write); err != nil {
out["rendered"] = map[string]any{"failed": err.Error()}
} else {
out["rendered"] = rendered
}
return out, nil
}
// ---- MCP servers ----------------------------------------------------------------------------------
// Registration is a server registered (or, with no entry, unregistered) through this module.
type Registration struct {
Name string
Entry map[string]any
// Nodes: nil for this node, ["all"] for every node running the module, or a list.
Nodes []string
}
// ServerState is the `servers` state as this module reaches it through the runtime.
type ServerState interface {
Put(key string, value any) error
Delete(key string) error
Keys() ([]string, error)
}
// ServerChange is one change to the `servers` state, as a watch hands it over.
type ServerChange struct {
Key string
Op string // put | delete
Value map[string]any
}
// KeyOf is the key a registration lives at: `all.<server>` for every node, `<node>.<server>` for one.
func KeyOf(scope, name string) string { return scope + "." + name }
// ServerView is what this node takes from the `servers` state: the entries for every node and for this
// one, kept in memory from the watch and written through to the module's own file whenever what applies
// here changes, so the managed directory renders without the bus.
type ServerView struct {
p Paths
mu sync.Mutex
entries map[string]map[string]any
}
// NewServerView is an empty view for this node.
func NewServerView(p Paths) *ServerView {
return &ServerView{p: p, entries: map[string]map[string]any{}}
}
// Take takes one change, and answers whether what applies to this node changed.
func (v *ServerView) Take(c ServerChange) bool {
scope, name, ok := strings.Cut(c.Key, ".")
if !ok || scope == "" || (scope != "all" && scope != v.p.Node) {
return false
}
v.mu.Lock()
if c.Op == "put" && c.Value != nil && EntryProblem(name, c.Value) == "" {
v.entries[c.Key] = c.Value
} else {
delete(v.entries, c.Key)
}
v.mu.Unlock()
return v.writeThrough()
}
// Effective is what applies here: every node's entries, with this node's own laid over them by name.
func (v *ServerView) Effective() Servers {
v.mu.Lock()
defer v.mu.Unlock()
out := Servers{}
for _, scope := range []string{"all", v.p.Node} {
for key, entry := range v.entries {
if name, ok := strings.CutPrefix(key, scope+"."); ok {
out[name] = entry
}
}
}
return out
}
func (v *ServerView) writeThrough() bool {
now, _ := indented(v.Effective())
before, _ := os.ReadFile(v.p.registry())
if string(before) == string(now) {
return false
}
_ = os.WriteFile(v.p.registry(), now, 0o600)
return true
}
// OnServerChange takes a change from the watch, and renders when what applies here changed.
func OnServerChange(v *ServerView, c ServerChange, p Paths, write WriteManaged) (string, error) {
if !v.Take(c) {
return "", nil
}
if _, err := RenderNow(p, write); err != nil {
return "", err
}
what := "registered"
if c.Op != "put" {
what = "unregistered"
}
return what + " " + c.Key, nil
}
// RegisterServer registers (or, with no entry, unregisters) a server: a put (or delete) per scope in the
// `servers` state, taken into this node's view at once so the answer says what it did here; every other
// node takes it from its watch, and a node that joins later from the current state.
func RegisterServer(p Paths, r Registration, servers ServerState, v *ServerView, write WriteManaged,
others func() ([]string, error)) (map[string]any, error) {
if r.Entry != nil {
if problem := EntryProblem(r.Name, r.Entry); problem != "" {
return map[string]any{"registered": false, "reason": problem}, nil
}
}
scopes := r.Nodes
if len(scopes) == 0 {
scopes = []string{p.Node}
}
// Compared before and after rather than read from Take: this node's own watch may hand the view the
// same change first, and then Take here finds nothing new although this call made it.
before, _ := json.Marshal(v.Effective())
for _, scope := range scopes {
key := KeyOf(scope, r.Name)
var err error
if r.Entry != nil {
err = servers.Put(key, r.Entry)
} else {
err = servers.Delete(key)
}
if err != nil {
return nil, err
}
op := "put"
if r.Entry == nil {
op = "delete"
}
v.Take(ServerChange{Key: key, Op: op, Value: r.Entry})
}
after, _ := json.Marshal(v.Effective())
changed := string(before) != string(after)
here := false
for _, s := range scopes {
here = here || s == "all" || s == p.Node
}
verb := "registered"
if r.Entry == nil {
verb = "unregistered"
}
answer := map[string]any{verb: r.Name, "on": scopes}
switch {
case !here:
answer["here"] = "not this node"
case changed:
answer["here"] = "changed"
default:
answer["here"] = "already so"
}
if changed {
rendered, err := RenderNow(p, write)
if err != nil {
return nil, err
}
answer["rendered"] = rendered
}
if r.Entry == nil {
if _, still := v.Effective()[r.Name]; still {
answer["still"] = r.Name + " still applies here from another registration (for every node, or for this one); unregister that too"
}
}
if len(r.Nodes) == 0 {
// The question the operator wanted asked: here only, or more?
var elsewhere []string
if nodes, err := others(); err == nil {
for _, n := range nodes {
if n != p.Node {
elsewhere = append(elsewhere, n)
}
}
}
if len(elsewhere) > 0 {
answer["also"] = fmt.Sprintf("claude-code also runs on %s. To %s it there too, call again with nodes: \"all\" or a list of those nodes.",
strings.Join(elsewhere, ", "), map[bool]string{true: "register", false: "unregister"}[r.Entry != nil])
} else {
answer["also"] = "To do the same on every node running claude-code, call again with nodes: \"all\"."
}
}
return answer, nil
}
// stamp is a time as the status answers it.
func stamp(ms int64) string { return time.UnixMilli(ms).UTC().Format(time.RFC3339) }
@@ -1,121 +0,0 @@
package main
// What the module writes into the agent's machine-wide managed directory (novox/hq design 36 §1–§4).
// Pure: composed from the facts the mesh rendered, the settings the operator set and the licence the node
// holds, so what lands under /etc is tested without a machine.
//
// Three files, owned whole by this module:
//
// managed-mcp.json the tool servers every session loads: the mesh's console as `mesh`, and the
// servers the operator declared or registered through this module. Exclusive by
// the vendor's rule — a server not listed here does not load (operator's choice,
// 2026-10-03).
// managed-settings.json the mesh's keys only: the repositories' attribution convention, the claude.ai
// connectors kept beside the managed servers, and — for an API-key licence only —
// the key-helper. A person's preferences are theirs.
// CLAUDE.md how a session on this mesh works, who this node is, the conventions.
import (
"bytes"
"encoding/json"
"fmt"
"regexp"
"strings"
)
// ManagedDir is the agent's machine-wide managed directory.
const ManagedDir = "/etc/claude-code"
const meshEntry = "mesh"
// Facts are what the mesh rendered for this node.
type Facts struct {
Node string `json:"node"`
Console string `json:"console"`
}
// Settings are the operator's, for the mesh or this node.
type Settings struct {
Role string `json:"role"`
MCPServers map[string]map[string]any `json:"mcp_servers"`
}
// Binding is the licence this node holds, as it was last applied.
type Binding struct {
Licence string `json:"licence"`
Kind string `json:"kind"` // subscription | api-key
Generation int64 `json:"generation,omitempty"`
}
// Servers are tool server entries by name, in the vendor's `.mcp.json` shape.
type Servers map[string]map[string]any
var serverName = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// EntryProblem says why the vendor's managed file would not take an entry, or "" when it would: a name of
// letters, digits, `-` and `_`, and an http/sse server with a url or a stdio server with a command.
func EntryProblem(name string, entry map[string]any) string {
if !serverName.MatchString(name) {
return fmt.Sprintf("%q is not a name the agent takes: letters, digits, - and _", name)
}
if name == meshEntry {
return fmt.Sprintf("%q is the mesh's own entry", meshEntry)
}
kind, _ := entry["type"].(string)
if kind == "" {
kind = "stdio"
}
switch kind {
case "http", "sse", "streamable-http":
if u, _ := entry["url"].(string); u != "" {
return ""
}
return fmt.Sprintf("an %s server needs a url", kind)
case "stdio":
if c, _ := entry["command"].(string); c != "" {
return ""
}
return "a stdio server needs a command"
}
return fmt.Sprintf("%q is not a server type the agent knows (http, sse, stdio)", kind)
}
// jsonFile is a value as the managed files are written: two-space indent, a trailing newline, nothing
// escaped that need not be.
func jsonFile(v any) string {
var b bytes.Buffer
enc := json.NewEncoder(&b)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
_ = enc.Encode(v)
return b.String()
}
// Render composes the three files. registered — what was registered through this module and applies
// here — is laid over the servers the operator set in its settings.
func Render(facts Facts, settings Settings, binding *Binding, helperPath string, registered Servers) map[string]string {
servers := map[string]any{}
for _, layer := range []map[string]map[string]any{settings.MCPServers, registered} {
for name, entry := range layer {
if EntryProblem(name, entry) != "" {
continue // the mesh's own entry, or one the agent would refuse
}
servers[name] = entry
}
}
servers[meshEntry] = map[string]any{"type": "http", "url": facts.Console}
managed := map[string]any{"attribution": map[string]any{"commit": "", "pr": ""}, "allowAllClaudeAiMcps": true}
if binding != nil && binding.Kind == "api-key" {
managed["apiKeyHelper"] = helperPath
}
role := strings.TrimSpace(settings.Role)
if role == "" {
role = "not stated — set it in this module's settings for the node"
}
return map[string]string{
"managed-mcp.json": jsonFile(map[string]any{"mcpServers": servers}),
"managed-settings.json": jsonFile(managed),
"CLAUDE.md": instructionsText(facts.Node, role),
}
}
-189
View File
@@ -1,189 +0,0 @@
package main
// Sealing to one recipient (novox/hq ADR 0183, ADR 0206): the manager seals what it hands a consumer to
// the key that consumer sent, and a node seals a waiting login to the key the manager gives. The same box
// the agent module's TypeScript makes and opens, byte for byte — X25519 for the agreement, HKDF-SHA256 for
// the key, AES-256-GCM for the box — so `testdata/sealed-by-typescript.json` is opened here, and a test
// reopens what this seals with the same derivation.
//
// A box is `{ v: 1, eph, iv, tag, ct }`, every field base64; `eph` is the one-time public key as SPKI DER,
// and the key is bound to it and to the recipient's raw public key, so a box cannot be re-addressed.
import (
"crypto/aes"
"crypto/cipher"
"crypto/ecdh"
"crypto/hkdf"
"crypto/rand"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/pem"
"errors"
"fmt"
)
// SealedBox is a value sealed to one recipient.
type SealedBox struct {
V int `json:"v"`
Eph string `json:"eph"`
IV string `json:"iv"`
Tag string `json:"tag"`
Ct string `json:"ct"`
}
// KeyPair is a recipient's keypair as the two PEM strings it is kept and sent as.
type KeyPair struct {
PublicKey string `json:"publicKey"`
PrivateKey string `json:"privateKey"`
}
const sealInfo = "novox-mesh sealed box v1"
// GenerateKeyPair makes an X25519 keypair, PEM-encoded as the agent module's are.
func GenerateKeyPair() (KeyPair, error) {
priv, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
return KeyPair{}, err
}
pubDER, err := x509.MarshalPKIXPublicKey(priv.PublicKey())
if err != nil {
return KeyPair{}, err
}
privDER, err := x509.MarshalPKCS8PrivateKey(priv)
if err != nil {
return KeyPair{}, err
}
return KeyPair{
PublicKey: string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: pubDER})),
PrivateKey: string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privDER})),
}, nil
}
func publicFromPEM(p string) (*ecdh.PublicKey, error) {
block, _ := pem.Decode([]byte(p))
if block == nil {
return nil, errors.New("not a PEM public key")
}
k, err := x509.ParsePKIXPublicKey(block.Bytes)
if err != nil {
return nil, err
}
pub, ok := k.(*ecdh.PublicKey)
if !ok || pub.Curve() != ecdh.X25519() {
return nil, errors.New("not an X25519 public key")
}
return pub, nil
}
func privateFromPEM(p string) (*ecdh.PrivateKey, error) {
block, _ := pem.Decode([]byte(p))
if block == nil {
return nil, errors.New("not a PEM private key")
}
k, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, err
}
priv, ok := k.(*ecdh.PrivateKey)
if !ok || priv.Curve() != ecdh.X25519() {
return nil, errors.New("not an X25519 private key")
}
return priv, nil
}
func boxKey(secret, ephDER, recipientRaw []byte) ([]byte, error) {
salt := append(append([]byte{}, ephDER...), recipientRaw...)
return hkdf.Key(sha256.New, secret, salt, sealInfo, 32)
}
// Seal seals plaintext to the recipient's public key.
func Seal(plaintext, recipientPEM string) (SealedBox, error) {
recipient, err := publicFromPEM(recipientPEM)
if err != nil {
return SealedBox{}, err
}
eph, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
return SealedBox{}, err
}
secret, err := eph.ECDH(recipient)
if err != nil {
return SealedBox{}, err
}
ephDER, err := x509.MarshalPKIXPublicKey(eph.PublicKey())
if err != nil {
return SealedBox{}, err
}
key, err := boxKey(secret, ephDER, recipient.Bytes())
if err != nil {
return SealedBox{}, err
}
gcm, err := newGCM(key)
if err != nil {
return SealedBox{}, err
}
iv := make([]byte, 12)
if _, err := rand.Read(iv); err != nil {
return SealedBox{}, err
}
out := gcm.Seal(nil, iv, []byte(plaintext), nil)
ct, tag := out[:len(out)-gcm.Overhead()], out[len(out)-gcm.Overhead():]
b64 := base64.StdEncoding.EncodeToString
return SealedBox{V: 1, Eph: b64(ephDER), IV: b64(iv), Tag: b64(tag), Ct: b64(ct)}, nil
}
// Open opens a box with the recipient's private key; it fails for a box to another key or one tampered with.
func Open(box SealedBox, privatePEM string) (string, error) {
if box.V != 1 {
return "", errors.New("not a sealed box this module can open")
}
priv, err := privateFromPEM(privatePEM)
if err != nil {
return "", err
}
d := base64.StdEncoding.DecodeString
ephDER, err := d(box.Eph)
if err != nil {
return "", fmt.Errorf("the box's eph: %w", err)
}
ephKey, err := x509.ParsePKIXPublicKey(ephDER)
if err != nil {
return "", err
}
eph, ok := ephKey.(*ecdh.PublicKey)
if !ok {
return "", errors.New("the box's eph is not an X25519 key")
}
secret, err := priv.ECDH(eph)
if err != nil {
return "", err
}
key, err := boxKey(secret, ephDER, priv.PublicKey().Bytes())
if err != nil {
return "", err
}
iv, err1 := d(box.IV)
tag, err2 := d(box.Tag)
ct, err3 := d(box.Ct)
if err := errors.Join(err1, err2, err3); err != nil {
return "", err
}
gcm, err := newGCM(key)
if err != nil {
return "", err
}
plain, err := gcm.Open(nil, iv, append(ct, tag...), nil)
if err != nil {
return "", errors.New("the box does not open with this key")
}
return string(plain), nil
}
func newGCM(key []byte) (cipher.AEAD, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
return cipher.NewGCM(block)
}
@@ -1,42 +0,0 @@
{
"facts": {
"node": "workstation",
"console": "http://127.0.0.1:4270/mcp"
},
"settings": {
"role": "the laptop",
"mcp_servers": {
"search": {
"type": "http",
"url": "https://s.example/mcp"
},
"docs": {
"type": "stdio",
"command": "docs-mcp",
"args": [
"--x"
]
}
}
},
"registered": {
"anton": {
"type": "stdio",
"command": "node",
"args": [
"/a/b.js"
],
"env": {}
}
},
"withKey": {
"managed-mcp.json": "{\n \"mcpServers\": {\n \"anton\": {\n \"type\": \"stdio\",\n \"command\": \"node\",\n \"args\": [\n \"/a/b.js\"\n ],\n \"env\": {}\n },\n \"docs\": {\n \"type\": \"stdio\",\n \"command\": \"docs-mcp\",\n \"args\": [\n \"--x\"\n ]\n },\n \"mesh\": {\n \"type\": \"http\",\n \"url\": \"http://127.0.0.1:4270/mcp\"\n },\n \"search\": {\n \"type\": \"http\",\n \"url\": \"https://s.example/mcp\"\n }\n }\n}\n",
"managed-settings.json": "{\n \"attribution\": {\n \"commit\": \"\",\n \"pr\": \"\"\n },\n \"allowAllClaudeAiMcps\": true,\n \"apiKeyHelper\": \"/state/api-key-helper\"\n}\n",
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** the laptop\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
},
"plain": {
"managed-mcp.json": "{\n \"mcpServers\": {\n \"mesh\": {\n \"type\": \"http\",\n \"url\": \"http://127.0.0.1:4270/mcp\"\n }\n }\n}\n",
"managed-settings.json": "{\n \"attribution\": {\n \"commit\": \"\",\n \"pr\": \"\"\n },\n \"allowAllClaudeAiMcps\": true\n}\n",
"CLAUDE.md": "# This machine is a node of a Novox mesh\n\nWritten by the mesh's `claude-code` module. Edit the module's settings or the catalogue, never this file:\nit is rewritten whenever the module renders.\n\n## Who this node is\n\n- **Node:** `workstation`\n- **Role:** not stated — set it in this module's settings for the node\n- The other nodes, their roles and what runs where: ask the controller (`mesh-controller.nodes`,\n `mesh-controller.node`). Nothing here lists them, because a copy drifts.\n\n## How a session on this mesh works\n\nThe console is the only way to the mesh: the MCP server named `mesh`. It offers five tools, and\neverything else is an address you find and call through them:\n\n- `mesh_search` — words in, matching addresses out. `mesh_describe` — one address's arguments.\n- `mesh_call` — call an address. A seat the mesh holds once is `<seat>.<verb>` (the mesh's own verbs\n are `mesh-controller.<verb>`: `status`, `plan`, `node`, `assign`, `push`, `settings`);\n a module on a machine is `<node>/<module>.<tool>`.\n- `mesh_overview` and `mesh_machine` — the mesh's seats and machines, and what one machine runs.\n\n- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the\n literal text before forming a hypothesis: the records module's `records_search`, then\n `records_read`.\n- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.\n- **A licence** through the `anthropic-licence-manager` seat's verbs. Never edit the agent's credentials\n file by hand, never print or ask for a token.\n\n## Hard rules\n\n- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If\n unsure, `mesh-controller.plan` for the node says what the mesh writes there.\n- Never write to a store's database by hand; schema changes are numbered migrations.\n- Never push to a main branch: a branch, a pull request, and a human approval for every merge.\n- The mesh creates no symlinks, and nobody else does either.\n- A package is declared in a module, never installed by hand.\n\n## Conventions\n\n- Commit messages are concise, in the imperative, about why.\n- Test before pushing: nodes update unattended.\n- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.\n"
}
}
-5
View File
@@ -1,5 +0,0 @@
module claude-code
go 1.25.0
require git.novox.be/novox/mesh-sdk/go v0.1.7
-2
View File
@@ -1,2 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
-94
View File
@@ -1,94 +0,0 @@
{
"module": "claude-code",
"version": "1",
"slug": "agent",
"capabilities": [
"package-manager"
],
"requires": [
"mcp-endpoint"
],
"binds": {
"mcp-endpoint": "${dir:state}/mcp-endpoint.json"
},
"state": [
"servers",
"holdings"
],
"reads": [
"claude-licence-manager.bindings"
],
"tools": [
"claude_code_status",
"claude_code_render",
"claude_code_pull",
"claude_code_grant",
"claude_code_mcp_list",
"claude_code_mcp_register",
"claude_code_mcp_unregister"
],
"resources": [
{
"id": "package",
"type": "package",
"package": "claude-code"
},
{
"id": "managed",
"type": "directory",
"path": "/etc/claude-code",
"mode": "0755"
},
{
"id": "agent-home",
"type": "directory",
"path": "${machine:account-home}/.claude",
"mode": "0700",
"owner": "${machine:account}"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "${machine:account}",
"place": "."
},
{
"id": "facts",
"type": "file",
"path": "${dir:state}/facts.json",
"mode": "0600",
"owner": "${machine:account}",
"content": "{\n \"node\": \"${machine:name}\",\n \"console\": \"http://127.0.0.1:${bound:mcp-endpoint:port}/mcp\"\n}\n"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"mode": "0600",
"owner": "${machine:account}",
"merge": "json",
"content": "{\n \"role\": \"\",\n \"mcp_servers\": {}\n}\n"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/claude-code",
"binary": "claude-code",
"loads": [
"claude-code"
],
"env": {
"MESH_CLAUDE_CODE_STATE": "${dir:state}",
"MESH_CLAUDE_CODE_FACTS": "${dir:state}/facts.json",
"MESH_CLAUDE_CODE_SETTINGS": "${dir:state}/settings.json"
}
}
]
}
}
-55
View File
@@ -1,55 +0,0 @@
# claude-licence-manager
Holds the `anthropic-licence-manager` seat: every Anthropic licence the mesh has, kept alive by one
rotation source, and handed to each consumer sealed (novox/hq ADR 0183, ADR 0206, design 39).
## How a licence comes to exist
Nothing is configured. Every node running `claude-code` reports what it holds as that module's
`holdings` state — the account, fingerprints and expiries, never a token. This module reads every report
when it starts and watches them:
1. A report with a refresh token it does not hold is a **candidate**.
2. It asks that node's `claude_code_grant`, giving its public key, and receives the grant sealed to it.
3. **It refreshes it.** If the vendor exchanges the token, the grant is this module's — encrypted at rest
with the key the vault made for it — and from then on it is the only refresher. If not, the candidate
is recorded dead and nothing is adopted.
4. Several nodes logged in to one account: newest login first; the rest are never exchanged.
5. A node reporting that account and bound to nothing is bound to it.
Each node is then handed an access token only, so the agent there never refreshes, and a refresh token
appearing on a node later can only be a person's login — which wins if it refreshes.
An API key enters through `adopt`, from a file on this module's node.
## What each consumer holds
This module's `bindings` state: one key per consumer (a node's name) with the licence, its kind and a
generation that grows with every rotation and switch. `claude-code` watches its own key and, on a newer
generation, asks `current` with its public key.
## The seat's verbs
`licences`, `bindings`, `bind`, `switch`, `release`, `refresh`, `usage`, `adopt`, `current` — through
the console as `anthropic-licence-manager.<verb>`. No answer carries a token.
## Settings
`settings.json` in the state directory: `cadence_minutes` (240), `floor_minutes` (60),
`failures_to_notify` (3), `cooldown_hours` (24), `refresh_warn_days` (3).
## Events
`licence.adopted`, `licence.refused`, `licence.failing`, `usage.read` — none carries a secret.
## Code and tests
Go, one binary (`cmd/claude-licence-manager`): the seat's verbs and the daemon in one launched bundle,
`prepare` as the run-once preparation step. The sealed box is `claude-code`'s own format, byte for byte —
the two modules carry the same `seal.go` — and a test opens one sealed by the TypeScript agent module the
Go one replaced, so the format is the one already on the machines.
go test ./...
# the store against a real postgres:
docker run -d --rm --name licmgr-pg -e POSTGRES_PASSWORD=t -p 15498:5432 postgres:16-alpine
MESH_TEST_POSTGRES=postgres://postgres:t@127.0.0.1:15498/postgres go test ./...
@@ -1,64 +0,0 @@
package main
// The grants at rest (novox/hq ADR 0183): encrypted with a key the vault made for this module, so the
// store holds ciphertext and only this module, reading its own secret, can open a row. AES-256-GCM, the
// key derived from the vault's secret by SHA-256 so a secret of any length serves.
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"errors"
"os"
"strings"
)
// Crypt seals and opens what the store keeps.
type Crypt struct{ key []byte }
// NewCrypt is the store's cipher from the vault's secret.
func NewCrypt(secret string) (*Crypt, error) {
secret = strings.TrimSpace(secret)
if secret == "" {
return nil, errors.New("the key the grants are encrypted with is empty: the vault has not delivered it yet")
}
sum := sha256.Sum256([]byte(secret))
return &Crypt{key: sum[:]}, nil
}
// CryptFromFile reads the vault's secret from the file the mesh delivered it to.
func CryptFromFile(path string) (*Crypt, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return NewCrypt(string(raw))
}
// Seal encrypts a plaintext as `v1.<iv>.<ciphertext and tag>`.
func (c *Crypt) Seal(plaintext string) string {
gcm, _ := newGCM(c.key)
iv := make([]byte, 12)
_, _ = rand.Read(iv)
b64 := base64.StdEncoding.EncodeToString
return "v1." + b64(iv) + "." + b64(gcm.Seal(nil, iv, []byte(plaintext), nil))
}
// Open decrypts what Seal made with the same key.
func (c *Crypt) Open(sealed string) (string, error) {
parts := strings.Split(sealed, ".")
if len(parts) != 3 || parts[0] != "v1" {
return "", errors.New("not a grant this module sealed")
}
iv, err1 := base64.StdEncoding.DecodeString(parts[1])
ct, err2 := base64.StdEncoding.DecodeString(parts[2])
if err := errors.Join(err1, err2); err != nil {
return "", err
}
gcm, _ := newGCM(c.key)
plain, err := gcm.Open(nil, iv, ct, nil)
if err != nil {
return "", errors.New("the grant does not open with this module's key")
}
return string(plain), nil
}
@@ -1,348 +0,0 @@
// claude-licence-manager (novox/hq ADR 0183, ADR 0206, design 39): one binary, launched by the control
// node's runtime and speaking MCP to it over stdio through the Go SDK (ADR 0193, ADR 0198). It serves the
// `anthropic-licence-manager` seat's verbs and, beside them, runs long: it watches what every node reports
// holding and adopts a login it does not hold, keeps every grant alive under a lease, and reads usage.
//
// `claude-licence-manager prepare` is the preparation step (ADR 0135): the host runs it once before the
// version that needs it, with the module's words and no bus, and it brings the store's schema to shape.
//
// stdout is the MCP channel; everything this module says, it says on stderr.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Seat is the role this module holds.
const Seat = "anthropic-licence-manager"
func say(format string, args ...any) {
fmt.Fprintf(os.Stderr, "[claude-licence-manager] "+format+"\n", args...)
}
func main() {
if len(os.Args) > 1 && os.Args[1] == "prepare" {
if err := prepare(); err != nil {
say("preparing the store failed: %v", err)
os.Exit(1)
}
say("the store's schema is what this version needs")
return
}
go daemon()
if err := stdio.Serve("", tools()); err != nil {
say("%v", err)
os.Exit(1)
}
}
func prepare() error {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
store, err := PgStoreFromEnv(ctx)
if err != nil {
return err
}
defer store.Close()
return store.Migrate(ctx)
}
// ---- what the binary is handed -----------------------------------------------------------------------
var (
built *Manager
buildMu sync.Mutex
)
// manager builds the rules' dependencies once, from the module's words (ADR 0192): file paths, never
// values. A failure is said and tried again on the next call, so a database that arrives late is not fatal.
func manager() (*Manager, error) {
buildMu.Lock()
defer buildMu.Unlock()
if built != nil {
return built, nil
}
dir, keyFile := os.Getenv("MESH_LICENCE_STATE"), os.Getenv("MESH_LICENCE_KEY_FILE")
if dir == "" || keyFile == "" {
return nil, errors.New("MESH_LICENCE_STATE and MESH_LICENCE_KEY_FILE are not set: the mesh renders them for this module")
}
crypt, err := CryptFromFile(keyFile)
if err != nil {
return nil, err
}
keys, err := keypair(dir)
if err != nil {
return nil, err
}
store, err := PgStoreFromEnv(context.Background())
if err != nil {
return nil, err
}
node, _ := os.Hostname()
if n := os.Getenv("MESH_NODE"); n != "" {
node = n
}
bindings := stdio.State("bindings")
built = &Manager{
Store: store,
Vendor: LiveVendor(),
Crypt: crypt,
Keys: keys,
AskGrant: func(_ context.Context, node, publicKey string) (GrantAnswer, error) {
var a GrantAnswer
raw, err := stdio.Ask("claude-code.claude_code_grant@"+node, map[string]any{"public_key": publicKey})
if err != nil {
return a, err
}
return a, json.Unmarshal(raw, &a)
},
PutBinding: func(_ context.Context, consumer string, b BindingState) error {
_, err := bindings.Put(consumer, b)
return err
},
DeleteBinding: func(_ context.Context, consumer string) error { return bindings.Delete(consumer) },
Emit: func(event string, body map[string]any) error { return stdio.Emit(event, body) },
Now: time.Now,
Log: say,
Holder: fmt.Sprintf("%s:%d", node, os.Getpid()),
Settings: SettingsFrom(os.Getenv("MESH_LICENCE_SETTINGS")),
}
return built, nil
}
// keypair is this module's own, made once in its state directory; the private half never leaves it.
// Written whole, then linked into place, so a second process reads the first's key and never half of it.
func keypair(dir string) (KeyPair, error) {
file := filepath.Join(dir, "manager-key.json")
if _, err := os.Stat(file); errors.Is(err, os.ErrNotExist) {
k, err := GenerateKeyPair()
if err != nil {
return KeyPair{}, err
}
raw, _ := json.Marshal(k)
tmp := filepath.Join(dir, fmt.Sprintf(".manager-key.%d.json", os.Getpid()))
if err := os.WriteFile(tmp, raw, 0o600); err != nil {
return KeyPair{}, err
}
_ = os.Link(tmp, file) // fails when another made it first, which is right
_ = os.Remove(tmp)
}
raw, err := os.ReadFile(file)
if err != nil {
return KeyPair{}, err
}
var k KeyPair
return k, json.Unmarshal(raw, &k)
}
// ---- the long-running half ---------------------------------------------------------------------------
func daemon() {
var mu sync.Mutex
reports := map[string]Holdings{}
var passing sync.Mutex
pass := func() {
passing.Lock() // one pass at a time: each account is leased, and a pass is cheap
defer passing.Unlock()
m, err := manager()
if err != nil {
say("not ready: %v", err)
return
}
mu.Lock()
all := make([]Holdings, 0, len(reports))
for _, r := range reports {
all = append(all, r)
}
mu.Unlock()
sort.Slice(all, func(i, j int) bool { return all[i].Node < all[j].Node })
adopted, err := m.Consider(context.Background(), all)
if err != nil {
say("considering the reports failed: %v", err)
}
if len(adopted) > 0 {
say("adopted %s", strings.Join(adopted, ", "))
}
}
// What every node holds (ADR 0206): the whole current set, then each change. Asked again until it
// answers — the channel to the runtime opens as the bundle starts, and the agent module's state may
// arrive after this one.
go func() {
waits := []time.Duration{2 * time.Second, 5 * time.Second, 10 * time.Second, 30 * time.Second}
for attempt := 0; ; attempt++ {
err := stdio.State("claude-code.holdings").Watch("", func(c stdio.StateChange) error {
mu.Lock()
if c.Op == "put" {
var h Holdings
if json.Unmarshal(c.Value, &h) == nil {
reports[c.Key] = h
}
} else {
delete(reports, c.Key)
}
mu.Unlock()
// During the current values the pass waits for the whole set: newest login first needs all.
if !c.Current {
go pass()
}
return nil
})
if err == nil {
mu.Lock()
n := len(reports)
mu.Unlock()
say("watching what %d node(s) hold", n)
pass()
return
}
pause := time.Minute
if attempt < len(waits) {
pause = waits[attempt]
}
say("what the nodes hold cannot be watched yet (%v); asking again in %s", err, pause)
time.Sleep(pause)
}
}()
refresh := time.NewTicker(time.Minute)
usage := time.NewTicker(5 * time.Minute)
for {
select {
case <-refresh.C:
if m, err := manager(); err == nil {
out, err := m.RotateDue(context.Background())
for _, r := range out {
b, _ := json.Marshal(r)
say("%s", b)
}
if err != nil {
say("refreshing failed: %v", err)
}
}
pass() // a login whose node did not answer last time is asked again
case <-usage.C:
if m, err := manager(); err == nil {
if err := m.ReadUsage(context.Background()); err != nil {
say("reading usage failed: %v", err)
}
}
}
}
}
// ---- the seat's verbs --------------------------------------------------------------------------------
func text(a map[string]any, k string) (string, error) {
v, _ := a[k].(string)
if strings.TrimSpace(v) == "" {
return "", fmt.Errorf("%s is required", k)
}
return strings.TrimSpace(v), nil
}
// verb is one of the seat's verbs: listed as `<seat>.<verb>`, so the runtime serves it on the seat's subject.
func verb(name, description string, input map[string]any, run func(ctx context.Context, m *Manager, a map[string]any) (any, error)) stdio.Tool {
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input,
Run: func(a map[string]any) (any, error) {
m, err := manager()
if err != nil {
return nil, err
}
return run(context.Background(), m, a)
}}
}
func str(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
func two(a map[string]any, k1, k2 string) (string, string, error) {
v1, err1 := text(a, k1)
v2, err2 := text(a, k2)
return v1, v2, errors.Join(err1, err2)
}
func tools() []stdio.Tool {
consumer := str("the node's name")
return []stdio.Tool{
verb("licences", "Every licence the manager holds — account, kind, when its token and its refresh token expire, failures in a row, which consumers are bound to it. Never a token.",
nil, func(ctx context.Context, m *Manager, _ map[string]any) (any, error) { return m.Licences(ctx) }),
verb("bindings", "Which licence each consumer (a node's agent, by the node's name) is bound to, and the generation it was last given.",
nil, func(ctx context.Context, m *Manager, _ map[string]any) (any, error) { return m.Store.Bindings(ctx) }),
verb("bind", "Bind a consumer — a node's agent, by the node's name — to a licence. Its node fetches the licence's token at once.",
map[string]any{"consumer": consumer, "licence": str("a licence, as `licences` names it")},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
c, l, err := two(a, "consumer", "licence")
if err != nil {
return nil, err
}
return m.Bind(ctx, c, l, "bind")
}),
verb("switch", "Move a consumer to another licence. Its node fetches the new licence's token at once and points the agent's account at it.",
map[string]any{"consumer": consumer, "licence": str("the licence to move to")},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
c, l, err := two(a, "consumer", "licence")
if err != nil {
return nil, err
}
return m.Bind(ctx, c, l, "switch")
}),
verb("release", "Unbind a consumer. Its node keeps its last token, which expires within hours.",
map[string]any{"consumer": consumer},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
c, err := text(a, "consumer")
if err != nil {
return nil, err
}
return m.Release(ctx, c, "release")
}),
verb("refresh", "Refresh a licence now, or every due licence when none is named; under each licence's lease, so it never races the daemon. Answers the outcome, never a token.",
map[string]any{"licence": str("a licence; absent for every due one")},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
if l, _ := a["licence"].(string); l != "" {
return m.Rotate(ctx, l, true)
}
return m.RotateDue(ctx)
}),
verb("usage", "Usage readings, the latest first, for every licence or one.",
map[string]any{"licence": str("a licence; absent for all"), "limit": map[string]any{"type": "number", "description": "how many readings (default 20)"}},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
l, _ := a["licence"].(string)
limit := 20
if v, ok := a["limit"].(float64); ok && v > 0 {
limit = int(v)
}
return m.Store.Usage(ctx, l, limit)
}),
verb("adopt", "Adopt an API key from a file on the manager's node, never as an argument. Subscriptions are adopted from the nodes' logins by themselves.",
map[string]any{"name": str("the licence's name"), "file": str("a file on the manager's node holding the key")},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
n, f, err := two(a, "name", "file")
if err != nil {
return nil, err
}
return m.AdoptKey(ctx, n, f)
}),
verb("current", "For a consumer's agent module (ADR 0206): its token, sealed to the public key it sends, with the licence, kind and generation. Null when it is bound to nothing.",
map[string]any{"consumer": consumer, "public_key": str("the consumer's public key, PEM")},
func(ctx context.Context, m *Manager, a map[string]any) (any, error) {
c, k, err := two(a, "consumer", "public_key")
if err != nil {
return nil, err
}
return m.Current(ctx, c, k)
}),
}
}
@@ -1,640 +0,0 @@
package main
// The Anthropic licence manager's rules (novox/hq ADR 0183, ADR 0206, design 39), written against what it
// is handed — a store, the vendor, a way to ask a node, its own state, a way to emit — so every rule is
// tested without a bus, a database or the vendor.
//
// - A licence is an account, learned from what the nodes report (`holdings`, the agent module's state).
// A report with a refresh token the manager does not hold is a candidate.
// - The secret is asked for, sealed to this module's key, never published.
// - Adopting is refreshing: newest login first, once per account; a failure adopts nothing.
// - What each consumer should hold is this module's `bindings` state, with a generation that grows with
// every rotation and switch; the consumer fetches its token by asking `current`.
// - One rotation source: every exchange with the vendor runs under a lease.
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"regexp"
"sort"
"strings"
"time"
)
// Fingerprint names a token without being one: the agent module's own fingerprint, so the two compare.
func Fingerprint(s string) string {
sum := sha256.Sum256([]byte(s))
return "sha256:" + hex.EncodeToString(sum[:])[:16]
}
// Identity is the account a grant belongs to, as the agent's own state file names it.
type Identity struct {
AccountUUID string `json:"accountUuid"`
EmailAddress string `json:"emailAddress,omitempty"`
OrganizationUUID string `json:"organizationUuid,omitempty"`
}
// Holdings is one node's report, as the agent module writes it to its `holdings` state (ADR 0206).
type Holdings struct {
Node string `json:"node"`
Identity *Identity `json:"identity"`
Kind string `json:"kind"`
Refresh struct {
Present bool `json:"present"`
Fingerprint string `json:"fingerprint"`
} `json:"refresh"`
ChangedAt string `json:"changedAt"`
}
// BindingState is what `bindings` holds for one consumer: no secret, only what it should hold and which
// generation.
type BindingState struct {
Licence string `json:"licence"`
Kind string `json:"kind"`
Generation int64 `json:"generation"`
}
// Settings are the manager's own, declared with defaults (design 39 §8).
type Settings struct {
Cadence time.Duration // rotate a grant once older than this
Floor time.Duration // refresh in any case with less than this left
FailuresToNotify int
Cooldown time.Duration // at most one notification per licence in this window
RefreshWarn time.Duration // warn this long before a refresh token itself expires
}
// Defaults are the settings a fresh mesh runs with.
var Defaults = Settings{Cadence: 4 * time.Hour, Floor: time.Hour, FailuresToNotify: 3, Cooldown: 24 * time.Hour, RefreshWarn: 72 * time.Hour}
// SettingsFrom reads the settings file, keeping a default for anything absent or not positive.
func SettingsFrom(path string) Settings {
s := Defaults
raw, err := os.ReadFile(path)
if err != nil {
return s
}
var f map[string]float64
if json.Unmarshal(raw, &f) != nil {
return s
}
set := func(k string, unit time.Duration, into *time.Duration) {
if v := f[k]; v > 0 {
*into = time.Duration(v * float64(unit))
}
}
set("cadence_minutes", time.Minute, &s.Cadence)
set("floor_minutes", time.Minute, &s.Floor)
set("cooldown_hours", time.Hour, &s.Cooldown)
if v := f["refresh_warn_days"]; v > 0 {
s.RefreshWarn = time.Duration(v * float64(24*time.Hour))
}
if v := f["failures_to_notify"]; v > 0 {
s.FailuresToNotify = int(v)
}
return s
}
// GrantAnswer is what a node's `claude_code_grant` answers: a login sealed to the key given, or nothing.
type GrantAnswer struct {
Sealed *SealedBox `json:"sealed"`
Identity *Identity `json:"identity"`
Fingerprint string `json:"fingerprint"`
}
// Manager is the rules and what they are handed.
type Manager struct {
Store Store
Vendor Vendor
Crypt *Crypt
Keys KeyPair
// AskGrant asks a node's agent module for the grant a login left there, sealed to publicKey. An error
// is the node not answering; a nil Sealed is no login waiting.
AskGrant func(ctx context.Context, node, publicKey string) (GrantAnswer, error)
// PutBinding and DeleteBinding are this module's `bindings` state.
PutBinding func(ctx context.Context, consumer string, b BindingState) error
DeleteBinding func(ctx context.Context, consumer string) error
Emit func(event string, body map[string]any) error
Now func() time.Time
Log func(format string, args ...any)
// Holder names this process in a lease, so a second run is told apart.
Holder string
Settings Settings
}
const leaseFor = 2 * time.Minute
// NameFor is a licence's name: the account's address where it has one, else its id.
func NameFor(id Identity) string {
if e := strings.TrimSpace(id.EmailAddress); e != "" {
return e
}
return id.AccountUUID
}
// ---- learning licences from what the nodes hold ------------------------------------------------------
// Candidate is a report the manager should try.
type Candidate struct {
Node string
Identity Identity
Fingerprint string
ChangedAt int64
}
// CandidatesIn is the candidates in a set of reports by account, newest login first (ADR 0206 §2, §4). A
// report without an identity is not one: a grant is filed under its account or not at all.
func (m *Manager) CandidatesIn(ctx context.Context, reports []Holdings) (map[string][]Candidate, error) {
out := map[string][]Candidate{}
for _, r := range reports {
if !r.Refresh.Present || r.Refresh.Fingerprint == "" || r.Identity == nil || r.Identity.AccountUUID == "" {
continue
}
settled, err := m.Store.Outcome(ctx, r.Refresh.Fingerprint)
if err != nil {
return nil, err
}
if settled != "" {
continue // adopted, dead, skipped, refused or gone: settled once
}
held, err := m.Store.LicenceForAccount(ctx, r.Identity.AccountUUID)
if err != nil {
return nil, err
}
if held != nil && held.RefreshFingerprint == r.Refresh.Fingerprint {
continue
}
var changed int64
if t, err := time.Parse(time.RFC3339Nano, r.ChangedAt); err == nil {
changed = t.UnixMilli()
}
// The latest login wins: one no newer than the grant held is not a newer login.
if held != nil && changed <= held.AdoptedAt {
continue
}
out[r.Identity.AccountUUID] = append(out[r.Identity.AccountUUID],
Candidate{Node: r.Node, Identity: *r.Identity, Fingerprint: r.Refresh.Fingerprint, ChangedAt: changed})
}
for _, list := range out {
sort.SliceStable(list, func(i, j int) bool { return list[i].ChangedAt > list[j].ChangedAt })
}
return out, nil
}
// Consider every report (ADR 0206): for each account with candidates, under that account's lease, try them
// newest first; the first that refreshes is adopted and the rest are settled as skipped without being
// exchanged. Answers what was adopted.
func (m *Manager) Consider(ctx context.Context, reports []Holdings) ([]string, error) {
byAccount, err := m.CandidatesIn(ctx, reports)
if err != nil {
return nil, err
}
accounts := make([]string, 0, len(byAccount))
for a := range byAccount {
accounts = append(accounts, a)
}
sort.Strings(accounts)
var adopted []string
for _, account := range accounts {
list := byAccount[account]
key := "account:" + account
ok, err := m.Store.Lease(ctx, key, m.Holder, leaseFor)
if err != nil || !ok {
continue
}
for i, c := range list {
name, err := m.adoptOne(ctx, c, reports)
if err != nil {
m.Log("adopting %s's login failed: %v", c.Node, err)
continue
}
if name != "" {
adopted = append(adopted, name)
for _, rest := range list[i+1:] {
_ = m.Store.RecordOutcome(ctx, rest.Fingerprint, rest.Node, account, Skipped, c.Node+"'s newer login was adopted first")
}
break
}
}
_ = m.Store.Unlease(ctx, key, m.Holder)
}
return adopted, nil
}
func (m *Manager) adoptOne(ctx context.Context, c Candidate, reports []Holdings) (string, error) {
answer, err := m.AskGrant(ctx, c.Node, m.Keys.PublicKey)
if err != nil {
// Not answering is not an answer: asked again on the next pass.
m.Log("%s did not hand over its login: %v", c.Node, err)
return "", nil
}
if answer.Sealed == nil {
return "", m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Gone, "no login was waiting when asked")
}
plain, err := Open(*answer.Sealed, m.Keys.PrivateKey)
if err != nil {
return "", m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Refused, "the grant did not open with this module's key")
}
var offered FullGrant
if err := json.Unmarshal([]byte(plain), &offered); err != nil || offered.RefreshToken == "" {
return "", m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Refused, "the grant holds no refresh token")
}
if Fingerprint(offered.RefreshToken) != c.Fingerprint {
m.Log("%s's login changed while it was asked for; waiting for its next report", c.Node)
return "", nil
}
// Adopting is refreshing (ADR 0206 §4): the exchange is the check, and from here this module is the
// only holder of a live refresh token for the account.
r := m.Vendor.Refresh(ctx, offered)
if !r.OK {
_ = m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Dead, fmt.Sprintf("%d %s", r.Status, r.Reason))
_ = m.Store.Audit(ctx, "refused", map[string]any{"node": c.Node, "account": c.Identity.AccountUUID, "status": r.Status, "reason": r.Reason})
_ = m.Emit("licence.refused", map[string]any{"node": c.Node, "account": NameFor(c.Identity),
"reason": fmt.Sprintf("the login's refresh token did not refresh (%d)", r.Status)})
m.Log("%s's login for %s did not refresh: %d %s", c.Node, NameFor(c.Identity), r.Status, r.Reason)
return "", nil
}
// The identity guard, on two sources (ADR 0206 §8).
if r.Account != "" && r.Account != c.Identity.AccountUUID {
_ = m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Refused,
"the node says "+c.Identity.AccountUUID+", the vendor says "+r.Account)
_ = m.Store.Audit(ctx, "refused", map[string]any{"node": c.Node, "reported": c.Identity.AccountUUID, "vendor": r.Account})
_ = m.Emit("licence.refused", map[string]any{"node": c.Node, "account": NameFor(c.Identity),
"reason": "the account the node reported is not the one the vendor answered for"})
return "", nil
}
held, err := m.Store.LicenceForAccount(ctx, c.Identity.AccountUUID)
if err != nil {
return "", err
}
now := m.Now().UnixMilli()
l := Licence{Name: NameFor(c.Identity), Kind: "subscription", AccountUUID: c.Identity.AccountUUID,
Email: c.Identity.EmailAddress, OrganizationUUID: c.Identity.OrganizationUUID}
if held != nil {
l.Name, l.NotifiedAt = held.Name, held.NotifiedAt
if l.Email == "" {
l.Email = held.Email
}
if l.OrganizationUUID == "" {
l.OrganizationUUID = held.OrganizationUUID
}
}
m.keep(&l, r.Grant)
l.AdoptedAt = max(now, c.ChangedAt)
if err := m.Store.SaveLicence(ctx, l); err != nil {
return "", err
}
why := "a new licence"
if held != nil {
why = "a newer login"
}
_ = m.Store.RecordOutcome(ctx, c.Fingerprint, c.Node, c.Identity.AccountUUID, Adopted, why)
_ = m.Store.Audit(ctx, "adopted", map[string]any{"licence": l.Name, "node": c.Node, "account": c.Identity.AccountUUID,
"vendorNamedAccount": r.Account != ""})
// A first binding follows the login (ADR 0206 §7): every node reporting this account and bound to nothing.
for _, rep := range reports {
if rep.Identity == nil || rep.Identity.AccountUUID != c.Identity.AccountUUID {
continue
}
if b, err := m.Store.Binding(ctx, rep.Node); err != nil || b != nil {
continue
}
if _, err := m.Store.Bind(ctx, rep.Node, l.Name); err == nil {
_ = m.Store.Audit(ctx, "bound", map[string]any{"consumer": rep.Node, "licence": l.Name, "by": "its login"})
}
}
if err := m.publishAdvance(ctx, l); err != nil {
return "", err
}
_ = m.Emit("licence.adopted", map[string]any{"licence": l.Name, "from": c.Node, "replaced": held != nil})
m.Log("adopted %s from %s (%s)", l.Name, c.Node, why)
return l.Name, nil
}
// keep stores a grant on its licence: encrypted, fingerprinted, its expiries, fresh.
func (m *Manager) keep(l *Licence, g FullGrant) {
raw, _ := json.Marshal(g)
l.Sealed = m.Crypt.Seal(string(raw))
l.RefreshFingerprint = Fingerprint(g.RefreshToken)
l.AccessExpiresAt = g.ExpiresAt
if g.RefreshTokenExpiresAt != nil {
l.RefreshExpiresAt = *g.RefreshTokenExpiresAt
}
l.Failures = 0
l.RotatedAt = m.Now().UnixMilli()
}
// publishAdvance gives every consumer of a licence a new generation, and tells each in the state.
func (m *Manager) publishAdvance(ctx context.Context, l Licence) error {
bindings, err := m.Store.Advance(ctx, l.Name)
if err != nil {
return err
}
for _, b := range bindings {
if err := m.PutBinding(ctx, b.Consumer, BindingState{Licence: b.Licence, Kind: l.Kind, Generation: b.Generation}); err != nil {
return err
}
}
return nil
}
// ---- keeping grants alive ----------------------------------------------------------------------------
// Due says whether a licence needs a refresh now: near expiry, or older than the cadence.
func Due(l Licence, now time.Time, s Settings) bool {
if l.Kind != "subscription" || l.Sealed == "" {
return false
}
ms := now.UnixMilli()
if l.AccessExpiresAt != 0 && l.AccessExpiresAt-ms < s.Floor.Milliseconds() {
return true
}
return l.RotatedAt == 0 || ms-l.RotatedAt >= s.Cadence.Milliseconds()
}
// Rotate refreshes one licence under its lease (design 39 §3). A second run started together finds the
// lease live and does nothing; one started just after finds a fresh grant and is not due. force refreshes
// whatever the age — the seat's `refresh` verb.
func (m *Manager) Rotate(ctx context.Context, name string, force bool) (map[string]any, error) {
key := "licence:" + name
ok, err := m.Store.Lease(ctx, key, m.Holder, leaseFor)
if err != nil {
return nil, err
}
if !ok {
return map[string]any{"licence": name, "refreshed": false, "reason": "another run holds its lease"}, nil
}
defer func() { _ = m.Store.Unlease(ctx, key, m.Holder) }()
l, err := m.Store.Licence(ctx, name)
if err != nil {
return nil, err
}
if l == nil {
return map[string]any{"licence": name, "refreshed": false, "reason": "no such licence"}, nil
}
if l.Kind != "subscription" || l.Sealed == "" {
return map[string]any{"licence": name, "refreshed": false, "reason": "an API key does not refresh"}, nil
}
now := m.Now()
if !force && !Due(*l, now, m.Settings) {
return map[string]any{"licence": name, "refreshed": false, "reason": "not due"}, nil
}
plain, err := m.Crypt.Open(l.Sealed)
if err != nil {
return nil, err
}
var g FullGrant
if err := json.Unmarshal([]byte(plain), &g); err != nil {
return nil, err
}
r := m.Vendor.Refresh(ctx, g)
if !r.OK {
l.Failures++
m.Log("%s did not refresh (%d in a row): %d %s", name, l.Failures, r.Status, r.Reason)
_ = m.Store.Audit(ctx, "failed", map[string]any{"licence": name, "status": r.Status, "reason": r.Reason, "failures": l.Failures})
m.notify(l, fmt.Sprintf("refresh failed %d time(s) in a row: %d", l.Failures, r.Status), l.Failures >= m.Settings.FailuresToNotify)
if err := m.Store.SaveLicence(ctx, *l); err != nil {
return nil, err
}
return map[string]any{"licence": name, "refreshed": false, "reason": fmt.Sprintf("%d %s", r.Status, r.Reason), "failures": l.Failures}, nil
}
m.keep(l, r.Grant)
if l.RefreshExpiresAt != 0 {
left := time.Duration(l.RefreshExpiresAt-now.UnixMilli()) * time.Millisecond
m.notify(l, fmt.Sprintf("its refresh token expires in %.1f day(s): a person must log in again", left.Hours()/24),
left < m.Settings.RefreshWarn)
}
if err := m.Store.SaveLicence(ctx, *l); err != nil {
return nil, err
}
_ = m.Store.Audit(ctx, "rotated", map[string]any{"licence": name, "expiresAt": l.AccessExpiresAt})
if err := m.publishAdvance(ctx, *l); err != nil {
return nil, err
}
return map[string]any{"licence": name, "refreshed": true, "expiresAt": time.UnixMilli(l.AccessExpiresAt).UTC().Format(time.RFC3339)}, nil
}
// notify emits `licence.failing` at most once per cooldown (design 39 §3); it carries no secret.
func (m *Manager) notify(l *Licence, why string, when bool) {
if !when {
return
}
now := m.Now().UnixMilli()
if l.NotifiedAt != 0 && now-l.NotifiedAt < m.Settings.Cooldown.Milliseconds() {
return
}
l.NotifiedAt = now
_ = m.Emit("licence.failing", map[string]any{"licence": l.Name, "why": why})
}
// RotateDue refreshes every licence that is due.
func (m *Manager) RotateDue(ctx context.Context) ([]map[string]any, error) {
all, err := m.Store.Licences(ctx)
if err != nil {
return nil, err
}
var out []map[string]any
for _, l := range all {
if Due(l, m.Now(), m.Settings) {
r, err := m.Rotate(ctx, l.Name, false)
if err != nil {
return out, err
}
out = append(out, r)
}
}
return out, nil
}
// ReadUsage reads and records each subscription's usage (ADR 0054); the event names the licence and numbers.
func (m *Manager) ReadUsage(ctx context.Context) error {
all, err := m.Store.Licences(ctx)
if err != nil {
return err
}
for _, l := range all {
if l.Kind != "subscription" || l.Sealed == "" {
continue
}
plain, err := m.Crypt.Open(l.Sealed)
if err != nil {
return err
}
var g FullGrant
_ = json.Unmarshal([]byte(plain), &g)
raw, err := m.Vendor.Usage(ctx, g.AccessToken)
if err != nil || raw == nil {
continue
}
reading := FlattenUsage(raw)
if err := m.Store.RecordUsage(ctx, l.Name, m.Now().UnixMilli(), reading, raw); err != nil {
return err
}
_ = m.Emit("usage.read", map[string]any{"licence": l.Name, "sessionPct": reading.SessionPct,
"weeklyPct": reading.WeeklyPct, "sonnetPct": reading.SonnetPct})
}
return nil
}
// ---- the seat's verbs --------------------------------------------------------------------------------
// Current is a consumer's token sealed to the key it sent (ADR 0206 §6): for a subscription the access
// token and its expiries only — never the refresh token, which no node holds. Nil when it is bound to
// nothing.
func (m *Manager) Current(ctx context.Context, consumer, publicKey string) (map[string]any, error) {
if !strings.Contains(publicKey, "PUBLIC KEY") {
return nil, errors.New("current seals to the consumer's public key, and none was given")
}
b, err := m.Store.Binding(ctx, consumer)
if err != nil || b == nil {
return nil, err
}
l, err := m.Store.Licence(ctx, b.Licence)
if err != nil || l == nil || l.Sealed == "" {
return nil, err
}
plain, err := m.Crypt.Open(l.Sealed)
if err != nil {
return nil, err
}
handed := plain
if l.Kind == "subscription" {
var g FullGrant
if err := json.Unmarshal([]byte(plain), &g); err != nil {
return nil, err
}
access, _ := json.Marshal(map[string]any{"accessToken": g.AccessToken, "expiresAt": g.ExpiresAt,
"refreshTokenExpiresAt": g.RefreshTokenExpiresAt, "scopes": g.Scopes,
"subscriptionType": g.SubscriptionType, "rateLimitTier": g.RateLimitTier})
handed = string(access)
}
box, err := Seal(handed, publicKey)
if err != nil {
return nil, err
}
out := map[string]any{"licence": l.Name, "kind": l.Kind, "generation": b.Generation, "sealed": box}
if l.AccountUUID != "" {
out["identity"] = Identity{AccountUUID: l.AccountUUID, EmailAddress: l.Email, OrganizationUUID: l.OrganizationUUID}
}
return out, nil
}
// Bind binds or switches a consumer: a person's act (ADR 0183), told to the consumer as a new generation.
func (m *Manager) Bind(ctx context.Context, consumer, licence, by string) (map[string]any, error) {
l, err := m.Store.Licence(ctx, licence)
if err != nil {
return nil, err
}
if l == nil {
return nil, fmt.Errorf("there is no licence %s; `licences` lists them", licence)
}
before, err := m.Store.Binding(ctx, consumer)
if err != nil {
return nil, err
}
b, err := m.Store.Bind(ctx, consumer, licence)
if err != nil {
return nil, err
}
from, what := "", "bound"
if before != nil {
from, what = before.Licence, "switched"
}
_ = m.Store.Audit(ctx, what, map[string]any{"consumer": consumer, "licence": licence, "from": from, "by": by})
if err := m.PutBinding(ctx, consumer, BindingState{Licence: licence, Kind: l.Kind, Generation: b.Generation}); err != nil {
return nil, err
}
return map[string]any{"consumer": consumer, "licence": licence, "generation": b.Generation, "from": from}, nil
}
// Release unbinds a consumer; its node keeps its last token, which expires within hours.
func (m *Manager) Release(ctx context.Context, consumer, by string) (map[string]any, error) {
was, err := m.Store.Binding(ctx, consumer)
if err != nil {
return nil, err
}
if ok, err := m.Store.Unbind(ctx, consumer); err != nil || !ok {
return map[string]any{"consumer": consumer, "released": false, "reason": "it was bound to nothing"}, err
}
if err := m.DeleteBinding(ctx, consumer); err != nil {
return nil, err
}
_ = m.Store.Audit(ctx, "released", map[string]any{"consumer": consumer, "licence": was.Licence, "by": by})
return map[string]any{"consumer": consumer, "released": true, "was": was.Licence}, nil
}
var licenceName = regexp.MustCompile(`^[A-Za-z0-9@._-]+$`)
// AdoptKey adopts an API key from a file on this node — never an argument (design 39 §6).
func (m *Manager) AdoptKey(ctx context.Context, name, file string) (map[string]any, error) {
if !licenceName.MatchString(name) {
return nil, fmt.Errorf("%q is not a licence name: letters, digits and @._-", name)
}
raw, err := os.ReadFile(file)
if err != nil {
return nil, err
}
key := strings.TrimSpace(string(raw))
if key == "" {
return nil, fmt.Errorf("%s is empty", file)
}
held, err := m.Store.Licence(ctx, name)
if err != nil {
return nil, err
}
if held != nil && held.Kind != "api-key" {
return nil, fmt.Errorf("%s is a subscription; an API key needs a name of its own", name)
}
l := Licence{Name: name, Kind: "api-key", Sealed: m.Crypt.Seal(key), AdoptedAt: m.Now().UnixMilli()}
if err := m.Store.SaveLicence(ctx, l); err != nil {
return nil, err
}
_ = m.Store.Audit(ctx, "adopted", map[string]any{"licence": name, "kind": "api-key", "from": "a file"})
if err := m.publishAdvance(ctx, l); err != nil {
return nil, err
}
_ = m.Emit("licence.adopted", map[string]any{"licence": name, "from": "a file", "replaced": held != nil})
return map[string]any{"licence": name, "kind": "api-key", "adopted": true, "fingerprint": Fingerprint(key)}, nil
}
// Licences is each licence as a person reads it: health and who is bound, never a token.
func (m *Manager) Licences(ctx context.Context) ([]map[string]any, error) {
all, err := m.Store.Licences(ctx)
if err != nil {
return nil, err
}
bindings, err := m.Store.Bindings(ctx)
if err != nil {
return nil, err
}
stamp := func(ms int64) any {
if ms == 0 {
return nil
}
return time.UnixMilli(ms).UTC().Format(time.RFC3339)
}
out := []map[string]any{}
for _, l := range all {
bound := []string{}
for _, b := range bindings {
if b.Licence == l.Name {
bound = append(bound, b.Consumer)
}
}
account := l.Email
if account == "" {
account = l.AccountUUID
}
out = append(out, map[string]any{"name": l.Name, "kind": l.Kind, "account": account,
"accessExpiresAt": stamp(l.AccessExpiresAt), "refreshExpiresAt": stamp(l.RefreshExpiresAt),
"rotatedAt": stamp(l.RotatedAt), "failures": l.Failures, "bound": bound})
}
return out, nil
}
@@ -1,338 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
var t0 = time.Date(2026, 10, 4, 12, 0, 0, 0, time.UTC)
// stubVendor rotates like the real one is presumed to: each refresh token exchanges once.
type stubVendor struct {
mu sync.Mutex
live map[string]bool
exchanged []string
issued int
account string
now func() time.Time
}
func (v *stubVendor) Refresh(_ context.Context, g FullGrant) Refreshed {
v.mu.Lock()
defer v.mu.Unlock()
v.exchanged = append(v.exchanged, g.RefreshToken)
if !v.live[g.RefreshToken] {
return Refreshed{Status: 400, Reason: `{"error":"invalid_grant"}`}
}
delete(v.live, g.RefreshToken)
v.issued++
next := fmt.Sprintf("rt-%d", v.issued)
v.live[next] = true
g.AccessToken = fmt.Sprintf("at-%d", v.issued)
g.RefreshToken = next
g.ExpiresAt = v.now().Add(8 * time.Hour).UnixMilli()
exp := v.now().Add(30 * 24 * time.Hour).UnixMilli()
g.RefreshTokenExpiresAt = &exp
return Refreshed{OK: true, Grant: g, Account: v.account}
}
func (v *stubVendor) Usage(context.Context, string) (map[string]any, error) {
return map[string]any{"five_hour": map[string]any{"utilization": 12.0}}, nil
}
type miniMesh struct {
m *Manager
store *MemoryStore
vendor *stubVendor
logins map[string]FullGrant // node → what its credentials file holds
state map[string]BindingState
events []string
now time.Time
keys KeyPair
askDown bool
}
func newMesh(t *testing.T) *miniMesh {
t.Helper()
mm := &miniMesh{logins: map[string]FullGrant{}, state: map[string]BindingState{}, now: t0}
now := func() time.Time { return mm.now }
mm.store = NewMemoryStore(now)
mm.vendor = &stubVendor{live: map[string]bool{}, now: now}
crypt, _ := NewCrypt("a key the vault made")
mm.keys, _ = GenerateKeyPair()
mm.m = &Manager{
Store: mm.store, Vendor: mm.vendor, Crypt: crypt, Keys: mm.keys,
AskGrant: func(_ context.Context, node, publicKey string) (GrantAnswer, error) {
if mm.askDown {
return GrantAnswer{}, fmt.Errorf("503 no responders")
}
g, ok := mm.logins[node]
if !ok {
return GrantAnswer{}, nil
}
raw, _ := json.Marshal(g)
box, err := Seal(string(raw), publicKey)
return GrantAnswer{Sealed: &box, Fingerprint: Fingerprint(g.RefreshToken)}, err
},
PutBinding: func(_ context.Context, c string, b BindingState) error { mm.state[c] = b; return nil },
DeleteBinding: func(_ context.Context, c string) error { delete(mm.state, c); return nil },
Emit: func(event string, body map[string]any) error {
raw, _ := json.Marshal(body)
mm.events = append(mm.events, event+" "+string(raw))
return nil
},
Now: now, Log: func(string, ...any) {}, Holder: "test", Settings: Defaults,
}
return mm
}
func (mm *miniMesh) report(node, rt string, at time.Time, account string) Holdings {
h := Holdings{Node: node, Identity: &Identity{AccountUUID: account, EmailAddress: account + "@example.org"},
Kind: "subscription", ChangedAt: at.Format(time.RFC3339Nano)}
if rt != "" {
h.Refresh.Present, h.Refresh.Fingerprint = true, Fingerprint(rt)
}
return h
}
func (mm *miniMesh) login(node, rt string, valid bool, at time.Time) Holdings {
mm.logins[node] = FullGrant{AccessToken: "local-" + node, RefreshToken: rt, ExpiresAt: mm.now.Add(time.Hour).UnixMilli()}
if valid {
mm.vendor.live[rt] = true
}
return mm.report(node, rt, at, "acct-1")
}
const licence1 = "acct-1@example.org"
func TestAManagerWithNoLicenceAdoptsTheNewestLoginThatRefreshesAndNeverExchangesTheRest(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
older := mm.login("server", "rt-server", true, t0.Add(-time.Hour))
newest := mm.login("laptop", "rt-laptop", true, t0.Add(-time.Minute))
adopted, err := mm.m.Consider(ctx, []Holdings{older, newest})
if err != nil || len(adopted) != 1 || adopted[0] != licence1 {
t.Fatalf("adopted %v, %v", adopted, err)
}
if strings.Join(mm.vendor.exchanged, ",") != "rt-laptop" {
t.Fatalf("exchanged %v: an older login was exchanged although a newer one refreshed", mm.vendor.exchanged)
}
if o, _ := mm.store.Outcome(ctx, Fingerprint("rt-server")); o != Skipped {
t.Fatalf("the older login is %q, not skipped", o)
}
// A first binding follows the login: both nodes reported this account and were bound to nothing.
bs, _ := mm.store.Bindings(ctx)
if len(bs) != 2 || mm.state["laptop"].Licence != licence1 || mm.state["server"].Licence != licence1 {
t.Fatalf("bindings %v, state %v", bs, mm.state)
}
if !strings.HasPrefix(strings.Join(mm.events, "|"), "licence.adopted") {
t.Fatalf("events %v", mm.events)
}
}
func TestALoginThatDoesNotRefreshAdoptsNothingAndIsNotTriedAgain(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
dead := mm.login("laptop", "rt-dead", false, t0)
if adopted, _ := mm.m.Consider(ctx, []Holdings{dead}); len(adopted) != 0 {
t.Fatalf("adopted %v", adopted)
}
if o, _ := mm.store.Outcome(ctx, Fingerprint("rt-dead")); o != Dead {
t.Fatalf("outcome %q", o)
}
if ls, _ := mm.store.Licences(ctx); len(ls) != 0 {
t.Fatalf("licences %v", ls)
}
if !strings.Contains(strings.Join(mm.events, "|"), "licence.refused") {
t.Fatalf("events %v", mm.events)
}
_, _ = mm.m.Consider(ctx, []Holdings{dead})
if len(mm.vendor.exchanged) != 1 {
t.Fatalf("a dead refresh token was exchanged %d times", len(mm.vendor.exchanged))
}
}
func TestANewerLoginReplacesTheGrantHeldAndAnOlderOneDoesNot(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0.Add(-time.Minute))})
before, _ := mm.store.Licence(ctx, licence1)
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("server", "rt-old", true, t0.Add(-24*time.Hour))})
if strings.Join(mm.vendor.exchanged, ",") != "rt-a" {
t.Fatalf("an older login was exchanged: %v", mm.vendor.exchanged)
}
mm.now = t0.Add(10 * time.Minute)
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("desktop", "rt-new", true, t0.Add(10*time.Minute))})
after, _ := mm.store.Licence(ctx, licence1)
if after.RefreshFingerprint == before.RefreshFingerprint || mm.vendor.exchanged[len(mm.vendor.exchanged)-1] != "rt-new" {
t.Fatalf("a newer login did not win: %v", mm.vendor.exchanged)
}
if ls, _ := mm.store.Licences(ctx); len(ls) != 1 {
t.Fatalf("one account became %d licences", len(ls))
}
}
func TestAReportNamingAnotherAccountThanTheVendorAnsweredForIsRefused(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
mm.vendor.account = "someone-else"
if adopted, _ := mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0)}); len(adopted) != 0 {
t.Fatalf("adopted %v", adopted)
}
if o, _ := mm.store.Outcome(ctx, Fingerprint("rt-a")); o != Refused {
t.Fatalf("outcome %q", o)
}
}
func TestTwoRefreshRunsStartedTogetherRotateAGrantOnce(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0)})
mm.now = t0.Add(5 * time.Hour)
second := *mm.m
second.Holder = "another run"
var wg sync.WaitGroup
results := make([]map[string]any, 2)
for i, m := range []*Manager{mm.m, &second} {
wg.Add(1)
go func() { defer wg.Done(); results[i], _ = m.Rotate(ctx, licence1, false) }()
}
wg.Wait()
n := 0
for _, r := range results {
if r["refreshed"] == true {
n++
}
}
if n != 1 {
t.Fatalf("rotated %d times: %v", n, results)
}
if r, _ := second.Rotate(ctx, licence1, false); r["reason"] != "not due" {
t.Fatalf("a run just after was %v", r)
}
}
func TestARotationAndASwitchEachGiveANewerGeneration(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0)})
g0 := mm.state["laptop"].Generation
_, _ = mm.m.Rotate(ctx, licence1, true)
g1 := mm.state["laptop"].Generation
if g1 <= g0 {
t.Fatalf("a rotation went from %d to %d", g0, g1)
}
file := filepath.Join(t.TempDir(), "key")
_ = os.WriteFile(file, []byte("sk-ant-api-key\n"), 0o600)
if _, err := mm.m.AdoptKey(ctx, "api", file); err != nil {
t.Fatal(err)
}
if _, err := mm.m.Bind(ctx, "laptop", "api", "test"); err != nil {
t.Fatal(err)
}
if mm.state["laptop"].Licence != "api" || mm.state["laptop"].Generation <= g1 {
t.Fatalf("a switch to a licence rotated less often went backwards: %v", mm.state["laptop"])
}
if _, err := mm.m.Release(ctx, "laptop", "test"); err != nil {
t.Fatal(err)
}
if _, ok := mm.state["laptop"]; ok {
t.Fatal("a released consumer still has a binding in the state")
}
}
func TestCurrentHandsAnAccessTokenOnlySealedToTheConsumersKey(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0)})
node, _ := GenerateKeyPair()
answer, err := mm.m.Current(ctx, "laptop", node.PublicKey)
if err != nil || answer["kind"] != "subscription" {
t.Fatalf("%v %v", answer, err)
}
box := answer["sealed"].(SealedBox)
plain, err := Open(box, node.PrivateKey)
if err != nil {
t.Fatal(err)
}
var handed map[string]any
_ = json.Unmarshal([]byte(plain), &handed)
if !strings.HasPrefix(handed["accessToken"].(string), "at-") || handed["refreshToken"] != nil {
t.Fatalf("handed %v", handed)
}
other, _ := GenerateKeyPair()
if _, err := Open(box, other.PrivateKey); err == nil {
t.Fatal("the hand-over opened with another key")
}
if a, _ := mm.m.Current(ctx, "nobody", node.PublicKey); a != nil {
t.Fatalf("a consumer bound to nothing was answered %v", a)
}
}
func TestNothingTheManagerPublishesKeepsOrListsCarriesAToken(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-secret-1", true, t0)})
_, _ = mm.m.Rotate(ctx, licence1, true)
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("server", "rt-secret-dead", false, t0.Add(time.Hour))})
_ = mm.m.ReadUsage(ctx)
ls, _ := mm.m.Licences(ctx)
bs, _ := mm.store.Bindings(ctx)
everything, _ := json.Marshal([]any{mm.events, mm.state, ls, bs})
for _, token := range []string{"rt-secret", "rt-1", "rt-2", "at-1", "at-2", "local-"} {
if strings.Contains(string(everything), token) {
t.Fatalf("%s was published: %s", token, everything)
}
}
row, _ := mm.store.Licence(ctx, licence1)
if strings.Contains(row.Sealed, "rt-") {
t.Fatal("the grant is stored in the clear")
}
}
func TestANodeThatDoesNotAnswerIsAskedAgainAndOneWithNoLoginIsSettled(t *testing.T) {
mm := newMesh(t)
ctx := context.Background()
r := mm.login("laptop", "rt-a", true, t0)
mm.askDown = true
if adopted, _ := mm.m.Consider(ctx, []Holdings{r}); len(adopted) != 0 {
t.Fatalf("adopted %v from a node that did not answer", adopted)
}
if o, _ := mm.store.Outcome(ctx, Fingerprint("rt-a")); o != "" {
t.Fatalf("a node that was down was settled %q", o)
}
mm.askDown = false
if adopted, _ := mm.m.Consider(ctx, []Holdings{r}); len(adopted) != 1 {
t.Fatalf("adopted %v on the next pass", adopted)
}
gone := mm.report("server", "rt-gone", t0.Add(time.Second), "acct-2")
_, _ = mm.m.Consider(ctx, []Holdings{gone})
if o, _ := mm.store.Outcome(ctx, Fingerprint("rt-gone")); o != Gone {
t.Fatalf("outcome %q", o)
}
}
func TestAReportIsReadAsTheAgentModuleWritesIt(t *testing.T) {
// The agent module's own report shape (claude-code's holdingsOf), parsed here.
raw := `{"node":"laptop","identity":{"accountUuid":"u-1","emailAddress":"a@example.org"},"kind":"subscription",
"refresh":{"present":true,"fingerprint":"sha256:0123456789abcdef","expiresAt":null},
"access":{"fingerprint":"sha256:fedcba9876543210","expiresAt":1},"licence":null,"generation":0,
"changedAt":"2026-10-04T11:00:00.000Z"}`
var h Holdings
if err := json.Unmarshal([]byte(raw), &h); err != nil {
t.Fatal(err)
}
if h.Node != "laptop" || h.Identity.AccountUUID != "u-1" || !h.Refresh.Present || h.Refresh.Fingerprint != "sha256:0123456789abcdef" {
t.Fatalf("%+v", h)
}
if _, err := time.Parse(time.RFC3339Nano, h.ChangedAt); err != nil {
t.Fatalf("the report's time does not parse: %v", err)
}
}
@@ -1,281 +0,0 @@
package main
// The store on the mesh's postgres (novox/hq design 39 §1), the database the mesh provisioned for this
// module. The schema is brought to this version's shape by the preparation step (ADR 0135), each
// statement idempotent.
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// Schema is what this version needs.
var Schema = []string{
`create table if not exists licence (
name text primary key,
kind text not null check (kind in ('subscription', 'api-key')),
account_uuid text unique,
email text,
organization_uuid text,
sealed text,
refresh_fingerprint text,
access_expires_at bigint,
refresh_expires_at bigint,
failures integer not null default 0,
notified_at bigint,
adopted_at bigint not null,
rotated_at bigint)`,
`create sequence if not exists binding_generation`,
`create table if not exists binding (
consumer text primary key,
licence text not null references licence(name),
generation bigint not null)`,
`create table if not exists lease (
key text primary key,
holder text not null,
until timestamptz not null)`,
`create table if not exists offered (
fingerprint text primary key,
node text not null,
account_uuid text,
outcome text not null,
why text not null,
at timestamptz not null default now())`,
`create table if not exists usage (
licence text not null,
at bigint not null,
reading jsonb not null,
raw jsonb not null)`,
`create index if not exists usage_by_licence on usage (licence, at desc)`,
`create table if not exists audit (
at timestamptz not null default now(),
what text not null,
detail jsonb not null)`,
}
// PgStore is the store on postgres.
type PgStore struct{ pool *pgxpool.Pool }
// PgStoreFromEnv opens the store the mesh provisioned, its URL in the file DATABASE_URL_FILE names.
func PgStoreFromEnv(ctx context.Context) (*PgStore, error) {
file := os.Getenv("DATABASE_URL_FILE")
if file == "" {
return nil, errors.New("DATABASE_URL_FILE is not set: the manager's database is a requirement the mesh resolves")
}
raw, err := os.ReadFile(file)
if err != nil {
return nil, err
}
return OpenPgStore(ctx, strings.TrimSpace(string(raw)))
}
// OpenPgStore opens a store at a URL.
func OpenPgStore(ctx context.Context, url string) (*PgStore, error) {
pool, err := pgxpool.New(ctx, url)
if err != nil {
return nil, err
}
return &PgStore{pool: pool}, nil
}
// Migrate brings the schema to this version's shape.
func (s *PgStore) Migrate(ctx context.Context) error {
for _, q := range Schema {
if _, err := s.pool.Exec(ctx, q); err != nil {
return fmt.Errorf("%s: %w", strings.Fields(q)[0:6], err)
}
}
return nil
}
const licenceColumns = `name, kind, coalesce(account_uuid,''), coalesce(email,''), coalesce(organization_uuid,''),
coalesce(sealed,''), coalesce(refresh_fingerprint,''), coalesce(access_expires_at,0), coalesce(refresh_expires_at,0),
failures, coalesce(notified_at,0), adopted_at, coalesce(rotated_at,0)`
func scanLicence(row pgx.Row) (*Licence, error) {
var l Licence
err := row.Scan(&l.Name, &l.Kind, &l.AccountUUID, &l.Email, &l.OrganizationUUID, &l.Sealed, &l.RefreshFingerprint,
&l.AccessExpiresAt, &l.RefreshExpiresAt, &l.Failures, &l.NotifiedAt, &l.AdoptedAt, &l.RotatedAt)
if errors.Is(err, pgx.ErrNoRows) {
return nil, nil
}
return &l, err
}
func (s *PgStore) Licences(ctx context.Context) ([]Licence, error) {
rows, err := s.pool.Query(ctx, `select `+licenceColumns+` from licence order by name`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Licence
for rows.Next() {
l, err := scanLicence(rows)
if err != nil {
return nil, err
}
out = append(out, *l)
}
return out, rows.Err()
}
func (s *PgStore) Licence(ctx context.Context, name string) (*Licence, error) {
return scanLicence(s.pool.QueryRow(ctx, `select `+licenceColumns+` from licence where name = $1`, name))
}
func (s *PgStore) LicenceForAccount(ctx context.Context, account string) (*Licence, error) {
return scanLicence(s.pool.QueryRow(ctx, `select `+licenceColumns+` from licence where account_uuid = $1`, account))
}
func nullable(s string) any {
if s == "" {
return nil
}
return s
}
func nullableInt(v int64) any {
if v == 0 {
return nil
}
return v
}
func (s *PgStore) SaveLicence(ctx context.Context, l Licence) error {
_, err := s.pool.Exec(ctx, `insert into licence (name, kind, account_uuid, email, organization_uuid, sealed, refresh_fingerprint,
access_expires_at, refresh_expires_at, failures, notified_at, adopted_at, rotated_at)
values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13)
on conflict (name) do update set kind = excluded.kind, account_uuid = excluded.account_uuid, email = excluded.email,
organization_uuid = excluded.organization_uuid, sealed = excluded.sealed, refresh_fingerprint = excluded.refresh_fingerprint,
access_expires_at = excluded.access_expires_at, refresh_expires_at = excluded.refresh_expires_at,
failures = excluded.failures, notified_at = excluded.notified_at, adopted_at = excluded.adopted_at,
rotated_at = excluded.rotated_at`,
l.Name, l.Kind, nullable(l.AccountUUID), nullable(l.Email), nullable(l.OrganizationUUID), nullable(l.Sealed),
nullable(l.RefreshFingerprint), nullableInt(l.AccessExpiresAt), nullableInt(l.RefreshExpiresAt), l.Failures,
nullableInt(l.NotifiedAt), l.AdoptedAt, nullableInt(l.RotatedAt))
return err
}
// Lease is taken in the store before a row is read (design 39 §3): a second run started together finds
// it live and does nothing.
func (s *PgStore) Lease(ctx context.Context, key, holder string, d time.Duration) (bool, error) {
tag, err := s.pool.Exec(ctx, `insert into lease (key, holder, until) values ($1, $2, now() + make_interval(secs => $3))
on conflict (key) do update set holder = excluded.holder, until = excluded.until
where lease.until < now() or lease.holder = excluded.holder`, key, holder, d.Seconds())
if err != nil {
return false, err
}
return tag.RowsAffected() == 1, nil
}
func (s *PgStore) Unlease(ctx context.Context, key, holder string) error {
_, err := s.pool.Exec(ctx, `delete from lease where key = $1 and holder = $2`, key, holder)
return err
}
func (s *PgStore) bindingsWhere(ctx context.Context, q string, args ...any) ([]Binding, error) {
rows, err := s.pool.Query(ctx, q, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Binding
for rows.Next() {
var b Binding
if err := rows.Scan(&b.Consumer, &b.Licence, &b.Generation); err != nil {
return nil, err
}
out = append(out, b)
}
return out, rows.Err()
}
func (s *PgStore) Bindings(ctx context.Context) ([]Binding, error) {
return s.bindingsWhere(ctx, `select consumer, licence, generation from binding order by consumer`)
}
func (s *PgStore) Binding(ctx context.Context, consumer string) (*Binding, error) {
out, err := s.bindingsWhere(ctx, `select consumer, licence, generation from binding where consumer = $1`, consumer)
if err != nil || len(out) == 0 {
return nil, err
}
return &out[0], nil
}
func (s *PgStore) Bind(ctx context.Context, consumer, licence string) (Binding, error) {
out, err := s.bindingsWhere(ctx, `insert into binding (consumer, licence, generation) values ($1, $2, nextval('binding_generation'))
on conflict (consumer) do update set licence = excluded.licence, generation = excluded.generation
returning consumer, licence, generation`, consumer, licence)
if err != nil {
return Binding{}, err
}
return out[0], nil
}
func (s *PgStore) Unbind(ctx context.Context, consumer string) (bool, error) {
tag, err := s.pool.Exec(ctx, `delete from binding where consumer = $1`, consumer)
return err == nil && tag.RowsAffected() == 1, err
}
func (s *PgStore) Advance(ctx context.Context, licence string) ([]Binding, error) {
return s.bindingsWhere(ctx, `update binding set generation = nextval('binding_generation') where licence = $1
returning consumer, licence, generation`, licence)
}
func (s *PgStore) Outcome(ctx context.Context, fp string) (Outcome, error) {
var o string
err := s.pool.QueryRow(ctx, `select outcome from offered where fingerprint = $1`, fp).Scan(&o)
if errors.Is(err, pgx.ErrNoRows) {
return "", nil
}
return Outcome(o), err
}
func (s *PgStore) RecordOutcome(ctx context.Context, fp, node, account string, o Outcome, why string) error {
_, err := s.pool.Exec(ctx, `insert into offered (fingerprint, node, account_uuid, outcome, why) values ($1,$2,$3,$4,$5)
on conflict (fingerprint) do update set outcome = excluded.outcome, why = excluded.why, at = now()`,
fp, node, nullable(account), string(o), why)
return err
}
func (s *PgStore) RecordUsage(ctx context.Context, licence string, at int64, r UsageReading, raw map[string]any) error {
reading, _ := json.Marshal(r)
rawJSON, _ := json.Marshal(raw)
_, err := s.pool.Exec(ctx, `insert into usage (licence, at, reading, raw) values ($1,$2,$3,$4)`, licence, at, reading, rawJSON)
return err
}
func (s *PgStore) Usage(ctx context.Context, licence string, limit int) ([]UsageRow, error) {
rows, err := s.pool.Query(ctx, `select licence, at, reading from usage where ($1 = '' or licence = $1) order by at desc limit $2`, licence, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var out []UsageRow
for rows.Next() {
var u UsageRow
var reading []byte
if err := rows.Scan(&u.Licence, &u.At, &reading); err != nil {
return nil, err
}
_ = json.Unmarshal(reading, &u.Reading)
out = append(out, u)
}
return out, rows.Err()
}
func (s *PgStore) Audit(ctx context.Context, what string, detail map[string]any) error {
raw, _ := json.Marshal(detail)
_, err := s.pool.Exec(ctx, `insert into audit (what, detail) values ($1, $2)`, what, raw)
return err
}
func (s *PgStore) Close() { s.pool.Close() }
@@ -1,189 +0,0 @@
package main
// Sealing to one recipient (novox/hq ADR 0183, ADR 0206): the manager seals what it hands a consumer to
// the key that consumer sent, and a node seals a waiting login to the key the manager gives. The same box
// the agent module's TypeScript makes and opens, byte for byte — X25519 for the agreement, HKDF-SHA256 for
// the key, AES-256-GCM for the box — so `testdata/sealed-by-typescript.json` is opened here, and a test
// reopens what this seals with the same derivation.
//
// A box is `{ v: 1, eph, iv, tag, ct }`, every field base64; `eph` is the one-time public key as SPKI DER,
// and the key is bound to it and to the recipient's raw public key, so a box cannot be re-addressed.
import (
"crypto/aes"
"crypto/cipher"
"crypto/ecdh"
"crypto/hkdf"
"crypto/rand"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/pem"
"errors"
"fmt"
)
// SealedBox is a value sealed to one recipient.
type SealedBox struct {
V int `json:"v"`
Eph string `json:"eph"`
IV string `json:"iv"`
Tag string `json:"tag"`
Ct string `json:"ct"`
}
// KeyPair is a recipient's keypair as the two PEM strings it is kept and sent as.
type KeyPair struct {
PublicKey string `json:"publicKey"`
PrivateKey string `json:"privateKey"`
}
const sealInfo = "novox-mesh sealed box v1"
// GenerateKeyPair makes an X25519 keypair, PEM-encoded as the agent module's are.
func GenerateKeyPair() (KeyPair, error) {
priv, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
return KeyPair{}, err
}
pubDER, err := x509.MarshalPKIXPublicKey(priv.PublicKey())
if err != nil {
return KeyPair{}, err
}
privDER, err := x509.MarshalPKCS8PrivateKey(priv)
if err != nil {
return KeyPair{}, err
}
return KeyPair{
PublicKey: string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: pubDER})),
PrivateKey: string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privDER})),
}, nil
}
func publicFromPEM(p string) (*ecdh.PublicKey, error) {
block, _ := pem.Decode([]byte(p))
if block == nil {
return nil, errors.New("not a PEM public key")
}
k, err := x509.ParsePKIXPublicKey(block.Bytes)
if err != nil {
return nil, err
}
pub, ok := k.(*ecdh.PublicKey)
if !ok || pub.Curve() != ecdh.X25519() {
return nil, errors.New("not an X25519 public key")
}
return pub, nil
}
func privateFromPEM(p string) (*ecdh.PrivateKey, error) {
block, _ := pem.Decode([]byte(p))
if block == nil {
return nil, errors.New("not a PEM private key")
}
k, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, err
}
priv, ok := k.(*ecdh.PrivateKey)
if !ok || priv.Curve() != ecdh.X25519() {
return nil, errors.New("not an X25519 private key")
}
return priv, nil
}
func boxKey(secret, ephDER, recipientRaw []byte) ([]byte, error) {
salt := append(append([]byte{}, ephDER...), recipientRaw...)
return hkdf.Key(sha256.New, secret, salt, sealInfo, 32)
}
// Seal seals plaintext to the recipient's public key.
func Seal(plaintext, recipientPEM string) (SealedBox, error) {
recipient, err := publicFromPEM(recipientPEM)
if err != nil {
return SealedBox{}, err
}
eph, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
return SealedBox{}, err
}
secret, err := eph.ECDH(recipient)
if err != nil {
return SealedBox{}, err
}
ephDER, err := x509.MarshalPKIXPublicKey(eph.PublicKey())
if err != nil {
return SealedBox{}, err
}
key, err := boxKey(secret, ephDER, recipient.Bytes())
if err != nil {
return SealedBox{}, err
}
gcm, err := newGCM(key)
if err != nil {
return SealedBox{}, err
}
iv := make([]byte, 12)
if _, err := rand.Read(iv); err != nil {
return SealedBox{}, err
}
out := gcm.Seal(nil, iv, []byte(plaintext), nil)
ct, tag := out[:len(out)-gcm.Overhead()], out[len(out)-gcm.Overhead():]
b64 := base64.StdEncoding.EncodeToString
return SealedBox{V: 1, Eph: b64(ephDER), IV: b64(iv), Tag: b64(tag), Ct: b64(ct)}, nil
}
// Open opens a box with the recipient's private key; it fails for a box to another key or one tampered with.
func Open(box SealedBox, privatePEM string) (string, error) {
if box.V != 1 {
return "", errors.New("not a sealed box this module can open")
}
priv, err := privateFromPEM(privatePEM)
if err != nil {
return "", err
}
d := base64.StdEncoding.DecodeString
ephDER, err := d(box.Eph)
if err != nil {
return "", fmt.Errorf("the box's eph: %w", err)
}
ephKey, err := x509.ParsePKIXPublicKey(ephDER)
if err != nil {
return "", err
}
eph, ok := ephKey.(*ecdh.PublicKey)
if !ok {
return "", errors.New("the box's eph is not an X25519 key")
}
secret, err := priv.ECDH(eph)
if err != nil {
return "", err
}
key, err := boxKey(secret, ephDER, priv.PublicKey().Bytes())
if err != nil {
return "", err
}
iv, err1 := d(box.IV)
tag, err2 := d(box.Tag)
ct, err3 := d(box.Ct)
if err := errors.Join(err1, err2, err3); err != nil {
return "", err
}
gcm, err := newGCM(key)
if err != nil {
return "", err
}
plain, err := gcm.Open(nil, iv, append(ct, tag...), nil)
if err != nil {
return "", errors.New("the box does not open with this key")
}
return string(plain), nil
}
func newGCM(key []byte) (cipher.AEAD, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
return cipher.NewGCM(block)
}
@@ -1,43 +0,0 @@
package main
import (
"encoding/json"
"os"
"testing"
)
// A box the agent module's TypeScript sealed opens here: the two implementations are one format.
func TestABoxSealedInTypeScriptOpensInGo(t *testing.T) {
raw, err := os.ReadFile("testdata/sealed-by-typescript.json")
if err != nil {
t.Fatal(err)
}
var f struct {
PrivateKey string `json:"privateKey"`
Box SealedBox `json:"box"`
Plaintext string `json:"plaintext"`
}
if err := json.Unmarshal(raw, &f); err != nil {
t.Fatal(err)
}
got, err := Open(f.Box, f.PrivateKey)
if err != nil || got != f.Plaintext {
t.Fatalf("opened %q, %v", got, err)
}
}
// What Go seals opens with its own key and no other.
func TestABoxOpensOnlyForItsRecipient(t *testing.T) {
a, _ := GenerateKeyPair()
b, _ := GenerateKeyPair()
box, err := Seal("a token", a.PublicKey)
if err != nil {
t.Fatal(err)
}
if got, err := Open(box, a.PrivateKey); err != nil || got != "a token" {
t.Fatalf("opened %q, %v", got, err)
}
if _, err := Open(box, b.PrivateKey); err == nil {
t.Fatal("a box opened for another key")
}
}
@@ -1,263 +0,0 @@
package main
// The manager's store (novox/hq ADR 0183, design 39 §1): licences with their grants encrypted, bindings
// with a generation, what became of each login it was offered, usage readings, and the audit. One
// interface, two implementations — postgres for the mesh (pgstore.go), memory for the tests — so every
// rule is tested without a database, and the database is asked only to keep rows.
import (
"context"
"sort"
"sync"
"time"
)
// Licence is one licence: an account, or an API key.
type Licence struct {
Name string `json:"name"`
Kind string `json:"kind"` // subscription | api-key
AccountUUID string `json:"accountUuid,omitempty"`
Email string `json:"email,omitempty"`
OrganizationUUID string `json:"organizationUuid,omitempty"`
Sealed string `json:"-"` // the grant or the key, encrypted at rest
RefreshFingerprint string `json:"-"`
AccessExpiresAt int64 `json:"accessExpiresAt,omitempty"`
RefreshExpiresAt int64 `json:"refreshExpiresAt,omitempty"`
Failures int `json:"failures"`
NotifiedAt int64 `json:"-"`
AdoptedAt int64 `json:"adoptedAt"`
RotatedAt int64 `json:"rotatedAt,omitempty"`
}
// Binding is one consumer's binding and the generation it was last given (ADR 0206).
type Binding struct {
Consumer string `json:"consumer"`
Licence string `json:"licence"`
Generation int64 `json:"generation"`
}
// Outcome is what became of a login the manager was offered, by its refresh token's fingerprint.
type Outcome string
const (
Adopted Outcome = "adopted"
Dead Outcome = "dead"
Skipped Outcome = "skipped"
Refused Outcome = "refused"
Gone Outcome = "gone"
)
// UsageRow is one usage reading.
type UsageRow struct {
Licence string `json:"licence"`
At int64 `json:"at"`
Reading UsageReading `json:"reading"`
}
// Store is what the manager keeps.
type Store interface {
Licences(ctx context.Context) ([]Licence, error)
Licence(ctx context.Context, name string) (*Licence, error)
LicenceForAccount(ctx context.Context, account string) (*Licence, error)
SaveLicence(ctx context.Context, l Licence) error
// Lease takes a lease for d, or answers false while another holder's is live.
Lease(ctx context.Context, key, holder string, d time.Duration) (bool, error)
Unlease(ctx context.Context, key, holder string) error
Bindings(ctx context.Context) ([]Binding, error)
Binding(ctx context.Context, consumer string) (*Binding, error)
// Bind binds (or switches) a consumer at the next generation.
Bind(ctx context.Context, consumer, licence string) (Binding, error)
Unbind(ctx context.Context, consumer string) (bool, error)
// Advance gives every consumer of a licence a new generation: what a rotation is to them.
Advance(ctx context.Context, licence string) ([]Binding, error)
Outcome(ctx context.Context, fingerprint string) (Outcome, error)
RecordOutcome(ctx context.Context, fingerprint, node, account string, o Outcome, why string) error
RecordUsage(ctx context.Context, licence string, at int64, r UsageReading, raw map[string]any) error
Usage(ctx context.Context, licence string, limit int) ([]UsageRow, error)
Audit(ctx context.Context, what string, detail map[string]any) error
Close()
}
// MemoryStore is the tests' store.
type MemoryStore struct {
mu sync.Mutex
now func() time.Time
rows map[string]Licence
binds map[string]Binding
leases map[string]struct {
holder string
until time.Time
}
outcomes map[string]Outcome
usage []UsageRow
Audits []map[string]any
generation int64
}
// NewMemoryStore is an empty store whose leases age by now.
func NewMemoryStore(now func() time.Time) *MemoryStore {
return &MemoryStore{now: now, rows: map[string]Licence{}, binds: map[string]Binding{},
leases: map[string]struct {
holder string
until time.Time
}{}, outcomes: map[string]Outcome{}}
}
func (m *MemoryStore) Licences(context.Context) ([]Licence, error) {
m.mu.Lock()
defer m.mu.Unlock()
out := make([]Licence, 0, len(m.rows))
for _, l := range m.rows {
out = append(out, l)
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out, nil
}
func (m *MemoryStore) Licence(_ context.Context, name string) (*Licence, error) {
m.mu.Lock()
defer m.mu.Unlock()
if l, ok := m.rows[name]; ok {
return &l, nil
}
return nil, nil
}
func (m *MemoryStore) LicenceForAccount(_ context.Context, account string) (*Licence, error) {
m.mu.Lock()
defer m.mu.Unlock()
for _, l := range m.rows {
if l.AccountUUID == account {
return &l, nil
}
}
return nil, nil
}
func (m *MemoryStore) SaveLicence(_ context.Context, l Licence) error {
m.mu.Lock()
defer m.mu.Unlock()
m.rows[l.Name] = l
return nil
}
func (m *MemoryStore) Lease(_ context.Context, key, holder string, d time.Duration) (bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
if held, ok := m.leases[key]; ok && held.until.After(m.now()) && held.holder != holder {
return false, nil
}
m.leases[key] = struct {
holder string
until time.Time
}{holder, m.now().Add(d)}
return true, nil
}
func (m *MemoryStore) Unlease(_ context.Context, key, holder string) error {
m.mu.Lock()
defer m.mu.Unlock()
if m.leases[key].holder == holder {
delete(m.leases, key)
}
return nil
}
func (m *MemoryStore) Bindings(context.Context) ([]Binding, error) {
m.mu.Lock()
defer m.mu.Unlock()
out := make([]Binding, 0, len(m.binds))
for _, b := range m.binds {
out = append(out, b)
}
sort.Slice(out, func(i, j int) bool { return out[i].Consumer < out[j].Consumer })
return out, nil
}
func (m *MemoryStore) Binding(_ context.Context, consumer string) (*Binding, error) {
m.mu.Lock()
defer m.mu.Unlock()
if b, ok := m.binds[consumer]; ok {
return &b, nil
}
return nil, nil
}
func (m *MemoryStore) Bind(_ context.Context, consumer, licence string) (Binding, error) {
m.mu.Lock()
defer m.mu.Unlock()
m.generation++
b := Binding{Consumer: consumer, Licence: licence, Generation: m.generation}
m.binds[consumer] = b
return b, nil
}
func (m *MemoryStore) Unbind(_ context.Context, consumer string) (bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
_, ok := m.binds[consumer]
delete(m.binds, consumer)
return ok, nil
}
func (m *MemoryStore) Advance(_ context.Context, licence string) ([]Binding, error) {
m.mu.Lock()
defer m.mu.Unlock()
var out []Binding
for c, b := range m.binds {
if b.Licence != licence {
continue
}
m.generation++
b.Generation = m.generation
m.binds[c] = b
out = append(out, b)
}
sort.Slice(out, func(i, j int) bool { return out[i].Consumer < out[j].Consumer })
return out, nil
}
func (m *MemoryStore) Outcome(_ context.Context, fp string) (Outcome, error) {
m.mu.Lock()
defer m.mu.Unlock()
return m.outcomes[fp], nil
}
func (m *MemoryStore) RecordOutcome(_ context.Context, fp, _, _ string, o Outcome, _ string) error {
m.mu.Lock()
defer m.mu.Unlock()
m.outcomes[fp] = o
return nil
}
func (m *MemoryStore) RecordUsage(_ context.Context, licence string, at int64, r UsageReading, _ map[string]any) error {
m.mu.Lock()
defer m.mu.Unlock()
m.usage = append(m.usage, UsageRow{Licence: licence, At: at, Reading: r})
return nil
}
func (m *MemoryStore) Usage(_ context.Context, licence string, limit int) ([]UsageRow, error) {
m.mu.Lock()
defer m.mu.Unlock()
var out []UsageRow
for i := len(m.usage) - 1; i >= 0 && len(out) < limit; i-- {
if licence == "" || m.usage[i].Licence == licence {
out = append(out, m.usage[i])
}
}
return out, nil
}
func (m *MemoryStore) Audit(_ context.Context, what string, detail map[string]any) error {
m.mu.Lock()
defer m.mu.Unlock()
d := map[string]any{"what": what}
for k, v := range detail {
d[k] = v
}
m.Audits = append(m.Audits, d)
return nil
}
func (m *MemoryStore) Close() {}
@@ -1,125 +0,0 @@
package main
import (
"context"
"os"
"strings"
"testing"
"time"
)
// Against a real postgres, because the questions are the database's: does the schema apply twice, does a
// lease refuse a second holder, does a generation only grow. Skipped unless one is named:
//
// docker run -d --rm --name licmgr-pg -e POSTGRES_PASSWORD=t -p 15498:5432 postgres:16-alpine
// MESH_TEST_POSTGRES=postgres://postgres:t@127.0.0.1:15498/postgres go test ./...
func TestTheStoreOnPostgres(t *testing.T) {
url := os.Getenv("MESH_TEST_POSTGRES")
if url == "" {
t.Skip("MESH_TEST_POSTGRES unset")
}
ctx := context.Background()
s, err := OpenPgStore(ctx, url)
if err != nil {
t.Fatal(err)
}
defer s.Close()
for _, table := range []string{"binding", "licence", "lease", "offered", "usage", "audit"} {
_, _ = s.pool.Exec(ctx, "drop table if exists "+table+" cascade")
}
_, _ = s.pool.Exec(ctx, "drop sequence if exists binding_generation")
for i := 0; i < 2; i++ {
if err := s.Migrate(ctx); err != nil {
t.Fatalf("migration %d: %v", i+1, err)
}
}
l := Licence{Name: "a@example.org", Kind: "subscription", AccountUUID: "u-1", Email: "a@example.org", Sealed: "v1.x.y",
RefreshFingerprint: "sha256:1", AccessExpiresAt: 1, RefreshExpiresAt: 2, AdoptedAt: 3}
if err := s.SaveLicence(ctx, l); err != nil {
t.Fatal(err)
}
l.Failures = 2
_ = s.SaveLicence(ctx, l)
if got, _ := s.LicenceForAccount(ctx, "u-1"); got == nil || got.Failures != 2 || got.OrganizationUUID != "" {
t.Fatalf("%+v", got)
}
if none, err := s.Licence(ctx, "nobody"); none != nil || err != nil {
t.Fatalf("%v %v", none, err)
}
lease := func(holder string) bool {
ok, err := s.Lease(ctx, "licence:a", holder, time.Minute)
if err != nil {
t.Fatal(err)
}
return ok
}
if !lease("one") || lease("two") || !lease("one") {
t.Fatal("a lease did not refuse a second holder, or its own holder could not renew it")
}
_ = s.Unlease(ctx, "licence:a", "one")
if !lease("two") {
t.Fatal("a released lease was not taken")
}
b1, _ := s.Bind(ctx, "laptop", l.Name)
adv, _ := s.Advance(ctx, l.Name)
b3, _ := s.Bind(ctx, "laptop", l.Name)
if len(adv) != 1 || !(b1.Generation < adv[0].Generation && adv[0].Generation < b3.Generation) {
t.Fatalf("generations %d %v %d", b1.Generation, adv, b3.Generation)
}
_ = s.RecordOutcome(ctx, "sha256:x", "laptop", "u-1", Dead, "400")
if o, _ := s.Outcome(ctx, "sha256:x"); o != Dead {
t.Fatalf("outcome %q", o)
}
if o, _ := s.Outcome(ctx, "sha256:none"); o != "" {
t.Fatalf("an unknown login is %q", o)
}
pct := 1.0
_ = s.RecordUsage(ctx, l.Name, 5, UsageReading{SessionPct: &pct}, map[string]any{})
if u, _ := s.Usage(ctx, "", 5); len(u) != 1 || *u[0].Reading.SessionPct != 1 {
t.Fatalf("usage %v", u)
}
if err := s.Audit(ctx, "bound", map[string]any{"consumer": "laptop"}); err != nil {
t.Fatal(err)
}
if ok, _ := s.Unbind(ctx, "laptop"); !ok {
t.Fatal("unbind")
}
all, _ := s.Licences(ctx)
if len(all) != 1 || !strings.HasPrefix(all[0].Sealed, "v1.") {
t.Fatalf("%v", all)
}
}
func TestARefreshThatDoesNotRotateKeepsTheRefreshToken(t *testing.T) {
prev := FullGrant{AccessToken: "a", RefreshToken: "r", ExpiresAt: 1}
in := int64(60)
kept := NextGrant(prev, tokenResponse{AccessToken: "a2", ExpiresIn: &in}, 1000)
if !kept.OK || kept.Grant.RefreshToken != "r" || kept.Grant.ExpiresAt != 61_000 {
t.Fatalf("%+v", kept)
}
rotated := NextGrant(prev, tokenResponse{AccessToken: "a3", RefreshToken: "r2"}, 0)
if rotated.Grant.RefreshToken != "r2" {
t.Fatalf("%+v", rotated)
}
if NextGrant(prev, tokenResponse{}, 0).OK {
t.Fatal("an answer with no access token was a grant")
}
}
func TestAGrantAtRestOpensOnlyWithItsKey(t *testing.T) {
a, _ := NewCrypt("one key")
b, _ := NewCrypt("another key")
sealed := a.Seal(`{"refreshToken":"rt"}`)
if strings.Contains(sealed, "rt") {
t.Fatal("stored in the clear")
}
if got, err := a.Open(sealed); err != nil || got != `{"refreshToken":"rt"}` {
t.Fatalf("%q %v", got, err)
}
if _, err := b.Open(sealed); err == nil {
t.Fatal("opened with another key")
}
if _, err := NewCrypt(" "); err == nil {
t.Fatal("an empty key was accepted")
}
}
@@ -1,12 +0,0 @@
{
"privateKey": "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VuBCIEIAi2NK/bN+p7cqYUwv/kz72TgLdmJUfOHCDZTrMpQzpS\n-----END PRIVATE KEY-----\n",
"publicKey": "-----BEGIN PUBLIC KEY-----\nMCowBQYDK2VuAyEARYvD/w+9ah0KWS9T9pd6Ea6CBymUE48vEVk983QPKyY=\n-----END PUBLIC KEY-----\n",
"box": {
"v": 1,
"eph": "MCowBQYDK2VuAyEAIYlSGrJvF8qSjR1aTFWbEQM/NFbZXrarglN0aRLZZ0E=",
"iv": "ABqT/hMukn6+xpjh",
"tag": "POzmsWTPHSn9xLMMJJ9Akg==",
"ct": "m2u0kuQ0PwrsGelM99Z5aBw6FCd6lFISUbwiK5TzzDw4ZrGtsHEvxytoIeFC"
},
"plaintext": "a grant sealed by the TypeScript agent module"
}
@@ -1,187 +0,0 @@
package main
// The only file that talks to Anthropic (novox/hq ADR 0183): the token endpoint, which this module alone
// calls — one rotation source — and the usage endpoint. Ported from the predecessor's manager, whose
// client id and error handling were each earned by an incident.
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
)
// The public Claude Code client's id: not a secret, and a hard-won constant — a metadata URL in its place
// answers 400, which the predecessor once misdiagnosed as a dead grant.
const clientID = "9d1c250a-e61b-44d9-88ed-5944d1962f5e"
func tokenEndpoint() string {
if v := os.Getenv("MESH_ANTHROPIC_TOKEN_ENDPOINT"); v != "" {
return v
}
return "https://platform.claude.com/v1/oauth/token"
}
func usageEndpoint() string {
if v := os.Getenv("MESH_ANTHROPIC_USAGE_ENDPOINT"); v != "" {
return v
}
return "https://api.anthropic.com/api/oauth/usage"
}
// FullGrant is a subscription's grant as this module keeps it: what the agent's credentials file calls
// `claudeAiOauth`.
type FullGrant struct {
AccessToken string `json:"accessToken"`
RefreshToken string `json:"refreshToken"`
ExpiresAt int64 `json:"expiresAt"`
RefreshTokenExpiresAt *int64 `json:"refreshTokenExpiresAt,omitempty"`
Scopes []string `json:"scopes,omitempty"`
SubscriptionType string `json:"subscriptionType,omitempty"`
RateLimitTier string `json:"rateLimitTier,omitempty"`
}
// Refreshed is what a refresh came to: the next grant and the account the vendor answered for, or why not.
type Refreshed struct {
OK bool
Grant FullGrant
Account string // empty when the vendor named none
Status int
Reason string
}
// Vendor is the vendor as the manager reaches it; a test stubs it.
type Vendor interface {
Refresh(ctx context.Context, g FullGrant) Refreshed
Usage(ctx context.Context, accessToken string) (map[string]any, error)
}
type tokenResponse struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn *int64 `json:"expires_in"`
RefreshTokenExpiresIn *int64 `json:"refresh_token_expires_in"`
Scope string `json:"scope"`
Scopes []string `json:"scopes"`
SubscriptionType string `json:"subscription_type"`
Account *struct {
UUID string `json:"uuid"`
} `json:"account"`
}
// NextGrant is the grant a refresh answered, laid over the one refreshed: a refresh token the vendor did
// not rotate is kept, so a rotating vendor and one that does not are both handled.
func NextGrant(prev FullGrant, r tokenResponse, nowMs int64) Refreshed {
if r.AccessToken == "" {
return Refreshed{Status: 200, Reason: "the vendor answered without an access token"}
}
g := prev
g.AccessToken = r.AccessToken
if r.RefreshToken != "" {
g.RefreshToken = r.RefreshToken
}
if r.ExpiresIn != nil {
g.ExpiresAt = nowMs + *r.ExpiresIn*1000
}
if r.RefreshTokenExpiresIn != nil {
v := nowMs + *r.RefreshTokenExpiresIn*1000
g.RefreshTokenExpiresAt = &v
}
if len(r.Scopes) > 0 {
g.Scopes = r.Scopes
} else if r.Scope != "" {
g.Scopes = strings.Fields(r.Scope)
}
if r.SubscriptionType != "" {
g.SubscriptionType = r.SubscriptionType
}
out := Refreshed{OK: true, Grant: g}
if r.Account != nil {
out.Account = r.Account.UUID
}
return out
}
type liveVendor struct{ client *http.Client }
// LiveVendor is the vendor over the network.
func LiveVendor() Vendor { return liveVendor{client: &http.Client{Timeout: 30 * time.Second}} }
func (v liveVendor) Refresh(ctx context.Context, g FullGrant) Refreshed {
form := url.Values{"grant_type": {"refresh_token"}, "refresh_token": {g.RefreshToken}, "client_id": {clientID}}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenEndpoint(), strings.NewReader(form.Encode()))
if err != nil {
return Refreshed{Reason: err.Error()}
}
req.Header.Set("content-type", "application/x-www-form-urlencoded")
resp, err := v.client.Do(req)
if err != nil {
return Refreshed{Reason: "the token endpoint did not answer: " + err.Error()}
}
defer resp.Body.Close()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if resp.StatusCode/100 != 2 {
// The body, never only the status: a malformed request and a revoked grant both answer 400.
reason := string(body)
if len(reason) > 400 {
reason = reason[:400]
}
return Refreshed{Status: resp.StatusCode, Reason: reason}
}
var r tokenResponse
if err := json.Unmarshal(body, &r); err != nil {
return Refreshed{Status: resp.StatusCode, Reason: "the vendor's answer is not JSON"}
}
return NextGrant(g, r, time.Now().UnixMilli())
}
func (v liveVendor) Usage(ctx context.Context, accessToken string) (map[string]any, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, usageEndpoint(), nil)
if err != nil {
return nil, err
}
req.Header.Set("authorization", "Bearer "+accessToken)
resp, err := v.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
return nil, fmt.Errorf("the usage endpoint answered %d", resp.StatusCode)
}
var out map[string]any
return out, json.NewDecoder(resp.Body).Decode(&out)
}
// UsageReading is the licence-grain reading (ADR 0054), flattened from the vendor's windows.
type UsageReading struct {
SessionPct *float64 `json:"sessionPct"`
SessionResetsAt string `json:"sessionResetsAt,omitempty"`
WeeklyPct *float64 `json:"weeklyPct"`
SonnetPct *float64 `json:"sonnetPct"`
}
// FlattenUsage reads the windows the predecessor read.
func FlattenUsage(u map[string]any) UsageReading {
window := func(k string) (*float64, string) {
w, ok := u[k].(map[string]any)
if !ok {
return nil, ""
}
pct, ok := w["utilization"].(float64)
resets, _ := w["resets_at"].(string)
if !ok {
return nil, resets
}
return &pct, resets
}
s, resets := window("five_hour")
w, _ := window("seven_day")
so, _ := window("seven_day_sonnet")
return UsageReading{SessionPct: s, SessionResetsAt: resets, WeeklyPct: w, SonnetPct: so}
}
-16
View File
@@ -1,16 +0,0 @@
module claude-licence-manager
go 1.25.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.7
github.com/jackc/pgx/v5 v5.11.0
)
require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/text v0.29.0 // indirect
)
-28
View File
@@ -1,28 +0,0 @@
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-130
View File
@@ -1,130 +0,0 @@
{
"module": "claude-licence-manager",
"version": "1",
"slug": "licmgr",
"requires": [
"postgres-database",
"secret"
],
"contributes": {
"postgres-database": {
"name": "claude_licences"
}
},
"binds": {
"postgres-database": "${dir:state}/database.json"
},
"secrets": {
"postgres-database": "${dir:state}/database.secret",
"secret": {
"grant-key": "${dir:state}/grant.key"
}
},
"seats": [
{
"name": "anthropic-licence-manager",
"scope": "mesh",
"serves": [
"licences",
"bindings",
"bind",
"switch",
"release",
"refresh",
"usage",
"adopt",
"current"
]
}
],
"claims": [
{
"name": "anthropic-licence-manager",
"scope": "mesh",
"serves": [
"licences",
"bindings",
"bind",
"switch",
"release",
"refresh",
"usage",
"adopt",
"current"
]
}
],
"emits": [
"licence.adopted",
"licence.refused",
"licence.failing",
"usage.read"
],
"state": [
"bindings"
],
"reads": [
"claude-code.holdings"
],
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "database-url",
"type": "file",
"path": "${dir:state}/database.url",
"mode": "0600",
"content": "postgresql://${bound:postgres-database:as}:${secret:postgres-database}@${bound:postgres-database:at}:${bound:postgres-database:port}/${bound:postgres-database:as}\n"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"mode": "0600",
"merge": "json",
"content": "{\n \"cadence_minutes\": 240,\n \"floor_minutes\": 60,\n \"failures_to_notify\": 3,\n \"cooldown_hours\": 24,\n \"refresh_warn_days\": 3\n}\n"
},
{
"id": "prepare",
"type": "process",
"name": "claude-licence-manager-prepare",
"artifact": "code",
"run": [
"./claude-licence-manager",
"prepare"
],
"run-once": true,
"env": {
"DATABASE_URL_FILE": "${dir:state}/database.url"
},
"restart-on": [
"database-url"
]
}
],
"build": {
"artifacts": [
{
"name": "code",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/claude-licence-manager",
"binary": "claude-licence-manager",
"loads": [
"claude-licence-manager"
],
"env": {
"DATABASE_URL_FILE": "${dir:state}/database.url",
"MESH_LICENCE_STATE": "${dir:state}",
"MESH_LICENCE_KEY_FILE": "${dir:state}/grant.key",
"MESH_LICENCE_SETTINGS": "${dir:state}/settings.json"
}
}
]
}
}
+24
View File
@@ -0,0 +1,24 @@
# cloudflare-dns's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/cloudflare-dns
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts provisioner/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/cloudflare-dns/dist /app/modules/cloudflare-dns/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/cloudflare-dns/dist/tools/index.js,/app/modules/cloudflare-dns/dist/provisioner/index.js
+48 -22
View File
@@ -12,61 +12,87 @@
"public-dns": {} "public-dns": {}
}, },
"grants": { "grants": {
"public-dns": "${dir:grants}" "public-dns": "/var/lib/cloudflare-dns/grants"
}, },
"receives": { "receives": {
"public-dns": "${dir:grants}/mesh.json" "public-dns": "/var/lib/cloudflare-dns/grants/mesh.json"
}, },
"own-secrets": { "own-secrets": {
"token": "${dir:state}/token" "token": "/var/lib/cloudflare-dns/token",
"broker": "/var/lib/mesh/cloudflare-dns/broker"
}, },
"emits": [ "emits": [
"record.created", "record.created",
"record.removed" "record.removed"
], ],
"resources": [ "resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/cloudflare-dns",
"mode": "0700"
},
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/cloudflare-dns",
"place": "." "mode": "0700"
}, },
{ {
"id": "grants", "id": "grants",
"type": "directory", "type": "directory",
"path": "/var/lib/cloudflare-dns/grants",
"mode": "0700" "mode": "0700"
}, },
{ {
"id": "config", "id": "config",
"type": "file", "type": "file",
"path": "${dir:state}/config.json", "path": "/var/lib/cloudflare-dns/config.json",
"merge": "json", "merge": "json",
"content": "{}", "content": "{}",
"mode": "0600" "mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-cloudflare-dns",
"network": "host",
"volumes": [
"/var/lib/cloudflare-dns/config.json:/run/config/config.json:ro",
"/var/lib/cloudflare-dns/grants:/grants",
"/var/lib/cloudflare-dns/token:/run/secrets/token:ro",
"/var/lib/mesh/cloudflare-dns/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_CLOUDFLARE_TOKEN_FILE": "/run/secrets/token",
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_CLOUDFLARE_CONFIG_FILE": "/run/config/config.json",
"MESH_RECEIVES": "/var/lib/cloudflare-dns/grants/mesh.json"
},
"artifact": "runtime"
} }
], ],
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"tools/index.js",
"provisioner/index.js"
],
"loads": [
"tools/index.js",
"provisioner/index.js"
],
"env": {
"MESH_CLOUDFLARE_TOKEN_FILE": "${dir:state}/token",
"MESH_CLOUDFLARE_CONFIG_FILE": "${dir:state}/config.json",
"MESH_RECEIVES": "${dir:grants}/mesh.json"
}
} }
] ]
} }
+24
View File
@@ -0,0 +1,24 @@
# confluence's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/confluence
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/confluence/dist /app/modules/confluence/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/confluence/dist/tools/index.js
+43 -17
View File
@@ -3,43 +3,69 @@
"version": "1", "version": "1",
"slug": "confl", "slug": "confl",
"own-secrets": { "own-secrets": {
"token": "${dir:state}/token" "token": "/var/lib/confluence/token",
"broker": "/var/lib/mesh/confluence/broker"
}, },
"resources": [ "resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/confluence",
"mode": "0700"
},
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/confluence",
"place": "." "mode": "0700"
}, },
{ {
"id": "config", "id": "config",
"type": "file", "type": "file",
"path": "${dir:state}/config.json", "path": "/var/lib/confluence/config.json",
"merge": "json", "merge": "json",
"content": "{}", "content": "{}",
"mode": "0600" "mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-runtime-confluence",
"network": "host",
"volumes": [
"/var/lib/confluence/config.json:/run/config/config.json:ro",
"/var/lib/confluence/token:/run/secrets/token:ro",
"/var/lib/mesh/confluence/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_CONFLUENCE_TOKEN_FILE": "/run/secrets/token",
"MESH_CONFLUENCE_CONFIG_FILE": "/run/config/config.json",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"artifact": "runtime"
} }
], ],
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "tools", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_CONFLUENCE_TOKEN_FILE": "${dir:state}/token",
"MESH_CONFLUENCE_CONFIG_FILE": "${dir:state}/config.json"
}
} }
] ]
} }
+9 -12
View File
@@ -15,11 +15,11 @@
} }
}, },
"binds": { "binds": {
"route": "${dir:state}/route.json" "route": "/var/lib/de-spiegel/route.json"
}, },
"own-secrets": { "own-secrets": {
"smtp-user": "${dir:state}/smtp-user.secret", "smtp-user": "/var/lib/de-spiegel/smtp-user.secret",
"smtp-pass": "${dir:state}/smtp-pass.secret" "smtp-pass": "/var/lib/de-spiegel/smtp-pass.secret"
}, },
"listens": [ "listens": [
{ {
@@ -27,20 +27,20 @@
"port": 35621, "port": 35621,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "the de-spiegel site and its /contact endpoint over http; its public name is a route grant, and route-proxy reaches it on this published port" "why": "the de-spiegel site and its /contact endpoint over http; the public name de-spiegel.novox.be is a route grant, and route-proxy reaches it on this published port"
} }
], ],
"resources": [ "resources": [
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/de-spiegel",
"place": "." "mode": "0700"
}, },
{ {
"id": "server-env", "id": "server-env",
"type": "file", "type": "file",
"path": "${dir:state}/server.env", "path": "/var/lib/de-spiegel/server.env",
"mode": "0600", "mode": "0600",
"content": "SMTP_AUTH_USER=${secret:smtp-user}\nSMTP_AUTH_PASS=${secret:smtp-pass}\n" "content": "SMTP_AUTH_USER=${secret:smtp-user}\nSMTP_AUTH_PASS=${secret:smtp-pass}\n"
}, },
@@ -56,15 +56,12 @@
"image": "registry-api.novox.be/novox/de-spiegel@sha256:e144b72ce9c145870470d765343549f2c60211728cd118b9ff0e4029f36342ba", "image": "registry-api.novox.be/novox/de-spiegel@sha256:e144b72ce9c145870470d765343549f2c60211728cd118b9ff0e4029f36342ba",
"network": "de-spiegel", "network": "de-spiegel",
"env-file": [ "env-file": [
"${dir:state}/server.env" "/var/lib/de-spiegel/server.env"
], ],
"ports": [ "ports": [
"35621" "35621"
], ],
"secrets-in-environment": "the application's own code reads SMTP_AUTH_USER/PASS from the environment (de-spiegel server/index.js); converting is that repository's change", "secrets-in-environment": "the application's own code reads SMTP_AUTH_USER/PASS from the environment (de-spiegel server/index.js); converting is that repository's change"
"names-on-purpose": {
"registry-api.novox.be": "built outside the mesh, from the application's own repository, and pulled from the registry that built it; moves when that repository is a build source on the git seat (novox/hq ADR 0155, issue 122)"
}
} }
] ]
} }
+1 -2
View File
@@ -3,8 +3,7 @@
"version": "1", "version": "1",
"capabilities": [ "capabilities": [
"package-manager", "package-manager",
"service-manager", "service-manager"
"uplink-dhcpcd"
], ],
"claims": [ "claims": [
{ {
+2 -5
View File
@@ -9,7 +9,7 @@
], ],
"claims": [ "claims": [
{ {
"name": "mesh-artifact-store", "name": "the-artifact-store",
"scope": "mesh" "scope": "mesh"
} }
], ],
@@ -59,10 +59,7 @@
], ],
"volumes": [ "volumes": [
"/var/lib/mesh-registry:/var/lib/registry" "/var/lib/mesh-registry:/var/lib/registry"
], ]
"env": {
"REGISTRY_STORAGE_DELETE_ENABLED": "true"
}
} }
] ]
} }
File diff suppressed because one or more lines are too long
+36 -221
View File
@@ -1,236 +1,51 @@
// fail2ban's own code, in the module (novox/hq ADR 0039). The jails are composed by the mesh from // fail2ban's own code, in the module (novox/hq ADR 0039). The jails and the daemon are declared
// the modules a machine runs (to-be 31) and written as declared resources; the daemon is kept // resources — the mesh writes /etc/fail2ban/jail.d/* and keeps fail2ban.service running (see
// running by one. This code exists only to read and steer the *live* state the daemon owns: who is // module.json). This code exists only to read and steer the *live* state the daemon owns at
// banned now and until when, and the ban or release an operator asks for — the node-intrusion- // runtime: which IPs are banned right now, and the manual ban/unban an operator reaches for. That
// prevention seat's four verbs (ADR 0179). The daemon's state is fail2ban's, not the mesh's: the // state (the running bans, /var/lib/fail2ban's sqlite) is fail2ban's, not the mesh's — the mesh
// mesh composes the jails and never writes the ban list. // reconciles the config, never the ban list.
//
// Spoken through fail2ban-client over the daemon's socket. Client and daemon come from the one
// package this module declares on the machine, and the socket is root's: root is the module's
// concern (ADR 0175 §4), and the runtime loading this bundle runs as the operator's account (to-be
// 38 WP4), so the client is run through sudo without a prompt where the account is not root.
import { execFile } from "node:child_process"; import { execFile } from "node:child_process";
import { accessSync, constants } from "node:fs";
import { isIP } from "node:net";
import { delimiter, join } from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
const execFileP = promisify(execFile); const run = promisify(execFile);
/** A command runner, so the verbs can be tested without a daemon. */
export type Runner = (cmd: string, args: string[]) => Promise<string>;
/** The command as it is run: as given when this process is root, else through sudo without a
* prompt. The daemon's socket answers only to root. */
export function escalated(cmd: string, args: string[], uid: number | undefined = process.getuid?.()): [string, string[]] {
if (uid === 0) return [cmd, args];
return ["sudo", ["-n", cmd, ...args]];
}
/** Whether a tool is on this machine: an executable of that name on the path, or where the
* system keeps its administration. */
export function installed(tool: string, path: string = process.env.PATH ?? ""): boolean {
const dirs = [...path.split(delimiter), "/usr/sbin", "/sbin", "/usr/bin"].filter((d) => d !== "");
return dirs.some((dir) => {
try {
accessSync(join(dir, tool), constants.X_OK);
return true;
} catch {
return false;
}
});
}
export const execRunner: Runner = async (cmd, args) => {
if (!installed(cmd)) throw new Error(`${cmd} is not installed on this machine`);
const [program, argv] = escalated(cmd, args);
try {
const { stdout } = await execFileP(program, argv, { maxBuffer: 16 * 1024 * 1024 });
return stdout;
} catch (err) {
const e = err as { code?: string | number; stderr?: string; stdout?: string; message?: string };
const said = `${e.stdout ?? ""}${e.stderr ?? ""}`.trim();
// What failed is named by how it failed: sudo missing is a spawn error, sudo refusing speaks
// on its own stderr line, and the rest is the client's own answer.
if (program === "sudo") {
if (e.code === "ENOENT") throw new Error(`${cmd} needs root, and sudo is not installed here for the runtime's account to escalate with`);
if (/^sudo:/m.test(said)) throw new Error(`${cmd} needs root and the runtime's account may not run it without a prompt: ${said}`);
}
if (/Failed to access socket path|Is fail2ban running|Permission denied to socket/i.test(said)) {
throw new Error("fail2ban is not running on this machine, or its socket does not answer the runtime's account");
}
// fail2ban-client's own last line is the one a person reads ("Sorry but the jail 'x' does not exist").
const lines = said.split("\n").map((l) => l.trim()).filter(Boolean);
throw new Error(lines.length ? lines[lines.length - 1] : (e.message ?? `${cmd} failed`));
}
};
/** One jail as the daemon reports it. */
export interface JailStatus {
jail: string;
/** What the jail is reading: files or journal matches, as fail2ban names them. */
watching: string[];
/** Addresses with failures counted against them right now, and all failures since the jail started. */
failing: { now: number; total: number };
/** Addresses held right now, and all bans since the jail started. */
banned: { now: number; total: number; addresses: string[] };
}
/** One ban as the daemon holds it. */
export interface Ban {
ip: string;
jail: string;
/** When the ban was placed, in the machine's local time as fail2ban prints it. */
since: string;
/** When the ban ends; "never" for a permanent ban. */
until: string;
}
export interface JailSettings {
jail: string;
bantime: string;
findtime: string;
maxretry: number;
ignoreip: string[];
actions: string[];
/** The log files the jail reads, when it reads files. */
logpath: string[];
/** The journal match the jail reads, when it reads the journal. */
journalmatch: string;
}
export class Fail2banClient { export class Fail2banClient {
private readonly run: Runner; static fromEnv(_env: NodeJS.ProcessEnv = process.env): Fail2banClient {
constructor(run: Runner = execRunner) {
this.run = run;
}
/** The daemon as this machine has it, through its own client. */
static onThisMachine(): Fail2banClient {
return new Fail2banClient(); return new Fail2banClient();
} }
private client(...args: string[]): Promise<string> { /** Overview of every jail, or the detailed status of one — currently-banned IPs and totals. */
return this.run("fail2ban-client", args); async status(jail?: string): Promise<string> {
}
/** The jails the daemon runs, by name. */
async jails(): Promise<string[]> {
const out = await this.client("status");
const m = out.match(/Jail list:\s*(.*)/);
if (!m) return [];
return m[1].split(",").map((j) => j.trim()).filter(Boolean);
}
/** Every jail with what it watches and holds, or one jail's detail. */
async status(jail?: string): Promise<{ jails: JailStatus[] }> {
const names = jail ? [jail] : await this.jails();
const jails: JailStatus[] = [];
for (const name of names) {
jails.push(parseJailStatus(name, await this.client("status", name)));
}
return { jails };
}
/** Every address banned now, with the jail holding it and when the ban ends. */
async banned(jail?: string): Promise<{ banned: Ban[] }> {
const names = jail ? [jail] : await this.jails();
const banned: Ban[] = [];
for (const name of names) {
banned.push(...parseBans(name, await this.client("get", name, "banip", "--with-time")));
}
banned.sort((a, b) => a.until.localeCompare(b.until) || a.ip.localeCompare(b.ip));
return { banned };
}
/** Ban one address in one jail now. The daemon's own answer is how many addresses it added. */
async ban(ip: string, jail: string): Promise<{ banned: Ban | null; added: number }> {
address(ip);
name(jail);
const out = await this.client("set", jail, "banip", ip);
const added = Number.parseInt(out.trim(), 10) || 0;
const held = (await this.banned(jail)).banned.find((b) => b.ip === ip) ?? null;
return { banned: held, added };
}
/** Let one address go, from one jail or from every jail. The daemon's answer is how many it released. */
async unban(ip: string, jail?: string): Promise<{ released: number; ip: string; jail: string | "every jail" }> {
address(ip);
let out: string;
if (jail) { if (jail) {
name(jail); const { stdout } = await run("sudo", ["fail2ban-client", "status", jail]);
out = await this.client("set", jail, "unbanip", ip); return stdout;
} else {
out = await this.client("unban", ip);
} }
return { released: Number.parseInt(out.trim(), 10) || 0, ip, jail: jail ?? "every jail" }; const { stdout: overview } = await run("sudo", ["fail2ban-client", "status"]);
const match = overview.match(/Jail list:\s*(.+)/);
if (!match) return overview;
const jails = match[1].split(",").map((j) => j.trim()).filter(Boolean);
const parts: string[] = [overview.trimEnd(), ""];
for (const j of jails) {
const { stdout } = await run("sudo", ["fail2ban-client", "status", j]);
parts.push(`=== ${j} ===`, stdout.trimEnd(), "");
}
return parts.join("\n");
} }
/** One jail's effective settings — the module's own tool, beside the seat's verbs. */ /** Manually ban an IP in a jail. Mutates live state, not a mesh-managed file. */
async settings(jail: string): Promise<JailSettings> { async ban(jail: string, ip: string): Promise<string> {
name(jail); const { stdout } = await run("sudo", ["fail2ban-client", "set", jail, "banip", ip]);
const get = (key: string) => this.client("get", jail, key); return stdout;
const [bantime, findtime, maxretry, ignoreip, actions, logpath, journalmatch] = await Promise.all([ }
get("bantime"), get("findtime"), get("maxretry"), get("ignoreip"), get("actions"), get("logpath"),
get("journalmatch"), /** Unban an IP from one jail, or from every jail when no jail is given. */
]); async unban(ip: string, jail?: string): Promise<string> {
return { const args = jail
jail, ? ["fail2ban-client", "set", jail, "unbanip", ip]
bantime: bantime.trim(), : ["fail2ban-client", "unban", ip];
findtime: findtime.trim(), const { stdout } = await run("sudo", args);
maxretry: Number.parseInt(maxretry.trim(), 10), return stdout;
ignoreip: listed(ignoreip),
actions: actions.split("\n").slice(1).map((l) => l.trim()).filter(Boolean),
logpath: /No file is currently monitored/.test(logpath) ? [] : listed(logpath),
journalmatch: journalmatch.split("\n").slice(1).map((l) => l.trim()).filter(Boolean).join(" "),
};
} }
} }
/** fail2ban's tree listings: lines like "|- 127.0.0.0/8" and "`- ::1", after a heading. */
function listed(out: string): string[] {
return out
.split("\n")
.map((l) => l.replace(/^[\s|`-]+/, "").trim())
.filter((l, i) => i > 0 && l.length > 0);
}
export function parseJailStatus(jail: string, out: string): JailStatus {
const field = (label: string) => {
const m = out.match(new RegExp(label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") + ":\\t?\\s*(.*)"));
return m ? m[1].trim() : "";
};
const num = (label: string) => Number.parseInt(field(label), 10) || 0;
const watching = [field("File list"), field("Journal matches")].filter(Boolean);
return {
jail,
watching,
failing: { now: num("Currently failed"), total: num("Total failed") },
banned: {
now: num("Currently banned"),
total: num("Total banned"),
addresses: field("Banned IP list").split(/\s+/).filter(Boolean),
},
};
}
/** `get <jail> banip --with-time` prints one ban per line: "IP \tsince + seconds = until". */
export function parseBans(jail: string, out: string): Ban[] {
const bans: Ban[] = [];
for (const line of out.split("\n")) {
const m = line.match(/^(\S+)\s+(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) \+ (-?\d+) = (\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}|\S+)/);
if (!m) continue;
bans.push({ ip: m[1], jail, since: m[2], until: Number(m[3]) < 0 ? "never" : m[4] });
}
return bans;
}
function address(ip: string): void {
if (!isIP(ip)) throw new Error(`${JSON.stringify(ip)} is not an address`);
}
function name(jail: string): void {
if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/.test(jail)) throw new Error(`${JSON.stringify(jail)} is not a jail's name`);
}
+6 -44
View File
@@ -7,22 +7,9 @@
"claims": [ "claims": [
{ {
"name": "node-intrusion-prevention", "name": "node-intrusion-prevention",
"scope": "node", "scope": "node"
"serves": [
"status",
"banned",
"ban",
"unban"
]
} }
], ],
"tools": [
"fail2ban_settings"
],
"jailing": {
"into": "/etc/fail2ban/jail.d/mesh.conf",
"filter-into": "/etc/fail2ban/filter.d"
},
"resources": [ "resources": [
{ {
"id": "package", "id": "package",
@@ -41,31 +28,19 @@
"path": "/etc/fail2ban/action.d", "path": "/etc/fail2ban/action.d",
"mode": "0755" "mode": "0755"
}, },
{
"id": "filter-d",
"type": "directory",
"path": "/etc/fail2ban/filter.d",
"mode": "0755"
},
{
"id": "run-dir",
"type": "directory",
"path": "/var/run/fail2ban",
"mode": "0755"
},
{ {
"id": "jail-local", "id": "jail-local",
"type": "file", "type": "file",
"path": "/etc/fail2ban/jail.local", "path": "/etc/fail2ban/jail.local",
"mode": "0644", "mode": "0644",
"content": "[INCLUDES]\n\nbefore = paths-arch.conf\n\n[DEFAULT]\n\n# Never act on the machine itself or on a tunnel peer: the mesh's private range is\n# ${machine:mesh-range}, named here rather than written as a value the module cannot\n# know (novox/hq ADR 0112). Without this, fail2ban could ban the mesh's own nodes.\n# **A ban list never holds a neighbour.** The mesh's own range is named rather than written\n# (novox/hq ADR 0112), and every private range beside it: a source on one is somebody's own\n# network, not the internet. On a machine behind a router that reflects local traffic, every\n# client in the house arrives as the gateway's address — so one mistyped local request banned\n# 192.168.1.1 on the home server and would have cut the whole house off from it (ADR 0186).\nignoreip = 127.0.0.1/8 ::1 ${machine:mesh-range} 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 169.254.0.0/16 fc00::/7 fe80::/10\n\n# Three failures in a day ban for a day (novox/hq ADR 0179). The attackers this mesh sees pace\n# themselves at one try every ten minutes, under any ten-minute window; a day's window counts\n# them, and a day's ban costs a person who mistyped three times once, from one address, while\n# the mesh's own range is never banned at all.\nbantime = 1d\nfindtime = 1d\nmaxretry = 3\n\n# Ban through iptables, not through a firewall front-end the machine may not have. ufw is\n# installed on two of this mesh's machines and absent on the other two, and fail2ban finds out\n# only at ban time: the service reports healthy, the jail counts the attempt, the ban command\n# exits 127, and nothing is blocked. Proven on 2026-09-28 -- 'ufw: command not found' on a\n# machine the mesh reported as protected.\n#\n# The action below is this module's own, already used by the recidive jail on every machine\n# here, and it bans in DOCKER-USER as well as INPUT, so a container's published port is\n# covered too.\nbanaction = iptables-allports-dualchain\nbanaction_allports = iptables-allports-dualchain\n\n[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\n" "content": "[INCLUDES]\n\nbefore = paths-arch.conf\n\n[DEFAULT]\n\n# Never act on the machine itself or on a tunnel peer: the mesh's private range is\n# ${machine:mesh-range}, named here rather than written as a value the module cannot\n# know (novox/hq ADR 0112). Without this, fail2ban could ban the mesh's own nodes.\nignoreip = 127.0.0.1/8 ::1 ${machine:mesh-range}\n\nbantime = 10m\nfindtime = 10m\nmaxretry = 5\n\n# Ban through iptables, not through a firewall front-end the machine may not have. ufw is\n# installed on two of this mesh's machines and absent on the other two, and fail2ban finds out\n# only at ban time: the service reports healthy, the jail counts the attempt, the ban command\n# exits 127, and nothing is blocked. Proven on 2026-09-28 -- 'ufw: command not found' on a\n# machine the mesh reported as protected.\n#\n# The action below is this module's own, already used by the recidive jail on every machine\n# here, and it bans in DOCKER-USER as well as INPUT, so a container's published port is\n# covered too.\nbanaction = iptables-allports-dualchain\nbanaction_allports = iptables-allports-dualchain\n\n[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\n"
}, },
{ {
"id": "jail-sshd", "id": "jail-sshd",
"type": "file", "type": "file",
"path": "/etc/fail2ban/jail.d/sshd.conf", "path": "/etc/fail2ban/jail.d/sshd.conf",
"mode": "0644", "mode": "0644",
"content": "[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\nmaxretry = 3\nfindtime = 1d\nbantime = 1d\n" "content": "[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\nmaxretry = 5\n"
}, },
{ {
"id": "log", "id": "log",
@@ -80,7 +55,7 @@
"type": "file", "type": "file",
"path": "/etc/fail2ban/jail.d/recidive.conf", "path": "/etc/fail2ban/jail.d/recidive.conf",
"mode": "0644", "mode": "0644",
"content": "[recidive]\nenabled = true\nlogpath = /var/log/fail2ban.log\n# Ban in both INPUT (host services like SSH) and DOCKER-USER (container services)\nbanaction = iptables-allports-dualchain\n# Banned twice in two weeks, by any jail, is banned for four (novox/hq ADR 0179).\nbantime = 4w\nfindtime = 2w\nmaxretry = 2\n" "content": "[recidive]\nenabled = true\nlogpath = /var/log/fail2ban.log\n# Ban in both INPUT (host services like SSH) and DOCKER-USER (container services)\nbanaction = iptables-allports-dualchain\nbantime = 1w\nfindtime = 1d\n"
}, },
{ {
"id": "action-dualchain", "id": "action-dualchain",
@@ -106,21 +81,8 @@
"jail-local", "jail-local",
"jail-sshd", "jail-sshd",
"jail-recidive", "jail-recidive",
"action-dualchain", "action-dualchain"
"composed-jails"
] ]
} }
], ]
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
]
}
]
}
} }
+2 -6
View File
@@ -1,18 +1,14 @@
{ {
"name": "@novox/module-fail2ban", "name": "@novox/module-fail2ban",
"version": "0.1.0", "version": "0.1.0",
"description": "fail2ban \u2014 intrusion prevention: the mesh composes the jails and keeps the daemon running; this module holds the node-intrusion-prevention seat and serves its verbs status, banned, ban and unban (novox/hq to-be 31, ADR 0179).", "description": "fail2ban — intrusion prevention: the mesh declares the jails and keeps the daemon running; its ban/unban/status tools live here.",
"type": "module", "type": "module",
"private": true, "private": true,
"dependencies": { "dependencies": {
"@novox/mesh-sdk": "^0.1.1" "@novox/mesh-sdk": "^0.1.0"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^22.0.0", "@types/node": "^22.0.0",
"typescript": "^5.6.0" "typescript": "^5.6.0"
},
"scripts": {
"build": "tsc client.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . --outDir dist",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
} }
} }
-114
View File
@@ -1,114 +0,0 @@
// The intrusion prevention's verbs over a fake daemon, with the shapes fail2ban-client 1.1.0 printed
// on the control node on 2026-10-02 (novox/hq ADR 0179).
import { test } from "node:test";
import assert from "node:assert/strict";
import { Fail2banClient, escalated, installed, parseBans, parseJailStatus, type Runner } from "../client.ts";
const STATUS = "Status\n|- Number of jail:\t2\n`- Jail list:\trecidive, sshd\n";
const RECIDIVE =
"Status for the jail: recidive\n|- Filter\n| |- Currently failed:\t36\n| |- Total failed:\t149\n" +
"| `- File list:\t/var/log/fail2ban.log\n`- Actions\n |- Currently banned:\t9\n |- Total banned:\t13\n" +
" `- Banned IP list:\t195.178.110.30 45.148.10.240 92.118.39.71\n";
const SSHD =
"Status for the jail: sshd\n|- Filter\n| |- Currently failed:\t5\n| |- Total failed:\t11776\n" +
"| `- Journal matches:\t_SYSTEMD_UNIT=sshd.service + _COMM=sshd\n`- Actions\n |- Currently banned:\t0\n" +
" |- Total banned:\t150\n `- Banned IP list:\t\n";
const WITH_TIME =
"195.178.110.30 \t2026-09-26 23:18:47 + 604800 = 2026-10-03 23:18:47\n" +
"92.118.39.71 \t2026-09-28 10:33:49 + 604800 = 2026-10-05 10:33:49\n";
function fake(answers: Record<string, string>, calls: string[][] = []): Runner {
return async (cmd, args) => {
calls.push([cmd, ...args]);
const key = args.join(" ");
if (key in answers) return answers[key];
throw new Error(`unexpected ${cmd} ${key}`);
};
}
test("a jail's status is read into numbers, what it watches and who it holds", () => {
const s = parseJailStatus("recidive", RECIDIVE);
assert.deepEqual(s, {
jail: "recidive",
watching: ["/var/log/fail2ban.log"],
failing: { now: 36, total: 149 },
banned: { now: 9, total: 13, addresses: ["195.178.110.30", "45.148.10.240", "92.118.39.71"] },
});
const j = parseJailStatus("sshd", SSHD);
assert.deepEqual(j.watching, ["_SYSTEMD_UNIT=sshd.service + _COMM=sshd"]);
assert.deepEqual(j.banned, { now: 0, total: 150, addresses: [] });
});
test("status covers every jail the daemon lists, or the one named", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({ status: STATUS, "status recidive": RECIDIVE, "status sshd": SSHD }, calls));
const all = await f.status();
assert.deepEqual(all.jails.map((j) => j.jail), ["recidive", "sshd"]);
const one = await f.status("sshd");
assert.equal(one.jails.length, 1);
assert.deepEqual(calls[calls.length - 1], ["fail2ban-client", "status", "sshd"]);
});
test("bans are read with when they were placed and when they end, a permanent one as never", () => {
const bans = parseBans("recidive", WITH_TIME + "203.0.113.9 \t2026-10-01 00:00:00 + -1 = never\n");
assert.equal(bans.length, 3);
assert.deepEqual(bans[0], { ip: "195.178.110.30", jail: "recidive", since: "2026-09-26 23:18:47", until: "2026-10-03 23:18:47" });
assert.equal(bans[2].until, "never");
assert.deepEqual(parseBans("sshd", "\n"), []);
});
test("banned gathers every jail's bans, soonest to end first", async () => {
const f = new Fail2banClient(fake({
status: STATUS,
"get recidive banip --with-time": WITH_TIME,
"get sshd banip --with-time": "198.51.100.7 \t2026-10-02 15:06:58 + 600 = 2026-10-02 15:16:58\n",
}));
const { banned } = await f.banned();
assert.deepEqual(banned.map((b) => `${b.ip}@${b.jail}`), ["198.51.100.7@sshd", "195.178.110.30@recidive", "92.118.39.71@recidive"]);
});
test("ban asks the daemon by jail and answers with the ban as held; a non-address is refused before anything runs", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({
"set recidive banip 198.51.100.7": "1\n",
"get recidive banip --with-time": WITH_TIME + "198.51.100.7 \t2026-10-02 17:00:00 + 604800 = 2026-10-09 17:00:00\n",
}, calls));
const r = await f.ban("198.51.100.7", "recidive");
assert.equal(r.added, 1);
assert.equal(r.banned?.until, "2026-10-09 17:00:00");
assert.deepEqual(calls[0], ["fail2ban-client", "set", "recidive", "banip", "198.51.100.7"]);
await assert.rejects(() => f.ban("not-an-ip", "recidive"), /is not an address/);
await assert.rejects(() => f.ban("198.51.100.7", "a jail; rm"), /is not a jail's name/);
assert.equal(calls.length, 2);
});
test("unban releases from one jail or from every jail", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({ "set sshd unbanip 198.51.100.7": "1\n", "unban 198.51.100.7": "2\n" }, calls));
assert.deepEqual(await f.unban("198.51.100.7", "sshd"), { released: 1, ip: "198.51.100.7", jail: "sshd" });
assert.deepEqual(await f.unban("198.51.100.7"), { released: 2, ip: "198.51.100.7", jail: "every jail" });
assert.deepEqual(calls[1], ["fail2ban-client", "unban", "198.51.100.7"]);
});
test("a jail's settings are read from the daemon's listings", async () => {
const f = new Fail2banClient(fake({
"get sshd bantime": "86400\n", "get sshd findtime": "86400\n", "get sshd maxretry": "3\n",
"get sshd ignoreip": "These IP addresses/networks are ignored:\n|- 127.0.0.0/8\n|- 10.10.0.0/24\n`- ::1\n",
"get sshd actions": "The jail sshd has the following actions:\niptables-allports-dualchain\n",
"get sshd logpath": "No file is currently monitored\n",
"get sshd journalmatch": "Current match filter:\n_SYSTEMD_UNIT=sshd.service + _COMM=sshd\n",
}));
assert.deepEqual(await f.settings("sshd"), {
jail: "sshd", bantime: "86400", findtime: "86400", maxretry: 3,
ignoreip: ["127.0.0.0/8", "10.10.0.0/24", "::1"], actions: ["iptables-allports-dualchain"],
logpath: [], journalmatch: "_SYSTEMD_UNIT=sshd.service + _COMM=sshd",
});
});
test("the client runs as given by root and through sudo without a prompt by anyone else", () => {
assert.deepEqual(escalated("fail2ban-client", ["status"], 0), ["fail2ban-client", ["status"]]);
assert.deepEqual(escalated("fail2ban-client", ["set", "sshd", "banip", "198.51.100.7"], 1000),
["sudo", ["-n", "fail2ban-client", "set", "sshd", "banip", "198.51.100.7"]]);
assert.equal(installed("sh"), true);
assert.equal(installed("no-such-client-of-the-mesh"), false);
});
+43 -50
View File
@@ -1,62 +1,55 @@
// The intrusion prevention's tools: the node-intrusion-prevention seat's four verbs — who is banned, // fail2ban's tools — reading and steering the live ban state. The jails themselves are declared
// the jails' state, ban one, let one go — and the module's own reading of a jail's settings // resources (module.json); these three touch what the running daemon holds: what is banned now,
// (novox/hq to-be 31, ADR 0179). The jails themselves are composed by the mesh from the modules a // and the manual ban/unban an operator reaches for. The daemon's state is fail2ban's own, so this
// machine runs and written as declared resources; these touch only what the running daemon holds. // is the only way to see or change it — the mesh reconciles the config, not the bans.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools"; import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { Fail2banClient } from "../client.js"; import { Fail2banClient } from "../client.js";
export function getSeatVerbs(fail2ban: Fail2banClient): ToolDefinition[] {
return [
{
name: "status",
description:
"Every jail on this machine with what it watches, how many addresses it is counting failures against and holding now, and the totals since it started; one jail's detail when named.",
input: { jail: { type: "string", description: "one jail (optional)" } },
run: async (args) => fail2ban.status(args.jail ? String(args.jail) : undefined),
},
{
name: "banned",
description: "Every address banned on this machine right now, with the jail that holds it, when it was banned and when the ban ends.",
input: { jail: { type: "string", description: "one jail (optional)" } },
run: async (args) => fail2ban.banned(args.jail ? String(args.jail) : undefined),
},
{
name: "ban",
description:
"Ban one address in one jail now, for the jail's ban time — an operator's act on the live ban list, which the mesh never writes itself.",
input: {
ip: { type: "string", description: "the address" },
jail: { type: "string", description: "the jail to hold it (recidive for the long ban)" },
},
run: async (args) => fail2ban.ban(String(args.ip ?? ""), String(args.jail ?? "")),
},
{
name: "unban",
description: "Let one address go, from one jail or from every jail when none is named.",
input: {
ip: { type: "string", description: "the address" },
jail: { type: "string", description: "one jail (optional)" },
},
run: async (args) => fail2ban.unban(String(args.ip ?? ""), args.jail ? String(args.jail) : undefined),
},
];
}
export function getFail2banTools(fail2ban: Fail2banClient): ToolDefinition[] { export function getFail2banTools(fail2ban: Fail2banClient): ToolDefinition[] {
return [ return [
{ {
name: "fail2ban_settings", name: "fail2ban_status",
description: description:
"One jail's effective settings on this machine: ban time, window, tries, the addresses it never bans, its actions and what it reads.", "fail2ban status on this node — the jails and their live bans. Omit `jail` for every jail, or name one for its detail.",
input: { jail: { type: "string", description: "the jail" } }, input: {
run: async (args) => fail2ban.settings(String(args.jail ?? "")), type: "object",
properties: {
jail: {
type: "string",
description: "A specific jail (e.g. sshd, recidive); omit for the overview of all jails.",
},
},
},
run: async (args) => ({ status: await fail2ban.status(args.jail as string | undefined) }),
},
{
name: "fail2ban_ban",
description: "Manually ban an IP address in a jail — a live change to the running daemon, not a mesh-managed file.",
input: {
type: "object",
properties: {
jail: { type: "string", description: "Jail name (e.g. sshd, recidive)." },
ip: { type: "string", description: "IP address to ban." },
},
required: ["jail", "ip"],
},
run: async (args) => ({ result: await fail2ban.ban(args.jail as string, args.ip as string) }),
},
{
name: "fail2ban_unban",
description: "Unban an IP address from one jail, or from every jail when `jail` is omitted.",
input: {
type: "object",
properties: {
ip: { type: "string", description: "IP address to unban." },
jail: { type: "string", description: "A specific jail; omit to unban from all jails." },
},
required: ["ip"],
},
run: async (args) => ({ result: await fail2ban.unban(args.ip as string, args.jail as string | undefined) }),
}, },
]; ];
} }
const fail2ban = Fail2banClient.onThisMachine(); registerModuleTools("fail2ban", () => getFail2banTools(Fail2banClient.fromEnv()));
// The seat's verbs under the seat's name: the runtime serves them on the seat's subjects where this
// module holds it (ADR 0159, 0160). The module's own under its own.
registerModuleTools("node-intrusion-prevention", () => getSeatVerbs(fail2ban));
registerModuleTools("fail2ban", () => getFail2banTools(fail2ban));
+37
View File
@@ -0,0 +1,37 @@
# gitea's runtime: the tool runtime, carrying this module's compiled provisioner, tools and event
# consumer.
#
# **Built from this module's own directory and nothing else.** The sdk is in the base image, so
# nothing is copied out of a neighbouring checkout — which is what lets the mesh build this from a
# repository and a path (novox/hq ADR 0069) rather than only on a workstation that happens to have
# the siblings.
#
# Two bases, named rather than pinned: the image this is COMPILED in, and the image it RUNS in.
# They are different images on purpose — the first carries a compiler and the second must not, or
# every running container would carry one it never invokes. The mesh answers both with the copies it
# holds, because a fingerprint written here would name one particular copy and no other mesh has it
# (novox/hq issue 044). Declared in module.json's `build.on`; deliberately no defaults, so a build
# nobody told stops here and says which module to build first.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the sdk it will run against.
WORKDIR /app/modules/gitea
COPY . .
# The compiler is invoked by its real path rather than through node_modules/.bin, whose entries are
# symlinks to a launcher that requires its library relatively — resolved away when the base image
# was assembled.
RUN node /app/node_modules/typescript/bin/tsc client.ts token.ts index.ts provisioner/index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
# **No apt packages.** gitea's provisioner talks to the forge over HTTP (the gitea REST API), not
# through a CLI the way postgres drives psql — so the runtime base holds everything this needs.
COPY --from=build /app/modules/gitea/dist /app/modules/gitea/dist
# What a tool host should load from this module: its event consumer and its tools, which are
# separate entrypoints because they are loaded by different things. The provisioner is the third,
# and is not listed here — the declaration names it in the container's `args`, because it is what
# this module's own container runs. One image, because they are one module and share a client.
ENV MESH_TOOL_MODULES=/app/modules/gitea/dist/index.js,/app/modules/gitea/dist/tools/index.js,/app/modules/gitea/dist/provisioner/index.js
-73
View File
@@ -45,14 +45,6 @@ export interface GiteaPull {
html_url: string; html_url: string;
} }
export interface GiteaComment {
id: number;
user?: string;
body: string;
created_at?: string;
html_url: string;
}
export interface GiteaLabel { export interface GiteaLabel {
id: number; id: number;
name: string; name: string;
@@ -103,13 +95,6 @@ export class GiteaClient {
if (res.status === 401) { if (res.status === 401) {
token = await this.tokens.renew(token); token = await this.tokens.renew(token);
res = await this.send(path, options, token); res = await this.send(path, options, token);
} else if (res.status === 403) {
// A kept token minted before a scope was added lacks it. The forge says so; the source
// re-mints with the whole list and the call is retried once. Any other 403 stays a 403.
const text = await res.text();
if (!MintedToken.lacksScope(res.status, text)) throw new Error(`Gitea API ${path}: 403 ${text}`);
token = await this.tokens.renew(token);
res = await this.send(path, options, token);
} }
if (!res.ok) throw new Error(`Gitea API ${path}: ${res.status} ${await res.text()}`); if (!res.ok) throw new Error(`Gitea API ${path}: ${res.status} ${await res.text()}`);
if (res.status === 204) return null as T; if (res.status === 204) return null as T;
@@ -265,64 +250,6 @@ export class GiteaClient {
); );
} }
/** Close or reopen a pull request without merging it. A pull request is an issue to the forge's
* state machine, and the pulls endpoint takes the same `state`. */
async setPullState(owner: string, repo: string, index: number, state: "open" | "closed"): Promise<GiteaPull> {
return GiteaClient.mapPull(
await this.request<any>(`/repos/${owner}/${repo}/pulls/${index}`, { method: "PATCH", body: JSON.stringify({ state }) }),
);
}
/** Change a pull request's title or body; a field left undefined is left alone. */
async updatePullRequest(owner: string, repo: string, index: number, data: { title?: string; body?: string }): Promise<GiteaPull> {
return GiteaClient.mapPull(
await this.request<any>(`/repos/${owner}/${repo}/pulls/${index}`, { method: "PATCH", body: JSON.stringify(data) }),
);
}
/** The unified diff of a pull request, as text. */
async pullDiff(owner: string, repo: string, index: number): Promise<string> {
return this.requestText(`/repos/${owner}/${repo}/pulls/${index}.diff`);
}
/** Every comment on an issue or pull request, oldest first. */
async listComments(owner: string, repo: string, index: number): Promise<GiteaComment[]> {
const raw = await this.request<any[]>(`/repos/${owner}/${repo}/issues/${index}/comments`);
return (raw ?? []).map((c) => ({
id: Number(c?.id ?? 0),
user: c?.user?.login,
body: String(c?.body ?? ""),
created_at: c?.created_at,
html_url: String(c?.html_url ?? ""),
}));
}
/** One file's contents at a ref (default the repository's default branch), decoded. */
async getFile(owner: string, repo: string, path: string, ref?: string): Promise<{ path: string; ref?: string; sha: string; size: number; content: string }> {
const qs = ref ? `?ref=${encodeURIComponent(ref)}` : "";
const f = await this.request<any>(`/repos/${owner}/${repo}/contents/${path.split("/").map(encodeURIComponent).join("/")}${qs}`);
if (!f || f.type !== "file") throw new Error(`Gitea API: ${path} is not a file`);
const content = f.encoding === "base64" ? Buffer.from(String(f.content ?? ""), "base64").toString("utf8") : String(f.content ?? "");
return { path, ref, sha: String(f.sha ?? ""), size: Number(f.size ?? content.length), content };
}
async listBranches(owner: string, repo: string): Promise<{ name: string; commit: string; protected: boolean }[]> {
const raw = await this.request<any[]>(`/repos/${owner}/${repo}/branches?limit=100`);
return (raw ?? []).map((b) => ({ name: String(b?.name ?? ""), commit: String(b?.commit?.id ?? ""), protected: Boolean(b?.protected) }));
}
async deleteBranch(owner: string, repo: string, branch: string): Promise<void> {
await this.request(`/repos/${owner}/${repo}/branches/${encodeURIComponent(branch)}`, { method: "DELETE" });
}
/** A request whose answer is text, not JSON — a diff. Same token handling as request(). */
private async requestText(path: string): Promise<string> {
const token = await this.tokens.current();
const res = await this.send(path, { headers: { Accept: "text/plain" } }, token);
if (!res.ok) throw new Error(`Gitea API ${path}: ${res.status} ${await res.text()}`);
return res.text();
}
async mergePullRequest(owner: string, repo: string, index: number, method = "merge", deleteBranch = false): Promise<void> { async mergePullRequest(owner: string, repo: string, index: number, method = "merge", deleteBranch = false): Promise<void> {
await this.request(`/repos/${owner}/${repo}/pulls/${index}/merge`, { await this.request(`/repos/${owner}/${repo}/pulls/${index}/merge`, {
method: "POST", method: "POST",
+50 -35
View File
@@ -85,17 +85,20 @@
"scope": "mesh" "scope": "mesh"
} }
], ],
"own-secrets": {
"broker": "/var/lib/mesh/gitea/broker"
},
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/gitea",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "runtime-state", "id": "runtime-state",
"type": "directory", "type": "directory",
"path": "${dir:mesh-state}/state", "path": "/var/lib/mesh/gitea/state",
"mode": "0700" "mode": "0700"
}, },
{ {
@@ -142,8 +145,7 @@
"volumes": [ "volumes": [
"${dir:data}:/data" "${dir:data}:/data"
], ],
"secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it", "secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it"
"logging": "journald"
}, },
{ {
"id": "admin-bootstrap", "id": "admin-bootstrap",
@@ -173,10 +175,36 @@
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/gitea/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n",
"merge": "json" "merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-gitea",
"network": "host",
"volumes": [
"/var/lib/mesh/gitea/broker:/run/secrets/broker:ro",
"/var/lib/mesh/gitea/config.json:/run/config/config.json:ro",
"${dir:grants}:${dir:grants}:ro",
"${dir:state}/admin.secret:/run/secrets/admin:ro",
"/var/lib/mesh/gitea/state:/run/state"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GITEA_CONFIG_FILE": "/run/config/config.json",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "/run/secrets/admin",
"MESH_GITEA_STATE_DIR": "/run/state",
"MESH_RECEIVES": "${dir:grants}/npm.json"
},
"artifact": "runtime",
"restart-on": [
"runtime-config"
]
} }
], ],
"provides": [ "provides": [
@@ -190,37 +218,24 @@
} }
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"loads": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"env": {
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GITEA_CONFIG_FILE": "${dir:mesh-state}/config.json",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_STATE_DIR": "${dir:runtime-state}",
"MESH_RECEIVES": "${dir:grants}/npm.json"
}
} }
] ]
}, }
"jails": [
{
"name": "gitea",
"failregex": "^.*Failed authentication attempt for .* from <HOST>(?::\\d+)?\\s*$\n ^.*Invalid user .* from <HOST> port \\d+\\s*$\n ^.*User \\S+ from <HOST> not allowed because .*$",
"jail": "backend = systemd\njournalmatch = CONTAINER_NAME=gitea\nport = http,https,222\nmaxretry = 3\nfindtime = 1d\nbantime = 1d"
}
]
} }
+2 -140
View File
@@ -29,11 +29,7 @@ interface Forge {
mints: number; mints: number;
lastScopes: string[] | null; lastScopes: string[] | null;
tokens: Map<string, string>; tokens: Map<string, string>;
scopesOf: Map<string, string[]>;
admins: Map<string, string>; admins: Map<string, string>;
pullState: string;
pullTitle: string;
branchDeleted: boolean;
close(): Promise<void>; close(): Promise<void>;
} }
@@ -44,9 +40,6 @@ function fakeForge(): Promise<Forge> {
tokens: new Map<string, string>(), // name -> value tokens: new Map<string, string>(), // name -> value
scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does
admins: new Map([[ADMIN, PASSWORD]]), admins: new Map([[ADMIN, PASSWORD]]),
pullState: "open",
pullTitle: "The console shipped",
branchDeleted: false,
}; };
// write:X implies read:X — gitea's own rule (models/auth/access_token_scope.go). // write:X implies read:X — gitea's own rule (models/auth/access_token_scope.go).
const covers = (scopes: string[], required: string): boolean => const covers = (scopes: string[], required: string): boolean =>
@@ -92,51 +85,6 @@ function fakeForge(): Promise<Forge> {
} }
return json(res, 405, { message: "method not allowed" }); return json(res, 405, { message: "method not allowed" });
} }
const tokenOf = (): string => {
const h = req.headers.authorization ?? "";
return h.startsWith("token ") ? h.slice(6) : "";
};
const pull = url.pathname.match(/^\/api\/v1\/repos\/novox\/hq\/pulls\/(\d+)(\.diff)?$/);
if (pull) {
if (![...forge.tokens.values()].includes(tokenOf())) return json(res, 401, { message: "token is required" });
if (pull[2]) {
res.writeHead(200, { "Content-Type": "text/plain" });
return res.end("diff --git a/x b/x\n--- a/x\n+++ b/x\n@@ -1 +1 @@\n-old\n+new\n");
}
if (req.method === "PATCH") {
const patch = await body(req);
forge.pullState = patch?.state ?? forge.pullState;
forge.pullTitle = patch?.title ?? forge.pullTitle;
}
return json(res, 200, { number: Number(pull[1]), title: forge.pullTitle, state: forge.pullState, merged: false,
user: { login: "mesh-admin" }, head: { ref: "feat/x" }, base: { ref: "main" }, html_url: "http://fake/novox/hq/pulls/" + pull[1] });
}
if (url.pathname === "/api/v1/repos/novox/hq/issues/223/comments") {
return json(res, 200, [{ id: 1, user: { login: "jochen" }, body: "landed elsewhere", created_at: "2026-10-01T00:00:00Z", html_url: "http://fake/c/1" }]);
}
if (url.pathname === "/api/v1/repos/novox/hq/contents/README.md") {
return json(res, 200, { type: "file", encoding: "base64", sha: "abc", size: 5, content: Buffer.from("hello").toString("base64") });
}
if (url.pathname === "/api/v1/repos/novox/hq/branches") {
return json(res, 200, [{ name: "main", protected: true, commit: { id: "aaaa" } }, { name: "feat/x", protected: false, commit: { id: "bbbb" } }]);
}
if (url.pathname === "/api/v1/repos/novox/hq/branches/feat%2Fx" || url.pathname === "/api/v1/repos/novox/hq/branches/feat/x") {
if (req.method === "DELETE") { forge.branchDeleted = true; return json(res, 204, null); }
}
if (url.pathname === "/api/v1/repos/search") {
// The client lists through the search endpoint since 2026-09-28 (the forge's whole view);
// it sits under `repository`, which write:repository covers.
const h = req.headers.authorization ?? "";
const value = h.startsWith("token ") ? h.slice(6) : "";
if (![...forge.tokens.values()].includes(value)) return json(res, 401, { message: "token is required" });
if (!covers(forge.scopesOf.get(value) ?? [], "read:repository")) {
return json(res, 403, { message: `token does not have at least one of required scope(s), required=[read:repository]` });
}
return json(res, 200, {
ok: true,
data: [{ full_name: "novox/hq", name: "hq", owner: { login: "novox" }, private: true, html_url: "http://fake/novox/hq" }],
});
}
if (url.pathname === "/api/v1/user/repos") { if (url.pathname === "/api/v1/user/repos") {
const h = req.headers.authorization ?? ""; const h = req.headers.authorization ?? "";
const value = h.startsWith("token ") ? h.slice(6) : ""; const value = h.startsWith("token ") ? h.slice(6) : "";
@@ -153,21 +101,6 @@ function fakeForge(): Promise<Forge> {
{ full_name: "novox/hq", name: "hq", owner: { login: "novox" }, private: true, html_url: "http://fake/novox/hq" }, { full_name: "novox/hq", name: "hq", owner: { login: "novox" }, private: true, html_url: "http://fake/novox/hq" },
]); ]);
} }
const adminUser = url.pathname.match(/^\/api\/v1\/admin\/users\/([^/]+)$/);
if (adminUser && req.method === "PATCH") {
const h = req.headers.authorization ?? "";
const value = h.startsWith("token ") ? h.slice(6) : "";
if (![...forge.tokens.values()].includes(value)) return json(res, 401, { message: "token is required" });
if (!covers(forge.scopesOf.get(value) ?? [], "write:admin")) {
return json(res, 403, {
message: `token does not have at least one of required scope(s), required=[write:admin]`,
});
}
const login = decodeURIComponent(adminUser[1]);
if (login === "untouchable") return json(res, 403, { message: "user untouchable may not be edited" });
const patch = await body(req);
return json(res, 200, { login, is_admin: patch?.admin === true });
}
return json(res, 404, { message: "no such route in the fake" }); return json(res, 404, { message: "no such route in the fake" });
}); });
return new Promise((resolve) => { return new Promise((resolve) => {
@@ -177,11 +110,7 @@ function fakeForge(): Promise<Forge> {
url: `http://127.0.0.1:${port}`, url: `http://127.0.0.1:${port}`,
get mints() { return forge.mints; }, get mints() { return forge.mints; },
get lastScopes() { return forge.lastScopes; }, get lastScopes() { return forge.lastScopes; },
get pullState() { return forge.pullState; },
get pullTitle() { return forge.pullTitle; },
get branchDeleted() { return forge.branchDeleted; },
tokens: forge.tokens, tokens: forge.tokens,
scopesOf: forge.scopesOf,
admins: forge.admins, admins: forge.admins,
close: () => new Promise((r) => server.close(() => r())), close: () => new Promise((r) => server.close(() => r())),
}); });
@@ -223,14 +152,14 @@ function minted(env: NodeJS.ProcessEnv, logs: string[]): GiteaClient {
const forge = await fakeForge(); const forge = await fakeForge();
after(() => forge.close()); after(() => forge.close());
test("first start: mints with the admin account, keeps the token at 0600, asks for the tools' scopes only", async () => { test("first start: mints with the admin account, keeps the token at 0600, asks for two scopes only", async () => {
const { env, file, logs } = await delivered(forge); const { env, file, logs } = await delivered(forge);
const repos = await minted(env, logs).listRepos(); const repos = await minted(env, logs).listRepos();
assert.equal(repos[0]?.full_name, "novox/hq"); assert.equal(repos[0]?.full_name, "novox/hq");
assert.equal(forge.mints, 1); assert.equal(forge.mints, 1);
assert.deepEqual(forge.lastScopes, ["write:repository", "write:issue", "read:user", "write:admin"]); assert.deepEqual(forge.lastScopes, ["write:repository", "write:issue", "read:user"]);
assert.deepEqual(forge.lastScopes, [...TOKEN_SCOPES]); assert.deepEqual(forge.lastScopes, [...TOKEN_SCOPES]);
const token = forge.tokens.get("mesh-tools")!; const token = forge.tokens.get("mesh-tools")!;
assert.equal(await readFile(file, "utf8"), token + "\n"); assert.equal(await readFile(file, "utf8"), token + "\n");
@@ -268,34 +197,6 @@ test("the forge rejects the kept token (its data was restored): minted afresh, o
assert.ok(logs.some((l) => l.startsWith("the forge rejected the kept token")), logs.join("\n")); assert.ok(logs.some((l) => l.startsWith("the forge rejected the kept token")), logs.join("\n"));
}); });
test("a kept token from before write:admin: the forge refuses the admin route for the scope, the token is re-minted with the whole list, and the call goes through", async () => {
const { env, file, logs } = await delivered(forge);
const client = minted(env, logs);
await client.listRepos();
const before = forge.mints;
const old = forge.tokens.get("mesh-tools")!;
forge.scopesOf.set(old, ["write:repository", "write:issue", "read:user"]); // minted by the previous build
const user = await client.api<{ login: string; is_admin: boolean }>("/admin/users/mesh_novox_builder", {
method: "PATCH",
body: JSON.stringify({ admin: true }),
});
assert.equal(user.is_admin, true);
assert.equal(forge.mints, before + 1);
assert.deepEqual(forge.lastScopes, [...TOKEN_SCOPES]);
assert.notEqual(forge.tokens.get("mesh-tools"), old);
assert.equal(await readFile(file, "utf8"), forge.tokens.get("mesh-tools") + "\n");
assert.ok(logs.some((l) => l.startsWith("the forge rejected the kept token")), logs.join("\n"));
// A 403 that is not about scopes is the forge's answer, not a reason to mint.
const again = forge.mints;
await assert.rejects(
client.api("/admin/users/untouchable", { method: "PATCH", body: JSON.stringify({ admin: true }) }),
/403 .*untouchable/,
);
assert.equal(forge.mints, again);
});
test("the kept file is gone but the forge still holds a token by that name: replaced, not refused", async () => { test("the kept file is gone but the forge still holds a token by that name: replaced, not refused", async () => {
const { env, file, logs } = await delivered(forge); const { env, file, logs } = await delivered(forge);
await minted(env, logs).listRepos(); await minted(env, logs).listRepos();
@@ -400,16 +301,6 @@ test("the tools register once there is a way to a token, and the first call mint
"gitea_list_repos", "gitea_create_repo", "gitea_delete_repo", "gitea_list_repos", "gitea_create_repo", "gitea_delete_repo",
"gitea_list_issues", "gitea_get_issue", "gitea_create_issue", "gitea_close_issue", "gitea_add_comment", "gitea_list_issues", "gitea_get_issue", "gitea_create_issue", "gitea_close_issue", "gitea_add_comment",
"gitea_list_pull_requests", "gitea_get_pull_request", "gitea_create_pull_request", "gitea_merge_pull_request", "gitea_list_pull_requests", "gitea_get_pull_request", "gitea_create_pull_request", "gitea_merge_pull_request",
"gitea_close_pull_request",
"gitea_reopen_pull_request",
"gitea_update_pull_request",
"gitea_pull_request_files",
"gitea_pull_request_diff",
"gitea_list_comments",
"gitea_reopen_issue",
"gitea_get_file",
"gitea_list_branches",
"gitea_delete_branch",
"gitea_list_labels", "gitea_create_label", "gitea_list_labels", "gitea_create_label",
"gitea_api", "gitea_api",
], ],
@@ -420,32 +311,3 @@ test("the tools register once there is a way to a token, and the first call mint
assert.equal(result.repos.length, 1); assert.equal(result.repos.length, 1);
assert.equal(forge.mints, before + 1); assert.equal(forge.mints, before + 1);
}); });
// The forge's tools reach every action a review needs without a checkout and without the API
// escape hatch: close a pull request whose work landed elsewhere, read its diff, its comments, a
// file, the branches, and delete the branch left behind. Against the fake forge, through the
// compiled tools, the way the console calls them.
test("a pull request can be closed, read and cleaned up through the tools", async () => {
const { env } = await delivered(forge);
const tools = collectTools(env).find((c) => c.module === "gitea")!.tools;
const tool = (name: string) => tools.find((t) => t.name === name)!;
for (const name of ["gitea_close_pull_request", "gitea_reopen_pull_request", "gitea_update_pull_request", "gitea_pull_request_files",
"gitea_pull_request_diff", "gitea_list_comments", "gitea_reopen_issue", "gitea_get_file", "gitea_list_branches", "gitea_delete_branch"]) {
assert.ok(tool(name), `${name} is not a tool`);
}
const closed = (await tool("gitea_close_pull_request").run({ owner: "novox", repo: "hq", number: 223 })) as { pull: { state: string } };
assert.equal(closed.pull.state, "closed");
assert.equal(forge.pullState, "closed");
const renamed = (await tool("gitea_update_pull_request").run({ owner: "novox", repo: "hq", number: 223, title: "Superseded" })) as { pull: { title: string } };
assert.equal(renamed.pull.title, "Superseded");
const diff = (await tool("gitea_pull_request_diff").run({ owner: "novox", repo: "hq", number: 223 })) as { diff: string };
assert.match(diff.diff, /^diff --git/);
const comments = (await tool("gitea_list_comments").run({ owner: "novox", repo: "hq", number: 223 })) as { comments: { body: string }[] };
assert.equal(comments.comments[0].body, "landed elsewhere");
const file = (await tool("gitea_get_file").run({ owner: "novox", repo: "hq", path: "README.md" })) as { file: { content: string } };
assert.equal(file.file.content, "hello");
const branches = (await tool("gitea_list_branches").run({ owner: "novox", repo: "hq" })) as { branches: { name: string }[] };
assert.deepEqual(branches.branches.map((b) => b.name), ["main", "feat/x"]);
await tool("gitea_delete_branch").run({ owner: "novox", repo: "hq", branch: "feat/x" });
assert.equal(forge.branchDeleted, true);
});
+3 -14
View File
@@ -34,21 +34,15 @@ export const TOKEN_NAME = "mesh-tools";
* It sits under the `user` category despite listing repositories, not `repository` * It sits under the `user` category despite listing repositories, not `repository`
* — confirmed against the running forge (1.27.3), which answered * — confirmed against the running forge (1.27.3), which answered
* `required=[read:user]` to a token carrying only the other two. * `required=[read:user]` to a token carrying only the other two.
* write:admin — /admin/users: the forge's own users are the mesh's to settle, such as making * Nothing under /admin, /orgs or write:user — the escape-hatch tool reaches only what these three cover.
* the builder's login a site admin so every repository the mesh may build is
* clonable (novox/hq 229). Nothing under /orgs or write:user.
*
* A token kept from before a scope was added lacks it: the forge answers such a call with
* `403 token does not have at least one of required scope(s)`, and the client treats that like a
* 401 — the source re-mints by name, with the whole list, and the call is retried once.
*/ */
export const TOKEN_SCOPES: readonly string[] = ["write:repository", "write:issue", "read:user", "write:admin"]; export const TOKEN_SCOPES: readonly string[] = ["write:repository", "write:issue", "read:user"];
/** Where a client's token comes from, and what to do when the forge says it is wrong. */ /** Where a client's token comes from, and what to do when the forge says it is wrong. */
export interface TokenSource { export interface TokenSource {
/** The token to authenticate with now; minted, read or configured. */ /** The token to authenticate with now; minted, read or configured. */
current(): Promise<string>; current(): Promise<string>;
/** The forge answered 401 to `rejected`, or 403 for a scope it lacks. A fresh token, or a plain error when there is nothing to renew with. */ /** The forge answered 401 to `rejected`. A fresh token, or a plain error when there is nothing to renew with. */
renew(rejected: string): Promise<string>; renew(rejected: string): Promise<string>;
} }
@@ -176,11 +170,6 @@ export class MintedToken implements TokenSource {
return this.mint("the forge rejected the kept token — minting a fresh one"); return this.mint("the forge rejected the kept token — minting a fresh one");
} }
/** What the forge's scoped tokens say when a kept token predates a scope the tools now need. */
static lacksScope(status: number, body: string): boolean {
return status === 403 && /required scope/i.test(body);
}
/** One mint at a time: concurrent first calls share it, rather than each minting its own. */ /** One mint at a time: concurrent first calls share it, rather than each minting its own. */
private mint(why: string): Promise<string> { private mint(why: string): Promise<string> {
if (this.inflight === null) { if (this.inflight === null) {
-125
View File
@@ -254,131 +254,6 @@ export function getGiteaTools(gitea: GiteaClient): ToolDefinition[] {
}, },
}, },
{
name: "gitea_close_pull_request",
description: "Close a pull request without merging it — one whose work landed elsewhere, or was abandoned.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
pull: await gitea.setPullState(String(args.owner), String(args.repo), Number(args.number), "closed"),
}),
},
{
name: "gitea_reopen_pull_request",
description: "Reopen a closed, unmerged pull request.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
pull: await gitea.setPullState(String(args.owner), String(args.repo), Number(args.number), "open"),
}),
},
{
name: "gitea_update_pull_request",
description: "Change a pull request's title or body; a field not given is left as it is.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
title: { type: "string", description: "the new title (optional)" },
body: { type: "string", description: "the new body, markdown (optional)" },
},
run: async (args) => ({
pull: await gitea.updatePullRequest(String(args.owner), String(args.repo), Number(args.number), {
title: args.title === undefined ? undefined : String(args.title),
body: args.body === undefined ? undefined : String(args.body),
}),
}),
},
{
name: "gitea_pull_request_files",
description: "The files a pull request changes, as paths from the repository's root (up to 100; says when there are more).",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => gitea.listPullFiles(String(args.owner), String(args.repo), Number(args.number)),
},
{
name: "gitea_pull_request_diff",
description: "A pull request's unified diff, as text — for reviewing it without a checkout.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
diff: await gitea.pullDiff(String(args.owner), String(args.repo), Number(args.number)),
}),
},
{
name: "gitea_list_comments",
description: "Every comment on an issue or pull request, oldest first.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the issue or PR number" },
},
run: async (args) => ({
comments: await gitea.listComments(String(args.owner), String(args.repo), Number(args.number)),
}),
},
{
name: "gitea_reopen_issue",
description: "Reopen a closed issue.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the issue number" },
},
run: async (args) => ({
issue: await gitea.setIssueState(String(args.owner), String(args.repo), Number(args.number), "open"),
}),
},
// ---- Contents and branches ----
{
name: "gitea_get_file",
description: "One file's contents from a repository, decoded, at a branch, tag or commit (default the repository's default branch).",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
path: { type: "string", description: "the file's path from the repository's root" },
ref: { type: "string", description: "branch, tag or commit (optional)" },
},
run: async (args) => ({
file: await gitea.getFile(String(args.owner), String(args.repo), String(args.path), args.ref ? String(args.ref) : undefined),
}),
},
{
name: "gitea_list_branches",
description: "Every branch of a repository with the commit it points at.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
},
run: async (args) => ({ branches: await gitea.listBranches(String(args.owner), String(args.repo)) }),
},
{
name: "gitea_delete_branch",
description: "Delete a branch — a feature branch whose pull request was closed rather than merged. Refused by the forge for a protected branch.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
branch: { type: "string", description: "the branch name" },
},
run: async (args) => {
await gitea.deleteBranch(String(args.owner), String(args.repo), String(args.branch));
return { deleted: true, branch: String(args.branch) };
},
},
// ---- Labels ---- // ---- Labels ----
{ {
name: "gitea_list_labels", name: "gitea_list_labels",
+24
View File
@@ -0,0 +1,24 @@
# gitlab's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/gitlab
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/gitlab/dist /app/modules/gitlab/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/gitlab/dist/tools/index.js
+43 -17
View File
@@ -2,43 +2,69 @@
"module": "gitlab", "module": "gitlab",
"version": "1", "version": "1",
"own-secrets": { "own-secrets": {
"token": "${dir:state}/token" "token": "/var/lib/gitlab/token",
"broker": "/var/lib/mesh/gitlab/broker"
}, },
"resources": [ "resources": [
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/gitlab",
"mode": "0700"
},
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/gitlab",
"place": "." "mode": "0700"
}, },
{ {
"id": "config", "id": "config",
"type": "file", "type": "file",
"path": "${dir:state}/config.json", "path": "/var/lib/gitlab/config.json",
"merge": "json", "merge": "json",
"content": "{}", "content": "{}",
"mode": "0600" "mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-runtime-gitlab",
"network": "host",
"volumes": [
"/var/lib/gitlab/config.json:/run/config/config.json:ro",
"/var/lib/gitlab/token:/run/secrets/token:ro",
"/var/lib/mesh/gitlab/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_GITLAB_TOKEN_FILE": "/run/secrets/token",
"MESH_GITLAB_CONFIG_FILE": "/run/config/config.json",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"artifact": "runtime"
} }
], ],
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "tools", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_GITLAB_TOKEN_FILE": "${dir:state}/token",
"MESH_GITLAB_CONFIG_FILE": "${dir:state}/config.json"
}
} }
] ]
} }
+24
View File
@@ -0,0 +1,24 @@
# grafana's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/grafana
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/grafana/dist /app/modules/grafana/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/grafana/dist/index.js,/app/modules/grafana/dist/tools/index.js
+53 -91
View File
@@ -5,7 +5,8 @@
"alert.firing" "alert.firing"
], ],
"own-secrets": { "own-secrets": {
"admin": "${dir:mesh-state}/admin" "admin": "/var/lib/grafana-module/admin.secret",
"broker": "/var/lib/mesh/grafana/broker"
}, },
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
@@ -23,142 +24,103 @@
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/grafana",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/grafana-module",
"place": "." "mode": "0700"
}, },
{ {
"id": "data", "id": "data",
"type": "directory", "type": "directory",
"path": "/services/grafana/data",
"mode": "0700", "mode": "0700",
"owner": "472:472" "owner": "472:472"
}, },
{ {
"id": "admin-secret", "id": "server-env",
"type": "file", "type": "file",
"path": "${dir:state}/admin.secret", "path": "/var/lib/grafana-module/server.env",
"mode": "0400", "mode": "0600",
"owner": "472:472", "content": "GF_SECURITY_ADMIN_PASSWORD=${secret:admin}\n"
"content": "${secret:admin}"
},
{
"id": "oidc-secret",
"type": "file",
"path": "${dir:state}/oidc-client.secret",
"mode": "0400",
"owner": "472:472",
"content": "${secret:oidc-client}"
},
{
"id": "oidc-env",
"type": "file",
"path": "${dir:state}/oidc.env",
"mode": "0644",
"content": "GF_SERVER_ROOT_URL=https://${bound:route:name}\nGF_AUTH_GENERIC_OAUTH_ENABLED=true\nGF_AUTH_GENERIC_OAUTH_NAME=Keycloak\nGF_AUTH_GENERIC_OAUTH_CLIENT_ID=${bound:oidc-client:as}\nGF_AUTH_GENERIC_OAUTH_CLIENT_SECRET__FILE=/run/secrets/oidc-client\nGF_AUTH_GENERIC_OAUTH_SCOPES=openid email profile roles\nGF_AUTH_GENERIC_OAUTH_AUTH_URL=${bound:oidc-client:issuer}${bound:oidc-client:authorization-path}\nGF_AUTH_GENERIC_OAUTH_TOKEN_URL=${bound:oidc-client:issuer}${bound:oidc-client:token-path}\nGF_AUTH_GENERIC_OAUTH_API_URL=${bound:oidc-client:issuer}${bound:oidc-client:userinfo-path}\nGF_AUTH_GENERIC_OAUTH_ROLE_ATTRIBUTE_PATH=contains(roles[*], 'admin') && 'Admin' || contains(realm_access.roles[*], 'admin') && 'Admin' || 'Viewer'\nGF_AUTH_GENERIC_OAUTH_USE_PKCE=true\nGF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=true\nGF_AUTH_GENERIC_OAUTH_ALLOW_ASSIGN_GRAFANA_ADMIN=true\n"
},
{
"id": "influxdb-secret",
"type": "file",
"path": "${dir:state}/influxdb-api.secret",
"mode": "0400",
"owner": "472:472",
"content": "${secret:influxdb-api}"
},
{
"id": "influxdb-datasource",
"type": "file",
"path": "${dir:state}/datasource-influxdb.yaml",
"mode": "0644",
"content": "apiVersion: 1\n# Written by the mesh from grafana's influxdb-api binding; grafana reads it at start. Its own name and\n# uid, so a data source somebody made in the UI is never overwritten, and read-only in the UI because\n# the mesh resets it. The password is read from the file the mesh delivers, never written here.\ndatasources:\n - name: InfluxDB (mesh)\n uid: mesh-influxdb-api\n type: influxdb\n access: proxy\n url: ${bound:influxdb-api:scheme}://${bound:influxdb-api:at}:${bound:influxdb-api:port}\n user: ${bound:influxdb-api:as}\n isDefault: false\n editable: false\n jsonData:\n dbName: ${bound:influxdb-api:bucket}\n httpMode: POST\n secureJsonData:\n password: $__file{/run/secrets/influxdb-api}\n"
}, },
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "grafana", "name": "grafana",
"image": "grafana/grafana@sha256:ac461fb352abc50da10a51c7d02462e9c05488f11f53f14b3ad79a8145f638a0", "image": "grafana/grafana@sha256:f772d434e8fab0049deb2b1b30abd43342bcfca1537614aa8d36080232cf4283",
"ports": [ "ports": [
"3000" "3000"
], ],
"volumes": [ "volumes": [
"${dir:data}:/var/lib/grafana", "/services/grafana/data:/var/lib/grafana"
"${dir:state}/admin.secret:/run/secrets/admin:ro",
"${dir:state}/oidc-client.secret:/run/secrets/oidc-client:ro",
"${dir:state}/influxdb-api.secret:/run/secrets/influxdb-api:ro",
"${dir:state}/datasource-influxdb.yaml:/etc/grafana/provisioning/datasources/mesh-influxdb.yaml:ro"
], ],
"env": {
"GF_SECURITY_ADMIN_PASSWORD__FILE": "/run/secrets/admin"
},
"env-file": [ "env-file": [
"${dir:state}/oidc.env" "/var/lib/grafana-module/server.env"
], ],
"restart-on": [ "secrets-in-environment": "grafana honours GF_SECURITY_ADMIN_PASSWORD__FILE; convertible, awaiting a bed that exercises the admin password (assigned-grafana serves tools only)"
"oidc-env",
"oidc-secret",
"influxdb-datasource",
"influxdb-secret"
]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/grafana/config.json",
"mode": "0600", "mode": "0600",
"content": "{\n \"user\": \"admin\",\n \"password\": \"${secret:admin}\"\n}\n", "content": "{}\n",
"merge": "json" "merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-grafana",
"network": "host",
"volumes": [
"/var/lib/mesh/grafana/broker:/run/secrets/broker:ro",
"/var/lib/mesh/grafana/config.json:/run/config/config.json:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_GRAFANA_URL": "http://127.0.0.1:3000",
"MESH_GRAFANA_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
} }
], ],
"requires": [ "requires": [
"route", "route"
"oidc-client",
"influxdb-api"
], ],
"contributes": { "contributes": {
"route": { "route": {
"label": "grafana", "label": "grafana",
"endpoint": "web" "endpoint": "web"
},
"oidc-client": {
"label": "grafana",
"endpoint": "web",
"callback": "/login/generic_oauth"
},
"influxdb-api": {
"access": "read"
} }
}, },
"binds": { "binds": {
"route": "${dir:state}/route.json", "route": "/var/lib/mesh/grafana/route.json"
"oidc-client": "${dir:state}/oidc.json",
"influxdb-api": "${dir:state}/influxdb.json"
},
"secrets": {
"oidc-client": "${dir:mesh-state}/oidc-client",
"influxdb-api": "${dir:mesh-state}/influxdb-api"
}, },
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"index.js",
"tools/index.js"
],
"loads": [
"index.js",
"tools/index.js"
],
"env": {
"MESH_GRAFANA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GRAFANA_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
} }
] ]
} }
+5 -5
View File
@@ -15,7 +15,7 @@
} }
}, },
"binds": { "binds": {
"route": "${dir:state}/route.json" "route": "/var/lib/hello-web/route.json"
}, },
"listens": [ "listens": [
{ {
@@ -30,13 +30,13 @@
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/hello-web",
"place": "." "mode": "0700"
}, },
{ {
"id": "page", "id": "page",
"type": "file", "type": "file",
"path": "${dir:state}/index.html", "path": "/var/lib/hello-web/index.html",
"mode": "0644", "mode": "0644",
"content": "hello from hello-web, routed by the mesh\n" "content": "hello from hello-web, routed by the mesh\n"
}, },
@@ -54,7 +54,7 @@
"8080" "8080"
], ],
"volumes": [ "volumes": [
"${dir:state}/index.html:/www/index.html:ro" "/var/lib/hello-web/index.html:/www/index.html:ro"
], ],
"args": [ "args": [
"sh", "sh",
+24
View File
@@ -0,0 +1,24 @@
# home-assistant's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/home-assistant
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/home-assistant/dist /app/modules/home-assistant/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/home-assistant/dist/index.js,/app/modules/home-assistant/dist/tools/index.js
+47 -99
View File
@@ -9,7 +9,8 @@
"state.changed" "state.changed"
], ],
"own-secrets": { "own-secrets": {
"token": "${dir:mesh-state}/token" "broker": "/var/lib/mesh/home-assistant/broker",
"token": "/var/lib/mesh/home-assistant/token"
}, },
"listens": [ "listens": [
{ {
@@ -17,151 +18,98 @@
"port": 8123, "port": 8123,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "the dashboard, the API and the companion apps" "why": "the dashboard and the API"
},
{
"name": "sonos-events",
"port": 1400,
"protocol": "tcp",
"from": "mesh",
"why": "the Sonos integration's event callback: speakers push their state changes here"
},
{
"name": "webrtc",
"port": 18555,
"protocol": "tcp",
"from": "mesh",
"why": "the bundled go2rtc's WebRTC port, which camera streams to a browser use"
} }
], ],
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/home-assistant",
"place": "mesh" "mode": "0700"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
}, },
{ {
"id": "config", "id": "config",
"type": "directory", "type": "directory",
"mode": "0700" "path": "/services/home-assistant/config",
}, "mode": "0700",
{ "owner": "1000:1000"
"id": "written",
"type": "directory",
"mode": "0700"
}, },
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "home-assistant", "name": "home-assistant",
"image": "ghcr.io/home-assistant/home-assistant@sha256:d8922685169707fd91e8b9729902d975f06157d005e422874d201e0261dda196", "image": "ghcr.io/home-assistant/home-assistant@sha256:14931c6b13756317849f46da1d01b45937a1150db66c081cfe529d48215943fe",
"network": "host", "network": "host",
"env": { "env": {
"TZ": "Etc/UTC" "TZ": "Etc/UTC"
}, },
"volumes": [ "volumes": [
"${dir:config}:/config" "/services/home-assistant/config:/config"
] ]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/home-assistant/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n",
"merge": "json" "merge": "json"
}, },
{ {
"id": "provisions-env", "id": "runtime",
"type": "file", "type": "container",
"path": "${dir:state}/provisions.env", "name": "mesh-home-assistant",
"mode": "0600", "network": "host",
"content": "MESH_HOMEASSISTANT_URL=http://127.0.0.1:${port:8123}\nMESH_HOMEASSISTANT_TOKEN_FILE=${dir:mesh-state}/token\nMESH_PROVISIONS_DIR=${dir:state}\nMESH_WRITTEN_DIR=${dir:written}\n" "volumes": [
}, "/var/lib/mesh/home-assistant/broker:/run/secrets/broker:ro",
{ "/var/lib/mesh/home-assistant/token:/run/secrets/token:ro",
"id": "provisions", "/var/lib/mesh/home-assistant/config.json:/run/config/config.json:ro",
"type": "process", "/services/home-assistant/config:/var/lib/home-assistant/config:ro"
"name": "home-assistant-provisions",
"artifact": "code",
"run": [
"node",
"provisions/index.js"
],
"run-once": true,
"env-file": [
"${dir:state}/provisions.env"
], ],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_HOMEASSISTANT_URL": "http://127.0.0.1:8123",
"MESH_HOMEASSISTANT_TOKEN_FILE": "/run/secrets/token",
"MESH_HOMEASSISTANT_CONFIG_FILE": "/run/config/config.json",
"MESH_HOMEASSISTANT_CONFIG_DIR": "/var/lib/home-assistant/config"
},
"restart-on": [ "restart-on": [
"provisions-env", "runtime-config"
"bound-mqtt-topic", ],
"secret-mqtt-topic", "artifact": "runtime"
"bound-sonarr-api",
"secret-sonarr-api",
"bound-radarr-api",
"secret-radarr-api",
"bound-lidarr-api",
"secret-lidarr-api"
]
} }
], ],
"requires": [ "requires": [
"lidarr-api", "route"
"mqtt-topic",
"radarr-api",
"route",
"sonarr-api"
], ],
"contributes": { "contributes": {
"mqtt-topic": {
"topics": [
"#"
]
},
"route": { "route": {
"label": "home-assistant", "label": "home-assistant",
"endpoint": "web" "endpoint": "web"
} }
}, },
"binds": { "binds": {
"route": "${dir:state}/route.json", "route": "/var/lib/mesh/home-assistant/route.json"
"mqtt-topic": "${dir:state}/mqtt-topic.json",
"sonarr-api": "${dir:state}/sonarr-api.json",
"radarr-api": "${dir:state}/radarr-api.json",
"lidarr-api": "${dir:state}/lidarr-api.json"
},
"secrets": {
"mqtt-topic": "${dir:state}/mqtt-topic.secret",
"sonarr-api": "${dir:state}/sonarr-api.secret",
"radarr-api": "${dir:state}/radarr-api.secret",
"lidarr-api": "${dir:state}/lidarr-api.secret"
}, },
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"index.js",
"tools/index.js",
"provisions/index.js"
],
"loads": [
"index.js",
"tools/index.js"
],
"env": {
"MESH_HOMEASSISTANT_URL": "http://127.0.0.1:${port:8123}",
"MESH_HOMEASSISTANT_TOKEN_FILE": "${dir:mesh-state}/token",
"MESH_HOMEASSISTANT_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
} }
] ]
} }
+1 -6
View File
@@ -1,14 +1,9 @@
{ {
"name": "@novox/module-home-assistant", "name": "@novox/module-home-assistant",
"version": "0.1.0", "version": "0.1.0",
"description": "home-assistant \u2014 home automation platform. Its API client, tools and events live here (novox/hq ADR 0039).", "description": "home-assistant — home automation platform. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module", "type": "module",
"private": true, "private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts provisions/hass.ts provisions/probe.ts provisions/connections.ts provisions/mesh.ts provisions/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": { "dependencies": {
"@novox/mesh-sdk": "^0.1.0" "@novox/mesh-sdk": "^0.1.0"
}, },
@@ -1,486 +0,0 @@
// How home-assistant's provisions step brings Home Assistant's integrations in line with what the
// mesh bound: the MQTT integration to `mqtt-topic`, the Sonarr, Radarr and Lidarr integrations to
// `sonarr-api`, `radarr-api` and `lidarr-api`. Pure logic over two seams — Home Assistant's config
// flows (hass.ts) and the broker/apps — so it is tested against fakes (test/provisions.test.ts).
//
// The half that reads files and talks HTTP lives beside it (mesh.ts, hass.ts, probe.ts, index.ts).
import { createHash } from "node:crypto";
import type { Hass, SchemaField } from "./hass.js";
import type { Probe } from "./probe.js";
/** What the mesh wrote at `binds.<provision>` (the controller's binding document). */
export interface Binding {
provision?: string;
from?: string;
at?: string;
as?: string;
serves?: Record<string, unknown>;
}
/** How one provision came out. Never carries a credential. */
export type Outcome =
| { what: string; result: "unchanged"; note?: string }
| { what: string; result: "written"; fields: string[]; note?: string }
| { what: string; result: "equivalent"; note: string }
| { what: string; result: "refused"; problem: string };
/** A port the binding serves, or undefined when it names none usable. */
export function portOf(serves: Record<string, unknown> | undefined): number | undefined {
const port = Number(serves?.port);
return Number.isInteger(port) && port > 0 && port <= 65535 ? port : undefined;
}
/** A host as it goes into a URL: an IPv6 literal bracketed. */
export function urlHost(host: string): string {
return host.includes(":") && !host.startsWith("[") ? `[${host}]` : host;
}
/**
* What this step last wrote, per target, as a digest: the only way to know "already as the mesh
* says" for a credential Home Assistant will not show back. A sha256 over the target and the values,
* never the values; kept in the module's own placed directory.
*/
export interface Marks {
get(name: string): Promise<string | undefined>;
set(name: string, digest: string): Promise<void>;
}
export function digest(...parts: (string | number)[]): string {
return createHash("sha256").update(parts.map(String).join("\u0000")).digest("hex");
}
/** An error as text with the credential taken out, raw and URL-encoded. */
export function scrub(err: unknown, ...secrets: (string | undefined)[]): string {
let text = err instanceof Error ? err.message : String(err);
for (const s of secrets) {
if (!s) continue;
for (const form of new Set([s, encodeURIComponent(s)])) text = text.split(form).join("***");
}
return text;
}
/**
* What a form would submit if a person pressed "submit" without touching it: each field's
* suggested value (what Home Assistant pre-fills from the entry), else its default; a section's
* fields nested under its name. The step lays only the connection fields over this, so every other
* choice the entry carries is sent back exactly as Home Assistant showed it.
*/
export function formValues(schema: readonly SchemaField[] | null | undefined): Record<string, unknown> {
const out: Record<string, unknown> = {};
for (const field of schema ?? []) {
if (Array.isArray(field.schema)) {
out[field.name] = formValues(field.schema);
continue;
}
const suggested = field.description?.suggested_value;
if (suggested !== undefined && suggested !== null) out[field.name] = suggested;
else if (field.default !== undefined) out[field.name] = field.default;
}
return out;
}
/** Whether a form has a field of this name at its top level. */
export function hasField(schema: readonly SchemaField[] | null | undefined, name: string): boolean {
return (schema ?? []).some((f) => f.name === name);
}
// ---- MQTT ----
// Home Assistant's MQTT integration, pointed at the broker the mesh bound — `mqtt-topic`.
//
// **Why a step.** Home Assistant keeps its broker, login and password in its MQTT config entry
// (`.storage/core.config_entries`), not in a file the mesh could fill with `${bound:mqtt-topic:at}`.
// So this reads the binding and the pair credential and makes the entry say the same thing, through
// the MQTT integration's own reconfigure flow — the flow its "Reconfigure" button runs, which tests
// the connection itself and saves nothing it could not connect with.
//
// **Only the connection, and only when it differs.** Broker, port, username, password. The protocol
// version, client id, keepalive, TLS choices and discovery options the entry holds are sent back
// exactly as Home Assistant pre-filled them. Whether the password already matches cannot be read
// back (Home Assistant never shows a stored password), so the step keeps a digest of what it last
// wrote: equal broker/port/username and an equal digest is "already as the mesh says".
//
// **Nothing loses its connection without someone seeing it.** Before Home Assistant is touched the
// broker itself is asked whether it takes the delivered login (the provisioner creates it within
// seconds of the grant): if not, nothing is written and the step fails saying why, and Home
// Assistant keeps the login it has — the carried `luffy` on ace, which mosquitto keeps. If Home
// Assistant's own connection test refuses the new settings, the flow saves nothing, and the step
// fails with Home Assistant's reason. A login that may not subscribe to the discovery topics is said
// as a warning: discovery would find nothing.
export const MQTT_PROVISION = "mqtt-topic";
/** Home Assistant's discovery prefix, subscribed to whenever discovery is on (the default). */
export const DISCOVERY_FILTER = "homeassistant/#";
export interface MqttWanted {
host: string;
port: number;
username: string;
password: string;
}
export type Wanted = { ok: true; want: MqttWanted } | { ok: false; problem: string };
/** The broker, port and login the mesh says Home Assistant uses. */
export function wantedMqtt(binding: Binding | undefined, credential: string | undefined): Wanted {
if (!binding) return { ok: false, problem: `no binding for ${MQTT_PROVISION} was delivered — the mesh writes it before this step runs` };
const host = typeof binding.at === "string" ? binding.at.trim() : "";
if (!host) return { ok: false, problem: `the ${MQTT_PROVISION} binding names no host (at)` };
const port = portOf(binding.serves);
if (port === undefined) return { ok: false, problem: `the ${MQTT_PROVISION} binding serves no usable port (${String(binding.serves?.port)})` };
const scheme = binding.serves?.scheme;
if (scheme !== undefined && scheme !== "mqtt") {
return { ok: false, problem: `the ${MQTT_PROVISION} binding serves scheme ${String(scheme)}; this step writes plain MQTT` };
}
const username = typeof binding.as === "string" ? binding.as.trim() : "";
if (!username) return { ok: false, problem: `the ${MQTT_PROVISION} binding names no login (as)` };
const password = (credential ?? "").replace(/\n$/, "");
if (!password) return { ok: false, problem: `the ${MQTT_PROVISION} credential is empty or was not delivered` };
return { ok: true, want: { host, port, username, password } };
}
export interface MqttDeps {
hass: Hass;
probe: Probe;
marks: Marks;
}
const markFor = (entryId: string, w: MqttWanted): string => digest("mqtt", entryId, w.host, w.port, w.username, w.password);
/** Bring Home Assistant's MQTT entry in line with the mesh. Never throws: every failure is an outcome. */
export async function reconcileMqtt(deps: MqttDeps, binding: Binding | undefined, credential: string | undefined): Promise<Outcome> {
const what = "mqtt";
const w = wantedMqtt(binding, credential);
if ("problem" in w) return { what, result: "refused", problem: w.problem };
const want = w.want;
// The broker first: a login it does not take is never written into Home Assistant.
let note: string | undefined;
try {
const probe = await deps.probe(want.host, want.port, want.username, want.password, DISCOVERY_FILTER);
if (probe.connack === 4 || probe.connack === 5) {
return {
what,
result: "refused",
problem:
`the broker at ${want.host}:${want.port} does not (yet) take the login ${want.username} with the delivered ` +
`password (CONNACK ${probe.connack}); mosquitto's provisioner creates it from the grant — nothing was ` +
`written, and Home Assistant keeps the broker login it has`,
};
}
if (probe.connack !== 0) {
return { what, result: "refused", problem: `the broker at ${want.host}:${want.port} answered CONNACK ${probe.connack}; nothing was written` };
}
if (probe.suback === 0x80) {
note =
`warning: ${want.username} may not subscribe to ${DISCOVERY_FILTER} — MQTT discovery will find nothing; ` +
`grant it with the mqtt-topic contribution's \`topics\``;
}
} catch (err) {
return {
what,
result: "refused",
problem: `the broker at ${want.host}:${want.port} could not be asked: ${scrub(err, want.password)}; nothing was written`,
};
}
try {
const entries = (await deps.hass.entries("mqtt")).filter((e) => e.domain === "mqtt");
if (entries.length > 1) {
return { what, result: "refused", problem: `Home Assistant has ${entries.length} MQTT entries; which one the mesh owns is not guessed` };
}
if (entries.length === 0) return await createEntry(deps, want, note);
const entry = entries[0];
const flow = await deps.hass.startFlow("mqtt", entry.entry_id);
if (flow.type !== "form" || !flow.flow_id || !flow.data_schema) {
if (flow.flow_id) await deps.hass.abortFlow(flow.flow_id);
return { what, result: "refused", problem: `Home Assistant's MQTT reconfigure flow answered ${flow.type}${flow.reason ? ` (${flow.reason})` : ""}` };
}
const current = formValues(flow.data_schema);
const fields: string[] = [];
if (String(current.broker ?? "") !== want.host) fields.push("broker");
if (Number(current.port ?? 0) !== want.port) fields.push("port");
if (String(current.username ?? "") !== want.username) fields.push("username");
if ((await deps.marks.get("mqtt")) !== markFor(entry.entry_id, want)) fields.push("password");
if (fields.length === 0) {
await deps.hass.abortFlow(flow.flow_id);
return note ? { what, result: "unchanged", note } : { what, result: "unchanged" };
}
const saved = await deps.hass.stepFlow(flow.flow_id, {
...current,
broker: want.host,
port: want.port,
username: want.username,
password: want.password,
});
if (saved.type === "abort" && saved.reason === "reconfigure_successful") {
await deps.marks.set("mqtt", markFor(entry.entry_id, want));
return { what, result: "written", fields, ...(note ? { note } : {}) };
}
if (saved.flow_id) await deps.hass.abortFlow(saved.flow_id);
return {
what,
result: "refused",
problem:
`Home Assistant's own connection test refused ${want.username}@${want.host}:${want.port} ` +
`(${describe(saved)}); its MQTT entry is unchanged`,
};
} catch (err) {
return { what, result: "refused", problem: scrub(err, want.password) };
}
}
/** A fresh Home Assistant has no MQTT entry: made through the integration's user flow. */
async function createEntry(deps: MqttDeps, want: MqttWanted, note?: string): Promise<Outcome> {
const what = "mqtt";
let flow = await deps.hass.startFlow("mqtt");
if (flow.type === "form" && flow.step_id !== "broker" && flow.flow_id) {
// Anything before the broker form (none outside the Supervisor) is not this step's to answer.
await deps.hass.abortFlow(flow.flow_id);
return { what, result: "refused", problem: `Home Assistant's MQTT user flow asked ${flow.step_id} before the broker` };
}
if (flow.type !== "form" || !flow.flow_id) {
return { what, result: "refused", problem: `Home Assistant's MQTT user flow answered ${describe(flow)}` };
}
const shown = formValues(flow.data_schema);
// A new entry's form has no value for its two certificate choices (a reconfigure pre-fills them
// from the entry): plain MQTT, so neither a CA nor a client certificate.
const other = (shown.other_settings ?? {}) as Record<string, unknown>;
if (flow.data_schema?.some((f) => f.name === "other_settings")) {
shown.other_settings = { set_ca_cert: "off", set_client_cert: false, ...other };
}
flow = await deps.hass.stepFlow(flow.flow_id, {
...shown,
broker: want.host,
port: want.port,
username: want.username,
password: want.password,
});
if (flow.type === "create_entry") {
const id = (flow.result as { entry_id?: string } | undefined)?.entry_id;
if (id) await deps.marks.set("mqtt", markFor(id, want));
return { what, result: "written", fields: ["entry"], ...(note ? { note } : {}) };
}
if (flow.flow_id) await deps.hass.abortFlow(flow.flow_id);
return { what, result: "refused", problem: `Home Assistant refused a new MQTT entry for ${want.host}:${want.port} (${describe(flow)})` };
}
export function describe(r: { type: string; reason?: string; errors?: Record<string, string> | null }): string {
const errors = r.errors ? Object.entries(r.errors).map(([k, v]) => `${k}: ${v}`).join(", ") : "";
return [r.type, r.reason, errors].filter(Boolean).join(" — ");
}
// ---- Sonarr, Radarr, Lidarr ----
// Home Assistant's Sonarr, Radarr and Lidarr integrations, pointed at the apps the mesh bound —
// `sonarr-api`, `radarr-api`, `lidarr-api` (their providers: mesh-catalog #156).
//
// **What Home Assistant lets anyone change, and what it does not.** Each integration keeps a URL and
// an API key in its config entry. None of the three has a reconfigure flow: Home Assistant changes
// them only through the flow its UI runs —
// - a **user flow** makes a new entry (validated against the app);
// - a **reauth flow**, which Home Assistant starts by itself when the app refuses the key it holds,
// takes a new key (Sonarr) or a new URL and key (Radarr, Lidarr);
// - anything else — the URL of a working entry — only by removing the integration and adding it
// again, which throws away its entities' names, areas and history links. **This step never
// removes an entry.**
// So, per app:
// 1. The bound key is tried against the bound app first. Refused, nothing is written: until the
// operator accepts the app's own key for this pair, the mesh delivers a value it minted, which
// no Servarr app takes (novox/hq ADR 0092) — the failure names the `secret accept` that fixes it.
// 2. No entry: one is made through the user flow.
// 3. A reauth flow Home Assistant started for the entry: finished with the bound key (and URL,
// where the integration's reauth asks for one).
// 4. An entry whose URL (read from the device the integration registered, `configuration_url`)
// is the bound one and which is loaded: already as the mesh says. The key needs no digest here:
// a Servarr app has one key, so an entry loaded against the app holds the key the app took.
// 5. A working entry at a different URL that reaches **the same app** — the same process, by the
// app's own status (start time, data folder, version) — is left as it is and said: ace's entries
// say `127.0.0.1:<port>` and the binding says `ace.internal:<port>`, one Sonarr either way.
// 6. Anything else is refused, loudly, with what the operator can do; nothing is removed.
export interface ServarrApp {
/** The integration's domain, also the app. */
domain: "sonarr" | "radarr" | "lidarr";
/** The provision it is required as: the `requires`, `binds` and `secrets` key. */
provision: string;
/** The app's status endpoint: answers 401 to a wrong key, and says which process answered. */
statusPath: string;
}
export const APPS: readonly ServarrApp[] = [
{ domain: "sonarr", provision: "sonarr-api", statusPath: "/api/v3/system/status" },
{ domain: "radarr", provision: "radarr-api", statusPath: "/api/v3/system/status" },
{ domain: "lidarr", provision: "lidarr-api", statusPath: "/api/v1/system/status" },
];
/** The HTTP the step needs toward the apps, so a test can stand fakes in. */
export interface Http {
fetch(url: string, init?: { method?: string; headers?: Record<string, string> }): Promise<{ status: number; text(): Promise<string> }>;
}
export type AppWanted = { ok: true; url: string; key: string; from: string } | { ok: false; problem: string };
/** The URL and key the mesh says Home Assistant uses for this app. */
export function wantedApp(spec: ServarrApp, binding: Binding | undefined, credential: string | undefined): AppWanted {
if (!binding) return { ok: false, problem: `no binding for ${spec.provision} was delivered — the mesh writes it before this step runs` };
const at = typeof binding.at === "string" ? binding.at.trim() : "";
if (!at) return { ok: false, problem: `the ${spec.provision} binding names no host (at)` };
const port = portOf(binding.serves);
if (port === undefined) return { ok: false, problem: `the ${spec.provision} binding serves no usable port (${String(binding.serves?.port)})` };
const scheme = typeof binding.serves?.scheme === "string" && binding.serves.scheme ? binding.serves.scheme : "http";
if (scheme !== "http" && scheme !== "https") return { ok: false, problem: `the ${spec.provision} binding serves scheme ${scheme}` };
const base = typeof binding.serves?.["url-base"] === "string" ? String(binding.serves["url-base"]).trim().replace(/^\/+|\/+$/g, "") : "";
const key = (credential ?? "").trim();
if (!key) return { ok: false, problem: `the ${spec.provision} credential is empty or was not delivered` };
return {
ok: true,
url: `${scheme}://${urlHost(at)}:${port}${base ? `/${base}` : ""}`,
key,
from: typeof binding.from === "string" ? binding.from : "",
};
}
/** Two URLs naming the same place: scheme, host, port (explicit or default) and base path. */
export function sameUrl(a: string | null | undefined, b: string): boolean {
if (!a) return false;
try {
const x = new URL(a);
const y = new URL(b);
const port = (u: URL) => u.port || (u.protocol === "https:" ? "443" : "80");
const path = (u: URL) => u.pathname.replace(/\/+$/, "");
return x.protocol === y.protocol && x.hostname.toLowerCase() === y.hostname.toLowerCase() && port(x) === port(y) && path(x) === path(y);
} catch {
return false;
}
}
type Status = { taken: true; status: Record<string, unknown> } | { taken: false };
/** The app's status with this key: `taken: false` when it refuses the key; throws when it cannot be asked. */
export async function appStatus(http: Http, spec: ServarrApp, url: string, key: string): Promise<Status> {
const res = await http.fetch(`${url.replace(/\/+$/, "")}${spec.statusPath}`, {
method: "GET",
headers: { "X-Api-Key": key, Accept: "application/json" },
});
if (res.status === 401 || res.status === 403) return { taken: false };
if (res.status < 200 || res.status >= 300) throw new Error(`${spec.domain} answered ${res.status} at ${spec.statusPath}`);
return { taken: true, status: JSON.parse(await res.text()) as Record<string, unknown> };
}
/** Whether two status answers came from one running app. */
export function sameInstance(a: Record<string, unknown>, b: Record<string, unknown>): boolean {
const facts = ["startTime", "appData", "version"];
return facts.every((k) => a[k] !== undefined && a[k] !== null && a[k] === b[k]);
}
/** The remedy for a refused key, in the controller's words (ADR 0092). */
export function acceptRemedy(spec: ServarrApp, from: string): string {
return (
`${spec.domain} refuses the ${spec.provision} credential the mesh delivered, so nothing was written into ` +
`Home Assistant. A Servarr app has one API key and the mesh cannot make it: accept ${spec.domain}'s own key ` +
`for this pair — \`secret accept <this node> home-assistant ${spec.provision} --provider ${from || "<its node>"} ` +
`--from <file holding ${spec.domain}'s ApiKey>\``
);
}
export interface ServarrDeps {
hass: Hass;
http: Http;
}
/** The input a Servarr form takes: what it shows, with the URL (where asked) and the key laid over. */
function servarrInput(schema: readonly SchemaField[] | null | undefined, url: string, key: string): Record<string, unknown> {
const input = formValues(schema);
if (hasField(schema, "url")) input.url = url;
if (hasField(schema, "api_key")) input.api_key = key;
return input;
}
/** Bring Home Assistant's entry for one app in line with the mesh. Never throws. */
export async function reconcileApp(deps: ServarrDeps, spec: ServarrApp, binding: Binding | undefined, credential: string | undefined): Promise<Outcome> {
const what = spec.domain;
const w = wantedApp(spec, binding, credential);
if ("problem" in w) return { what, result: "refused", problem: w.problem };
let bound: Status;
try {
bound = await appStatus(deps.http, spec, w.url, w.key);
} catch (err) {
return { what, result: "refused", problem: `${spec.domain} could not be asked at ${w.url}: ${scrub(err, w.key)}` };
}
if (!bound.taken) return { what, result: "refused", problem: acceptRemedy(spec, w.from) };
try {
const entries = (await deps.hass.entries(spec.domain)).filter((e) => e.domain === spec.domain);
if (entries.length > 1) {
return { what, result: "refused", problem: `Home Assistant has ${entries.length} ${spec.domain} entries; which one the mesh owns is not guessed` };
}
// No entry: made, through the integration's own user flow, which validates the key itself.
if (entries.length === 0) {
const flow = await deps.hass.startFlow(spec.domain);
if (flow.type !== "form" || !flow.flow_id) return { what, result: "refused", problem: `Home Assistant's ${spec.domain} user flow answered ${describe(flow)}` };
const made = await deps.hass.stepFlow(flow.flow_id, servarrInput(flow.data_schema, w.url, w.key));
if (made.type === "create_entry") return { what, result: "written", fields: ["entry"] };
if (made.flow_id) await deps.hass.abortFlow(made.flow_id);
return { what, result: "refused", problem: `Home Assistant refused a new ${spec.domain} entry at ${w.url} (${describe(made)})` };
}
const entry = entries[0];
if (entry.disabled_by) return { what, result: "unchanged", note: `the ${spec.domain} entry is disabled (by ${entry.disabled_by}); left alone` };
// A reauth Home Assistant started because the app refused its key: finished with the bound one.
const reauth = (await deps.hass.flowsInProgress()).find(
(f) => f.handler === spec.domain && f.context?.source === "reauth" && f.context?.entry_id === entry.entry_id,
);
if (reauth) {
let step = await deps.hass.stepFlow(reauth.flow_id, {}); // reauth_confirm: a confirmation, no fields
if (step.type === "form" && step.flow_id && step.step_id !== "reauth_confirm") {
const input = servarrInput(step.data_schema, w.url, w.key);
const fields = ["api_key", ...(hasField(step.data_schema, "url") ? ["url"] : [])];
step = await deps.hass.stepFlow(step.flow_id, input);
if (step.type === "abort" && step.reason === "reauth_successful") return { what, result: "written", fields };
}
return { what, result: "refused", problem: `Home Assistant's ${spec.domain} reauth did not take the bound key and URL (${describe(step)})` };
}
const device = (await deps.hass.devices()).find((d) => d.config_entries?.includes(entry.entry_id) && d.configuration_url);
const current = device?.configuration_url ?? undefined;
if (entry.state === "loaded" && sameUrl(current, w.url)) return { what, result: "unchanged" };
if (entry.state === "loaded" && current) {
let there: Status | undefined;
try {
there = await appStatus(deps.http, spec, current, w.key);
} catch {
there = undefined;
}
if (there?.taken && sameInstance(there.status, bound.status)) {
return {
what,
result: "equivalent",
note:
`Home Assistant reaches ${spec.domain} at ${current}, the same running app the mesh bound at ${w.url}; ` +
`Home Assistant has no way to change a working ${spec.domain} entry's URL short of removing it, so it is left as it is`,
};
}
}
return {
what,
result: "refused",
problem:
`Home Assistant's ${spec.domain} entry (${entry.state ?? "unknown state"}) points at ${current ?? "an unknown URL"}, ` +
`not the ${spec.domain} the mesh bound at ${w.url}. Home Assistant only lets a working entry's URL change by ` +
`removing and re-adding the integration, which this step never does: remove it in Home Assistant ` +
`(Settings → Devices & services → ${spec.domain}) and the next run adds it at the bound URL`,
};
} catch (err) {
return { what, result: "refused", problem: scrub(err, w.key) };
}
}
-160
View File
@@ -1,160 +0,0 @@
// Home Assistant's own configuration API, as the provisions step uses it — the supported way to
// change an integration's connection. Home Assistant keeps every integration in
// `.storage/core.config_entries`, a file it owns and rewrites; the mesh may not write it, and it is
// not a file the mesh could merge into. What Home Assistant offers instead is the same thing its UI
// uses: **config flows** over REST (`/api/config/config_entries/flow`) — a user flow creates an
// entry, a reconfigure flow changes one, a reauth flow (which Home Assistant starts itself when a
// credential stops working) replaces its credential — each validated by the integration's own
// connection test before anything is saved. The two things REST does not answer (which flows Home
// Assistant has started, which device an entry made) come over its WebSocket API.
//
// Nothing here reads `.storage`. Authenticated with the module's accepted long-lived access token.
/** A config entry as `GET /api/config/config_entries/entry` lists it — no data, no credentials. */
export interface ConfigEntry {
entry_id: string;
domain: string;
title?: string;
source?: string;
state?: string;
disabled_by?: string | null;
}
/** One field of a flow's form, as Home Assistant serializes a voluptuous schema. */
export interface SchemaField {
name: string;
type?: string;
required?: boolean;
optional?: boolean;
default?: unknown;
description?: { suggested_value?: unknown } | null;
/** A section (`type: "expandable"`) carries its own fields. */
schema?: SchemaField[];
}
/** What a flow answered: another form, an entry made, or the flow ended (abort). */
export interface FlowResult {
type: string;
flow_id?: string;
handler?: string;
step_id?: string;
data_schema?: SchemaField[] | null;
errors?: Record<string, string> | null;
reason?: string;
result?: { entry_id?: string } | unknown;
}
/** A flow in progress that Home Assistant started itself (a reauth, a discovery). */
export interface FlowProgress {
flow_id: string;
handler: string;
step_id?: string;
context?: { source?: string; entry_id?: string };
}
/** A device from the device registry; an integration names where its app is as configuration_url. */
export interface DeviceEntry {
id: string;
config_entries?: string[];
configuration_url?: string | null;
}
export interface Hass {
entries(domain: string): Promise<ConfigEntry[]>;
/** A user flow for `handler`, or — given an entry — a reconfigure flow for it. */
startFlow(handler: string, entryId?: string): Promise<FlowResult>;
stepFlow(flowId: string, input: Record<string, unknown>): Promise<FlowResult>;
abortFlow(flowId: string): Promise<void>;
flowsInProgress(): Promise<FlowProgress[]>;
devices(): Promise<DeviceEntry[]>;
}
/** Home Assistant over HTTP: REST for entries and flows, one short WebSocket session per question. */
export class HassApi implements Hass {
private readonly base: string;
constructor(url: string, private readonly token: string) {
this.base = url.replace(/\/$/, "");
}
private async rest(method: string, path: string, body?: unknown): Promise<unknown> {
const res = await fetch(`${this.base}${path}`, {
method,
headers: {
Authorization: `Bearer ${this.token}`,
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) {
// Home Assistant's error text names fields, never echoes their values.
throw new Error(`Home Assistant ${method} ${path} answered ${res.status}${text ? `: ${text.slice(0, 200)}` : ""}`);
}
return text ? (JSON.parse(text) as unknown) : undefined;
}
async entries(domain: string): Promise<ConfigEntry[]> {
return ((await this.rest("GET", `/api/config/config_entries/entry?domain=${encodeURIComponent(domain)}`)) ??
[]) as ConfigEntry[];
}
async startFlow(handler: string, entryId?: string): Promise<FlowResult> {
return (await this.rest("POST", "/api/config/config_entries/flow", {
handler,
show_advanced_options: true,
...(entryId ? { entry_id: entryId } : {}),
})) as FlowResult;
}
async stepFlow(flowId: string, input: Record<string, unknown>): Promise<FlowResult> {
return (await this.rest("POST", `/api/config/config_entries/flow/${encodeURIComponent(flowId)}`, input)) as FlowResult;
}
async abortFlow(flowId: string): Promise<void> {
await this.rest("DELETE", `/api/config/config_entries/flow/${encodeURIComponent(flowId)}`).catch(() => undefined);
}
async flowsInProgress(): Promise<FlowProgress[]> {
return (await this.ws("config_entries/flow/progress")) as FlowProgress[];
}
async devices(): Promise<DeviceEntry[]> {
return (await this.ws("config/device_registry/list")) as DeviceEntry[];
}
/** One WebSocket command: connect, authenticate, ask, close. */
private ws(type: string): Promise<unknown> {
const url = `${this.base.replace(/^http/, "ws")}/api/websocket`;
return new Promise((resolve, reject) => {
const socket = new WebSocket(url);
const timer = setTimeout(() => {
socket.close();
reject(new Error(`Home Assistant's WebSocket did not answer ${type} within 30s`));
}, 30_000);
const done = (fn: () => void): void => {
clearTimeout(timer);
socket.close();
fn();
};
socket.onerror = () => done(() => reject(new Error(`Home Assistant's WebSocket at ${url} failed`)));
socket.onmessage = (event: { data: unknown }) => {
const msg = JSON.parse(String(event.data)) as {
type: string;
id?: number;
success?: boolean;
result?: unknown;
error?: { message?: string };
};
if (msg.type === "auth_required") socket.send(JSON.stringify({ type: "auth", access_token: this.token }));
else if (msg.type === "auth_invalid") done(() => reject(new Error("Home Assistant refused the token")));
else if (msg.type === "auth_ok") socket.send(JSON.stringify({ id: 1, type }));
else if (msg.type === "result" && msg.id === 1) {
if (msg.success) done(() => resolve(msg.result));
else done(() => reject(new Error(`Home Assistant ${type}: ${msg.error?.message ?? "failed"}`)));
}
};
});
}
}
@@ -1,84 +0,0 @@
// home-assistant's provisions step — run once by the host after Home Assistant starts, and again
// whenever a binding or pair credential it reads changes (the container's `restart-on`, novox/hq
// ADR 0099). It points Home Assistant's MQTT integration at the `mqtt-topic` broker and its Sonarr,
// Radarr and Lidarr integrations at the `sonarr-api`, `radarr-api` and `lidarr-api` apps, through
// Home Assistant's own config flows (connections.ts). It connects to no mesh broker.
//
// Exits non-zero when anything could not be put right, so the node reports the step failed and the
// host runs it again on the next apply. Declared last in the manifest, so its failing gates nothing
// else of home-assistant's (novox/hq ADR 0136). Never prints a key or password.
import { join } from "node:path";
import { APPS, MQTT_PROVISION, reconcileApp, reconcileMqtt, type Outcome } from "./connections.js";
import { HassApi } from "./hass.js";
import { marksIn, readBinding, readIfThere } from "./mesh.js";
import { probeBroker } from "./probe.js";
const dir = process.env.MESH_PROVISIONS_DIR ?? "/run/provisions";
const url = process.env.MESH_HOMEASSISTANT_URL ?? "http://127.0.0.1:8123";
const token = (await readIfThere(process.env.MESH_HOMEASSISTANT_TOKEN_FILE))?.trim() ?? "";
const marks = marksIn(process.env.MESH_WRITTEN_DIR ?? "/var/lib/home-assistant-provisions");
const waitSeconds = Number(process.env.MESH_HOMEASSISTANT_WAIT_SECONDS ?? "300");
if (!token) {
console.error("[hass-provisions] no Home Assistant token — home-assistant's own `token` secret has not been accepted");
process.exit(1);
}
/** Home Assistant answers /api/ with 200 once it is up and the token is good. */
async function ready(): Promise<boolean> {
const until = Date.now() + waitSeconds * 1000;
for (;;) {
try {
const res = await fetch(`${url.replace(/\/$/, "")}/api/`, { headers: { Authorization: `Bearer ${token}` } });
if (res.status === 200) return true;
if (res.status === 401 || res.status === 403) {
console.error("[hass-provisions] Home Assistant refuses the token — accept a long-lived access token it issued");
return false;
}
} catch {
// not listening yet
}
if (Date.now() >= until) return false;
await new Promise((r) => setTimeout(r, 3000));
}
}
if (!(await ready())) {
console.error(`[hass-provisions] Home Assistant did not answer at ${url} within ${waitSeconds}s`);
process.exit(1);
}
const hass = new HassApi(url, token);
const read = async (p: string) => [await readBinding(join(dir, `${p}.json`)), await readIfThere(join(dir, `${p}.secret`))] as const;
const outcomes: Outcome[] = [];
{
const [binding, secret] = await read(MQTT_PROVISION);
outcomes.push(await reconcileMqtt({ hass, probe: probeBroker, marks }, binding, secret));
}
for (const spec of APPS) {
const [binding, secret] = await read(spec.provision);
outcomes.push(await reconcileApp({ hass, http: { fetch: (u, init) => fetch(u, init) } }, spec, binding, secret));
}
let failed = 0;
for (const o of outcomes) {
switch (o.result) {
case "unchanged":
console.log(`[hass-provisions] ${o.what}: already as the mesh says${o.note ? ` — ${o.note}` : ""}`);
break;
case "written":
console.log(`[hass-provisions] ${o.what}: wrote ${o.fields.join(", ")}; Home Assistant's own test passed${o.note ? ` — ${o.note}` : ""}`);
break;
case "equivalent":
console.log(`[hass-provisions] ${o.what}: ${o.note}`);
break;
case "refused":
failed++;
console.error(`[hass-provisions] ${o.what}: ${o.problem}`);
break;
}
}
process.exitCode = failed > 0 ? 1 : 0;
-42
View File
@@ -1,42 +0,0 @@
// What the mesh delivered to home-assistant's provisions step, and the step's own small memory.
//
// Per provision it requires, the mesh writes two files beside each other (the manifest's `binds` and
// `secrets`): `<provision>.json`, the binding — where the provider is (`at`), what it serves (`port`,
// `scheme`, …) and the login this module presents (`as`) — and `<provision>.secret`, the pair
// credential. Nothing here guesses a host, a port or a key.
import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
import { join } from "node:path";
import type { Binding, Marks } from "./connections.js";
/** A file the mesh wrote, or undefined when it is not there. */
export async function readIfThere(path: string | undefined): Promise<string | undefined> {
if (!path) return undefined;
return readFile(path, "utf8").catch(() => undefined);
}
/** A binding file parsed, or undefined when absent or not JSON. */
export async function readBinding(path: string): Promise<Binding | undefined> {
const raw = await readIfThere(path);
if (raw === undefined) return undefined;
try {
return JSON.parse(raw) as Binding;
} catch {
return undefined;
}
}
export function marksIn(dir: string): Marks {
return {
async get(name) {
return (await readIfThere(join(dir, `${name}.digest`)))?.trim() || undefined;
},
async set(name, value) {
await mkdir(dir, { recursive: true, mode: 0o700 });
const path = join(dir, `${name}.digest`);
await writeFile(`${path}.tmp`, `${value}\n`, { mode: 0o600 });
await rename(`${path}.tmp`, path);
},
};
}
-117
View File
@@ -1,117 +0,0 @@
// Ask the broker, before Home Assistant is told anything, whether it takes the login and password
// the mesh delivered — and whether that login may subscribe to Home Assistant's discovery topics.
//
// One MQTT 3.1.1 session: CONNECT (clean, a throwaway client id, so Home Assistant's own session is
// never taken over), read the CONNACK, optionally SUBSCRIBE once and read the SUBACK, DISCONNECT.
// No dependency: the handful of bytes MQTT needs for this are written here.
import { randomBytes } from "node:crypto";
import { connect } from "node:net";
export interface ProbeResult {
/** 0 accepted; 4 bad username or password; 5 not authorised. */
connack: number;
/** The SUBACK return code for the filter asked about: 0–2 granted, 0x80 refused. */
suback?: number;
}
export type Probe = (host: string, port: number, username: string, password: string, subscribe?: string) => Promise<ProbeResult>;
function str(v: string): Buffer {
const b = Buffer.from(v, "utf8");
const len = Buffer.alloc(2);
len.writeUInt16BE(b.length);
return Buffer.concat([len, b]);
}
function packet(type: number, body: Buffer): Buffer {
let remaining = body.length;
const lenBytes: number[] = [];
do {
let byte = remaining % 128;
remaining = Math.floor(remaining / 128);
if (remaining > 0) byte |= 0x80;
lenBytes.push(byte);
} while (remaining > 0);
return Buffer.concat([Buffer.from([type, ...lenBytes]), body]);
}
/** The first complete packet in `buf`: its type byte, its body, and how many bytes it took. */
export function firstPacket(buf: Buffer): { type: number; body: Buffer; used: number } | undefined {
if (buf.length < 2) return undefined;
let length = 0;
let multiplier = 1;
let i = 1;
for (;;) {
if (i >= buf.length) return undefined;
const byte = buf[i++];
length += (byte & 0x7f) * multiplier;
if ((byte & 0x80) === 0) break;
multiplier *= 128;
if (i > 4) throw new Error("malformed MQTT remaining length");
}
if (buf.length < i + length) return undefined;
return { type: buf[0], body: buf.subarray(i, i + length), used: i + length };
}
export const probeBroker: Probe = (host, port, username, password, subscribe) => {
const connectBody = Buffer.concat([
str("MQTT"),
Buffer.from([4, 0xc2, 0, 10]), // level 4 (3.1.1); username + password + clean session; keepalive 10s
str(`mesh-probe-${randomBytes(6).toString("hex")}`),
str(username),
str(password),
]);
return new Promise((resolve, reject) => {
const socket = connect({ host, port });
let buf = Buffer.alloc(0);
const result: ProbeResult = { connack: -1 };
const timer = setTimeout(() => {
socket.destroy();
reject(new Error(`no answer from the broker at ${host}:${port} within 10s`));
}, 10_000);
const finish = (): void => {
clearTimeout(timer);
if (result.connack === 0) socket.end(Buffer.from([0xe0, 0]));
else socket.destroy();
resolve(result);
};
socket.on("connect", () => socket.write(packet(0x10, connectBody)));
socket.on("data", (chunk) => {
buf = Buffer.concat([buf, chunk]);
for (;;) {
let p;
try {
p = firstPacket(buf);
} catch (err) {
clearTimeout(timer);
socket.destroy();
reject(err);
return;
}
if (!p) return;
buf = buf.subarray(p.used);
const kind = p.type >> 4;
if (kind === 2) {
result.connack = p.body[1] ?? -1;
if (result.connack !== 0 || !subscribe) return finish();
// SUBSCRIBE, packet id 1, one filter at QoS 0.
socket.write(packet(0x82, Buffer.concat([Buffer.from([0, 1]), str(subscribe), Buffer.from([0])])));
} else if (kind === 9) {
result.suback = p.body[2];
return finish();
}
}
});
socket.on("error", (err) => {
clearTimeout(timer);
reject(err);
});
socket.on("close", () => {
if (result.connack === -1) {
clearTimeout(timer);
reject(new Error(`the broker at ${host}:${port} closed the connection without answering`));
}
});
});
};
@@ -1,293 +0,0 @@
// What holds home-assistant's provisions step (provisions/*.ts): Home Assistant's MQTT entry is
// made to use the broker, port and login the mesh bound — only after the broker takes that login,
// through the reconfigure flow, keeping every other setting as Home Assistant pre-filled it, and not
// again once it already says so; its Sonarr/Radarr/Lidarr entries are made, finished (reauth), left
// alone when they already reach the bound app, and never removed; a key the app refuses (the mesh's
// minted value before the operator accepts the app's) is never written.
//
// Home Assistant and the apps are fakes answering as the real ones do (flow shapes checked against
// ghcr.io/home-assistant/home-assistant 2026.9.3, the build ace runs).
import { test } from "node:test";
import assert from "node:assert/strict";
import type { ConfigEntry, DeviceEntry, FlowProgress, FlowResult, Hass, SchemaField } from "../provisions/hass.ts";
import type { Binding, Marks } from "../provisions/connections.ts";
import { APPS, formValues, reconcileApp, reconcileMqtt, sameUrl, type Http, type ServarrApp } from "../provisions/connections.ts";
import type { Probe } from "../provisions/probe.ts";
const PWD_NOT_CHANGED = "__**password_not_changed**__";
const MINTED = "mesh-minted-password";
function mqttBinding(): Binding {
return { provision: "mqtt-topic", from: "ace", at: "ace.internal", as: "mesh_ace_hass", serves: { scheme: "mqtt", port: 1883 } };
}
/** The MQTT reconfigure form as Home Assistant serializes it, pre-filled from an entry. */
function brokerForm(data: Record<string, unknown>): SchemaField[] {
return [
{ name: "broker", type: "string", required: true, description: { suggested_value: data.broker } },
{ name: "port", type: "integer", required: true, default: 1883, description: { suggested_value: data.port } },
{ name: "protocol", type: "select", required: true, default: "3.1.1", description: { suggested_value: data.protocol } },
{ name: "username", type: "string", optional: true, description: { suggested_value: data.username } },
{ name: "password", type: "string", optional: true, description: { suggested_value: data.password ? PWD_NOT_CHANGED : undefined } },
{
name: "other_settings",
type: "expandable",
required: true,
schema: [
{ name: "keepalive", type: "integer", optional: true, description: { suggested_value: 60 } },
{ name: "transport", type: "select", required: true, default: "tcp", description: { suggested_value: "tcp" } },
{ name: "set_ca_cert", type: "select", required: true, description: { suggested_value: "off" } },
{ name: "set_client_cert", type: "boolean", required: true, description: { suggested_value: false } },
],
},
];
}
interface FakeOpts {
entries?: Record<string, (ConfigEntry & { data: Record<string, unknown> })[]>;
/** What Home Assistant's own connection test accepts. */
accepts?: (data: Record<string, unknown>) => boolean;
reauth?: FlowProgress[];
devices?: DeviceEntry[];
}
function fakeHass(opts: FakeOpts = {}) {
const entries = opts.entries ?? {};
const calls: string[] = [];
const submitted: Record<string, unknown>[] = [];
const flows = new Map<string, { handler: string; entryId?: string; step: string; reauth?: boolean }>();
let n = 0;
const accepts = opts.accepts ?? (() => true);
const form = (id: string, step: string, schema: SchemaField[], errors?: Record<string, string>): FlowResult => ({
type: "form", flow_id: id, step_id: step, data_schema: schema, errors: errors ?? null,
});
const servarrUser: SchemaField[] = [
{ name: "url", type: "string", required: true },
{ name: "api_key", type: "string", required: true },
{ name: "more_options", type: "expandable", required: true, schema: [{ name: "verify_ssl", type: "boolean", optional: true, default: false }] },
];
const hass: Hass = {
async entries(domain) {
calls.push(`entries ${domain}`);
return (entries[domain] ?? []).map(({ data: _d, ...e }) => e);
},
async startFlow(handler, entryId) {
calls.push(`start ${handler}${entryId ? ` ${entryId}` : ""}`);
const id = `f${++n}`;
if (handler === "mqtt") {
const entry = entryId ? entries.mqtt.find((e) => e.entry_id === entryId) : undefined;
if (entryId && !entry) return { type: "abort", reason: "not_found" };
flows.set(id, { handler, entryId, step: "broker" });
return form(id, "broker", brokerForm(entry?.data ?? {}));
}
if (entryId) return { type: "abort", reason: "not_implemented" }; // no reconfigure for Servarr
flows.set(id, { handler, step: "user" });
return form(id, "user", servarrUser);
},
async stepFlow(flowId, input) {
calls.push(`step ${flowId}`);
const flow = flows.get(flowId);
if (!flow) throw new Error(`Home Assistant POST flow/${flowId} answered 404`);
if (flow.step === "reauth_confirm") {
flow.step = "user";
return form(flowId, "user", [
{ name: "url", type: "string", required: true, default: "http://old:1" },
{ name: "api_key", type: "string", optional: true },
{ name: "verify_ssl", type: "boolean", optional: true, default: false },
]);
}
submitted.push(input);
if (flow.handler === "mqtt") {
const entry = entries.mqtt?.find((e) => e.entry_id === flow.entryId);
const data = { ...input, ...(input.password === PWD_NOT_CHANGED ? { password: entry?.data.password } : {}) };
if (!accepts(data)) return form(flowId, "broker", brokerForm(data), { base: "cannot_connect" });
flows.delete(flowId);
if (entry) {
entry.data = data;
return { type: "abort", reason: "reconfigure_successful" };
}
(entries.mqtt ??= []).push({ entry_id: "new-mqtt", domain: "mqtt", state: "loaded", data });
return { type: "create_entry", result: { entry_id: "new-mqtt" } };
}
if (!accepts(input)) return form(flowId, "user", servarrUser, { base: "invalid_auth" });
flows.delete(flowId);
if (flow.reauth) return { type: "abort", reason: "reauth_successful" };
(entries[flow.handler] ??= []).push({ entry_id: `new-${flow.handler}`, domain: flow.handler, state: "loaded", data: input });
return { type: "create_entry", result: { entry_id: `new-${flow.handler}` } };
},
async abortFlow(flowId) {
calls.push(`abort ${flowId}`);
flows.delete(flowId);
},
async flowsInProgress() {
for (const f of opts.reauth ?? []) flows.set(f.flow_id, { handler: f.handler, entryId: f.context?.entry_id, step: "reauth_confirm", reauth: true });
return opts.reauth ?? [];
},
async devices() {
return opts.devices ?? [];
},
};
return { hass, calls, submitted, entries };
}
function memoryMarks(): Marks & { store: Map<string, string> } {
const store = new Map<string, string>();
return { store, get: async (k) => store.get(k), set: async (k, v) => void store.set(k, v) };
}
const takes = (suback = 0): Probe => async (_h, _p, user, pass) => ({ connack: user === "mesh_ace_hass" && pass === MINTED ? 0 : 5, suback });
const aceMqttEntry = () => ({
entry_id: "7d1e", domain: "mqtt", state: "loaded",
data: { broker: "127.0.0.1", port: 1883, protocol: "5", username: "luffy", password: "luffys-password" },
});
test("mqtt: the broker is asked first; a login it does not take is never written", async () => {
const f = fakeHass({ entries: { mqtt: [aceMqttEntry()] } });
const out = await reconcileMqtt({ hass: f.hass, probe: async () => ({ connack: 5 }), marks: memoryMarks() }, mqttBinding(), MINTED);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /does not \(yet\) take the login mesh_ace_hass/);
assert.deepEqual(f.calls, []); // Home Assistant not even asked
assert.equal(f.entries.mqtt[0].data.username, "luffy");
});
test("mqtt: ace's entry (127.0.0.1, luffy) is moved to the bound broker and login, every other setting kept", async () => {
const f = fakeHass({ entries: { mqtt: [aceMqttEntry()] } });
const marks = memoryMarks();
const out = await reconcileMqtt({ hass: f.hass, probe: takes(), marks }, mqttBinding(), `${MINTED}\n`);
assert.deepEqual(out, { what: "mqtt", result: "written", fields: ["broker", "username", "password"] });
assert.deepEqual(f.entries.mqtt[0].data, {
broker: "ace.internal", port: 1883, protocol: "5", username: "mesh_ace_hass", password: MINTED,
other_settings: { keepalive: 60, transport: "tcp", set_ca_cert: "off", set_client_cert: false },
});
assert.ok(marks.store.get("mqtt"));
assert.ok(![...marks.store.values()].some((v) => v.includes(MINTED)));
// Run again: nothing differs, the flow is opened to read and closed without submitting.
const before = f.submitted.length;
const again = await reconcileMqtt({ hass: f.hass, probe: takes(), marks }, mqttBinding(), MINTED);
assert.deepEqual(again, { what: "mqtt", result: "unchanged" });
assert.equal(f.submitted.length, before);
assert.match(f.calls.at(-1) ?? "", /^abort /);
});
test("mqtt: a new password alone is written (the digest tells)", async () => {
const f = fakeHass({ entries: { mqtt: [aceMqttEntry()] } });
const marks = memoryMarks();
await reconcileMqtt({ hass: f.hass, probe: takes(), marks }, mqttBinding(), MINTED);
const rotated: Probe = async () => ({ connack: 0, suback: 0 });
const out = await reconcileMqtt({ hass: f.hass, probe: rotated, marks }, mqttBinding(), "rotated");
assert.deepEqual(out, { what: "mqtt", result: "written", fields: ["password"] });
assert.equal(f.entries.mqtt[0].data.password, "rotated");
});
test("mqtt: Home Assistant's own connection test refusing saves nothing and fails loudly", async () => {
const f = fakeHass({ entries: { mqtt: [aceMqttEntry()] }, accepts: () => false });
const marks = memoryMarks();
const out = await reconcileMqtt({ hass: f.hass, probe: takes(), marks }, mqttBinding(), MINTED);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /cannot_connect.*unchanged/);
assert.equal(f.entries.mqtt[0].data.username, "luffy");
assert.equal(marks.store.size, 0);
});
test("mqtt: a fresh Home Assistant gets an entry; a grant without the discovery topics is warned about", async () => {
const f = fakeHass();
const out = await reconcileMqtt({ hass: f.hass, probe: takes(0x80), marks: memoryMarks() }, mqttBinding(), MINTED);
assert.equal(out.result, "written");
assert.match((out as { note?: string }).note ?? "", /may not subscribe to homeassistant\/#/);
assert.equal(f.entries.mqtt[0].data.broker, "ace.internal");
});
test("mqtt: two entries, or a binding without a port, are refused rather than guessed", async () => {
const f = fakeHass({ entries: { mqtt: [aceMqttEntry(), { ...aceMqttEntry(), entry_id: "other" }] } });
assert.equal((await reconcileMqtt({ hass: f.hass, probe: takes(), marks: memoryMarks() }, mqttBinding(), MINTED)).result, "refused");
const noPort = { ...mqttBinding(), serves: {} };
assert.match(((await reconcileMqtt({ hass: f.hass, probe: takes(), marks: memoryMarks() }, noPort, MINTED)) as { problem: string }).problem, /no usable port/);
});
// ---- Servarr ----
const SONARR = APPS.find((a) => a.domain === "sonarr") as ServarrApp;
const RADARR = APPS.find((a) => a.domain === "radarr") as ServarrApp;
const KEY = "the-apps-own-key";
const servarrBinding = (port: number, at = "ace.internal"): Binding => ({ provision: "sonarr-api", from: "ace", at, as: "mesh_ace_hass", serves: { scheme: "http", port, "url-base": "" } });
/** One running Sonarr, answering on several addresses (127.0.0.1 and ace.internal are one host). */
function apps(instances: Record<string, { startTime: string }>): Http & { asked: string[] } {
const asked: string[] = [];
return {
asked,
async fetch(url, init) {
asked.push(url);
const u = new URL(url);
const inst = instances[`${u.hostname}:${u.port}`];
if (!inst) throw new Error("connect ECONNREFUSED");
if (init?.headers?.["X-Api-Key"] !== KEY) return { status: 401, text: async () => "" };
return { status: 200, text: async () => JSON.stringify({ version: "4.0.15", appData: "/config", startTime: inst.startTime }) };
},
};
}
const oneSonarr = () => apps({ "ace.internal:8989": { startTime: "t1" }, "127.0.0.1:8989": { startTime: "t1" } });
const sonarrEntry = (state = "loaded") => ({ entry_id: "5a1d", domain: "sonarr", state, data: { url: "http://127.0.0.1:8989", api_key: KEY } });
test("servarr: the mesh's minted key is never written; the remedy names the accept", async () => {
const f = fakeHass({ entries: { sonarr: [sonarrEntry()] } });
const out = await reconcileApp({ hass: f.hass, http: oneSonarr() }, SONARR, servarrBinding(8989), "minted-by-the-mesh");
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /secret accept <this node> home-assistant sonarr-api --provider ace/);
assert.deepEqual(f.calls, []);
});
test("servarr: ace's entry at 127.0.0.1 reaches the same Sonarr the mesh bound at ace.internal — left, and said", async () => {
const f = fakeHass({ entries: { sonarr: [sonarrEntry()] }, devices: [{ id: "d", config_entries: ["5a1d"], configuration_url: "http://127.0.0.1:8989" }] });
const out = await reconcileApp({ hass: f.hass, http: oneSonarr() }, SONARR, servarrBinding(8989), KEY);
assert.equal(out.result, "equivalent");
assert.equal(f.submitted.length, 0);
});
test("servarr: an entry already at the bound URL is unchanged", async () => {
const f = fakeHass({ entries: { sonarr: [sonarrEntry()] }, devices: [{ id: "d", config_entries: ["5a1d"], configuration_url: "http://ace.internal:8989" }] });
assert.deepEqual(await reconcileApp({ hass: f.hass, http: oneSonarr() }, SONARR, servarrBinding(8989), KEY), { what: "sonarr", result: "unchanged" });
});
test("servarr: a working entry that reaches a different app is refused, and nothing is removed", async () => {
const f = fakeHass({ entries: { sonarr: [sonarrEntry()] }, devices: [{ id: "d", config_entries: ["5a1d"], configuration_url: "http://127.0.0.1:8989" }] });
const two = apps({ "ace.internal:8989": { startTime: "t1" }, "127.0.0.1:8989": { startTime: "another" } });
const out = await reconcileApp({ hass: f.hass, http: two }, SONARR, servarrBinding(8989), KEY);
assert.equal(out.result, "refused");
assert.match((out as { problem: string }).problem, /never does/);
assert.equal(f.entries.sonarr.length, 1);
});
test("servarr: no entry — one is made at the bound URL through the user flow", async () => {
const f = fakeHass({ entries: {} });
const out = await reconcileApp({ hass: f.hass, http: oneSonarr() }, SONARR, servarrBinding(8989), KEY);
assert.deepEqual(out, { what: "sonarr", result: "written", fields: ["entry"] });
assert.deepEqual(f.submitted[0], { url: "http://ace.internal:8989", api_key: KEY, more_options: { verify_ssl: false } });
});
test("servarr: a reauth Home Assistant started is finished with the bound URL and key", async () => {
const entry = { ...sonarrEntry("setup_error"), domain: "radarr", entry_id: "1955" };
const f = fakeHass({
entries: { radarr: [entry] },
reauth: [{ flow_id: "r1", handler: "radarr", step_id: "reauth_confirm", context: { source: "reauth", entry_id: "1955" } }],
});
const radarr = apps({ "ace.internal:7878": { startTime: "t" } });
const out = await reconcileApp({ hass: f.hass, http: radarr }, RADARR, { ...servarrBinding(7878), provision: "radarr-api" }, KEY);
assert.deepEqual(out, { what: "radarr", result: "written", fields: ["api_key", "url"] });
assert.deepEqual(f.submitted[0], { url: "http://ace.internal:7878", api_key: KEY, verify_ssl: false });
});
test("form values: suggested first, then default, sections nested", () => {
assert.deepEqual(formValues(brokerForm({ broker: "b", port: 1, protocol: "5", username: "u", password: "p" })), {
broker: "b", port: 1, protocol: "5", username: "u", password: PWD_NOT_CHANGED,
other_settings: { keepalive: 60, transport: "tcp", set_ca_cert: "off", set_client_cert: false },
});
assert.ok(sameUrl("http://ace.internal:8989/", "http://ace.internal:8989"));
assert.ok(sameUrl("http://ACE.internal", "http://ace.internal:80"));
assert.ok(!sameUrl("http://127.0.0.1:8989", "http://ace.internal:8989"));
});
+1 -10
View File
@@ -8,14 +8,5 @@
"skipLibCheck": true, "skipLibCheck": true,
"noEmit": true "noEmit": true
}, },
"include": [ "include": ["client.ts", "index.ts", "tools/index.ts"]
"client.ts",
"index.ts",
"tools/index.ts",
"provisions/hass.ts",
"provisions/probe.ts",
"provisions/connections.ts",
"provisions/mesh.ts",
"provisions/index.ts"
]
} }
+24
View File
@@ -0,0 +1,24 @@
# icecast's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/icecast
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/icecast/dist /app/modules/icecast/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/icecast/dist/index.js,/app/modules/icecast/dist/tools/index.js
+60 -63
View File
@@ -1,26 +1,6 @@
{ {
"module": "icecast", "module": "icecast",
"version": "1", "version": "1",
"requires": [
"route",
"secret"
],
"contributes": {
"route": {
"label": "icecast",
"endpoint": "stream"
}
},
"binds": {
"route": "${dir:state}/route.json"
},
"secrets": {
"secret": {
"source": "${dir:state}/source.secret",
"admin": "${dir:state}/admin.secret",
"relay": "${dir:state}/relay.secret"
}
},
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
@@ -28,91 +8,108 @@
"stream.started", "stream.started",
"stream.stopped" "stream.stopped"
], ],
"own-secrets": {
"broker": "/var/lib/mesh/icecast/broker"
},
"listens": [ "listens": [
{ {
"name": "stream", "name": "stream",
"port": 8000, "port": 8000,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "streams in from sources (HTTP PUT) and out to listeners, plus the status and admin pages; a public name is its route" "why": "streams in from sources and out to listeners"
} }
], ],
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/icecast",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/icecast-module",
"place": "." "mode": "0700"
}, },
{ {
"id": "logs", "id": "server-env",
"type": "directory",
"mode": "0700",
"owner": "100:101"
},
{
"id": "server-conf",
"type": "file", "type": "file",
"path": "${dir:state}/icecast.xml", "path": "/var/lib/icecast-module/server.env",
"mode": "0600", "mode": "0600",
"content": "<icecast>\n <!-- Written by the mesh (modules/icecast). Passwords arrive as secrets rendered into this file,\n never as environment: the image's entrypoint seds ICECAST_* variables into the file only\n when they are set, and none are. -->\n <location>Earth</location>\n <admin>icemaster@localhost</admin>\n <limits>\n <clients>100</clients>\n <sources>2</sources>\n <queue-size>524288</queue-size>\n <client-timeout>30</client-timeout>\n <header-timeout>15</header-timeout>\n <source-timeout>10</source-timeout>\n <burst-on-connect>1</burst-on-connect>\n <burst-size>65535</burst-size>\n </limits>\n <authentication>\n <source-password>${secret:source}</source-password>\n <relay-password>${secret:relay}</relay-password>\n <admin-user>admin</admin-user>\n <admin-password>${secret:admin}</admin-password>\n </authentication>\n <!-- The name icecast writes into playlists (.m3u/.xspf: http://<hostname>:<port>/<mount>) and\n would announce to YP (none configured). A machine's own name belongs to its assignment, and\n an assignment merges only into JSON; this XML cannot take it, so the neutral default stays. -->\n <hostname>localhost</hostname>\n <listen-socket>\n <port>8000</port>\n </listen-socket>\n <http-headers>\n <header name=\"Access-Control-Allow-Origin\" value=\"*\" />\n </http-headers>\n <fileserve>1</fileserve>\n <paths>\n <basedir>/usr/share/icecast</basedir>\n <logdir>/var/log/icecast</logdir>\n <webroot>/usr/share/icecast/web</webroot>\n <adminroot>/usr/share/icecast/admin</adminroot>\n <alias source=\"/\" destination=\"/status.xsl\"/>\n </paths>\n <logging>\n <accesslog>access.log</accesslog>\n <errorlog>error.log</errorlog>\n <loglevel>3</loglevel>\n <logsize>10000</logsize>\n </logging>\n <security>\n <chroot>0</chroot>\n <!-- Starts as root, reads this 0600 root-owned file, then drops to the image's icecast user\n (uid 100, group icecast 101) before serving. -->\n <changeowner>\n <user>icecast</user>\n <group>icecast</group>\n </changeowner>\n </security>\n</icecast>\n" "content": "ICECAST_SOURCE_PASSWORD=${secret:source}\nICECAST_ADMIN_PASSWORD=${secret:admin}\nICECAST_RELAY_PASSWORD=${secret:relay}\nICECAST_ADMIN_USERNAME=admin\n"
},
{
"id": "net",
"type": "network",
"name": "icecast"
}, },
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "icecast", "name": "icecast",
"image": "infiniteproject/icecast@sha256:cd506cf3dfe31ce05fd37d7e672dbd1213e7255cc93d28ecf5a3b547af4e162c", "image": "infiniteproject/icecast@sha256:cd506cf3dfe31ce05fd37d7e672dbd1213e7255cc93d28ecf5a3b547af4e162c",
"network": "icecast", "env-file": [
"/var/lib/icecast-module/server.env"
],
"ports": [ "ports": [
"8000" "8000"
], ],
"volumes": [ "secrets-in-environment": "the image seds ICECAST_*_PASSWORD into icecast.xml and has no _FILE; convertible by mounting a generated icecast.xml, not yet done"
"${dir:state}/icecast.xml:/etc/icecast.xml:ro",
"${dir:logs}:/var/log/icecast"
],
"restart-on": [
"server-conf"
]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/icecast/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n",
"merge": "json" "merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-icecast",
"network": "host",
"volumes": [
"/var/lib/mesh/icecast/broker:/run/secrets/broker:ro",
"/var/lib/mesh/icecast/config.json:/run/config/config.json:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_ICECAST_URL": "http://127.0.0.1:8000",
"MESH_ICECAST_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
} }
], ],
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"index.js",
"tools/index.js"
],
"loads": [
"index.js",
"tools/index.js"
],
"env": {
"MESH_ICECAST_URL": "http://127.0.0.1:${port:8000}",
"MESH_ICECAST_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
} }
] ]
},
"requires": [
"secret"
],
"secrets": {
"secret": {
"source": "/var/lib/icecast-module/source.secret",
"admin": "/var/lib/icecast-module/admin.secret",
"relay": "/var/lib/icecast-module/relay.secret"
}
} }
} }
+24
View File
@@ -0,0 +1,24 @@
# influxdb's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/influxdb
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/influxdb/dist /app/modules/influxdb/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/influxdb/dist/tools/index.js
+3 -118
View File
@@ -17,25 +17,6 @@ export interface InfluxBucket {
retentionSeconds?: number; retentionSeconds?: number;
} }
/** One permission of an authorization, as InfluxDB represents it: an action on a resource type,
* in one org, optionally narrowed to one resource by id (no id = every resource of that type). */
export interface InfluxPermission {
action: "read" | "write";
resource: { type: string; orgID?: string; id?: string; name?: string; org?: string };
}
/** A v1-compatibility ("legacy") authorization: a username (InfluxDB calls it `token`) and a
* password the caller chooses, scoped by permissions. The one credential InfluxDB 2.x lets a
* caller set to a value it did not generate — which is what a mesh-minted password needs. */
export interface LegacyAuthorization {
id: string;
token: string;
orgID: string;
status?: "active" | "inactive";
description?: string;
permissions: InfluxPermission[];
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */ /** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> { function meshConfig(file?: string): Record<string, string> {
if (!file) return {}; if (!file) return {};
@@ -43,20 +24,13 @@ function meshConfig(file?: string): Record<string, string> {
catch { return {}; } catch { return {}; }
} }
/** A secret delivered as a file, trimmed; undefined when there is none, so the caller can fall back. */
function tokenFromFile(file?: string): string | undefined {
if (!file) return undefined;
try { return readFileSync(file, "utf8").trim() || undefined; }
catch { return undefined; }
}
export class InfluxDBClient { export class InfluxDBClient {
readonly baseUrl: string; readonly baseUrl: string;
constructor( constructor(
url: string, url: string,
private readonly token: string, private readonly token: string,
readonly org: string, private readonly org: string,
) { ) {
this.baseUrl = url.replace(/\/$/, ""); this.baseUrl = url.replace(/\/$/, "");
} }
@@ -69,10 +43,8 @@ export class InfluxDBClient {
static fromEnv(env: NodeJS.ProcessEnv = process.env): InfluxDBClient { static fromEnv(env: NodeJS.ProcessEnv = process.env): InfluxDBClient {
const cfg = meshConfig(env.MESH_INFLUXDB_CONFIG_FILE); const cfg = meshConfig(env.MESH_INFLUXDB_CONFIG_FILE);
const url = cfg.url ?? env.MESH_INFLUXDB_URL ?? `http://127.0.0.1:${env.INFLUXDB_PORT ?? "8086"}`; const url = cfg.url ?? env.MESH_INFLUXDB_URL ?? `http://127.0.0.1:${env.INFLUXDB_PORT ?? "8086"}`;
// The token reaches the process as a file (novox/hq ADR 0086); the environment variable stays const token = cfg.token ?? env.MESH_INFLUXDB_TOKEN;
// only for a workstation running the tools by hand. if (!token) throw new Error("no InfluxDB token — set MESH_INFLUXDB_TOKEN");
const token = cfg.token ?? tokenFromFile(env.MESH_INFLUXDB_TOKEN_FILE) ?? env.MESH_INFLUXDB_TOKEN;
if (!token) throw new Error("no InfluxDB token — set MESH_INFLUXDB_TOKEN_FILE");
const org = cfg.org ?? env.MESH_INFLUXDB_ORG ?? "mesh"; const org = cfg.org ?? env.MESH_INFLUXDB_ORG ?? "mesh";
return new InfluxDBClient(url, token, org); return new InfluxDBClient(url, token, org);
} }
@@ -89,93 +61,6 @@ export class InfluxDBClient {
return res; return res;
} }
/** Like request, but the answer is returned whatever its status, for the caller to read. */
private async raw(path: string, init?: RequestInit): Promise<Response> {
return fetch(`${this.baseUrl}${path}`, {
...init,
headers: { Authorization: `Token ${this.token}`, ...(init?.headers ?? {}) },
});
}
private async send(path: string, method: string, body?: unknown): Promise<Response> {
return this.request(path, {
method,
headers: { "Content-Type": "application/json" },
body: body === undefined ? undefined : JSON.stringify(body),
});
}
/** The id of the org of this name, or undefined when there is none. */
async orgID(name: string): Promise<string | undefined> {
const res = await this.raw(`/api/v2/orgs?org=${encodeURIComponent(name)}`);
if (res.status === 404) return undefined;
if (!res.ok) throw new Error(`InfluxDB API /api/v2/orgs: ${res.status} ${await res.text()}`);
const body = (await res.json()) as { orgs?: { id: string; name: string }[] };
return body.orgs?.find((o) => o.name === name)?.id;
}
/** The bucket of exactly this name in the org, or undefined. */
async findBucket(orgID: string, name: string): Promise<InfluxBucket | undefined> {
const res = await this.raw(`/api/v2/buckets?orgID=${encodeURIComponent(orgID)}&name=${encodeURIComponent(name)}`);
if (res.status === 404) return undefined;
if (!res.ok) throw new Error(`InfluxDB API /api/v2/buckets: ${res.status} ${await res.text()}`);
const body = (await res.json()) as { buckets?: { id: string; name: string; orgID?: string }[] };
const b = body.buckets?.find((x) => x.name === name);
return b ? { id: b.id, name: b.name, orgID: b.orgID } : undefined;
}
/** Create a bucket that keeps its data for ever — retention is the operator's choice, never the mesh's. */
async createBucket(orgID: string, name: string, description: string): Promise<InfluxBucket> {
const b = (await (await this.send("/api/v2/buckets", "POST", {
orgID, name, description, retentionRules: [],
})).json()) as { id: string; name: string; orgID?: string };
return { id: b.id, name: b.name, orgID: b.orgID };
}
/** The v1 authorization whose username is exactly this, or undefined. */
async findLegacy(username: string): Promise<LegacyAuthorization | undefined> {
const path = `/private/legacy/authorizations?token=${encodeURIComponent(username)}`;
const res = await this.raw(path);
// InfluxDB answers a filter matching nothing with 404, not an empty list.
if (res.status === 404) return undefined;
if (!res.ok) throw new Error(`InfluxDB API ${path}: ${res.status} ${await res.text()}`);
const body = (await res.json()) as { authorizations?: LegacyAuthorization[] };
return body.authorizations?.find((a) => a.token === username);
}
async createLegacy(a: Omit<LegacyAuthorization, "id">): Promise<LegacyAuthorization> {
return (await (await this.send("/private/legacy/authorizations", "POST", a)).json()) as LegacyAuthorization;
}
/** Set a v1 authorization's password. InfluxDB keeps only a hash of it, so it can be set, never read. */
async setLegacyPassword(id: string, password: string): Promise<void> {
await this.send(`/private/legacy/authorizations/${encodeURIComponent(id)}/password`, "POST", { password });
}
async updateLegacy(id: string, patch: { status?: "active" | "inactive"; description?: string }): Promise<void> {
await this.send(`/private/legacy/authorizations/${encodeURIComponent(id)}`, "PATCH", patch);
}
async deleteLegacy(id: string): Promise<void> {
await this.send(`/private/legacy/authorizations/${encodeURIComponent(id)}`, "DELETE");
}
/**
* Whether this username and password sign in on the v1 API — the consumer's own view. Asked with
* a statement that reads nothing (`SHOW DATABASES` lists only what the credential may read), sent
* with Basic auth so the password is never in a URL. 401 is a wrong password or no such user;
* anything else that is not a server error means InfluxDB knew who was asking.
*/
async legacySignsIn(username: string, password: string): Promise<boolean> {
const res = await fetch(`${this.baseUrl}/query?q=${encodeURIComponent("SHOW DATABASES")}`, {
headers: { Authorization: `Basic ${Buffer.from(`${username}:${password}`).toString("base64")}` },
});
await res.arrayBuffer();
if (res.status === 401) return false;
if (res.status >= 500) throw new Error(`InfluxDB v1 /query: ${res.status}`);
return true;
}
/** Server health — the one endpoint that needs no token, but we send it anyway. */ /** Server health — the one endpoint that needs no token, but we send it anyway. */
async health(): Promise<InfluxHealth> { async health(): Promise<InfluxHealth> {
return (await (await this.request("/health")).json()) as InfluxHealth; return (await (await this.request("/health")).json()) as InfluxHealth;
-186
View File
@@ -1,186 +0,0 @@
// What the `influxdb-api` provision means in InfluxDB: one v1-compatibility authorization per
// consumer, in the org this module serves, under the username and password the mesh gave both ends,
// allowed exactly the access the consumer contributed. The provisioner (provisioner/index.ts) is the
// sdk harness calling these; they are here, apart from it, so they can be exercised against a fake
// InfluxDB without a broker or a contributions file.
//
// **Why a v1 authorization and not a v2 API token.** The mesh mints the consumer's password and
// hands it to both ends (novox/hq ADR 0048); the provider sets it, and never hands one back. An
// InfluxDB 2.x API token is generated by the server — `POST /api/v2/authorizations` ignores a token
// the caller sends — so a token could only ever be the operator's to accept, one per pair, by hand.
// A v1 authorization is a username and a password the caller chooses (8–72 characters; the mesh
// mints 40), stored hashed, and it reads and writes through InfluxQL (`/query`) and line protocol
// (`/write`), which every bucket answers under its own name as a database (InfluxDB maps each
// bucket to a database of the same name by itself). That is what grafana's InfluxDB data source
// speaks, and what Node-RED's influxdb nodes speak in their 1.x mode — so the mesh can make every
// consumer's credential, rotate it and withdraw it, with no person in the loop.
//
// **What a consumer contributes.** `access`: "read" (the default), "write" or "read-write".
// `buckets`: the buckets it may use, by name. A reader that names none may read every bucket of the
// org — a dashboard is pointed at data, it does not own it. A writer must name its buckets: writing
// everywhere, the org's system buckets included, is never what a consumer means. A named bucket
// that does not exist is created, keeping its data for ever; the mesh never deletes a bucket.
//
// **Only what the mesh made is touched.** An authorization this module creates is named with the
// mesh's identity prefix and its description starts with MARK. One with the same username that
// lacks the mark is somebody else's: it is refused, never adopted, never updated, never deleted.
// Every other authorization, token, user and bucket in the instance is left exactly as it was.
import type { InfluxDBClient, InfluxPermission, LegacyAuthorization } from "./client.js";
/** How a description marks an authorization as the mesh's own work. */
export const MARK = "[mesh]";
/** The prefix the mesh gives every consumer identity (novox/hq ADR 0049). */
const IDENTITY_PREFIX = "mesh_";
/** One consumer, as the harness hands it over. */
export interface ApiGrant {
readonly as: string;
readonly password: string;
readonly values: Readonly<Record<string, unknown>>;
readonly consumer?: string;
}
export type Access = "read" | "write" | "read-write";
/** What a contribution asks for, checked. Refused when it cannot be served as asked. */
export function askedFor(values: Readonly<Record<string, unknown>>): { access: Access; buckets: string[] } {
const access = values.access ?? "read";
if (access !== "read" && access !== "write" && access !== "read-write") {
throw new Error(`contributes an access of ${JSON.stringify(access)} — it is "read", "write" or "read-write"`);
}
const raw = values.buckets ?? [];
if (!Array.isArray(raw) || raw.some((b) => typeof b !== "string" || b.trim() === "")) {
throw new Error(`contributes buckets of ${JSON.stringify(raw)} — a list of bucket names`);
}
const buckets = [...new Set((raw as string[]).map((b) => b.trim()))].sort();
if (access !== "read" && buckets.length === 0) {
throw new Error(`asks to write and names no bucket (\`buckets\`) — a writer names what it writes to`);
}
if (buckets.some((b) => b.startsWith("_"))) {
throw new Error(`names a system bucket (${buckets.filter((b) => b.startsWith("_")).join(", ")}) — those are InfluxDB's own`);
}
return { access: access as Access, buckets };
}
/** The permissions a grant resolves to, given each named bucket's id. */
export function permissionsFor(orgID: string, access: Access, bucketIDs: string[]): InfluxPermission[] {
const actions: ("read" | "write")[] = access === "read-write" ? ["read", "write"] : [access];
const out: InfluxPermission[] = [];
for (const action of actions) {
if (bucketIDs.length === 0) {
out.push({ action, resource: { type: "buckets", orgID } });
continue;
}
for (const id of bucketIDs) out.push({ action, resource: { type: "buckets", orgID, id } });
}
return out;
}
/** A permission as a comparable string: what InfluxDB answers carries names and links besides. */
function key(p: InfluxPermission): string {
return `${p.action}:${p.resource.type}:${p.resource.orgID ?? ""}:${p.resource.id ?? "*"}`;
}
function samePermissions(a: readonly InfluxPermission[], b: readonly InfluxPermission[]): boolean {
const x = a.map(key).sort();
const y = b.map(key).sort();
return x.length === y.length && x.every((v, i) => v === y[i]);
}
export function marked(a: Pick<LegacyAuthorization, "token" | "description">): boolean {
return a.token.startsWith(IDENTITY_PREFIX) && (a.description ?? "").startsWith(MARK);
}
function describe(g: ApiGrant): string {
return `${MARK} made by the mesh for ${g.consumer ? `a module on ${g.consumer}` : "a consumer"} — do not edit; it is reset`;
}
export class ApiGrants {
constructor(private readonly influx: InfluxDBClient, readonly org: string) {}
private async orgID(): Promise<string> {
const id = await this.influx.orgID(this.org);
if (!id) throw new Error(`InfluxDB has no org ${JSON.stringify(this.org)} — the org this module serves must exist`);
return id;
}
/** The ids of the named buckets, creating any that are missing when `create` says so. Undefined
* when one is missing and may not be created (a read-only question). */
private async bucketIDs(orgID: string, names: string[], create: ApiGrant | undefined): Promise<string[] | undefined> {
const ids: string[] = [];
for (const name of names) {
let b = await this.influx.findBucket(orgID, name);
if (!b) {
if (!create) return undefined;
b = await this.influx.createBucket(orgID, name, `${MARK} made by the mesh for ${create.as}; the mesh never deletes it`);
}
ids.push(b.id);
}
return ids.sort();
}
/** Create the consumer's authorization, or bring the mesh's existing one back to what the grant
* says. Idempotent: a second apply of the same grant changes nothing beyond re-asserting the
* password, which InfluxDB can be told but never asked. */
async ensure(g: ApiGrant): Promise<"created" | "updated" | "unchanged"> {
if (!g.as.startsWith(IDENTITY_PREFIX)) {
throw new Error(`${g.as} is not a mesh identity — the mesh names every consumer ${IDENTITY_PREFIX}<node>_<module>`);
}
const { access, buckets } = askedFor(g.values);
const orgID = await this.orgID();
const found = await this.influx.findLegacy(g.as);
if (found && !marked(found)) {
throw new Error(
`InfluxDB already has a v1 authorization ${g.as} the mesh did not make — left alone; ` +
`delete it if the mesh should own that name`);
}
const want = permissionsFor(orgID, access, (await this.bucketIDs(orgID, buckets, g))!);
if (found && found.orgID === orgID && samePermissions(found.permissions, want)) {
// Only what differs is written. The password cannot be read back, so it is tried instead.
let changed = false;
if (found.status === "inactive") {
await this.influx.updateLegacy(found.id, { status: "active" });
changed = true;
}
if (!(await this.influx.legacySignsIn(g.as, g.password))) {
await this.influx.setLegacyPassword(found.id, g.password);
changed = true;
}
return changed ? "updated" : "unchanged";
}
// InfluxDB cannot change an authorization's permissions in place, so the mesh's own is made
// again. Only ever one the mesh made: a foreign one was refused above.
if (found) await this.influx.deleteLegacy(found.id);
const made = await this.influx.createLegacy({
token: g.as, orgID, status: "active", description: describe(g), permissions: want,
});
await this.influx.setLegacyPassword(made.id, g.password);
return found ? "updated" : "created";
}
/** Whether InfluxDB still holds this consumer's authorization exactly as the grant says: present,
* the mesh's, active, allowed what was asked and nothing more, and signing in with the mesh's
* password. Reads only — a missing bucket is "not held", never created here. */
async holds(g: ApiGrant): Promise<boolean> {
const { access, buckets } = askedFor(g.values);
const orgID = await this.influx.orgID(this.org);
if (!orgID) return false;
const found = await this.influx.findLegacy(g.as);
if (!found || !marked(found) || found.status === "inactive" || found.orgID !== orgID) return false;
const ids = await this.bucketIDs(orgID, buckets, undefined);
if (!ids || !samePermissions(found.permissions, permissionsFor(orgID, access, ids))) return false;
return this.influx.legacySignsIn(g.as, g.password);
}
/** Withdraw a consumer's authorization — only one the mesh made. Its buckets and their data stay. */
async remove(as: string): Promise<"removed" | "absent" | "not ours"> {
const found = await this.influx.findLegacy(as);
if (!found) return "absent";
if (!marked(found)) return "not ours";
await this.influx.deleteLegacy(found.id);
return "removed";
}
}
+66 -72
View File
@@ -1,18 +1,11 @@
{ {
"module": "influxdb", "module": "influxdb",
"version": "1", "version": "1",
"provides": [
{
"name": "influxdb-api",
"scope": "mesh"
}
],
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"own-secrets": { "own-secrets": {
"admin": "${dir:state}/admin.secret", "broker": "/var/lib/mesh/influxdb/broker"
"admin-token": "${dir:state}/admin-token.secret"
}, },
"listens": [ "listens": [
{ {
@@ -20,117 +13,118 @@
"port": 8086, "port": 8086,
"protocol": "tcp", "protocol": "tcp",
"from": "mesh", "from": "mesh",
"why": "queries, writes and the web UI, over http; consumers granted influxdb-api sign in with the mesh's credential, and a name is a route grant" "why": "queries and writes, over http"
} }
], ],
"serves": {
"influxdb-api": {
"scheme": "http",
"port": 8086,
"org": "mesh",
"bucket": "default"
}
},
"receives": {
"influxdb-api": "${dir:grants}/mesh.json"
},
"grants": {
"influxdb-api": "${dir:grants}"
},
"resources": [ "resources": [
{ {
"id": "mesh-state", "id": "mesh-state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/mesh/influxdb",
"place": "mesh" "mode": "0700"
}, },
{ {
"id": "state", "id": "state",
"type": "directory", "type": "directory",
"mode": "0700", "path": "/var/lib/influxdb-module",
"place": "." "mode": "0700"
},
{
"id": "server-env",
"type": "file",
"path": "/var/lib/influxdb-module/server.env",
"mode": "0600",
"content": "DOCKER_INFLUXDB_INIT_MODE=setup\nDOCKER_INFLUXDB_INIT_USERNAME=admin\nDOCKER_INFLUXDB_INIT_PASSWORD=${secret:admin}\nDOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${secret:admin-token}\nDOCKER_INFLUXDB_INIT_ORG=mesh\nDOCKER_INFLUXDB_INIT_BUCKET=default\n"
}, },
{ {
"id": "data", "id": "data",
"type": "directory", "type": "directory",
"path": "/services/influxdb/data",
"mode": "0700", "mode": "0700",
"owner": "1000:1000" "owner": "1000:1000"
}, },
{ {
"id": "config", "id": "config",
"type": "directory", "type": "directory",
"path": "/services/influxdb/config",
"mode": "0700", "mode": "0700",
"owner": "1000:1000" "owner": "1000:1000"
}, },
{
"id": "grants",
"type": "directory",
"mode": "0700"
},
{
"id": "server-env",
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "DOCKER_INFLUXDB_INIT_MODE=setup\nDOCKER_INFLUXDB_INIT_USERNAME=admin\nDOCKER_INFLUXDB_INIT_PASSWORD_FILE=/run/secrets/admin\nDOCKER_INFLUXDB_INIT_ADMIN_TOKEN_FILE=/run/secrets/admin-token\nDOCKER_INFLUXDB_INIT_ORG=mesh\nDOCKER_INFLUXDB_INIT_BUCKET=default\n"
},
{ {
"id": "server", "id": "server",
"type": "container", "type": "container",
"name": "influxdb", "name": "influxdb",
"image": "influxdb@sha256:f75e48af0598e8aec7986e991a848d19a119101a7d563a2e5db1dfaac9c45daa", "image": "influxdb@sha256:f75e48af0598e8aec7986e991a848d19a119101a7d563a2e5db1dfaac9c45daa",
"env-file": [ "env-file": [
"${dir:state}/server.env" "/var/lib/influxdb-module/server.env"
], ],
"ports": [ "ports": [
"8086" "8086"
], ],
"volumes": [ "volumes": [
"${dir:data}:/var/lib/influxdb2", "/services/influxdb/data:/var/lib/influxdb2",
"${dir:config}:/etc/influxdb2", "/services/influxdb/config:/etc/influxdb2"
"${dir:state}/admin.secret:/run/secrets/admin:ro", ],
"${dir:state}/admin-token.secret:/run/secrets/admin-token:ro" "secrets-in-environment": "the image honours DOCKER_INFLUXDB_INIT_PASSWORD_FILE and _ADMIN_TOKEN_FILE; convertible, awaiting a bed that proves it"
]
}, },
{ {
"id": "runtime-config", "id": "runtime-config",
"type": "file", "type": "file",
"path": "${dir:mesh-state}/config.json", "path": "/var/lib/mesh/influxdb/config.json",
"mode": "0600", "mode": "0600",
"content": "{}\n", "content": "{}\n",
"merge": "json" "merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-influxdb",
"network": "host",
"volumes": [
"/var/lib/mesh/influxdb/broker:/run/secrets/broker:ro",
"/var/lib/mesh/influxdb/config.json:/run/config/config.json:ro",
"/services/influxdb/config:/var/lib/influxdb/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_INFLUXDB_URL": "http://127.0.0.1:8086",
"MESH_INFLUXDB_CONFIG_FILE": "/run/config/config.json",
"MESH_INFLUXDB_CONFIG_DIR": "/var/lib/influxdb/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
} }
], ],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "influxdb",
"endpoint": "api"
}
},
"build": { "build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [ "artifacts": [
{ {
"name": "code", "name": "runtime",
"kind": "bundle", "kind": "image",
"language": "typescript", "from": "Dockerfile"
"entrypoints": [
"tools/index.js",
"provisioner/index.js"
],
"loads": [
"tools/index.js",
"provisioner/index.js"
],
"env": {
"MESH_INFLUXDB_URL": "http://127.0.0.1:${port:8086}",
"MESH_INFLUXDB_CONFIG_FILE": "${dir:mesh-state}/config.json",
"MESH_INFLUXDB_TOKEN_FILE": "${dir:state}/admin-token.secret",
"MESH_RECEIVES": "${dir:grants}/mesh.json"
}
} }
] ]
},
"requires": [
"secret"
],
"secrets": {
"secret": {
"admin": "/var/lib/influxdb-module/admin.secret",
"admin-token": "/var/lib/influxdb-module/admin-token.secret"
}
} }
} }
+1 -6
View File
@@ -1,14 +1,9 @@
{ {
"name": "@novox/module-influxdb", "name": "@novox/module-influxdb",
"version": "0.1.0", "version": "0.1.0",
"description": "influxdb — time-series database; provides the mesh influxdb-api interface. Its API client, provisioner and tools live here (novox/hq ADR 0039).", "description": "influxdb — time-series database. Its API client and tools live here (novox/hq ADR 0039).",
"type": "module", "type": "module",
"private": true, "private": true,
"scripts": {
"build": "tsc client.ts grants.ts provisioner/index.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"typecheck": "tsc -p tsconfig.json",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": { "dependencies": {
"@novox/mesh-sdk": "^0.1.0" "@novox/mesh-sdk": "^0.1.0"
}, },
-54
View File
@@ -1,54 +0,0 @@
// influxdb's provisioner — the adapter that makes influxdb a provider of the mesh `influxdb-api`
// interface. The reconcile loop, the contributions file and reading the mesh's minted secret are the
// sdk harness's; this writes only the per-service half: how InfluxDB creates, checks and removes a
// consumer's credential (novox/hq ADR 0039/0040/0048). What that credential is, and why it is a v1
// authorization, is in ../grants.ts.
//
// The `influxdb-api` interface: a consumer reaches `${bound:influxdb-api:scheme}://…:at:…:port`,
// signs in as `${bound:influxdb-api:as}` with the password the mesh minted for the pair, and reads
// or writes the org's buckets as databases of the same name — `${bound:influxdb-api:bucket}` being
// the one this instance serves by default. The org and the default bucket are the assignment's
// settings, which reach both what is served and this module's config.json, so the org a consumer is
// told and the org its credential is made in cannot disagree.
import { runProvisioner, type Provision } from "@novox/mesh-sdk/provisioner";
import { InfluxDBClient } from "../client.js";
import { ApiGrants } from "../grants.js";
let grants: ApiGrants | undefined;
try {
const influx = InfluxDBClient.fromEnv();
grants = new ApiGrants(influx, influx.org);
} catch (err) {
// No admin token: nothing can be provisioned, and the tools loaded beside this must still serve.
console.error(`[provisioner:influxdb-api] not started: ${err instanceof Error ? err.message : err}`);
}
if (grants) serve(grants);
function serve(grants: ApiGrants): void {
runProvisioner("influxdb-api", {
async create(p: Provision): Promise<void> {
const done = await grants.ensure(p);
if (done !== "unchanged") {
console.log(`[provisioner:influxdb-api] ${done} v1 authorization ${p.as} in org ${grants.org}`);
}
},
async remove(p: { as: string }): Promise<void> {
const done = await grants.remove(p.as);
if (done === "not ours") {
console.error(`[provisioner:influxdb-api] ${p.as}: an authorization of that name exists that the mesh did not make — left alone`);
} else if (done === "removed") {
console.log(`[provisioner:influxdb-api] removed v1 authorization ${p.as}; its buckets and their data stay`);
}
},
// Asked every minute by the harness: whether InfluxDB still holds this consumer's authorization
// exactly as the mesh gave it, so one deleted, disabled or re-passworded behind the mesh's back is
// made whole again (hq issue 120).
async holds(p: Provision): Promise<boolean> {
return grants.holds(p);
},
});
}

Some files were not shown because too many files have changed in this diff Show More