Compare commits

...
Author SHA1 Message Date
jschoubben ff697779ab The mesh's one resolver, what every node asks, and a node's hosts file (hq ADR 0194, 0196, 0199)
- dnsmasq holds mesh-dns-resolver: provides wildcard-resolution mesh-wide, forwards every declared
  zone (zones fact), listens on the private address and loopback only, reads no hosts file and no
  operator's files, and no longer writes the container runtime's dns.
- resolv-conf names the mesh's resolver by address, then 1.1.1.1, timeout 1, one attempt; it now
  holds the runtime's live-restore, which dnsmasq held and every node needs.
- resolved-split-dns routes the suffix to the mesh's resolver by address, not 127.0.0.1.
- hosts: new module holding node-hosts-file — the machine's own lines in its block of /etc/hosts,
  the operator's lines kept, changed by entries/add/remove through sudo -n.
2026-10-04 00:38:57 +02:00
mesh-admin efff54157b Merge pull request 'The seven say what the runtime loads from their tools bundle (hq ADR 0192)' (#243) from fix/0192-the-seven-say-what-the-runtime-loads into main 2026-10-03 13:58:27 +00:00
jochen 2000ec3f48 The seven say what the runtime loads from their tools bundle (hq ADR 0192)
None declares a tools list, so the composer had nothing saying the runtime loads from the bundle,
and delivered it nowhere: built, recorded, never sent. loads names tools/index.js.
2026-10-03 15:58:19 +02:00
mesh-admin 7c800705bf Merge pull request 'Seven tool containers move to bundles the node's runtime serves, given their words (hq ADR 0192)' (#242) from feat/0192-seven-tool-containers-move into main 2026-10-03 13:45:46 +00:00
jochen 19511012f8 Seven tool containers move to bundles the node's runtime serves, given their words (hq ADR 0192)
baserow, confluence, gitlab, jira, letta, searxng and unifi: each runtime container's
environment becomes its tools bundle's env, mount targets folded back into the host paths they
came from; baserow and letta reach their service on the published port rather than a container
network name. The container, base images, Dockerfile and the module's own bus credential go,
and the state directory where only that credential lived. Tool code is unchanged: every client
is built from the environment the contributor is handed.
2026-10-03 15:35:00 +02:00
jschoubben 0e8ad3a8e1 Merge pull request 'dnsmasq: restart on the mesh hosts region, which it reads only at start' (#241) from fix/dnsmasq-rereads-the-names-region into main 2026-10-03 13:29:29 +00:00
jschoubben b43e405947 dnsmasq: restart on the mesh's hosts region, which it reads only at start
The names region of /etc/hosts is written by mesh-wireguard and dnsmasq answers from it, but read
it once at start: after the mesh stopped publishing public names (hq ADR 0191) every machine's
hosts file was right and every resolver still answered mail.novox.be with a tunnel address. The
config comment that says so changes the file, which restarts dnsmasq once everywhere.
2026-10-03 15:29:28 +02:00
mesh-admin d5269c8662 Merge pull request 'fail2ban's tools are a bundle the node's runtime serves; its container goes (hq to-be 38 WP4)' (#240) from feat/fail2ban-tools-as-a-bundle into main 2026-10-03 13:07:35 +00:00
jochen aa5bf7d5ef fail2ban's tools are a bundle the node's runtime serves; its container goes (hq to-be 38 WP4)
The second holder follows the packet filter: the container, its base images, the Dockerfile,
and the bus credential and state directory only the container read are gone; the tools are a
TypeScript bundle node-tools loads. The daemon's socket answers only to root, so the client
runs through sudo without a prompt where the runtime's account is not root, naming sudo's
absence or refusal by how it failed; client and daemon are the one package the module declares.
2026-10-03 15:07:21 +02:00
mesh-admin 510de183b1 Merge pull request 'The packet filter's tools are a bundle the node's runtime serves; its container goes (hq to-be 38 WP4)' (#239) from feat/wp4-the-packet-filter-moves into main 2026-10-03 11:14:33 +00:00
jochen db5e7c80cf The packet filter's tools are a bundle the node's runtime serves; its container goes (hq to-be 38 WP4)
nftables drops its container, NET_ADMIN, the container-runtime capability, the runtime base
images, the Dockerfile, and the bus credential and state directory only the container read;
its tools are declared as a TypeScript bundle the toolchain compiles and node-tools loads on
every node, and the iptables package the image used to carry is declared on the host. The
runtime runs as the operator's account, so the tool runs the filter's commands through sudo
without a prompt when it is not root (ADR 0175 §4, to-be 38 WP4), naming sudo's absence or
refusal by how it failed; the filter file is the path the manifest's filtering names, held to
it by a test; a found firewall that is present but will not answer stops a removal rather
than passing for inactive; a legacy tool that is present but fails is said, not swallowed.
2026-10-03 12:59:07 +02:00
jschoubben 38be3ba7f0 Merge pull request 'ssh-client: own ~/.ssh, not the openssh package' (#238) from fix/ssh-client-owns-no-package into main 2026-10-03 10:43:46 +00:00
jschoubben a19638d112 ssh-client: own ~/.ssh, not the openssh package
A package has one owning module, and sshd already owns openssh on every node — assigning
ssh-client was refused for both declaring it, and the refusal left every node unresolvable
until it was unassigned. Owning it was wrong besides: unassigning ssh-client would have
removed the package sshd serves from. The client binary ships in the package sshd holds.
2026-10-03 12:43:35 +02:00
jschoubben 1cbddeb104 Merge pull request 'ssh-client module: the mesh owns ~/.ssh, config from the hub (to-be 29)' (#237) from feat/ssh-client-module into main 2026-10-03 10:40:13 +00:00
jschoubben 301aeda5f3 ssh-client module: the mesh owns ~/.ssh, config from the hub (to-be 29)
Requires openssh; creates ~/.ssh (0700, owned by the operator account via
${machine:account}); writes every other node's Host block (HostName + User
<account>) into a marked region of ~/.ssh/config (home-scoped, into:block), so
`ssh <node>` reaches each peer as the right account and the operator's own
config is kept. Universal-tier: assigned wherever a person logs in; a node with
no account gets no config.
2026-10-03 11:58:05 +02:00
mesh-admin 853ace3828 Merge pull request 'Retire builder: the build machine is the build-agent on every machine (hq ADR 0190)' (#236) from feat/retire-builder into main 2026-10-03 09:44:09 +00:00
jochen 8ccc6762d4 Retire builder: the build machine is the build-agent on every machine (hq ADR 0190)
build-agent holds node-build-agent on all four machines and the controller asks that seat; the one-holder
builder is unassigned and forgotten. Proven live before this: a catalogue module built on a workstation's
agent (step-ca, 2026-10-03 09:35).
2026-10-03 11:43:01 +02:00
mesh-admin 6d01007ea6 Merge pull request 'build-agent: a short slug, so its identifier on a machine fits a backend's 20-character key' (#235) from fix/build-agent-slug into main 2026-10-03 09:13:57 +00:00
jochen 02463ba55d build-agent: a short slug, so its identifier on a machine fits a backend's 20-character key
"mesh_novox_build_agent" is 22 characters; the mesh refused to send the control node its declaration
for it. "agent" keeps the identifier within what an S3 access key allows on every machine.
2026-10-03 11:09:24 +02:00
jschoubben 85af10ebad Merge pull request 'step-ca: stop offering acme-ca, so public names are certified by public-acme' (#234) from fix/step-ca-is-not-the-public-acme-ca into main 2026-10-03 08:57:25 +00:00
jschoubben 0778f8f0ae step-ca: stop offering acme-ca, so public names are certified by public-acme
step-ca and public-acme both offered acme-ca on novox, and route-proxy's pin names a node, not
a module — so which one certified the public names depended on provider order. After the
controller restart on 2026-10-03 it came out as step-ca, and every public site served a
certificate no browser trusts. step-ca's own names are already certified through
internal-acme-ca; acme-ca is the public authority's alone.
2026-10-03 10:54:43 +02:00
mesh-admin 6afc1160b6 Merge pull request 'build-agent: the build machine as a node role every machine can hold (hq ADR 0190)' (#231) from feat/build-agent into main 2026-10-03 01:52:04 +00:00
mesh-admin c32edcfc6f Merge pull request 'Retire mesh-console: the console is the node-tools runtime's serving mode (hq ADR 0175 §6, to-be 38 WP3)' (#230) from feat/retire-mesh-console into main 2026-10-03 00:40:37 +00:00
mesh-admin 740359ffd9 Merge pull request 'Remove portainer: deprecated, and unassigned everywhere' (#233) from jschoubben/remove-portainer into main 2026-10-02 21:26:03 +00:00
jschoubben a309deb479 Remove portainer: deprecated, and unassigned everywhere
It held the docker socket behind a public name. Nothing depends on it;
it is off both machines that ran it, with its data.
2026-10-02 23:25:45 +02:00
mesh-admin 0fd722e829 Merge pull request 'gitea: the jail also bans what gitea's sshd refuses' (#232) from jschoubben/gitea-ssh-jail into main 2026-10-02 21:23:44 +00:00
jschoubben 2e6cc71f7a gitea: the jail also bans what gitea's sshd refuses
The jail read gitea's container journal, which carries its sshd's lines,
but matched only the web login. 167 ssh attempts an hour from the
internet went unbanned. Two patterns, one per attempt: an unknown user,
and a user sshd refuses; tested against a day of the real log, 946
matches and none on an accepted login.
2026-10-02 23:23:35 +02:00
jochen 84403aec0c build-agent: the build machine as a node role every machine can hold (hq ADR 0190)
The builder's manifest with one change that matters: it claims node-build-agent, a node seat, so it
is assignable to every machine with a container runtime, and every holder pulls one build at a time
from the role's one work queue. A tier of many images is then built by as many machines as hold the
seat and are online. The builder module stays until this is assigned where it was; then it goes.
2026-10-02 22:28:48 +02:00
jochen b28b1b9f24 Retire mesh-console: the console is the node-tools runtime's serving mode (hq ADR 0175 §6, to-be 38 WP3)
The console was a container per node built on the runtime image, calling everything and serving
nothing. node-tools — the runtime as a module, in the mesh-tools repository — answers MCP on the
same loopback port from the same process that serves every module's tools, so the module that was
only that is gone. Merged once node-tools is assigned where mesh-console was, on every machine.
2026-10-02 21:58:21 +02:00
mesh-admin 810c7fbac3 Merge pull request 'A ban list never holds a neighbour (hq ADR 0186)' (#227) from fix/a-ban-list-never-holds-a-neighbour into main 2026-10-02 16:43:26 +00:00
jschoubben 304044da40 A ban list never holds a neighbour (hq ADR 0186)
The home server banned the house's own router within an hour of the first public jail: the router
reflects local traffic, so every client in the building arrives as the gateway's address. Every
private range joins the mesh's own in the never-ban list.
2026-10-02 18:42:16 +02:00
mesh-admin 419d92e810 Merge pull request 'The proxy's jail reads a refused name as well as a refused certificate (hq ADR 0179)' (#226) from fix/the-proxys-jail-reads-both-refusals into main 2026-10-02 15:23:48 +00:00
jschoubben 23112b111c The proxy's jail reads both refusals, each pattern naming the host once
fail2ban expands <HOST> to a named group, so two in one pattern is a duplicate group name and
the daemon refuses to start at all -- every jail on the machine, not just this one. Two patterns,
one <HOST> each: the certificate refused for an unserved name, and the request refused for one.
Caught live on the control node (hq ADR 0179).
2026-10-02 17:23:42 +02:00
jschoubben b547308e05 The proxy's jail reads a refused name as well as a refused certificate
The pattern ended at the line's end, which only the certificate refusal does; a request for
an unserved name carries trailing text and never matched. Caught against the live lines
before the jail counted anything (hq ADR 0179).
2026-10-02 17:20:54 +02:00
mesh-admin 3c3c5c6e03 Merge pull request 'fail2ban holds the intrusion seat's verbs and composes the jails; mail, forge and proxy declare theirs (hq ADR 0179, to-be 31)' (#225) from feat/the-intrusion-seat-serves-its-verbs into main 2026-10-02 15:19:20 +00:00
jschoubben 1601d5a335 fail2ban holds the intrusion seat's verbs and composes the jails; mail, forge and proxy declare theirs (hq ADR 0179, to-be 31)
The module gains a runtime carrying only the fail2ban client with the daemon's socket shared in,
serving status/banned/ban/unban and its own fail2ban_settings. It declares jailing, so the
controller's composition lands in jail.d/mesh.conf and filter.d; mailu, route-proxy and gitea log to
the journal and declare a jail reading it by container name. The base is strict: three in a day for
a day, twice banned in two weeks for four; the mesh's range stays never banned.
2026-10-02 17:02:49 +02:00
mesh-admin 96b3d60a4a Merge pull request 'nftables declares the ufw front end absent once its filter is loaded (hq ADR 0175)' (#223) from feat/the-found-front-end-is-uninstalled into main 2026-10-02 14:38:00 +00:00
jschoubben 3dfbad6f03 nftables declares the ufw front end absent once its filter is loaded (hq ADR 0175) 2026-10-02 16:27:34 +02:00
mesh-admin d5c5415756 Merge pull request 'lab: the image carries python3, file, iproute2 and sudo' (#222) from jschoubben/lab-image-tools into main 2026-10-02 13:24:00 +00:00
jschoubben 6dfd2401c9 lab: the image carries what the builds and the lab call: python3, file, iproute2, sudo 2026-10-02 15:23:52 +02:00
mesh-admin 31923f70e7 Merge pull request 'lab: a run resolves the @novox scope from the forge's package registry' (#221) from jschoubben/lab-npm-scope into main 2026-10-02 13:13:29 +00:00
jschoubben 8cd4f199f1 lab: a run resolves the @novox scope from the forge's package registry 2026-10-02 15:13:22 +02:00
mesh-admin 1b9b298827 Merge pull request 'lab: compile from the module's root, so the runtime finds its tools' (#220) from jschoubben/lab-tools-path into main 2026-10-02 13:08:14 +00:00
jschoubben bce7b3a551 lab: compile from the module's root, so the runtime finds its tools
Both sources sit in tools/, so tsc took tools/ as the root and wrote
dist/index.js, while the runtime loads dist/tools/index.js: the module
started and served nothing.
2026-10-02 15:08:01 +02:00
mesh-admin 17d3d3e63a Merge pull request 'lab: declares the virtualisation capability (hq ADR 0172)' (#218) from jschoubben/the-lab-is-a-module-2 into main 2026-10-02 12:53:43 +00:00
mesh-admin 5c961c446f Merge pull request 'Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered' (#219) from fix/adr-0170-cited into main 2026-10-02 12:53:10 +00:00
jschoubben b77582f6a6 Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered after a collision on hq main 2026-10-02 14:52:26 +02:00
jschoubben b3865d240f lab: declares the virtualisation capability, which grants its daemon's socket 2026-10-02 14:48:10 +02:00
mesh-admin a81b94d4ab Merge pull request 'lab: the lab as a module, running beds when the mesh asks (hq ADR 0172)' (#217) from jschoubben/the-lab-is-a-module into main 2026-10-02 12:17:14 +00:00
jschoubben 67d1a400e8 lab: the lab as a module, running beds when the mesh asks
Five tools on the machine the lab runs on: check, run beds against
branches on the forge, a run's status, its log, and stop. A run checks
out every repository the lab builds, side by side, and runs the suite;
one at a time, answered at once with an id (novox/hq ADR 0172).
2026-10-02 14:15:42 +02:00
mesh-admin 1c201d59c9 Merge pull request 'nftables holds the node-packet-filter seat: rules, reload and remove, from a runtime with NET_ADMIN (hq ADR 0169)' (#216) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 11:33:47 +00:00
jschoubben 663e8143d4 nftables holds the node-packet-filter seat: rules, reload and remove, from a runtime with NET_ADMIN (hq ADR 0169)
The seat's three verbs over the machine's own tools: the filter as enforced
(nftables and the legacy filter), the mesh's own table reloaded from its file,
and one rule set the mesh did not write removed by the name the host reports
it under (ADR 0168) — a predecessor's chain loses its jumps and goes, the
runtime's user chain is emptied back to its return, a table of the machine's
own goes whole; the mesh's tables, the runtime's chains, a built-in chain and
an active found firewall's chains are refused. Tested over the shapes two
machines of the first mesh reported live. The module's own tool stays.
2026-10-02 13:28:33 +02:00
mesh-admin 8ce4935132 Merge pull request 'unifi: list networks and set the DNS their DHCP hands out (hq issue 198)' (#215) from jschoubben/unifi-network-dns into main 2026-10-02 09:53:22 +00:00
jschoubben d1f8ab86d1 unifi: list networks and set the DNS their DHCP hands out
Which DNS server the home network's DHCP hands out could be changed only
in the controller's own interface or by hand against its API (novox/hq
issue 198).
2026-10-02 11:53:16 +02:00
mesh-admin 3020cd2312 Merge pull request 'dnsmasq: listen addresses are a setting, and docker's file takes none (hq issue 198)' (#214) from jschoubben/the-lans-dns-is-the-mesh-2 into main 2026-10-02 09:50:09 +00:00
jschoubben b72213261a dnsmasq: listen addresses are a setting, and docker's file takes none
The addresses dnsmasq listens on beside the machine's are a setting, so a
machine that answers its own LAN can say so (novox/hq issue 198). Docker's
daemon.json no longer merges the module's settings: it needs none, and a
setting reaching it is a key dockerd refuses. The host still merges it
into the existing file.
2026-10-02 11:49:57 +02:00
mesh-admin 7e5c98920e Merge pull request 'Revert dnsmasq's listen addresses as a setting (hq issue 198)' (#213) from jschoubben/revert-dnsmasq-listen into main 2026-10-02 09:48:49 +00:00
jschoubben 67f5236b01 Revert dnsmasq's listen addresses as a setting
A module's settings merge into every mergeable file it owns, so the
setting reached docker's daemon.json beside dnsmasq's config, where
dockerd would refuse it (novox/hq issue 198). Back to the fixed
loopback line until settings can be kept out of files they are not for.
2026-10-02 11:48:37 +02:00
mesh-admin e9876858a8 Merge pull request 'dnsmasq: the addresses it listens on beside the machine's are a setting (hq issue 198)' (#212) from jschoubben/the-lans-dns-is-the-mesh into main 2026-10-02 09:46:32 +00:00
jschoubben 56a22847f5 dnsmasq: the addresses it listens on beside the machine's are a setting
Loopback by default, as before. A machine that answers its own LAN adds
its LAN address, and its DNS endpoints' reach opens the filter (novox/hq
issue 198). The mesh-wide default must be set before this lands.
2026-10-02 11:42:41 +02:00
mesh-admin 1271f797e9 Merge pull request 'route-proxy: a bus account, to read its membership (hq ADR 0167, issue 191)' (#211) from jschoubben/an-internal-only-route into main 2026-10-01 23:49:50 +00:00
jschoubben 4f952ce771 route-proxy: a bus account, to read its membership
The proxy reads its routes and the mesh's addresses from its membership
on the bus rather than from a file alone (novox/hq ADR 0167, issue 191).
2026-10-02 01:46:18 +02:00
mesh-admin fec6d76fb3 Merge pull request 'mssql: the query runs as a read-only login, one line, no variables; sqlcmd is installed (hq #193)' (#210) from fix/193-mssql-reads-as-a-reader into main 2026-10-01 22:30:02 +00:00
mesh-admin da7355dce0 Merge pull request 'postgres: the store's query runs as a read-only login, never as the admin (hq #193)' (#209) from fix/193-the-store-reads-as-a-reader into main 2026-10-01 22:29:57 +00:00
jschoubben 400b2f9696 mssql: the query runs as a read-only login, one line, no variables; sqlcmd is installed (hq #193)
Proven on a throwaway server: as the administrator a caller's $(SQLCMDPASSWORD) returned
the sa password, and a line beginning ':!!' ran a program in the tools container. The
statement now runs as mesh_mssql_reader (CONNECT ANY DATABASE, SELECT ALL USER SECURABLES),
with substitution off (-x), after the module's own text on the first line, and a line break
is refused. go-sqlcmd v1.10.0 is installed at a pinned digest: the image never had sqlcmd,
so every mssql tool failed with spawn sqlcmd ENOENT.
2026-10-02 00:25:22 +02:00
jschoubben 160b5ad65a postgres: the store's query runs as a read-only login, never as the admin (hq #193)
The verb wrapped the caller's text in BEGIN READ ONLY ... ROLLBACK as the superuser, so
'COMMIT; ...' left the transaction and, proven on a throwaway server, COPY TO PROGRAM ran a
shell command on the database host. The statement now runs as mesh_store_reader:
pg_read_all_data, no other grant, read-only transactions by role and session, its password
an own-secret the mesh mints. Without that password the call is refused. -q drops the
command tags that came back as rows keyed by BEGIN.
2026-10-02 00:09:09 +02:00
mesh-admin ef44c502db Merge pull request 'route-proxy README: the node that runs a proxy carries public-acme (hq #258)' (#208) from docs/route-proxy-carries-public-acme into main 2026-10-01 15:32:05 +00:00
jschoubben 0651b63926 route-proxy README: the node that runs a proxy carries public-acme (hq #258) 2026-10-01 17:31:58 +02:00
mesh-admin 0c521ffa20 Merge pull request 'The vault's claim, its own event names, and the uplink holders' capabilities return' (#207) from fix/the-vaults-claim-and-events-return into main 2026-10-01 15:19:55 +00:00
jschoubben 01d68bda88 And the uplink holders' capabilities return
The same split lost them the other way round: the merge base held both changes, each branch had reset
the other's files, and the three-way merge kept neither. Both halves of hq ADR 0161 are on main again
with this.
2026-10-01 17:19:41 +02:00
jschoubben 89e0dde9e0 The vault's claim and its own event names return
The uplink branch was split from the vault's with the vault's files reset to a main that did not yet
hold #205; merging it afterwards took the older vault definition along (no claim, the refused event
names), and the vault could not be built. Restored to #205's state.
2026-10-01 17:19:05 +02:00
mesh-admin 5ebc89d89d Merge pull request 'Each uplink holder declares the manager it speaks for (hq ADR 0161)' (#206) from feat/each-uplink-holder-declares-the-manager-it-speaks-for into main 2026-10-01 15:11:31 +00:00
mesh-admin 32fa76fccb Merge pull request 'The vault claims mesh-vault, and each uplink holder declares the manager it speaks for (hq ADR 0161)' (#205) from feat/the-vault-claims-its-seat-and-the-uplinks-say-their-dialect into main 2026-10-01 14:53:15 +00:00
jschoubben 2e96d2f67d This branch carries the uplink capabilities alone (hq ADR 0161 rule 3); merges once every machine running a holder has reported uplink-<manager> 2026-10-01 16:52:47 +02:00
jschoubben 932efb5186 This branch carries the vault's claim alone; the uplink capabilities wait for every machine to report its profile 2026-10-01 16:52:46 +02:00
jschoubben 6ba61a8b4b The provisioner announces the vault's events by their new names 2026-10-01 16:51:13 +02:00
jschoubben 8368697744 The vault's events are its own: provisioned, rotated, deprovisioned
A module publishes under its own name only; secret.provisioned read as another module's event and the
builder refused the vault's definition today, so the seat claim could not be built. The three events lose
the prefix; nothing outside the vault listens for the old names.
2026-10-01 16:50:56 +02:00
jschoubben 966fed1829 The vault claims mesh-vault, and each uplink holder declares the manager it speaks for (hq ADR 0161)
The vault's claim makes a second provider of secret a second claimant, refused by name. The three
uplink definitions declare uplink-networkmanager, uplink-systemd-networkd and uplink-dhcpcd, which the
host reports for the manager it finds active, so the holder for a manager the machine does not run
is refused the way any missing capability is. Merges after the controller holds the seat and the host
reports the capability.
2026-10-01 15:58:05 +02:00
mesh-admin 67c834ac65 Merge pull request 'postgres serves the store seat's verbs, databases and query, and lists its tools (ADR 0159)' (#203) from feat/postgres-serves-the-stores-verbs into main 2026-10-01 13:53:05 +00:00
jschoubben c00dd04494 postgres implements the store seat's verbs under the seat's name, and its claim says so (hq ADR 0160)
The store's databases and query are registered under mesh-store, so the runtime serves them on the
seat's subjects wherever postgres holds the seat and never lists them as postgres's own; the claim
names them, so the mesh can judge the holder without postgres listing the seat's verbs among its
tools. Scoped to what the store enables: creating a database stays postgres's tool.
2026-10-01 15:19:35 +02:00
jschoubben abf5859415 Merge remote-tracking branch 'origin/main' into feat/postgres-serves-the-stores-verbs 2026-10-01 15:19:08 +02:00
mesh-admin fe8d6a25c0 Merge pull request 'The media chain's stale copies leave the catalogue' (#204) from chore/remove-stale-media-duplicates into main 2026-10-01 12:54:00 +00:00
jschoubben 738415710c The media chain's stale copies leave the catalogue
bazarr, bookshelf, lidarr, nzbget, ombi, plex, qbittorrent, radarr,
sonarr and tautulli live in novox/mesh-media-catalog (#195 moved jackett
and left these behind). A build of this repository at a commit today
registered plex and nzbget from these copies, which carry no provides,
and ace's plan stopped resolving. Nothing here depends on the directories:
home-assistant consumes their provisions by name.
2026-10-01 14:53:35 +02:00
jschoubben 4c7e438ea3 postgres serves the store seat's verbs, databases and query, and lists its tools (hq ADR 0159)
Named as the seat names them so the runtime finds them by name; the same calls as its own tools.
Its definition now lists its tools, which is what holding a seat with verbs demands at registration.
Merge before the controller declares the verbs on the mesh-store seat.
2026-10-01 14:02:41 +02:00
mesh-admin fd0fa75cc2 Merge pull request 'searxng says it reads its secret key at start, so the mesh may rotate it (hq 180)' (#202) from feat/searxng-says-how-its-secret-is-taken into main 2026-10-01 10:10:05 +00:00
jschoubben 50c08818d6 searxng says it reads its secret key at start, so the mesh may rotate it (hq 180)
The key signs sessions and nothing else holds it; it lands in the settings file the server
restarts on, so a rotation is a new value and a restart.
2026-10-01 12:09:47 +02:00
mesh-admin 1712670610 Merge pull request 'nodered says it reads its API token and admin password at start, so the mesh may rotate them (hq 180)' (#201) from feat/nodered-says-how-its-secrets-are-taken into main 2026-10-01 09:45:32 +00:00
jschoubben 6b0164c2ba nodered says it reads its API token and admin password at start, so the mesh may rotate them (hq 180)
Both land in settings.js and the runtime's config file, and the containers that read them restart
on those files; a rotation is a new value and a restart. The broker credential says nothing yet: its
other party is the bus, and that rotation is the two-party form.
2026-10-01 11:44:29 +02:00
mesh-admin 0966599c8a Merge pull request 'The forge's tools close and read pull requests, read files and branches, and delete a branch' (#200) from feat/the-forges-tools-close-and-read-pull-requests into main 2026-10-01 09:25:51 +00:00
jschoubben 171f8a03f6 The forge's tools close and read pull requests, read files and branches, and delete a branch
Ten tools the console lacked for the actions a review and a merge leave behind: close or reopen a
pull request whose work landed elsewhere, change its title or body, read its files, its diff and its
comments, reopen an issue, read one file at a ref, list branches, delete the branch a closed pull
request leaves. Each is the client's own call; `gitea_api` stays the escape hatch for the rest.
Tested against the fake forge through the compiled tools, the way the console calls them (13/13).
2026-10-01 11:25:34 +02:00
mesh-admin cb48c882a0 Merge pull request 'postgres: its server container is not named after the seat' (#177) from fix/postgres-is-not-named-after-the-seat into main 2026-10-01 09:25:05 +00:00
mesh-admin 3cbd98b14f Merge pull request 'n8n: its media library is an access placed by the assignment' (#199) from fix/n8n-media-access-by-id into main 2026-10-01 00:02:44 +00:00
jschoubben 9fc0d675cd n8n: its media library is an access placed by the assignment
The container mounted /services/media literally — one installation's path
(ADR 0112). The access is now declared by id and mounted as ${access:media};
the assignment says where the library is (ace: /storage/media, hq 153).
2026-10-01 02:02:30 +02:00
mesh-admin 1b0e3841e4 Merge pull request 'n8n: its own image built from source, placed data, and what its workflows use' (#172) from feat/n8n-for-ace into main 2026-09-30 23:59:29 +00:00
jschoubben 76443ec06d postgres: its server container is not named after the seat
The module's server container was called mesh-store and its data directory
/var/lib/mesh-store — the seat's name reused for the module's own resources,
a leftover from the first migration. On a machine whose postgres holds no
seat (ace, as a database provider only) that produced a container called
mesh-store holding nothing of the kind. The container is now named
postgres. The data directory keeps its path: a path change recreates a
running container on an empty directory (hq 126), and novox's store lives
there.

Rolling this out recreates novox's store container once (a restart on its
bind mount, no data moves). The two catalogue-test failures on this branch
(resolver_manifests_test) fail identically on main today.
2026-09-30 14:56:31 +02:00
149 changed files with 2699 additions and 5617 deletions
-24
View File
@@ -1,24 +0,0 @@
# baserow's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/baserow
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/baserow/dist /app/modules/baserow/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/baserow/dist/tools/index.js
+14 -36
View File
@@ -25,8 +25,7 @@
"postgres-database": "${dir:state}/database.secret"
},
"own-secrets": {
"admin": "${dir:state}/admin.secret",
"broker": "${dir:mesh-state}/broker"
"admin": "${dir:state}/admin.secret"
},
"listens": [
{
@@ -92,45 +91,24 @@
"mode": "0600",
"content": "{\n \"password\": \"${secret:admin}\",\n \"host\": \"${bound:route:name}\"\n}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-baserow",
"network": "baserow",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/config.json:/run/config/config.json:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BASEROW_URL": "http://baserow:80",
"MESH_BASEROW_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_BASEROW_URL": "http://127.0.0.1:${port:80}",
"MESH_BASEROW_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
}
]
}
-24
View File
@@ -1,24 +0,0 @@
# bazarr's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/bazarr
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/bazarr/dist /app/modules/bazarr/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/bazarr/dist/index.js,/app/modules/bazarr/dist/tools/index.js
-173
View File
@@ -1,173 +0,0 @@
// The Bazarr API client — bazarr's own code, living in the module (novox/hq ADR 0039). Bazarr
// manages subtitles for a Sonarr/Radarr library: it tracks which episodes and movies are still
// missing subtitles, searches providers for them, and records what it downloaded. This client
// talks its /api surface (keyed by an X-API-KEY header); bazarr's tools and events import it.
import { readFileSync } from "node:fs";
export interface WantedSubtitle {
kind: "episode" | "movie";
title: string; // series + episode, or movie title
path?: string;
seriesId?: number; // sonarr series id (episodes)
episodeId?: number; // sonarr episode id (episodes)
radarrId?: number; // radarr movie id (movies)
missing: string[]; // language names still missing
}
export interface ProviderSubtitle {
provider: string;
language: string;
hearingImpaired: boolean;
forced: boolean;
score?: number;
release?: string;
subtitle: string; // the opaque token Bazarr uses to download this exact result
}
export interface HistoryEntry {
kind: "episode" | "movie";
id: string; // stable dedup key across polls
title: string;
language?: string;
provider?: string;
path?: string;
timestamp?: string;
description?: string;
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> {
if (!file) return {};
try { return JSON.parse(readFileSync(file, "utf8")) as Record<string, string>; }
catch { return {}; }
}
/** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
* absent or unreadable yields undefined so callers fall back rather than crash. */
function readSecret(file?: string): string | undefined {
if (!file) return undefined;
try { return readFileSync(file, "utf8").trim(); }
catch { return undefined; }
}
export class BazarrClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/** Build from the module's resolved environment. Bazarr's API is keyed; without URL and key
* there is nothing to talk to, so this throws rather than run half-configured. */
static fromEnv(env: NodeJS.ProcessEnv = process.env): BazarrClient {
const cfg = meshConfig(env.MESH_BAZARR_CONFIG_FILE);
const url = cfg.url ?? env.MESH_BAZARR_URL;
const apiKey = cfg.apiKey ?? readSecret(env.MESH_BAZARR_API_KEY_FILE) ?? env.MESH_BAZARR_API_KEY;
if (!url) throw new Error("no Bazarr URL — set MESH_BAZARR_URL");
if (!apiKey) throw new Error("no Bazarr API key — set MESH_BAZARR_API_KEY");
return new BazarrClient(url, apiKey);
}
private async request(method: string, path: string, params: Record<string, string> = {}): Promise<any> {
const url = new URL(`${this.baseUrl}/api${path}`);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const res = await fetch(url.toString(), { method, headers: { "X-API-KEY": this.apiKey, Accept: "application/json" } });
if (!res.ok) throw new Error(`Bazarr API ${method} ${path}: ${res.status} ${await res.text()}`);
// Downloads/patches return an empty body; only GETs carry JSON.
const text = await res.text();
return text ? JSON.parse(text) : {};
}
private get(path: string, params?: Record<string, string>): Promise<any> {
return this.request("GET", path, params);
}
private languageNames(missing: any[]): string[] {
return (missing ?? []).map((m: any) => m?.name ?? m?.code2 ?? m?.code3).filter(Boolean);
}
/** Episodes and movies still missing subtitles — Bazarr's core "what's left to do" list. */
async getWanted(limit = 50): Promise<WantedSubtitle[]> {
const [eps, movies] = await Promise.all([
this.get("/episodes/wanted", { start: "0", length: String(limit) }),
this.get("/movies/wanted", { start: "0", length: String(limit) }),
]);
const episodes: WantedSubtitle[] = (eps?.data ?? []).map((e: any) => ({
kind: "episode" as const,
title: `${e.seriesTitle ?? e.series ?? "Unknown"} — ${e.episodeTitle ?? e.episode_title ?? ""}`.trim(),
path: e.path,
seriesId: e.sonarrSeriesId,
episodeId: e.sonarrEpisodeId,
missing: this.languageNames(e.missing_subtitles),
}));
const films: WantedSubtitle[] = (movies?.data ?? []).map((m: any) => ({
kind: "movie" as const,
title: m.title ?? "Unknown",
path: m.path,
radarrId: m.radarrId,
missing: this.languageNames(m.missing_subtitles),
}));
return [...episodes, ...films];
}
/** Ask providers what subtitles are available for one wanted episode — a manual search. */
async searchEpisode(episodeId: number): Promise<ProviderSubtitle[]> {
const raw = await this.get("/providers/episodes", { episodeid: String(episodeId) });
return this.mapProviderResults(raw);
}
/** Ask providers what subtitles are available for one movie — a manual search. */
async searchMovie(radarrId: number): Promise<ProviderSubtitle[]> {
const raw = await this.get("/providers/movies", { radarrid: String(radarrId) });
return this.mapProviderResults(raw);
}
private mapProviderResults(raw: any): ProviderSubtitle[] {
const list = Array.isArray(raw) ? raw : (raw?.data ?? []);
return list.map((r: any) => ({
provider: r.provider,
language: r.language?.name ?? r.language ?? "unknown",
hearingImpaired: Boolean(r.hearing_impaired ?? r.hi),
forced: Boolean(r.forced),
score: r.score,
release: r.release_info?.[0] ?? r.release_info,
subtitle: r.subtitle,
}));
}
/** Recent subtitle-download history, episodes and movies together, newest first. Each entry
* carries a stable id so the events poller can tell a fresh download from one already seen. */
async getHistory(limit = 40): Promise<HistoryEntry[]> {
const [eps, movies] = await Promise.all([
this.get("/episodes/history", { start: "0", length: String(limit) }),
this.get("/movies/history", { start: "0", length: String(limit) }),
]);
const key = (kind: string, r: any): string =>
`${kind}:${r.timestamp ?? r.parsed_timestamp ?? ""}:${r.subtitles_path ?? r.path ?? ""}:${r.language?.code3 ?? r.language ?? ""}`;
const episodes: HistoryEntry[] = (eps?.data ?? []).map((r: any) => ({
kind: "episode" as const,
id: key("episode", r),
title: `${r.seriesTitle ?? "Unknown"} — ${r.episodeTitle ?? ""}`.trim(),
language: r.language?.name ?? r.language,
provider: r.provider,
path: r.subtitles_path,
timestamp: r.timestamp,
description: r.description,
}));
const films: HistoryEntry[] = (movies?.data ?? []).map((r: any) => ({
kind: "movie" as const,
id: key("movie", r),
title: r.title ?? "Unknown",
language: r.language?.name ?? r.language,
provider: r.provider,
path: r.subtitles_path,
timestamp: r.timestamp,
description: r.description,
}));
return [...episodes, ...films];
}
}
-47
View File
@@ -1,47 +0,0 @@
// bazarr's events. The tool runtime imports this once the broker is bound. Bazarr's one genuinely
// observable thing is a subtitle arriving: it works away in the background, searching providers for
// the missing-subtitle list, and when it succeeds a subtitle appears in its history. That is worth
// announcing to the mesh.
//
// Emits (novox/hq ADR 0041/0042):
// module.bazarr.subtitle.downloaded — a subtitle was fetched for an episode or movie
//
// Bazarr has nothing on the mesh it usefully reacts to (a download completing is Sonarr/Radarr's
// business, and they trigger Bazarr directly), so it consumes nothing — a pure emitter.
//
// The event is observation-based: poll history and diff. Primed silently on the first look, or a
// restart would re-announce the whole recent history as freshly downloaded.
import { emit } from "@novox/mesh-sdk/events";
import { BazarrClient } from "./client.js";
const bazarr = BazarrClient.fromEnv();
const seen = new Set<string>();
let primed = false;
async function pollHistory(): Promise<void> {
const entries = await bazarr.getHistory(40);
for (const entry of entries) {
if (seen.has(entry.id)) continue;
if (primed) {
await emit("subtitle.downloaded", {
kind: entry.kind,
title: entry.title,
language: entry.language,
provider: entry.provider,
path: entry.path,
});
}
seen.add(entry.id);
}
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[bazarr] ${err}`));
setInterval(run, everyMs);
run();
};
tick(pollHistory, 60_000);
console.log("[bazarr] watching subtitle-download history");
-145
View File
@@ -1,145 +0,0 @@
{
"module": "bazarr",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"subtitle.downloaded"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker",
"api-key": "${dir:mesh-state}/api-key"
},
"listens": [
{
"name": "web",
"port": 6767,
"protocol": "tcp",
"from": "mesh",
"why": "managing subtitles"
}
],
"accesses": [
{
"id": "movies",
"path": "/services/media/movies",
"mode": "read-write"
},
{
"id": "series",
"path": "/services/media/series",
"mode": "read-write"
},
{
"id": "anime",
"path": "/services/media/anime",
"mode": "read-write"
},
{
"id": "downloads",
"path": "/services/media/downloads",
"mode": "read"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/bazarr/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "bazarr",
"image": "lscr.io/linuxserver/bazarr@sha256:3a820372f19fcb2981ea19fe4b5382934d67414afaba974bce831ddda0a64a02",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"6767"
],
"volumes": [
"/services/bazarr/config:/config",
"${access:movies}:/movies",
"${access:series}:/series",
"${access:anime}:/anime",
"${access:downloads}:/downloads"
]
},
{
"id": "runtime-config",
"type": "file",
"path": "${dir:mesh-state}/config.json",
"mode": "0600",
"content": "{}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-bazarr",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/api-key:/run/secrets/api-key:ro",
"${dir:mesh-state}/config.json:/run/config/config.json:ro",
"/services/bazarr/config:/var/lib/bazarr/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BAZARR_URL": "http://127.0.0.1:6767",
"MESH_BAZARR_API_KEY_FILE": "/run/secrets/api-key",
"MESH_BAZARR_CONFIG_FILE": "/run/config/config.json",
"MESH_BAZARR_CONFIG_DIR": "/var/lib/bazarr/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "subs",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:mesh-state}/route.json"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-bazarr",
"version": "0.1.0",
"description": "bazarr — subtitle management. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-57
View File
@@ -1,57 +0,0 @@
// bazarr's tools — its own code (novox/hq ADR 0039), importing bazarr's client. They return
// structured data; the mesh serves them through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { BazarrClient } from "../client.js";
export function getBazarrTools(bazarr: BazarrClient): ToolDefinition[] {
return [
{
name: "bazarr_wanted",
description: "Episodes and movies still missing subtitles, with the languages each still needs.",
input: { limit: { type: "number", description: "max items per kind (default 50)" } },
run: async (args) => {
const wanted = await bazarr.getWanted(args.limit ? Number(args.limit) : 50);
return { count: wanted.length, wanted };
},
},
{
name: "bazarr_search_subtitles",
description: "Manually search subtitle providers for one wanted item — pass an episodeId or a radarrId.",
input: {
episodeId: { type: "number", description: "a Sonarr episode id (from bazarr_wanted)" },
radarrId: { type: "number", description: "a Radarr movie id (from bazarr_wanted)" },
},
run: async (args) => {
if (args.episodeId !== undefined) {
const results = await bazarr.searchEpisode(Number(args.episodeId));
return { kind: "episode", episodeId: Number(args.episodeId), count: results.length, results };
}
if (args.radarrId !== undefined) {
const results = await bazarr.searchMovie(Number(args.radarrId));
return { kind: "movie", radarrId: Number(args.radarrId), count: results.length, results };
}
throw new Error("pass either episodeId or radarrId");
},
},
{
name: "bazarr_history",
description: "Recent subtitle-download history — what was downloaded, for which title, from which provider.",
input: { limit: { type: "number", description: "max entries per kind (default 40)" } },
run: async (args) => {
const history = await bazarr.getHistory(args.limit ? Number(args.limit) : 40);
return { count: history.length, history };
},
},
];
}
// Exposed only when Bazarr is configured; otherwise bazarr contributes no tools rather than
// failing the whole runtime.
registerModuleTools("bazarr", (env) => {
try {
return getBazarrTools(BazarrClient.fromEnv(env));
} catch {
return [];
}
});
-12
View File
@@ -1,12 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
-24
View File
@@ -1,24 +0,0 @@
# bookshelf's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/bookshelf
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/bookshelf/dist /app/modules/bookshelf/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/bookshelf/dist/index.js,/app/modules/bookshelf/dist/tools/index.js
-136
View File
@@ -1,136 +0,0 @@
// The Bookshelf API client — bookshelf's own code, living in the module (novox/hq ADR 0039).
// Ported from the shared hal `arr` client, but self-contained: in nox each Servarr app owns its own
// copy, so a change to Bookshelf's API rebuilds only bookshelf and nothing else. Both this module's
// tools and its events entrypoint import it, and nothing outside bookshelf does.
//
// Bookshelf is a Readarr fork (ghcr.io/pennydreadful/bookshelf). It speaks the Servarr v1 API; its
// content is "book". Unlike Sonarr/Radarr it exposes no calendar endpoint, so there is no calendar
// tool here — matching hal, which excluded bookshelf from its calendar-capable apps.
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
// Bookshelf speaks the v1 API; its content is "book".
const API_VERSION = "v1";
const CONTENT_ENDPOINT = "book";
const APP_NAME = "Bookshelf";
export interface BookshelfQueueItem {
/** The queue record id — stable while the item is in the queue, so events can diff on it. */
id: number;
title: string;
status: string;
size: string;
sizeleft: string;
timeleft?: string;
}
export interface BookshelfContentItem {
title: string;
author?: string;
year?: number;
status?: string;
monitored: boolean;
}
export class BookshelfClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/**
* Build from the module's resolved environment. The URL defaults to the server on this node (the
* runtime shares its network), and the API key is read from MESH_BOOKSHELF_API_KEY or, failing
* that, discovered from the server's own config.xml under MESH_BOOKSHELF_CONFIG_DIR — the same
* file Bookshelf writes it to, so a running server needs nothing configured by hand. Throws when
* no key can be found, so the tools/events simply do not load (the harness treats the throw as
* "exposes nothing").
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): BookshelfClient {
const url = env.MESH_BOOKSHELF_URL ?? `http://127.0.0.1:${env.MESH_BOOKSHELF_PORT ?? "8787"}`;
const configDir = env.MESH_BOOKSHELF_CONFIG_DIR ?? "/config";
const apiKey = env.MESH_BOOKSHELF_API_KEY ?? BookshelfClient.detectApiKey(configDir);
if (!apiKey) {
throw new Error("Bookshelf not configured — set MESH_BOOKSHELF_API_KEY or make the config dir readable");
}
return new BookshelfClient(url, apiKey);
}
/** Discover the API key from the server's config.xml, falling back to null. Every Servarr app
* writes <ApiKey> into config.xml at the root of its config directory. */
static detectApiKey(configDir: string): string | null {
const config = join(configDir, "config.xml");
if (existsSync(config)) {
const match = readFileSync(config, "utf8").match(/<ApiKey>([^<]+)<\/ApiKey>/);
if (match) return match[1];
}
return null;
}
private async get(endpoint: string, params?: Record<string, string>): Promise<unknown> {
const url = new URL(`${this.baseUrl}/api/${API_VERSION}/${endpoint}`);
if (params) {
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
}
const res = await fetch(url.toString(), { headers: { "X-Api-Key": this.apiKey } });
if (!res.ok) throw new Error(`${APP_NAME} API /${endpoint}: ${res.status} ${await res.text()}`);
return res.json();
}
async getStatus(): Promise<{ appName: string; version: string }> {
const data = (await this.get("system/status")) as { appName?: string; version?: string };
return { appName: data.appName || APP_NAME, version: data.version ?? "unknown" };
}
async getContent(limit?: number): Promise<BookshelfContentItem[]> {
const data = await this.get(CONTENT_ENDPOINT);
const items: any[] = Array.isArray(data) ? data : ((data as any)?.records ?? []);
const mapped = items.map((item) => ({
title: item.title ?? "Unknown",
author: item.author?.authorName ?? item.authorName,
year: item.releaseDate ? new Date(item.releaseDate).getFullYear() : item.year,
status: item.status,
monitored: item.monitored ?? true,
}));
return limit ? mapped.slice(0, limit) : mapped;
}
/** Library search is a filter over existing content, not an indexer lookup — same as hal's. */
async searchContent(term: string): Promise<BookshelfContentItem[]> {
const all = await this.getContent();
const lower = term.toLowerCase();
return all.filter(
(item) =>
item.title.toLowerCase().includes(lower) ||
(item.author?.toLowerCase().includes(lower) ?? false),
);
}
async getQueue(): Promise<{ totalRecords: number; items: BookshelfQueueItem[] }> {
const data = (await this.get("queue", { pageSize: "50" })) as { totalRecords?: number; records?: any[] };
const records = data.records ?? [];
return {
totalRecords: data.totalRecords ?? records.length,
items: records.map((r) => ({
id: r.id,
title: r.title ?? r.book?.title ?? r.author?.authorName ?? "Unknown",
status: r.status ?? "unknown",
size: formatBytes(r.size ?? 0),
sizeleft: formatBytes(r.sizeleft ?? 0),
timeleft: r.timeleft,
})),
};
}
}
function formatBytes(bytes: number): string {
if (bytes === 0) return "0 B";
const units = ["B", "KB", "MB", "GB", "TB"];
const i = Math.floor(Math.log(bytes) / Math.log(1024));
return `${(bytes / Math.pow(1024, i)).toFixed(1)} ${units[i]}`;
}
-74
View File
@@ -1,74 +0,0 @@
// bookshelf's events. The tool runtime imports this once the broker is bound. It watches the
// download queue and turns its comings and goings into mesh events — the same mechanism radarr uses,
// applied to a Servarr book manager.
//
// Emits (novox/hq ADR 0041/0042):
// module.bookshelf.book.grabbed — a release entered the queue (Bookshelf grabbed it)
// module.bookshelf.download.completed — a release left the queue, imported. This routing key is
// what the plex module consumes (module.*.download.completed)
// to rescan, so a new audiobook becomes a visible item.
// Consumes: none.
//
// NOTE: the hal bookshelf module emitted no events (its hooks only did install-time provisioning).
// This queue watcher is new in nox, modelled exactly on radarr's — bookshelf is a Servarr app with
// the same queue semantics, so the diff-and-emit pattern carries over unchanged.
//
// The queue is polled and diffed, primed silently on the first look (like plex's index.ts) so a
// restart mid-download does not re-announce everything already in flight as freshly grabbed.
import { emit } from "@novox/mesh-sdk/events";
import { BookshelfClient, type BookshelfQueueItem } from "./client.js";
// Building the client throws when Bookshelf has no URL/key yet. Like the tools (see tools/index.ts),
// the events entrypoint must not crash the runtime for that — it stays idle until configured.
function buildClient(): BookshelfClient | null {
try {
return BookshelfClient.fromEnv();
} catch {
return null;
}
}
const bookshelf = buildClient();
// Bookshelf removes an item from the queue once it has been imported; a "warning"/"failed" status is
// how a stuck or broken grab shows itself, so we do not call those a completion when they vanish.
const FAILED_STATUSES = new Set(["failed", "warning"]);
const inQueue = new Map<number, BookshelfQueueItem>();
let primed = false;
async function pollQueue(bookshelf: BookshelfClient): Promise<void> {
const { items } = await bookshelf.getQueue();
const now = new Map(items.map((i) => [i.id, i]));
if (primed) {
// Entered the queue since last look — Bookshelf grabbed a release.
for (const [id, item] of now) {
if (!inQueue.has(id)) await emit("book.grabbed", { title: item.title, status: item.status });
}
// Left the queue — imported and done, unless it was last seen failing.
for (const [id, item] of inQueue) {
if (!now.has(id) && !FAILED_STATUSES.has(item.status)) {
await emit("download.completed", { title: item.title });
}
}
}
inQueue.clear();
for (const [id, item] of now) inQueue.set(id, item);
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[bookshelf] ${err}`));
setInterval(run, everyMs);
run();
};
if (bookshelf) {
tick(() => pollQueue(bookshelf), 30_000);
console.log("[bookshelf] watching the download queue, emitting grabs and completions");
} else {
console.log("[bookshelf] not configured — events idle until an API key is available");
}
-120
View File
@@ -1,120 +0,0 @@
{
"module": "bookshelf",
"version": "1",
"slug": "books",
"capabilities": [
"container-runtime"
],
"emits": [
"book.grabbed",
"download.completed"
],
"consumes": [],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "web",
"port": 8787,
"protocol": "tcp",
"from": "mesh",
"why": "managing the ebook/audiobook library"
}
],
"accesses": [
{
"id": "books",
"path": "/services/media/books",
"mode": "read-write"
},
{
"id": "downloads",
"path": "/services/media/downloads",
"mode": "read-write"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/bookshelf/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "bookshelf",
"image": "ghcr.io/pennydreadful/bookshelf@sha256:388eecc94362580eae31ee0a454be6af516f8a311f8432a521c202fb475f4359",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"8787"
],
"volumes": [
"/services/bookshelf/config:/config",
"${access:books}:/books",
"${access:downloads}:/downloads"
]
},
{
"id": "runtime",
"type": "container",
"name": "mesh-bookshelf",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"/services/bookshelf/config:/var/lib/bookshelf/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_BOOKSHELF_URL": "http://127.0.0.1:8787",
"MESH_BOOKSHELF_CONFIG_DIR": "/var/lib/bookshelf/config"
},
"artifact": "runtime"
}
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "books",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:mesh-state}/route.json"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-bookshelf",
"version": "0.1.0",
"description": "bookshelf — ebook/audiobook management (Readarr fork). Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-69
View File
@@ -1,69 +0,0 @@
// bookshelf's tools — ported from the shared hal `arr` sdk (novox/hq ADR 0039), importing
// bookshelf's own client. They return structured data (not the pre-formatted text hal returned); the
// mesh serves them through the sdk's tool harness. Bookshelf has no calendar endpoint, so there is
// no calendar tool — matching hal, which excluded it from its calendar-capable apps.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { BookshelfClient } from "../client.js";
export function getBookshelfTools(bookshelf: BookshelfClient): ToolDefinition[] {
return [
{
name: "bookshelf_status",
description: "Bookshelf status overview: version, book count, monitored count, queue size.",
input: {},
run: async () => {
const [status, content, queue] = await Promise.all([
bookshelf.getStatus(),
bookshelf.getContent(),
bookshelf.getQueue(),
]);
return {
app: status.appName,
version: status.version,
books: content.length,
monitored: content.filter((c) => c.monitored).length,
queue: queue.totalRecords,
};
},
},
{
name: "bookshelf_library",
description: "List books from the Bookshelf library.",
input: { limit: { type: "number", description: "max items to return (default 50)" } },
run: async (args) => {
const items = await bookshelf.getContent(args.limit ? Number(args.limit) : 50);
return { count: items.length, books: items };
},
},
{
name: "bookshelf_search",
description:
"Search the Bookshelf library for books by title or author (filters existing content, not indexers).",
input: { query: { type: "string", description: "the search term" } },
run: async (args) => {
const query = String(args.query);
return { query, results: await bookshelf.searchContent(query) };
},
},
{
name: "bookshelf_queue",
description: "Show the Bookshelf download queue — what is downloading and how far along.",
input: {},
run: async () => {
const queue = await bookshelf.getQueue();
return { count: queue.totalRecords, items: queue.items };
},
},
];
}
// The tools exist only when Bookshelf is configured; without a URL and key, bookshelf contributes
// none rather than failing the whole runtime.
registerModuleTools("bookshelf", (env) => {
try {
return getBookshelfTools(BookshelfClient.fromEnv(env));
} catch {
return [];
}
});
-12
View File
@@ -1,12 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
@@ -1,6 +1,6 @@
ARG GO_BASE
ARG ALPINE_BASE
# builder's own image: the build machine itself, compiled into a container.
# build-agent's image: the build machine itself, compiled into a container (novox/hq ADR 0190).
#
# **The source is not vendored here.** builder's actual code — cmd/mesh-builder, internal/builder,
# internal/catalogue — lives in the mesh-controller repository, the same control plane it is one
+15
View File
@@ -0,0 +1,15 @@
# build-agent
The mesh's build machine as a role every machine can hold (novox/hq ADR 0190). It holds the node seat
`node-build-agent`: every holder pulls one build at a time from the role's one work queue when it is
idle, so a tier of many images is built by as many machines as hold the seat and are online, and a
machine that is off builds nothing and blocks nothing. The controller asks the role, never a machine;
the outcome names the machine that built it.
What a holding machine needs is what the builder always needed, said here once: a container runtime
(the socket is mounted), the artifact store and the package registry as provisions, a workspace, and
the bus credential. The code is `cmd/mesh-builder` in the mesh-controller repository, compiled from
that repository's main (`build.artifacts[].context`); this module ships the packaging.
Assign it to every machine with a container runtime. It replaces `builder`, the one-holder form of the
same thing; retire that once this is assigned where it was.
@@ -1,13 +1,14 @@
{
"module": "builder",
"module": "build-agent",
"version": "1",
"slug": "agent",
"capabilities": [
"container-runtime"
],
"claims": [
{
"name": "mesh-build-machine",
"scope": "mesh"
"name": "node-build-agent",
"scope": "node"
}
],
"requires": [
@@ -36,19 +37,19 @@
"mode": "0700"
},
{
"id": "builder-env",
"id": "agent-env",
"type": "file",
"path": "${dir:mesh-state}/builder.env",
"path": "${dir:mesh-state}/build-agent.env",
"mode": "0600",
"content": "MESH_BROKER_FILE=/run/mesh/broker\nMESH_NODE=${machine:name}\nMESH_REGISTRY=${bound:artifact-store:at}:${bound:artifact-store:port}\nMESH_PACKAGE_BINDING=/run/mesh/package-registry.json\nMESH_NPM_TOKEN_FILE=/run/mesh/package-registry.secret\nMESH_WORKSPACE=${dir:workspace}\n"
},
{
"id": "server",
"type": "container",
"name": "mesh-builder",
"name": "mesh-build-agent",
"artifact": "server",
"env-file": [
"${dir:mesh-state}/builder.env"
"${dir:mesh-state}/build-agent.env"
],
"volumes": [
"${dir:mesh-state}:/run/mesh:ro",
@@ -56,7 +57,7 @@
"/var/run/docker.sock:/var/run/docker.sock"
],
"restart-on": [
"builder-env"
"agent-env"
],
"network": "host"
}
-24
View File
@@ -1,24 +0,0 @@
# confluence's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/confluence
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/confluence/dist /app/modules/confluence/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/confluence/dist/tools/index.js
+14 -40
View File
@@ -3,16 +3,9 @@
"version": "1",
"slug": "confl",
"own-secrets": {
"token": "${dir:state}/token",
"broker": "${dir:mesh-state}/broker"
"token": "${dir:state}/token"
},
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -26,46 +19,27 @@
"merge": "json",
"content": "{}",
"mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-runtime-confluence",
"network": "host",
"volumes": [
"${dir:state}/config.json:/run/config/config.json:ro",
"${dir:state}/token:/run/secrets/token:ro",
"${dir:mesh-state}/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_CONFLUENCE_TOKEN_FILE": "/run/secrets/token",
"MESH_CONFLUENCE_CONFIG_FILE": "/run/config/config.json",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"artifact": "runtime"
}
],
"capabilities": [
"container-runtime"
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_CONFLUENCE_TOKEN_FILE": "${dir:state}/token",
"MESH_CONFLUENCE_CONFIG_FILE": "${dir:state}/config.json"
}
}
]
}
+2 -1
View File
@@ -3,7 +3,8 @@
"version": "1",
"capabilities": [
"package-manager",
"service-manager"
"service-manager",
"uplink-dhcpcd"
],
"claims": [
{
File diff suppressed because one or more lines are too long
+221 -36
View File
@@ -1,51 +1,236 @@
// fail2ban's own code, in the module (novox/hq ADR 0039). The jails and the daemon are declared
// resources — the mesh writes /etc/fail2ban/jail.d/* and keeps fail2ban.service running (see
// module.json). This code exists only to read and steer the *live* state the daemon owns at
// runtime: which IPs are banned right now, and the manual ban/unban an operator reaches for. That
// state (the running bans, /var/lib/fail2ban's sqlite) is fail2ban's, not the mesh's — the mesh
// reconciles the config, never the ban list.
// fail2ban's own code, in the module (novox/hq ADR 0039). The jails are composed by the mesh from
// the modules a machine runs (to-be 31) and written as declared resources; the daemon is kept
// running by one. This code exists only to read and steer the *live* state the daemon owns: who is
// banned now and until when, and the ban or release an operator asks for — the node-intrusion-
// prevention seat's four verbs (ADR 0179). The daemon's state is fail2ban's, not the mesh's: the
// mesh composes the jails and never writes the ban list.
//
// Spoken through fail2ban-client over the daemon's socket. Client and daemon come from the one
// package this module declares on the machine, and the socket is root's: root is the module's
// concern (ADR 0175 §4), and the runtime loading this bundle runs as the operator's account (to-be
// 38 WP4), so the client is run through sudo without a prompt where the account is not root.
import { execFile } from "node:child_process";
import { accessSync, constants } from "node:fs";
import { isIP } from "node:net";
import { delimiter, join } from "node:path";
import { promisify } from "node:util";
const run = promisify(execFile);
const execFileP = promisify(execFile);
/** A command runner, so the verbs can be tested without a daemon. */
export type Runner = (cmd: string, args: string[]) => Promise<string>;
/** The command as it is run: as given when this process is root, else through sudo without a
* prompt. The daemon's socket answers only to root. */
export function escalated(cmd: string, args: string[], uid: number | undefined = process.getuid?.()): [string, string[]] {
if (uid === 0) return [cmd, args];
return ["sudo", ["-n", cmd, ...args]];
}
/** Whether a tool is on this machine: an executable of that name on the path, or where the
* system keeps its administration. */
export function installed(tool: string, path: string = process.env.PATH ?? ""): boolean {
const dirs = [...path.split(delimiter), "/usr/sbin", "/sbin", "/usr/bin"].filter((d) => d !== "");
return dirs.some((dir) => {
try {
accessSync(join(dir, tool), constants.X_OK);
return true;
} catch {
return false;
}
});
}
export const execRunner: Runner = async (cmd, args) => {
if (!installed(cmd)) throw new Error(`${cmd} is not installed on this machine`);
const [program, argv] = escalated(cmd, args);
try {
const { stdout } = await execFileP(program, argv, { maxBuffer: 16 * 1024 * 1024 });
return stdout;
} catch (err) {
const e = err as { code?: string | number; stderr?: string; stdout?: string; message?: string };
const said = `${e.stdout ?? ""}${e.stderr ?? ""}`.trim();
// What failed is named by how it failed: sudo missing is a spawn error, sudo refusing speaks
// on its own stderr line, and the rest is the client's own answer.
if (program === "sudo") {
if (e.code === "ENOENT") throw new Error(`${cmd} needs root, and sudo is not installed here for the runtime's account to escalate with`);
if (/^sudo:/m.test(said)) throw new Error(`${cmd} needs root and the runtime's account may not run it without a prompt: ${said}`);
}
if (/Failed to access socket path|Is fail2ban running|Permission denied to socket/i.test(said)) {
throw new Error("fail2ban is not running on this machine, or its socket does not answer the runtime's account");
}
// fail2ban-client's own last line is the one a person reads ("Sorry but the jail 'x' does not exist").
const lines = said.split("\n").map((l) => l.trim()).filter(Boolean);
throw new Error(lines.length ? lines[lines.length - 1] : (e.message ?? `${cmd} failed`));
}
};
/** One jail as the daemon reports it. */
export interface JailStatus {
jail: string;
/** What the jail is reading: files or journal matches, as fail2ban names them. */
watching: string[];
/** Addresses with failures counted against them right now, and all failures since the jail started. */
failing: { now: number; total: number };
/** Addresses held right now, and all bans since the jail started. */
banned: { now: number; total: number; addresses: string[] };
}
/** One ban as the daemon holds it. */
export interface Ban {
ip: string;
jail: string;
/** When the ban was placed, in the machine's local time as fail2ban prints it. */
since: string;
/** When the ban ends; "never" for a permanent ban. */
until: string;
}
export interface JailSettings {
jail: string;
bantime: string;
findtime: string;
maxretry: number;
ignoreip: string[];
actions: string[];
/** The log files the jail reads, when it reads files. */
logpath: string[];
/** The journal match the jail reads, when it reads the journal. */
journalmatch: string;
}
export class Fail2banClient {
static fromEnv(_env: NodeJS.ProcessEnv = process.env): Fail2banClient {
private readonly run: Runner;
constructor(run: Runner = execRunner) {
this.run = run;
}
/** The daemon as this machine has it, through its own client. */
static onThisMachine(): Fail2banClient {
return new Fail2banClient();
}
/** Overview of every jail, or the detailed status of one — currently-banned IPs and totals. */
async status(jail?: string): Promise<string> {
private client(...args: string[]): Promise<string> {
return this.run("fail2ban-client", args);
}
/** The jails the daemon runs, by name. */
async jails(): Promise<string[]> {
const out = await this.client("status");
const m = out.match(/Jail list:\s*(.*)/);
if (!m) return [];
return m[1].split(",").map((j) => j.trim()).filter(Boolean);
}
/** Every jail with what it watches and holds, or one jail's detail. */
async status(jail?: string): Promise<{ jails: JailStatus[] }> {
const names = jail ? [jail] : await this.jails();
const jails: JailStatus[] = [];
for (const name of names) {
jails.push(parseJailStatus(name, await this.client("status", name)));
}
return { jails };
}
/** Every address banned now, with the jail holding it and when the ban ends. */
async banned(jail?: string): Promise<{ banned: Ban[] }> {
const names = jail ? [jail] : await this.jails();
const banned: Ban[] = [];
for (const name of names) {
banned.push(...parseBans(name, await this.client("get", name, "banip", "--with-time")));
}
banned.sort((a, b) => a.until.localeCompare(b.until) || a.ip.localeCompare(b.ip));
return { banned };
}
/** Ban one address in one jail now. The daemon's own answer is how many addresses it added. */
async ban(ip: string, jail: string): Promise<{ banned: Ban | null; added: number }> {
address(ip);
name(jail);
const out = await this.client("set", jail, "banip", ip);
const added = Number.parseInt(out.trim(), 10) || 0;
const held = (await this.banned(jail)).banned.find((b) => b.ip === ip) ?? null;
return { banned: held, added };
}
/** Let one address go, from one jail or from every jail. The daemon's answer is how many it released. */
async unban(ip: string, jail?: string): Promise<{ released: number; ip: string; jail: string | "every jail" }> {
address(ip);
let out: string;
if (jail) {
const { stdout } = await run("sudo", ["fail2ban-client", "status", jail]);
return stdout;
name(jail);
out = await this.client("set", jail, "unbanip", ip);
} else {
out = await this.client("unban", ip);
}
const { stdout: overview } = await run("sudo", ["fail2ban-client", "status"]);
const match = overview.match(/Jail list:\s*(.+)/);
if (!match) return overview;
const jails = match[1].split(",").map((j) => j.trim()).filter(Boolean);
const parts: string[] = [overview.trimEnd(), ""];
for (const j of jails) {
const { stdout } = await run("sudo", ["fail2ban-client", "status", j]);
parts.push(`=== ${j} ===`, stdout.trimEnd(), "");
}
return parts.join("\n");
return { released: Number.parseInt(out.trim(), 10) || 0, ip, jail: jail ?? "every jail" };
}
/** Manually ban an IP in a jail. Mutates live state, not a mesh-managed file. */
async ban(jail: string, ip: string): Promise<string> {
const { stdout } = await run("sudo", ["fail2ban-client", "set", jail, "banip", ip]);
return stdout;
}
/** Unban an IP from one jail, or from every jail when no jail is given. */
async unban(ip: string, jail?: string): Promise<string> {
const args = jail
? ["fail2ban-client", "set", jail, "unbanip", ip]
: ["fail2ban-client", "unban", ip];
const { stdout } = await run("sudo", args);
return stdout;
/** One jail's effective settings — the module's own tool, beside the seat's verbs. */
async settings(jail: string): Promise<JailSettings> {
name(jail);
const get = (key: string) => this.client("get", jail, key);
const [bantime, findtime, maxretry, ignoreip, actions, logpath, journalmatch] = await Promise.all([
get("bantime"), get("findtime"), get("maxretry"), get("ignoreip"), get("actions"), get("logpath"),
get("journalmatch"),
]);
return {
jail,
bantime: bantime.trim(),
findtime: findtime.trim(),
maxretry: Number.parseInt(maxretry.trim(), 10),
ignoreip: listed(ignoreip),
actions: actions.split("\n").slice(1).map((l) => l.trim()).filter(Boolean),
logpath: /No file is currently monitored/.test(logpath) ? [] : listed(logpath),
journalmatch: journalmatch.split("\n").slice(1).map((l) => l.trim()).filter(Boolean).join(" "),
};
}
}
/** fail2ban's tree listings: lines like "|- 127.0.0.0/8" and "`- ::1", after a heading. */
function listed(out: string): string[] {
return out
.split("\n")
.map((l) => l.replace(/^[\s|`-]+/, "").trim())
.filter((l, i) => i > 0 && l.length > 0);
}
export function parseJailStatus(jail: string, out: string): JailStatus {
const field = (label: string) => {
const m = out.match(new RegExp(label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") + ":\\t?\\s*(.*)"));
return m ? m[1].trim() : "";
};
const num = (label: string) => Number.parseInt(field(label), 10) || 0;
const watching = [field("File list"), field("Journal matches")].filter(Boolean);
return {
jail,
watching,
failing: { now: num("Currently failed"), total: num("Total failed") },
banned: {
now: num("Currently banned"),
total: num("Total banned"),
addresses: field("Banned IP list").split(/\s+/).filter(Boolean),
},
};
}
/** `get <jail> banip --with-time` prints one ban per line: "IP \tsince + seconds = until". */
export function parseBans(jail: string, out: string): Ban[] {
const bans: Ban[] = [];
for (const line of out.split("\n")) {
const m = line.match(/^(\S+)\s+(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) \+ (-?\d+) = (\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}|\S+)/);
if (!m) continue;
bans.push({ ip: m[1], jail, since: m[2], until: Number(m[3]) < 0 ? "never" : m[4] });
}
return bans;
}
function address(ip: string): void {
if (!isIP(ip)) throw new Error(`${JSON.stringify(ip)} is not an address`);
}
function name(jail: string): void {
if (!/^[A-Za-z0-9][A-Za-z0-9._-]*$/.test(jail)) throw new Error(`${JSON.stringify(jail)} is not a jail's name`);
}
+44 -6
View File
@@ -7,9 +7,22 @@
"claims": [
{
"name": "node-intrusion-prevention",
"scope": "node"
"scope": "node",
"serves": [
"status",
"banned",
"ban",
"unban"
]
}
],
"tools": [
"fail2ban_settings"
],
"jailing": {
"into": "/etc/fail2ban/jail.d/mesh.conf",
"filter-into": "/etc/fail2ban/filter.d"
},
"resources": [
{
"id": "package",
@@ -28,19 +41,31 @@
"path": "/etc/fail2ban/action.d",
"mode": "0755"
},
{
"id": "filter-d",
"type": "directory",
"path": "/etc/fail2ban/filter.d",
"mode": "0755"
},
{
"id": "run-dir",
"type": "directory",
"path": "/var/run/fail2ban",
"mode": "0755"
},
{
"id": "jail-local",
"type": "file",
"path": "/etc/fail2ban/jail.local",
"mode": "0644",
"content": "[INCLUDES]\n\nbefore = paths-arch.conf\n\n[DEFAULT]\n\n# Never act on the machine itself or on a tunnel peer: the mesh's private range is\n# ${machine:mesh-range}, named here rather than written as a value the module cannot\n# know (novox/hq ADR 0112). Without this, fail2ban could ban the mesh's own nodes.\nignoreip = 127.0.0.1/8 ::1 ${machine:mesh-range}\n\nbantime = 10m\nfindtime = 10m\nmaxretry = 5\n\n# Ban through iptables, not through a firewall front-end the machine may not have. ufw is\n# installed on two of this mesh's machines and absent on the other two, and fail2ban finds out\n# only at ban time: the service reports healthy, the jail counts the attempt, the ban command\n# exits 127, and nothing is blocked. Proven on 2026-09-28 -- 'ufw: command not found' on a\n# machine the mesh reported as protected.\n#\n# The action below is this module's own, already used by the recidive jail on every machine\n# here, and it bans in DOCKER-USER as well as INPUT, so a container's published port is\n# covered too.\nbanaction = iptables-allports-dualchain\nbanaction_allports = iptables-allports-dualchain\n\n[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\n"
"content": "[INCLUDES]\n\nbefore = paths-arch.conf\n\n[DEFAULT]\n\n# Never act on the machine itself or on a tunnel peer: the mesh's private range is\n# ${machine:mesh-range}, named here rather than written as a value the module cannot\n# know (novox/hq ADR 0112). Without this, fail2ban could ban the mesh's own nodes.\n# **A ban list never holds a neighbour.** The mesh's own range is named rather than written\n# (novox/hq ADR 0112), and every private range beside it: a source on one is somebody's own\n# network, not the internet. On a machine behind a router that reflects local traffic, every\n# client in the house arrives as the gateway's address — so one mistyped local request banned\n# 192.168.1.1 on the home server and would have cut the whole house off from it (ADR 0186).\nignoreip = 127.0.0.1/8 ::1 ${machine:mesh-range} 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 169.254.0.0/16 fc00::/7 fe80::/10\n\n# Three failures in a day ban for a day (novox/hq ADR 0179). The attackers this mesh sees pace\n# themselves at one try every ten minutes, under any ten-minute window; a day's window counts\n# them, and a day's ban costs a person who mistyped three times once, from one address, while\n# the mesh's own range is never banned at all.\nbantime = 1d\nfindtime = 1d\nmaxretry = 3\n\n# Ban through iptables, not through a firewall front-end the machine may not have. ufw is\n# installed on two of this mesh's machines and absent on the other two, and fail2ban finds out\n# only at ban time: the service reports healthy, the jail counts the attempt, the ban command\n# exits 127, and nothing is blocked. Proven on 2026-09-28 -- 'ufw: command not found' on a\n# machine the mesh reported as protected.\n#\n# The action below is this module's own, already used by the recidive jail on every machine\n# here, and it bans in DOCKER-USER as well as INPUT, so a container's published port is\n# covered too.\nbanaction = iptables-allports-dualchain\nbanaction_allports = iptables-allports-dualchain\n\n[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\n"
},
{
"id": "jail-sshd",
"type": "file",
"path": "/etc/fail2ban/jail.d/sshd.conf",
"mode": "0644",
"content": "[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\nmaxretry = 5\n"
"content": "[sshd]\nenabled = true\nport = ssh\nlogpath = %(sshd_log)s\nbackend = %(sshd_backend)s\nmaxretry = 3\nfindtime = 1d\nbantime = 1d\n"
},
{
"id": "log",
@@ -55,7 +80,7 @@
"type": "file",
"path": "/etc/fail2ban/jail.d/recidive.conf",
"mode": "0644",
"content": "[recidive]\nenabled = true\nlogpath = /var/log/fail2ban.log\n# Ban in both INPUT (host services like SSH) and DOCKER-USER (container services)\nbanaction = iptables-allports-dualchain\nbantime = 1w\nfindtime = 1d\n"
"content": "[recidive]\nenabled = true\nlogpath = /var/log/fail2ban.log\n# Ban in both INPUT (host services like SSH) and DOCKER-USER (container services)\nbanaction = iptables-allports-dualchain\n# Banned twice in two weeks, by any jail, is banned for four (novox/hq ADR 0179).\nbantime = 4w\nfindtime = 2w\nmaxretry = 2\n"
},
{
"id": "action-dualchain",
@@ -81,8 +106,21 @@
"jail-local",
"jail-sshd",
"jail-recidive",
"action-dualchain"
"action-dualchain",
"composed-jails"
]
}
]
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
]
}
]
}
}
+6 -2
View File
@@ -1,14 +1,18 @@
{
"name": "@novox/module-fail2ban",
"version": "0.1.0",
"description": "fail2ban — intrusion prevention: the mesh declares the jails and keeps the daemon running; its ban/unban/status tools live here.",
"description": "fail2ban \u2014 intrusion prevention: the mesh composes the jails and keeps the daemon running; this module holds the node-intrusion-prevention seat and serves its verbs status, banned, ban and unban (novox/hq to-be 31, ADR 0179).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
"@novox/mesh-sdk": "^0.1.1"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
},
"scripts": {
"build": "tsc client.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . --outDir dist",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
}
}
+114
View File
@@ -0,0 +1,114 @@
// The intrusion prevention's verbs over a fake daemon, with the shapes fail2ban-client 1.1.0 printed
// on the control node on 2026-10-02 (novox/hq ADR 0179).
import { test } from "node:test";
import assert from "node:assert/strict";
import { Fail2banClient, escalated, installed, parseBans, parseJailStatus, type Runner } from "../client.ts";
const STATUS = "Status\n|- Number of jail:\t2\n`- Jail list:\trecidive, sshd\n";
const RECIDIVE =
"Status for the jail: recidive\n|- Filter\n| |- Currently failed:\t36\n| |- Total failed:\t149\n" +
"| `- File list:\t/var/log/fail2ban.log\n`- Actions\n |- Currently banned:\t9\n |- Total banned:\t13\n" +
" `- Banned IP list:\t195.178.110.30 45.148.10.240 92.118.39.71\n";
const SSHD =
"Status for the jail: sshd\n|- Filter\n| |- Currently failed:\t5\n| |- Total failed:\t11776\n" +
"| `- Journal matches:\t_SYSTEMD_UNIT=sshd.service + _COMM=sshd\n`- Actions\n |- Currently banned:\t0\n" +
" |- Total banned:\t150\n `- Banned IP list:\t\n";
const WITH_TIME =
"195.178.110.30 \t2026-09-26 23:18:47 + 604800 = 2026-10-03 23:18:47\n" +
"92.118.39.71 \t2026-09-28 10:33:49 + 604800 = 2026-10-05 10:33:49\n";
function fake(answers: Record<string, string>, calls: string[][] = []): Runner {
return async (cmd, args) => {
calls.push([cmd, ...args]);
const key = args.join(" ");
if (key in answers) return answers[key];
throw new Error(`unexpected ${cmd} ${key}`);
};
}
test("a jail's status is read into numbers, what it watches and who it holds", () => {
const s = parseJailStatus("recidive", RECIDIVE);
assert.deepEqual(s, {
jail: "recidive",
watching: ["/var/log/fail2ban.log"],
failing: { now: 36, total: 149 },
banned: { now: 9, total: 13, addresses: ["195.178.110.30", "45.148.10.240", "92.118.39.71"] },
});
const j = parseJailStatus("sshd", SSHD);
assert.deepEqual(j.watching, ["_SYSTEMD_UNIT=sshd.service + _COMM=sshd"]);
assert.deepEqual(j.banned, { now: 0, total: 150, addresses: [] });
});
test("status covers every jail the daemon lists, or the one named", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({ status: STATUS, "status recidive": RECIDIVE, "status sshd": SSHD }, calls));
const all = await f.status();
assert.deepEqual(all.jails.map((j) => j.jail), ["recidive", "sshd"]);
const one = await f.status("sshd");
assert.equal(one.jails.length, 1);
assert.deepEqual(calls[calls.length - 1], ["fail2ban-client", "status", "sshd"]);
});
test("bans are read with when they were placed and when they end, a permanent one as never", () => {
const bans = parseBans("recidive", WITH_TIME + "203.0.113.9 \t2026-10-01 00:00:00 + -1 = never\n");
assert.equal(bans.length, 3);
assert.deepEqual(bans[0], { ip: "195.178.110.30", jail: "recidive", since: "2026-09-26 23:18:47", until: "2026-10-03 23:18:47" });
assert.equal(bans[2].until, "never");
assert.deepEqual(parseBans("sshd", "\n"), []);
});
test("banned gathers every jail's bans, soonest to end first", async () => {
const f = new Fail2banClient(fake({
status: STATUS,
"get recidive banip --with-time": WITH_TIME,
"get sshd banip --with-time": "198.51.100.7 \t2026-10-02 15:06:58 + 600 = 2026-10-02 15:16:58\n",
}));
const { banned } = await f.banned();
assert.deepEqual(banned.map((b) => `${b.ip}@${b.jail}`), ["198.51.100.7@sshd", "195.178.110.30@recidive", "92.118.39.71@recidive"]);
});
test("ban asks the daemon by jail and answers with the ban as held; a non-address is refused before anything runs", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({
"set recidive banip 198.51.100.7": "1\n",
"get recidive banip --with-time": WITH_TIME + "198.51.100.7 \t2026-10-02 17:00:00 + 604800 = 2026-10-09 17:00:00\n",
}, calls));
const r = await f.ban("198.51.100.7", "recidive");
assert.equal(r.added, 1);
assert.equal(r.banned?.until, "2026-10-09 17:00:00");
assert.deepEqual(calls[0], ["fail2ban-client", "set", "recidive", "banip", "198.51.100.7"]);
await assert.rejects(() => f.ban("not-an-ip", "recidive"), /is not an address/);
await assert.rejects(() => f.ban("198.51.100.7", "a jail; rm"), /is not a jail's name/);
assert.equal(calls.length, 2);
});
test("unban releases from one jail or from every jail", async () => {
const calls: string[][] = [];
const f = new Fail2banClient(fake({ "set sshd unbanip 198.51.100.7": "1\n", "unban 198.51.100.7": "2\n" }, calls));
assert.deepEqual(await f.unban("198.51.100.7", "sshd"), { released: 1, ip: "198.51.100.7", jail: "sshd" });
assert.deepEqual(await f.unban("198.51.100.7"), { released: 2, ip: "198.51.100.7", jail: "every jail" });
assert.deepEqual(calls[1], ["fail2ban-client", "unban", "198.51.100.7"]);
});
test("a jail's settings are read from the daemon's listings", async () => {
const f = new Fail2banClient(fake({
"get sshd bantime": "86400\n", "get sshd findtime": "86400\n", "get sshd maxretry": "3\n",
"get sshd ignoreip": "These IP addresses/networks are ignored:\n|- 127.0.0.0/8\n|- 10.10.0.0/24\n`- ::1\n",
"get sshd actions": "The jail sshd has the following actions:\niptables-allports-dualchain\n",
"get sshd logpath": "No file is currently monitored\n",
"get sshd journalmatch": "Current match filter:\n_SYSTEMD_UNIT=sshd.service + _COMM=sshd\n",
}));
assert.deepEqual(await f.settings("sshd"), {
jail: "sshd", bantime: "86400", findtime: "86400", maxretry: 3,
ignoreip: ["127.0.0.0/8", "10.10.0.0/24", "::1"], actions: ["iptables-allports-dualchain"],
logpath: [], journalmatch: "_SYSTEMD_UNIT=sshd.service + _COMM=sshd",
});
});
test("the client runs as given by root and through sudo without a prompt by anyone else", () => {
assert.deepEqual(escalated("fail2ban-client", ["status"], 0), ["fail2ban-client", ["status"]]);
assert.deepEqual(escalated("fail2ban-client", ["set", "sshd", "banip", "198.51.100.7"], 1000),
["sudo", ["-n", "fail2ban-client", "set", "sshd", "banip", "198.51.100.7"]]);
assert.equal(installed("sh"), true);
assert.equal(installed("no-such-client-of-the-mesh"), false);
});
+45 -38
View File
@@ -1,55 +1,62 @@
// fail2ban's tools — reading and steering the live ban state. The jails themselves are declared
// resources (module.json); these three touch what the running daemon holds: what is banned now,
// and the manual ban/unban an operator reaches for. The daemon's state is fail2ban's own, so this
// is the only way to see or change it — the mesh reconciles the config, not the bans.
// The intrusion prevention's tools: the node-intrusion-prevention seat's four verbs — who is banned,
// the jails' state, ban one, let one go — and the module's own reading of a jail's settings
// (novox/hq to-be 31, ADR 0179). The jails themselves are composed by the mesh from the modules a
// machine runs and written as declared resources; these touch only what the running daemon holds.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { Fail2banClient } from "../client.js";
export function getFail2banTools(fail2ban: Fail2banClient): ToolDefinition[] {
export function getSeatVerbs(fail2ban: Fail2banClient): ToolDefinition[] {
return [
{
name: "fail2ban_status",
name: "status",
description:
"fail2ban status on this node — the jails and their live bans. Omit `jail` for every jail, or name one for its detail.",
input: {
type: "object",
properties: {
jail: {
type: "string",
description: "A specific jail (e.g. sshd, recidive); omit for the overview of all jails.",
},
},
},
run: async (args) => ({ status: await fail2ban.status(args.jail as string | undefined) }),
"Every jail on this machine with what it watches, how many addresses it is counting failures against and holding now, and the totals since it started; one jail's detail when named.",
input: { jail: { type: "string", description: "one jail (optional)" } },
run: async (args) => fail2ban.status(args.jail ? String(args.jail) : undefined),
},
{
name: "fail2ban_ban",
description: "Manually ban an IP address in a jail — a live change to the running daemon, not a mesh-managed file.",
input: {
type: "object",
properties: {
jail: { type: "string", description: "Jail name (e.g. sshd, recidive)." },
ip: { type: "string", description: "IP address to ban." },
},
required: ["jail", "ip"],
},
run: async (args) => ({ result: await fail2ban.ban(args.jail as string, args.ip as string) }),
name: "banned",
description: "Every address banned on this machine right now, with the jail that holds it, when it was banned and when the ban ends.",
input: { jail: { type: "string", description: "one jail (optional)" } },
run: async (args) => fail2ban.banned(args.jail ? String(args.jail) : undefined),
},
{
name: "fail2ban_unban",
description: "Unban an IP address from one jail, or from every jail when `jail` is omitted.",
name: "ban",
description:
"Ban one address in one jail now, for the jail's ban time — an operator's act on the live ban list, which the mesh never writes itself.",
input: {
type: "object",
properties: {
ip: { type: "string", description: "IP address to unban." },
jail: { type: "string", description: "A specific jail; omit to unban from all jails." },
},
required: ["ip"],
ip: { type: "string", description: "the address" },
jail: { type: "string", description: "the jail to hold it (recidive for the long ban)" },
},
run: async (args) => ({ result: await fail2ban.unban(args.ip as string, args.jail as string | undefined) }),
run: async (args) => fail2ban.ban(String(args.ip ?? ""), String(args.jail ?? "")),
},
{
name: "unban",
description: "Let one address go, from one jail or from every jail when none is named.",
input: {
ip: { type: "string", description: "the address" },
jail: { type: "string", description: "one jail (optional)" },
},
run: async (args) => fail2ban.unban(String(args.ip ?? ""), args.jail ? String(args.jail) : undefined),
},
];
}
registerModuleTools("fail2ban", () => getFail2banTools(Fail2banClient.fromEnv()));
export function getFail2banTools(fail2ban: Fail2banClient): ToolDefinition[] {
return [
{
name: "fail2ban_settings",
description:
"One jail's effective settings on this machine: ban time, window, tries, the addresses it never bans, its actions and what it reads.",
input: { jail: { type: "string", description: "the jail" } },
run: async (args) => fail2ban.settings(String(args.jail ?? "")),
},
];
}
const fail2ban = Fail2banClient.onThisMachine();
// The seat's verbs under the seat's name: the runtime serves them on the seat's subjects where this
// module holds it (ADR 0159, 0160). The module's own under its own.
registerModuleTools("node-intrusion-prevention", () => getSeatVerbs(fail2ban));
registerModuleTools("fail2ban", () => getFail2banTools(fail2ban));
+66
View File
@@ -45,6 +45,14 @@ export interface GiteaPull {
html_url: string;
}
export interface GiteaComment {
id: number;
user?: string;
body: string;
created_at?: string;
html_url: string;
}
export interface GiteaLabel {
id: number;
name: string;
@@ -257,6 +265,64 @@ export class GiteaClient {
);
}
/** Close or reopen a pull request without merging it. A pull request is an issue to the forge's
* state machine, and the pulls endpoint takes the same `state`. */
async setPullState(owner: string, repo: string, index: number, state: "open" | "closed"): Promise<GiteaPull> {
return GiteaClient.mapPull(
await this.request<any>(`/repos/${owner}/${repo}/pulls/${index}`, { method: "PATCH", body: JSON.stringify({ state }) }),
);
}
/** Change a pull request's title or body; a field left undefined is left alone. */
async updatePullRequest(owner: string, repo: string, index: number, data: { title?: string; body?: string }): Promise<GiteaPull> {
return GiteaClient.mapPull(
await this.request<any>(`/repos/${owner}/${repo}/pulls/${index}`, { method: "PATCH", body: JSON.stringify(data) }),
);
}
/** The unified diff of a pull request, as text. */
async pullDiff(owner: string, repo: string, index: number): Promise<string> {
return this.requestText(`/repos/${owner}/${repo}/pulls/${index}.diff`);
}
/** Every comment on an issue or pull request, oldest first. */
async listComments(owner: string, repo: string, index: number): Promise<GiteaComment[]> {
const raw = await this.request<any[]>(`/repos/${owner}/${repo}/issues/${index}/comments`);
return (raw ?? []).map((c) => ({
id: Number(c?.id ?? 0),
user: c?.user?.login,
body: String(c?.body ?? ""),
created_at: c?.created_at,
html_url: String(c?.html_url ?? ""),
}));
}
/** One file's contents at a ref (default the repository's default branch), decoded. */
async getFile(owner: string, repo: string, path: string, ref?: string): Promise<{ path: string; ref?: string; sha: string; size: number; content: string }> {
const qs = ref ? `?ref=${encodeURIComponent(ref)}` : "";
const f = await this.request<any>(`/repos/${owner}/${repo}/contents/${path.split("/").map(encodeURIComponent).join("/")}${qs}`);
if (!f || f.type !== "file") throw new Error(`Gitea API: ${path} is not a file`);
const content = f.encoding === "base64" ? Buffer.from(String(f.content ?? ""), "base64").toString("utf8") : String(f.content ?? "");
return { path, ref, sha: String(f.sha ?? ""), size: Number(f.size ?? content.length), content };
}
async listBranches(owner: string, repo: string): Promise<{ name: string; commit: string; protected: boolean }[]> {
const raw = await this.request<any[]>(`/repos/${owner}/${repo}/branches?limit=100`);
return (raw ?? []).map((b) => ({ name: String(b?.name ?? ""), commit: String(b?.commit?.id ?? ""), protected: Boolean(b?.protected) }));
}
async deleteBranch(owner: string, repo: string, branch: string): Promise<void> {
await this.request(`/repos/${owner}/${repo}/branches/${encodeURIComponent(branch)}`, { method: "DELETE" });
}
/** A request whose answer is text, not JSON — a diff. Same token handling as request(). */
private async requestText(path: string): Promise<string> {
const token = await this.tokens.current();
const res = await this.send(path, { headers: { Accept: "text/plain" } }, token);
if (!res.ok) throw new Error(`Gitea API ${path}: ${res.status} ${await res.text()}`);
return res.text();
}
async mergePullRequest(owner: string, repo: string, index: number, method = "merge", deleteBranch = false): Promise<void> {
await this.request(`/repos/${owner}/${repo}/pulls/${index}/merge`, {
method: "POST",
+10 -2
View File
@@ -145,7 +145,8 @@
"volumes": [
"${dir:data}:/data"
],
"secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it"
"secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it",
"logging": "journald"
},
{
"id": "admin-bootstrap",
@@ -237,5 +238,12 @@
"from": "Dockerfile"
}
]
}
},
"jails": [
{
"name": "gitea",
"failregex": "^.*Failed authentication attempt for .* from <HOST>(?::\\d+)?\\s*$\n ^.*Invalid user .* from <HOST> port \\d+\\s*$\n ^.*User \\S+ from <HOST> not allowed because .*$",
"jail": "backend = systemd\njournalmatch = CONTAINER_NAME=gitea\nport = http,https,222\nmaxretry = 3\nfindtime = 1d\nbantime = 1d"
}
]
}
+79
View File
@@ -31,6 +31,9 @@ interface Forge {
tokens: Map<string, string>;
scopesOf: Map<string, string[]>;
admins: Map<string, string>;
pullState: string;
pullTitle: string;
branchDeleted: boolean;
close(): Promise<void>;
}
@@ -41,6 +44,9 @@ function fakeForge(): Promise<Forge> {
tokens: new Map<string, string>(), // name -> value
scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does
admins: new Map([[ADMIN, PASSWORD]]),
pullState: "open",
pullTitle: "The console shipped",
branchDeleted: false,
};
// write:X implies read:X — gitea's own rule (models/auth/access_token_scope.go).
const covers = (scopes: string[], required: string): boolean =>
@@ -86,6 +92,37 @@ function fakeForge(): Promise<Forge> {
}
return json(res, 405, { message: "method not allowed" });
}
const tokenOf = (): string => {
const h = req.headers.authorization ?? "";
return h.startsWith("token ") ? h.slice(6) : "";
};
const pull = url.pathname.match(/^\/api\/v1\/repos\/novox\/hq\/pulls\/(\d+)(\.diff)?$/);
if (pull) {
if (![...forge.tokens.values()].includes(tokenOf())) return json(res, 401, { message: "token is required" });
if (pull[2]) {
res.writeHead(200, { "Content-Type": "text/plain" });
return res.end("diff --git a/x b/x\n--- a/x\n+++ b/x\n@@ -1 +1 @@\n-old\n+new\n");
}
if (req.method === "PATCH") {
const patch = await body(req);
forge.pullState = patch?.state ?? forge.pullState;
forge.pullTitle = patch?.title ?? forge.pullTitle;
}
return json(res, 200, { number: Number(pull[1]), title: forge.pullTitle, state: forge.pullState, merged: false,
user: { login: "mesh-admin" }, head: { ref: "feat/x" }, base: { ref: "main" }, html_url: "http://fake/novox/hq/pulls/" + pull[1] });
}
if (url.pathname === "/api/v1/repos/novox/hq/issues/223/comments") {
return json(res, 200, [{ id: 1, user: { login: "jochen" }, body: "landed elsewhere", created_at: "2026-10-01T00:00:00Z", html_url: "http://fake/c/1" }]);
}
if (url.pathname === "/api/v1/repos/novox/hq/contents/README.md") {
return json(res, 200, { type: "file", encoding: "base64", sha: "abc", size: 5, content: Buffer.from("hello").toString("base64") });
}
if (url.pathname === "/api/v1/repos/novox/hq/branches") {
return json(res, 200, [{ name: "main", protected: true, commit: { id: "aaaa" } }, { name: "feat/x", protected: false, commit: { id: "bbbb" } }]);
}
if (url.pathname === "/api/v1/repos/novox/hq/branches/feat%2Fx" || url.pathname === "/api/v1/repos/novox/hq/branches/feat/x") {
if (req.method === "DELETE") { forge.branchDeleted = true; return json(res, 204, null); }
}
if (url.pathname === "/api/v1/repos/search") {
// The client lists through the search endpoint since 2026-09-28 (the forge's whole view);
// it sits under `repository`, which write:repository covers.
@@ -140,6 +177,9 @@ function fakeForge(): Promise<Forge> {
url: `http://127.0.0.1:${port}`,
get mints() { return forge.mints; },
get lastScopes() { return forge.lastScopes; },
get pullState() { return forge.pullState; },
get pullTitle() { return forge.pullTitle; },
get branchDeleted() { return forge.branchDeleted; },
tokens: forge.tokens,
scopesOf: forge.scopesOf,
admins: forge.admins,
@@ -360,6 +400,16 @@ test("the tools register once there is a way to a token, and the first call mint
"gitea_list_repos", "gitea_create_repo", "gitea_delete_repo",
"gitea_list_issues", "gitea_get_issue", "gitea_create_issue", "gitea_close_issue", "gitea_add_comment",
"gitea_list_pull_requests", "gitea_get_pull_request", "gitea_create_pull_request", "gitea_merge_pull_request",
"gitea_close_pull_request",
"gitea_reopen_pull_request",
"gitea_update_pull_request",
"gitea_pull_request_files",
"gitea_pull_request_diff",
"gitea_list_comments",
"gitea_reopen_issue",
"gitea_get_file",
"gitea_list_branches",
"gitea_delete_branch",
"gitea_list_labels", "gitea_create_label",
"gitea_api",
],
@@ -370,3 +420,32 @@ test("the tools register once there is a way to a token, and the first call mint
assert.equal(result.repos.length, 1);
assert.equal(forge.mints, before + 1);
});
// The forge's tools reach every action a review needs without a checkout and without the API
// escape hatch: close a pull request whose work landed elsewhere, read its diff, its comments, a
// file, the branches, and delete the branch left behind. Against the fake forge, through the
// compiled tools, the way the console calls them.
test("a pull request can be closed, read and cleaned up through the tools", async () => {
const { env } = await delivered(forge);
const tools = collectTools(env).find((c) => c.module === "gitea")!.tools;
const tool = (name: string) => tools.find((t) => t.name === name)!;
for (const name of ["gitea_close_pull_request", "gitea_reopen_pull_request", "gitea_update_pull_request", "gitea_pull_request_files",
"gitea_pull_request_diff", "gitea_list_comments", "gitea_reopen_issue", "gitea_get_file", "gitea_list_branches", "gitea_delete_branch"]) {
assert.ok(tool(name), `${name} is not a tool`);
}
const closed = (await tool("gitea_close_pull_request").run({ owner: "novox", repo: "hq", number: 223 })) as { pull: { state: string } };
assert.equal(closed.pull.state, "closed");
assert.equal(forge.pullState, "closed");
const renamed = (await tool("gitea_update_pull_request").run({ owner: "novox", repo: "hq", number: 223, title: "Superseded" })) as { pull: { title: string } };
assert.equal(renamed.pull.title, "Superseded");
const diff = (await tool("gitea_pull_request_diff").run({ owner: "novox", repo: "hq", number: 223 })) as { diff: string };
assert.match(diff.diff, /^diff --git/);
const comments = (await tool("gitea_list_comments").run({ owner: "novox", repo: "hq", number: 223 })) as { comments: { body: string }[] };
assert.equal(comments.comments[0].body, "landed elsewhere");
const file = (await tool("gitea_get_file").run({ owner: "novox", repo: "hq", path: "README.md" })) as { file: { content: string } };
assert.equal(file.file.content, "hello");
const branches = (await tool("gitea_list_branches").run({ owner: "novox", repo: "hq" })) as { branches: { name: string }[] };
assert.deepEqual(branches.branches.map((b) => b.name), ["main", "feat/x"]);
await tool("gitea_delete_branch").run({ owner: "novox", repo: "hq", branch: "feat/x" });
assert.equal(forge.branchDeleted, true);
});
+125
View File
@@ -254,6 +254,131 @@ export function getGiteaTools(gitea: GiteaClient): ToolDefinition[] {
},
},
{
name: "gitea_close_pull_request",
description: "Close a pull request without merging it — one whose work landed elsewhere, or was abandoned.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
pull: await gitea.setPullState(String(args.owner), String(args.repo), Number(args.number), "closed"),
}),
},
{
name: "gitea_reopen_pull_request",
description: "Reopen a closed, unmerged pull request.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
pull: await gitea.setPullState(String(args.owner), String(args.repo), Number(args.number), "open"),
}),
},
{
name: "gitea_update_pull_request",
description: "Change a pull request's title or body; a field not given is left as it is.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
title: { type: "string", description: "the new title (optional)" },
body: { type: "string", description: "the new body, markdown (optional)" },
},
run: async (args) => ({
pull: await gitea.updatePullRequest(String(args.owner), String(args.repo), Number(args.number), {
title: args.title === undefined ? undefined : String(args.title),
body: args.body === undefined ? undefined : String(args.body),
}),
}),
},
{
name: "gitea_pull_request_files",
description: "The files a pull request changes, as paths from the repository's root (up to 100; says when there are more).",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => gitea.listPullFiles(String(args.owner), String(args.repo), Number(args.number)),
},
{
name: "gitea_pull_request_diff",
description: "A pull request's unified diff, as text — for reviewing it without a checkout.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the PR number" },
},
run: async (args) => ({
diff: await gitea.pullDiff(String(args.owner), String(args.repo), Number(args.number)),
}),
},
{
name: "gitea_list_comments",
description: "Every comment on an issue or pull request, oldest first.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the issue or PR number" },
},
run: async (args) => ({
comments: await gitea.listComments(String(args.owner), String(args.repo), Number(args.number)),
}),
},
{
name: "gitea_reopen_issue",
description: "Reopen a closed issue.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
number: { type: "number", description: "the issue number" },
},
run: async (args) => ({
issue: await gitea.setIssueState(String(args.owner), String(args.repo), Number(args.number), "open"),
}),
},
// ---- Contents and branches ----
{
name: "gitea_get_file",
description: "One file's contents from a repository, decoded, at a branch, tag or commit (default the repository's default branch).",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
path: { type: "string", description: "the file's path from the repository's root" },
ref: { type: "string", description: "branch, tag or commit (optional)" },
},
run: async (args) => ({
file: await gitea.getFile(String(args.owner), String(args.repo), String(args.path), args.ref ? String(args.ref) : undefined),
}),
},
{
name: "gitea_list_branches",
description: "Every branch of a repository with the commit it points at.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
},
run: async (args) => ({ branches: await gitea.listBranches(String(args.owner), String(args.repo)) }),
},
{
name: "gitea_delete_branch",
description: "Delete a branch — a feature branch whose pull request was closed rather than merged. Refused by the forge for a protected branch.",
input: {
owner: { type: "string", description: "the repository owner" },
repo: { type: "string", description: "the repository name" },
branch: { type: "string", description: "the branch name" },
},
run: async (args) => {
await gitea.deleteBranch(String(args.owner), String(args.repo), String(args.branch));
return { deleted: true, branch: String(args.branch) };
},
},
// ---- Labels ----
{
name: "gitea_list_labels",
-24
View File
@@ -1,24 +0,0 @@
# gitlab's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/gitlab
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/gitlab/dist /app/modules/gitlab/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/gitlab/dist/tools/index.js
+14 -40
View File
@@ -2,16 +2,9 @@
"module": "gitlab",
"version": "1",
"own-secrets": {
"token": "${dir:state}/token",
"broker": "${dir:mesh-state}/broker"
"token": "${dir:state}/token"
},
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -25,46 +18,27 @@
"merge": "json",
"content": "{}",
"mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-runtime-gitlab",
"network": "host",
"volumes": [
"${dir:state}/config.json:/run/config/config.json:ro",
"${dir:state}/token:/run/secrets/token:ro",
"${dir:mesh-state}/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_GITLAB_TOKEN_FILE": "/run/secrets/token",
"MESH_GITLAB_CONFIG_FILE": "/run/config/config.json",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"artifact": "runtime"
}
],
"capabilities": [
"container-runtime"
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_GITLAB_TOKEN_FILE": "${dir:state}/token",
"MESH_GITLAB_CONFIG_FILE": "${dir:state}/config.json"
}
}
]
}
+178
View File
@@ -0,0 +1,178 @@
// The hosts file's own code (novox/hq ADR 0199): read /etc/hosts as the machine has it, and change the
// operator's lines — every line outside a `# BEGIN … / # END …` block — leaving every block, the mesh's
// and any other tool's, byte for byte. The mesh writes this module's block; these verbs never touch it.
// Root is the module's concern (ADR 0175 §4): the runtime runs as the operator's account, so the file
// is written through sudo without a prompt where the account is not root, as the packet filter's is.
import { execFile } from "node:child_process";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { isIP } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
const execFileP = promisify(execFile);
/** Where the file is. The manifest's resource names the same path; a test holds the two together. */
export const HOSTS_FILE = "/etc/hosts";
/** A command runner, so the writes can be tested without a machine. */
export type Runner = (cmd: string, args: string[]) => Promise<string>;
export function escalated(cmd: string, args: string[], uid: number | undefined = process.getuid?.()): [string, string[]] {
if (uid === 0) return [cmd, args];
return ["sudo", ["-n", cmd, ...args]];
}
const run: Runner = async (cmd, args) => {
const [program, argv] = escalated(cmd, args);
const { stdout } = await execFileP(program, argv);
return stdout;
};
/** One line of the file, as a reader sees it. */
export interface Line {
/** The line exactly as it is in the file. */
text: string;
/** Whose it is: the block's id (`mesh hosts.own`, or another tool's) or "operator". */
owner: string;
/** For an entry: its address and names. Absent for a comment or blank line. */
address?: string;
names?: string[];
}
const BEGIN = /^#\s*BEGIN\s+(.+?)\s*$/;
const END = /^#\s*END\s+(.+?)\s*$/;
/** Every line of a hosts file, each marked whose it is. */
export function parse(text: string): Line[] {
const out: Line[] = [];
let block: string | null = null;
for (const raw of text.split("\n")) {
const begin = raw.match(BEGIN);
if (!block && begin) {
block = begin[1];
out.push({ text: raw, owner: block });
continue;
}
const owner = block ?? "operator";
const entry = raw.replace(/#.*/, "").trim().split(/\s+/).filter(Boolean);
const line: Line = { text: raw, owner };
if (entry.length >= 2 && isIP(entry[0])) {
line.address = entry[0];
line.names = entry.slice(1);
}
out.push(line);
const end = raw.match(END);
if (block && end && end[1] === block) block = null;
}
// A trailing newline splits into one empty last element; it is the file's ending, not a line.
if (out.length > 0 && out[out.length - 1].text === "" && text.endsWith("\n")) out.pop();
return out;
}
const NAME = /^(?=.{1,253}$)[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*\.?$/;
/** Refused input says why, so a caller is one edit from right. */
function checkAddress(address: string): void {
if (!isIP(address)) throw new Error(`${JSON.stringify(address)} is not an IPv4 or IPv6 address`);
}
function checkName(name: string): void {
if (!NAME.test(name)) throw new Error(`${JSON.stringify(name)} is not a host name`);
}
/** The file with one address and its names added to the operator's lines; unchanged when already there. */
export function withAdded(text: string, address: string, names: string[]): string {
checkAddress(address);
if (names.length === 0) throw new Error("add names at least one name for the address");
names.forEach(checkName);
const lines = parse(text);
const have = new Set(
lines.filter((l) => l.owner === "operator" && l.address === address).flatMap((l) => l.names ?? []),
);
const missing = names.filter((n) => !have.has(n));
if (missing.length === 0) return text;
const body = text.endsWith("\n") || text === "" ? text : text + "\n";
return body + `${address}\t${missing.join(" ")}\n`;
}
/** The file with one name, or every line of one address, taken out of the operator's lines. Blocks are
* never touched: a name only the mesh or another tool writes is refused, naming whose it is. */
export function withRemoved(text: string, what: string): { text: string; removed: number } {
const byAddress = isIP(what) !== 0;
if (!byAddress) checkName(what);
const lines = parse(text);
let removed = 0;
const kept: string[] = [];
for (const l of lines) {
if (l.owner !== "operator" || !l.address) {
kept.push(l.text);
continue;
}
if (byAddress && l.address === what) {
removed++;
continue;
}
if (!byAddress && l.names?.includes(what)) {
removed++;
const rest = l.names.filter((n) => n !== what);
if (rest.length > 0) kept.push(`${l.address}\t${rest.join(" ")}`);
continue;
}
kept.push(l.text);
}
if (removed === 0) {
const elsewhere = lines.find((l) => l.owner !== "operator" && (byAddress ? l.address === what : l.names?.includes(what)));
if (elsewhere) throw new Error(`${what} is written by ${elsewhere.owner}, not the operator; it is not this verb's to remove`);
}
return { text: kept.join("\n") + "\n", removed };
}
export class HostsFile {
private readonly path: string;
private readonly runner: Runner;
constructor(path: string = HOSTS_FILE, runner: Runner = run) {
this.path = path;
this.runner = runner;
}
static onThisMachine(): HostsFile {
return new HostsFile();
}
async read(): Promise<string> {
return readFile(this.path, "utf8");
}
async entries(): Promise<{ path: string; lines: Line[] }> {
return { path: this.path, lines: parse(await this.read()) };
}
async add(address: string, names: string[]): Promise<{ added: boolean; line?: string }> {
const before = await this.read();
const after = withAdded(before, address, names);
if (after === before) return { added: false };
await this.write(after);
return { added: true, line: after.slice(before.length).trim() };
}
async remove(what: string): Promise<{ removed: number }> {
const before = await this.read();
const { text, removed } = withRemoved(before, what);
if (removed > 0) await this.write(text);
return { removed };
}
/** Written whole through a copy beside it, so a reader never sees half a file. */
private async write(content: string): Promise<void> {
const dir = await mkdtemp(join(tmpdir(), "hosts-"));
const staged = join(dir, "hosts");
try {
await writeFile(staged, content, { mode: 0o644 });
await this.runner("install", ["-m", "0644", staged, this.path]);
} finally {
await rm(dir, { recursive: true, force: true });
}
}
}
+41
View File
@@ -0,0 +1,41 @@
{
"module": "hosts",
"version": "1",
"claims": [
{
"name": "node-hosts-file",
"scope": "node",
"serves": [
"entries",
"add",
"remove"
]
}
],
"resources": [
{
"id": "own",
"type": "file",
"path": "/etc/hosts",
"mode": "0644",
"into": "block",
"at": "start",
"content": "# The machine's own names (module hosts, novox/hq ADR 0199). Every line outside this block is the\n# operator's: kept across every push, changed through the node-hosts-file verbs add and remove, and\n# given back when this module goes. The mesh's names are not here: the mesh's resolver answers them.\n127.0.0.1\tlocalhost\n::1\tlocalhost\n"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
]
}
]
}
}
+18
View File
@@ -0,0 +1,18 @@
{
"name": "@novox/module-hosts",
"version": "0.1.0",
"description": "hosts — holds the node-hosts-file seat: writes the machine's own lines into /etc/hosts and serves the verbs entries, add and remove over the operator's lines (novox/hq ADR 0199).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
},
"scripts": {
"build": "tsc client.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . --outDir dist",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
}
}
+69
View File
@@ -0,0 +1,69 @@
// The hosts file's verbs over files shaped like the workstation's on 2026-10-03 (novox/hq ADR 0199):
// distribution lines, an operator's development names, the mesh's block and another tool's.
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { HOSTS_FILE, escalated, parse, withAdded, withRemoved } from "../client.ts";
const FILE =
"# Static table lookup for hostnames.\n" +
"127.0.0.1\tlocaldev.example.com\n" +
"127.0.0.1 a.example.com b.example.com\n" +
"# BEGIN mesh hosts.own\n" +
"127.0.0.1\tlocalhost\n" +
"::1\tlocalhost\n" +
"# END mesh hosts.own\n" +
"# BEGIN other-tool\n" +
"192.0.2.7\tproject.test\n" +
"# END other-tool\n";
const blocks = (text: string) => parse(text).filter((l) => l.owner !== "operator").map((l) => l.text);
test("every line says whose it is", () => {
const lines = parse(FILE);
assert.equal(lines.length, 10);
assert.deepEqual(lines[1], { text: "127.0.0.1\tlocaldev.example.com", owner: "operator", address: "127.0.0.1", names: ["localdev.example.com"] });
assert.equal(lines[4].owner, "mesh hosts.own");
assert.equal(lines[8].owner, "other-tool");
assert.deepEqual(lines[8].names, ["project.test"]);
});
test("add appends an operator line, and is a no-op when the names are there", () => {
const after = withAdded(FILE, "192.0.2.9", ["lab.test", "www.lab.test"]);
assert.ok(after.endsWith("192.0.2.9\tlab.test www.lab.test\n"));
assert.deepEqual(blocks(after), blocks(FILE));
assert.equal(withAdded(FILE, "127.0.0.1", ["a.example.com"]), FILE);
assert.ok(withAdded(FILE, "127.0.0.1", ["a.example.com", "c.example.com"]).endsWith("127.0.0.1\tc.example.com\n"));
});
test("add refuses what is not an address or a host name", () => {
assert.throws(() => withAdded(FILE, "not-an-ip", ["x.test"]), /not an IPv4 or IPv6 address/);
assert.throws(() => withAdded(FILE, "192.0.2.9", ["bad name\n10.0.0.1 evil"]), /not a host name/);
assert.throws(() => withAdded(FILE, "192.0.2.9", []), /at least one name/);
});
test("remove takes one name or one address from the operator's lines, and blocks stay byte for byte", () => {
const one = withRemoved(FILE, "a.example.com");
assert.equal(one.removed, 1);
assert.ok(one.text.includes("127.0.0.1\tb.example.com\n"));
assert.ok(!one.text.includes("a.example.com"));
assert.deepEqual(blocks(one.text), blocks(FILE));
const all = withRemoved(FILE, "127.0.0.1");
assert.equal(all.removed, 2);
assert.ok(all.text.includes("# BEGIN mesh hosts.own\n127.0.0.1\tlocalhost\n"), "the mesh's own localhost is not the operator's to remove");
});
test("remove refuses a name only a block writes, naming whose", () => {
assert.throws(() => withRemoved(FILE, "project.test"), /written by other-tool/);
assert.equal(withRemoved(FILE, "nowhere.test").removed, 0);
});
test("the file is written as root through sudo where the account is not root", () => {
assert.deepEqual(escalated("install", ["x"], 1000), ["sudo", ["-n", "install", "x"]]);
assert.deepEqual(escalated("install", ["x"], 0), ["install", ["x"]]);
});
test("the path the code writes is the path the manifest's resource declares", () => {
const manifest = JSON.parse(readFileSync(new URL("../module.json", import.meta.url), "utf8"));
assert.equal(manifest.resources.find((r: { id: string }) => r.id === "own").path, HOSTS_FILE);
});
+40
View File
@@ -0,0 +1,40 @@
// The hosts file's tools: the node-hosts-file seat's three verbs (novox/hq ADR 0199) — the file's lines
// with whose each is, add an operator's line, remove one. They change the machine's file and nothing
// else; the controller holds none of it.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { HostsFile } from "../client.js";
export function getSeatVerbs(hosts: HostsFile): ToolDefinition[] {
return [
{
name: "entries",
description:
"Every line of this machine's /etc/hosts, each marked whose it is: the operator's, or the block of the module or tool that writes it.",
input: {},
run: async () => hosts.entries(),
},
{
name: "add",
description:
"Add one address and its names to the operator's lines of this machine's /etc/hosts — a name for this machine's own programs, not the mesh's. Nothing changes when they are already there.",
input: {
address: { type: "string", description: "the IPv4 or IPv6 address" },
names: { type: "string", description: "the names for it, separated by spaces" },
},
run: async (args) =>
hosts.add(String(args.address ?? ""), String(args.names ?? "").split(/[\s,]+/).filter(Boolean)),
},
{
name: "remove",
description:
"Remove one name, or every line of one address, from the operator's lines of this machine's /etc/hosts. A line a module writes is refused, naming the module.",
input: { name: { type: "string", description: "a host name, or an address to remove every line of" } },
run: async (args) => hosts.remove(String(args.name ?? "")),
},
];
}
const hosts = HostsFile.onThisMachine();
// The seat's verbs under the seat's name: the runtime serves them as <node>/node-hosts-file.<verb>.
registerModuleTools("node-hosts-file", () => getSeatVerbs(hosts));
@@ -8,5 +8,8 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
"include": [
"client.ts",
"tools/index.ts"
]
}
-24
View File
@@ -1,24 +0,0 @@
# jira's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/jira
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/jira/dist /app/modules/jira/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/jira/dist/tools/index.js
+14 -40
View File
@@ -2,16 +2,9 @@
"module": "jira",
"version": "1",
"own-secrets": {
"token": "${dir:state}/token",
"broker": "${dir:mesh-state}/broker"
"token": "${dir:state}/token"
},
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -25,46 +18,27 @@
"merge": "json",
"content": "{}",
"mode": "0600"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-runtime-jira",
"network": "host",
"volumes": [
"${dir:state}/config.json:/run/config/config.json:ro",
"${dir:state}/token:/run/secrets/token:ro",
"${dir:mesh-state}/broker:/run/secrets/broker:ro"
],
"env": {
"MESH_JIRA_TOKEN_FILE": "/run/secrets/token",
"MESH_JIRA_CONFIG_FILE": "/run/config/config.json",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"artifact": "runtime"
}
],
"capabilities": [
"container-runtime"
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_JIRA_TOKEN_FILE": "${dir:state}/token",
"MESH_JIRA_CONFIG_FILE": "${dir:state}/config.json"
}
}
]
}
+31
View File
@@ -0,0 +1,31 @@
# lab's runtime: the tool runtime, carrying this module's code, and the toolchain the lab's suite
# builds the mesh with (novox/hq ADR 0172). It reaches the machine's virtualisation and container
# runtime through their sockets, so what it raises is what a hand run on this machine raises.
#
# Every download is pinned by its checksum: an image that builds the mesh is the last place to take
# whatever an upstream serves today.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/lab
COPY . .
RUN node /app/node_modules/typescript/bin/tsc tools/index.ts tools/runs.ts --rootDir . \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
RUN apt-get update \
&& apt-get install -y --no-install-recommends git make ca-certificates curl python3 file iproute2 sudo \
&& rm -rf /var/lib/apt/lists/*
RUN curl -fsSL -o /tmp/go.tgz https://go.dev/dl/go1.26.8.linux-amd64.tar.gz \
&& echo "d0f743b33e8d8945e6b1f432edd15785c70507121d6e2a723b21285eddf8b57b /tmp/go.tgz" | sha256sum -c - \
&& tar -C /usr/local -xzf /tmp/go.tgz && rm /tmp/go.tgz
RUN curl -fsSL -o /usr/local/bin/incus https://github.com/lxc/incus/releases/download/v7.5.1/bin.linux.incus.x86_64 \
&& echo "7bd6223b369f4d693fcde695bd8549a73b5b3d403735329212483702aa22c179 /usr/local/bin/incus" | sha256sum -c - \
&& chmod 0755 /usr/local/bin/incus
RUN curl -fsSL -o /tmp/docker.tgz https://download.docker.com/linux/static/stable/x86_64/docker-28.5.2.tgz \
&& echo "ea90cfd12e1eeb12aa1c971741adb8bd4ed88e2a574eaac13f5029a1dbc6300d /tmp/docker.tgz" | sha256sum -c - \
&& tar -C /tmp -xzf /tmp/docker.tgz docker/docker && mv /tmp/docker/docker /usr/local/bin/docker && rm -rf /tmp/docker /tmp/docker.tgz
ENV PATH=/usr/local/go/bin:$PATH
COPY --from=build /app/modules/lab/dist /app/modules/lab/dist
ENV MESH_TOOL_MODULES=/app/modules/lab/dist/tools/index.js
+82
View File
@@ -0,0 +1,82 @@
{
"module": "lab",
"version": "1",
"capabilities": [
"container-runtime",
"virtualisation"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "work",
"type": "directory",
"path": "/var/lib/mesh-lab-runs",
"mode": "0700"
},
{
"id": "runtime-env",
"type": "file",
"path": "${dir:state}/lab.env",
"mode": "0600",
"content": "MESH_LAB_FORGE=${setting:forge}\n"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-lab",
"network": "host",
"env-file": [
"${dir:state}/lab.env"
],
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:work}:${dir:work}",
"/var/run/docker.sock:/var/run/docker.sock",
"/var/lib/incus/unix.socket:/var/lib/incus/unix.socket"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_LAB_WORK": "${dir:work}"
},
"restart-on": [
"runtime-env"
],
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
+9
View File
@@ -0,0 +1,9 @@
{
"name": "@novox/module-lab",
"version": "0.1.0",
"description": "lab — the lab, as a module: runs beds against the forge's branches when the mesh asks (novox/hq ADR 0172).",
"type": "module",
"private": true,
"dependencies": { "@novox/mesh-sdk": "^0.1.0" },
"devDependencies": { "@types/node": "^22.0.0", "typescript": "^5.6.0" }
}
+86
View File
@@ -0,0 +1,86 @@
// lab's tools — the lab, as the mesh asks for it (novox/hq ADR 0172). They run on the machine the
// lab is assigned to, and only there: a bed raises virtual machines on that machine's virtualisation.
import { spawnSync } from "node:child_process";
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { listRuns, readStatus, REPOSITORIES, running, start, stop, tail } from "./runs.js";
export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
const work = env.MESH_LAB_WORK ?? "/var/lib/mesh-lab-runs";
const forge = (env.MESH_LAB_FORGE ?? "").replace(/\/+$/, "");
return [
{
name: "lab_check",
description: "Whether this machine can run the lab's beds: the lab's own check, against the forge's main branch.",
input: {},
run: async () => {
if (!forge) return { ok: false, output: "the lab's forge is not set: settings for lab, {\"forge\": \"<url>\"}" };
const dir = `${work}/check`;
spawnSync("rm", ["-rf", dir]);
const clone = spawnSync("git", ["clone", "--quiet", "--depth", "1", `${forge}/novox/mesh-lab.git`, dir], { encoding: "utf8" });
if (clone.status !== 0) return { ok: false, output: clone.stderr };
spawnSync("npm", ["ci", "--no-audit", "--no-fund", "--loglevel=error"], { cwd: dir, encoding: "utf8" });
const check = spawnSync("node", ["--experimental-strip-types", "src/cli.ts", "check"], { cwd: dir, encoding: "utf8" });
return { ok: check.status === 0, output: `${check.stdout}${check.stderr}`.trim() };
},
},
{
name: "lab_run",
description:
"Run the lab's beds against branches on the forge: fresh checkouts of every repository the lab builds, " +
"side by side, then the suite on the named test files. Answers at once with the run's id; lab_status " +
"and lab_log follow it. One run at a time.",
input: {
tests: { type: "string", description: "the bed test files, comma-separated, relative to mesh-lab (e.g. test/integration/mesh.test.ts)" },
refs: {
type: "string",
description: `a JSON object of repository to branch, for any of ${REPOSITORIES.join(", ")}; the rest run main`,
},
},
run: async (args) => {
if (!forge) return { started: false, reason: "the lab's forge is not set: settings for lab, {\"forge\": \"<url>\"}" };
const tests = String(args.tests ?? "").split(",").map((s) => s.trim()).filter(Boolean);
if (tests.length === 0) return { started: false, reason: "name at least one bed test file" };
let refs: Record<string, string> = {};
if (args.refs) {
try {
refs = JSON.parse(String(args.refs)) as Record<string, string>;
} catch {
return { started: false, reason: "refs is not a JSON object of repository to branch" };
}
}
const stranger = Object.keys(refs).filter((r) => !REPOSITORIES.includes(r));
if (stranger.length > 0) return { started: false, reason: `the lab does not build ${stranger.join(", ")}` };
const busy = running(work);
if (busy) return { started: false, reason: `${busy.id} is still ${busy.state}; one run at a time`, running: busy };
return { started: true, run: start(work, forge, tests, refs) };
},
},
{
name: "lab_status",
description: "A run's state, the commits it tested and how it ended — or every run, newest first, when no id is given.",
input: { id: { type: "string", description: "the run's id (optional)" } },
run: async (args) => {
if (args.id) return readStatus(work, String(args.id)) ?? { found: false, id: String(args.id) };
return { runs: listRuns(work).slice(0, 10) };
},
},
{
name: "lab_log",
description: "The last lines of a run's log.",
input: {
id: { type: "string", description: "the run's id" },
lines: { type: "number", description: "how many lines from the end (default 200)" },
},
run: async (args) => ({ id: String(args.id), log: tail(work, String(args.id), Number(args.lines ?? 200)) }),
},
{
name: "lab_stop",
description: "Stop a run and everything it started.",
input: { id: { type: "string", description: "the run's id" } },
run: async (args) => stop(work, String(args.id)) ?? { found: false, id: String(args.id) },
},
];
}
registerModuleTools("lab", (env) => getLabTools(env));
+175
View File
@@ -0,0 +1,175 @@
// A lab run: fresh checkouts of the named branches, side by side, then the lab's suite on the named
// beds (novox/hq ADR 0172).
//
// **A run is a detached script with its own process group**, so it outlives the tool call that started
// it and `stop` ends everything it started. It writes what it is doing to a status file beside its log,
// and that file is the whole of what the tools read back: a runtime that restarts mid-run still answers
// for it, and says it was lost rather than pretending it is still going.
import { spawn } from "node:child_process";
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
import { join } from "node:path";
/** The repositories a lab run checks out, side by side, as the lab expects its siblings. */
export const REPOSITORIES = ["mesh-lab", "mesh-controller", "mesh-host", "mesh-catalog", "mesh-tools", "mesh-sdk"];
export interface RunStatus {
id: string;
state: "checking-out" | "building" | "running" | "passed" | "failed" | "stopped" | "lost";
started: string;
ended?: string;
tests: string[];
refs: Record<string, string>;
commits?: Record<string, string>;
exit?: number;
pid?: number;
}
export function runDir(work: string, id: string): string {
return join(work, id);
}
function statusPath(work: string, id: string): string {
return join(runDir(work, id), "status.json");
}
export function readStatus(work: string, id: string): RunStatus | undefined {
try {
const s = JSON.parse(readFileSync(statusPath(work, id), "utf8")) as RunStatus;
// A run whose process is gone while its status still says it is going was lost — the runtime or
// the machine restarted under it. Said, rather than left reading as running for ever.
if (!["passed", "failed", "stopped", "lost"].includes(s.state) && s.pid && !alive(s.pid)) {
s.state = "lost";
}
return s;
} catch {
return undefined;
}
}
function alive(pid: number): boolean {
try {
process.kill(pid, 0);
return true;
} catch {
return false;
}
}
export function listRuns(work: string): RunStatus[] {
if (!existsSync(work)) return [];
return readdirSync(work)
.filter((d) => d.startsWith("run-"))
.map((id) => readStatus(work, id))
.filter((s): s is RunStatus => !!s)
.sort((a, b) => b.started.localeCompare(a.started));
}
/** The run still going, if any: one at a time, because two would contend for the same machine. */
export function running(work: string): RunStatus | undefined {
return listRuns(work).find((s) => !["passed", "failed", "stopped", "lost"].includes(s.state));
}
const shellQuote = (s: string) => `'${s.replace(/'/g, `'\\''`)}'`;
/**
* The script one run executes. Every step writes its state first, so a run that dies says where.
*
* The environment is the one the lab's README describes for a run against sibling checkouts, pointed
* at this run's own tree, so what is built and claimed is exactly what was checked out.
*/
export function script(work: string, id: string, forge: string, tests: string[], refs: Record<string, string>): string {
const dir = runDir(work, id);
const setState = (state: string) =>
`node -e ${shellQuote(
`const f=${JSON.stringify(join(dir, "status.json"))};const s=JSON.parse(require("fs").readFileSync(f,"utf8"));s.state=${JSON.stringify(state)};require("fs").writeFileSync(f,JSON.stringify(s,null,2))`,
)}`;
const clones = REPOSITORIES.map((repo) => {
const ref = refs[repo] ?? "main";
return [
`git clone --quiet --depth 50 --branch ${shellQuote(ref)} ${shellQuote(`${forge}/novox/${repo}.git`)} ${shellQuote(join(dir, repo))}`,
`echo "${repo} $(git -C ${shellQuote(join(dir, repo))} rev-parse HEAD)" >> ${shellQuote(join(dir, "commits.txt"))}`,
].join("\n");
}).join("\n");
const bin = join(dir, "bin");
return `set -euo pipefail
cd ${shellQuote(dir)}
${setState("checking-out")}
${clones}
node -e ${shellQuote(
`const fs=require("fs");const f=${JSON.stringify(join(dir, "status.json"))};const s=JSON.parse(fs.readFileSync(f,"utf8"));s.commits=Object.fromEntries(fs.readFileSync(${JSON.stringify(join(dir, "commits.txt"))},"utf8").trim().split("\\n").map(l=>l.split(" ")));fs.writeFileSync(f,JSON.stringify(s,null,2))`,
)}
${setState("building")}
# The @novox scope resolves from the mesh's own package registry on the forge, as the build machine
# resolves it; nothing else is asked of it.
printf '%s\n' ${shellQuote(`@novox:registry=${forge}/api/packages/novox/npm/`)} > ${shellQuote(join(dir, ".npmrc"))}
export NPM_CONFIG_USERCONFIG=${shellQuote(join(dir, ".npmrc"))}
for repo in mesh-sdk mesh-tools mesh-lab; do (cd ${shellQuote(dir)}/$repo && npm ci --no-audit --no-fund --loglevel=error); done
(cd ${shellQuote(dir)}/mesh-sdk && npm run build --if-present)
(cd ${shellQuote(dir)}/mesh-tools && npm run build --if-present)
mkdir -p ${shellQuote(bin)}
for p in postgres-provisioner objectstore-provisioner route-proxy; do
(cd ${shellQuote(dir)}/mesh-controller && CGO_ENABLED=0 go build -o ${shellQuote(bin)}/$p ./examples/$p)
done
export MESH_LAB_HOST_BINARY=${shellQuote(join(dir, "mesh-host", "mesh-host"))}
export MESH_LAB_BUNDLE=${shellQuote(join(dir, "mesh-host", "examples", "foundation-first-node-nats.lock"))}
export MESH_LAB_MODULES=${shellQuote(join(dir, "mesh-controller", "examples", "modules"))}
export MESH_LAB_BUILDER=${shellQuote(join(dir, "mesh-controller", "build", "mesh-builder"))}
export MESH_LAB_BOOTSTRAP_BINARY=${shellQuote(join(dir, "mesh-host", "mesh-bootstrap"))}
export MESH_LAB_CATALOG=${shellQuote(join(dir, "mesh-catalog", "modules"))}
export MESH_LAB_PROVISIONER=${shellQuote(join(bin, "postgres-provisioner"))}
export MESH_LAB_OBJECTSTORE_PROVISIONER=${shellQuote(join(bin, "objectstore-provisioner"))}
export MESH_LAB_ROUTE_PROXY=${shellQuote(join(bin, "route-proxy"))}
${setState("running")}
cd ${shellQuote(join(dir, "mesh-lab"))}
node --experimental-strip-types src/cli.ts suite ${tests.map(shellQuote).join(" ")}
`;
}
/** start begins a run and returns at once with its status. */
export function start(work: string, forge: string, tests: string[], refs: Record<string, string>): RunStatus {
const id = `run-${new Date().toISOString().replace(/[:.]/g, "-")}`;
const dir = runDir(work, id);
mkdirSync(dir, { recursive: true });
const status: RunStatus = { id, state: "checking-out", started: new Date().toISOString(), tests, refs };
writeFileSync(statusPath(work, id), JSON.stringify(status, null, 2));
writeFileSync(join(dir, "run.sh"), script(work, id, forge, tests, refs), { mode: 0o700 });
// The wrapper records how the run ended, then removes the checkouts and keeps the log and status: a
// run's tree is its own, and the next run starts from fresh ones (novox/hq ADR 0172).
const wrapper = `bash ${shellQuote(join(dir, "run.sh"))} > ${shellQuote(join(dir, "run.log"))} 2>&1; code=$?
node -e ${shellQuote(
`const f=${JSON.stringify(statusPath(work, id))};const s=JSON.parse(require("fs").readFileSync(f,"utf8"));if(s.state!=="stopped"){s.state=process.argv[1]==="0"?"passed":"failed"};s.exit=Number(process.argv[1]);s.ended=new Date().toISOString();require("fs").writeFileSync(f,JSON.stringify(s,null,2))`,
)} "$code"
cd ${shellQuote(dir)} && rm -rf ${REPOSITORIES.map(shellQuote).join(" ")} bin`;
const child = spawn("bash", ["-c", wrapper], { detached: true, stdio: "ignore" });
child.unref();
status.pid = child.pid;
writeFileSync(statusPath(work, id), JSON.stringify(status, null, 2));
return status;
}
/** stop ends a run and everything it started, by its process group. */
export function stop(work: string, id: string): RunStatus | undefined {
const s = readStatus(work, id);
if (!s || !s.pid) return s;
if (["passed", "failed", "stopped", "lost"].includes(s.state)) return s;
s.state = "stopped";
writeFileSync(statusPath(work, id), JSON.stringify(s, null, 2));
try {
process.kill(-s.pid, "SIGTERM");
} catch {
// Already gone between the read and the kill.
}
return s;
}
/** tail is the last lines of a run's log. */
export function tail(work: string, id: string, lines: number): string {
try {
const all = readFileSync(join(runDir(work, id), "run.log"), "utf8").split("\n");
return all.slice(-Math.max(1, lines)).join("\n");
} catch {
return "";
}
}
@@ -8,5 +8,5 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "tools/index.ts"]
"include": ["tools/index.ts", "tools/runs.ts"]
}
-24
View File
@@ -1,24 +0,0 @@
# letta's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/letta
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/letta/dist /app/modules/letta/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/letta/dist/tools/index.js
+14 -36
View File
@@ -26,8 +26,7 @@
},
"own-secrets": {
"server-password": "${dir:state}/server-password.secret",
"openai-api-key": "${dir:state}/openai-api-key.secret",
"broker": "${dir:mesh-state}/broker"
"openai-api-key": "${dir:state}/openai-api-key.secret"
},
"listens": [
{
@@ -84,45 +83,24 @@
"mode": "0600",
"content": "{\n \"password\": \"${secret:server-password}\"\n}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-letta",
"network": "letta",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/config.json:/run/config/config.json:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_LETTA_URL": "http://letta:8283",
"MESH_LETTA_CONFIG_FILE": "/run/config/config.json"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"loads": [
"tools/index.js"
],
"env": {
"MESH_LETTA_URL": "http://127.0.0.1:${port:8283}",
"MESH_LETTA_CONFIG_FILE": "${dir:mesh-state}/config.json"
}
}
]
}
-24
View File
@@ -1,24 +0,0 @@
# lidarr's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/lidarr
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/lidarr/dist /app/modules/lidarr/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/lidarr/dist/index.js,/app/modules/lidarr/dist/tools/index.js
-144
View File
@@ -1,144 +0,0 @@
// The Lidarr API client — lidarr's own code, living in the module (novox/hq ADR 0039). Ported from
// the shared hal `arr` client, but self-contained: in nox each Servarr app owns its own copy, so a
// change to Lidarr's API rebuilds only lidarr and nothing else. Both this module's tools and its
// events entrypoint import it, and nothing outside lidarr does.
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
// Lidarr speaks the v1 API (Radarr/Sonarr are v3); its content is the "artist".
const API_VERSION = "v1";
const CONTENT_ENDPOINT = "artist";
const APP_NAME = "Lidarr";
export interface LidarrQueueItem {
/** The queue record id — stable while the item is in the queue, so events can diff on it. */
id: number;
title: string;
status: string;
size: string;
sizeleft: string;
timeleft?: string;
}
export interface LidarrCalendarItem {
title: string;
date: string;
overview?: string;
}
export interface LidarrContentItem {
title: string;
status?: string;
monitored: boolean;
}
export class LidarrClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/**
* Build from the module's resolved environment. The URL defaults to the server on this node (the
* runtime shares its network), and the API key is read from MESH_LIDARR_API_KEY or, failing that,
* discovered from the server's own config.xml under MESH_LIDARR_CONFIG_DIR — the same file Lidarr
* writes it to, so a running server needs nothing configured by hand. Throws when no key can be
* found, so the tools/events simply do not load (the harness treats the throw as "exposes
* nothing").
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): LidarrClient {
const url = env.MESH_LIDARR_URL ?? `http://127.0.0.1:${env.MESH_LIDARR_PORT ?? "8686"}`;
const configDir = env.MESH_LIDARR_CONFIG_DIR ?? "/config";
const apiKey = env.MESH_LIDARR_API_KEY ?? LidarrClient.detectApiKey(configDir);
if (!apiKey) {
throw new Error("Lidarr not configured — set MESH_LIDARR_API_KEY or make the config dir readable");
}
return new LidarrClient(url, apiKey);
}
/** Discover the API key from the server's config.xml, falling back to null. Every Servarr app
* writes <ApiKey> into config.xml at the root of its config directory. */
static detectApiKey(configDir: string): string | null {
const config = join(configDir, "config.xml");
if (existsSync(config)) {
const match = readFileSync(config, "utf8").match(/<ApiKey>([^<]+)<\/ApiKey>/);
if (match) return match[1];
}
return null;
}
private async get(endpoint: string, params?: Record<string, string>): Promise<unknown> {
const url = new URL(`${this.baseUrl}/api/${API_VERSION}/${endpoint}`);
if (params) {
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
}
const res = await fetch(url.toString(), { headers: { "X-Api-Key": this.apiKey } });
if (!res.ok) throw new Error(`${APP_NAME} API /${endpoint}: ${res.status} ${await res.text()}`);
return res.json();
}
async getStatus(): Promise<{ appName: string; version: string }> {
const data = (await this.get("system/status")) as { appName?: string; version?: string };
return { appName: data.appName || APP_NAME, version: data.version ?? "unknown" };
}
async getContent(limit?: number): Promise<LidarrContentItem[]> {
const data = await this.get(CONTENT_ENDPOINT);
const items: any[] = Array.isArray(data) ? data : ((data as any)?.records ?? []);
const mapped = items.map((item) => ({
// Lidarr's content is an artist; its display name is artistName, not title.
title: item.artistName ?? item.title ?? "Unknown",
status: item.status,
monitored: item.monitored ?? true,
}));
return limit ? mapped.slice(0, limit) : mapped;
}
/** Library search is a filter over existing content, not an indexer lookup — same as hal's. */
async searchContent(term: string): Promise<LidarrContentItem[]> {
const all = await this.getContent();
const lower = term.toLowerCase();
return all.filter((item) => item.title.toLowerCase().includes(lower));
}
async getQueue(): Promise<{ totalRecords: number; items: LidarrQueueItem[] }> {
const data = (await this.get("queue", { pageSize: "50" })) as { totalRecords?: number; records?: any[] };
const records = data.records ?? [];
return {
totalRecords: data.totalRecords ?? records.length,
items: records.map((r) => ({
id: r.id,
title: r.title ?? r.artist?.artistName ?? r.album?.title ?? "Unknown",
status: r.status ?? "unknown",
size: formatBytes(r.size ?? 0),
sizeleft: formatBytes(r.sizeleft ?? 0),
timeleft: r.timeleft,
})),
};
}
async getCalendar(days = 7): Promise<LidarrCalendarItem[]> {
const start = new Date().toISOString().split("T")[0];
const end = new Date(Date.now() + days * 86400000).toISOString().split("T")[0];
const data = await this.get("calendar", { start, end });
const items: any[] = Array.isArray(data) ? data : [];
return items.map((item) => ({
// A Lidarr calendar entry is an album release.
title: item.title ?? item.artist?.artistName ?? "Unknown",
date: item.releaseDate ?? "",
overview: item.overview?.slice(0, 150),
}));
}
}
function formatBytes(bytes: number): string {
if (bytes === 0) return "0 B";
const units = ["B", "KB", "MB", "GB", "TB"];
const i = Math.floor(Math.log(bytes) / Math.log(1024));
return `${(bytes / Math.pow(1024, i)).toFixed(1)} ${units[i]}`;
}
-69
View File
@@ -1,69 +0,0 @@
// lidarr's events. The tool runtime imports this once the broker is bound. It watches the download
// queue and turns its comings and goings into mesh events.
//
// Emits (novox/hq ADR 0041/0042):
// module.lidarr.album.grabbed — a release entered the queue (Lidarr grabbed it)
// module.lidarr.download.completed — a release left the queue, imported. The download.completed
// routing key matches what a media consumer subscribes to
// (module.*.download.completed) to rescan its library.
// Consumes: none.
//
// The queue is polled and diffed, primed silently on the first look (like plex's index.ts) so a
// restart mid-download does not re-announce everything already in flight as freshly grabbed.
import { emit } from "@novox/mesh-sdk/events";
import { LidarrClient, type LidarrQueueItem } from "./client.js";
// Building the client throws when Lidarr has no URL/key yet. Like the tools (see tools/index.ts),
// the events entrypoint must not crash the runtime for that — it stays idle until configured.
function buildClient(): LidarrClient | null {
try {
return LidarrClient.fromEnv();
} catch {
return null;
}
}
const lidarr = buildClient();
// Lidarr removes an item from the queue once it has been imported; a "warning"/"failed" status is
// how a stuck or broken grab shows itself, so we do not call those a completion when they vanish.
const FAILED_STATUSES = new Set(["failed", "warning"]);
const inQueue = new Map<number, LidarrQueueItem>();
let primed = false;
async function pollQueue(lidarr: LidarrClient): Promise<void> {
const { items } = await lidarr.getQueue();
const now = new Map(items.map((i) => [i.id, i]));
if (primed) {
// Entered the queue since last look — Lidarr grabbed a release.
for (const [id, item] of now) {
if (!inQueue.has(id)) await emit("album.grabbed", { title: item.title, status: item.status });
}
// Left the queue — imported and done, unless it was last seen failing.
for (const [id, item] of inQueue) {
if (!now.has(id) && !FAILED_STATUSES.has(item.status)) {
await emit("download.completed", { title: item.title });
}
}
}
inQueue.clear();
for (const [id, item] of now) inQueue.set(id, item);
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[lidarr] ${err}`));
setInterval(run, everyMs);
run();
};
if (lidarr) {
tick(() => pollQueue(lidarr), 30_000);
console.log("[lidarr] watching the download queue, emitting grabs and completions");
} else {
console.log("[lidarr] not configured — events idle until an API key is available");
}
-119
View File
@@ -1,119 +0,0 @@
{
"module": "lidarr",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"album.grabbed",
"download.completed"
],
"consumes": [],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "web",
"port": 8686,
"protocol": "tcp",
"from": "mesh",
"why": "managing music"
}
],
"accesses": [
{
"id": "music",
"path": "/services/media/music",
"mode": "read-write"
},
{
"id": "downloads",
"path": "/services/media/downloads",
"mode": "read-write"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/lidarr/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "lidarr",
"image": "lscr.io/linuxserver/lidarr@sha256:6b38dd330b0c653351c2e23c8b962ea51c95683dd7acace9d106c922baf85f75",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"8686"
],
"volumes": [
"/services/lidarr/config:/config",
"${access:music}:/music",
"${access:downloads}:/downloads"
]
},
{
"id": "runtime",
"type": "container",
"name": "mesh-lidarr",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"/services/lidarr/config:/var/lib/lidarr/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_LIDARR_URL": "http://127.0.0.1:8686",
"MESH_LIDARR_CONFIG_DIR": "/var/lib/lidarr/config"
},
"artifact": "runtime"
}
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "lidarr",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:mesh-state}/route.json"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-lidarr",
"version": "0.1.0",
"description": "lidarr — music management. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-78
View File
@@ -1,78 +0,0 @@
// lidarr's tools — ported from the shared hal sdk (novox/hq ADR 0039), importing lidarr's own
// client. They return structured data (not pre-formatted text as hal did); the mesh serves them
// through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { LidarrClient } from "../client.js";
export function getLidarrTools(lidarr: LidarrClient): ToolDefinition[] {
return [
{
name: "lidarr_status",
description: "Lidarr status overview: version, artist count, monitored count, queue size.",
input: {},
run: async () => {
const [status, content, queue] = await Promise.all([
lidarr.getStatus(),
lidarr.getContent(),
lidarr.getQueue(),
]);
return {
app: status.appName,
version: status.version,
artists: content.length,
monitored: content.filter((c) => c.monitored).length,
queue: queue.totalRecords,
};
},
},
{
name: "lidarr_library",
description: "List artists from the Lidarr library.",
input: { limit: { type: "number", description: "max items to return (default 50)" } },
run: async (args) => {
const items = await lidarr.getContent(args.limit ? Number(args.limit) : 50);
return { count: items.length, artists: items };
},
},
{
name: "lidarr_search",
description: "Search the Lidarr library for artists by name (filters existing content, not indexers).",
input: { query: { type: "string", description: "the search term" } },
run: async (args) => {
const query = String(args.query);
return { query, results: await lidarr.searchContent(query) };
},
},
{
name: "lidarr_queue",
description: "Show the Lidarr download queue — what is downloading and how far along.",
input: {},
run: async () => {
const queue = await lidarr.getQueue();
return { count: queue.totalRecords, items: queue.items };
},
},
{
name: "lidarr_calendar",
description: "Upcoming album releases from the Lidarr calendar.",
input: { days: { type: "number", description: "how many days to look ahead (default 7)" } },
run: async (args) => {
const days = args.days ? Number(args.days) : 7;
const items = await lidarr.getCalendar(days);
items.sort((a, b) => a.date.localeCompare(b.date));
return { days, count: items.length, items };
},
},
];
}
// The tools exist only when Lidarr is configured; without a URL and key, lidarr contributes none
// rather than failing the whole runtime.
registerModuleTools("lidarr", (env) => {
try {
return getLidarrTools(LidarrClient.fromEnv(env));
} catch {
return [];
}
});
+10 -2
View File
@@ -462,7 +462,8 @@
],
"dns": [
"192.168.203.254"
]
],
"logging": "journald"
},
{
"id": "runtime-config",
@@ -561,5 +562,12 @@
},
"grants": {
"smtp": "${dir:grants}"
}
},
"jails": [
{
"name": "mailu-front",
"failregex": "^.*(?:imap|pop3|submission|managesieve)-login: .*\\(auth failed, \\d+ attempts(?: in \\d+ secs)?\\):.*rip=<HOST>(?:,|$)",
"jail": "backend = systemd\njournalmatch = CONTAINER_NAME=mailu-front\nport = smtp,submission,submissions,imap,imaps,pop3,pop3s\nmaxretry = 3\nfindtime = 1d\nbantime = 1d"
}
]
}
-13
View File
@@ -1,13 +0,0 @@
# The console (novox/hq ADR 0152, design 34): the mesh's tools for whoever is on a machine, served
# over MCP on that machine's loopback.
#
# **Nothing is compiled here.** The console is the tool runtime's own client — `mesh serve` — which
# the runtime image already carries beside the runtime it runs modules with. This recipe changes the
# program the image starts and nothing else, so the console is exactly the client a person can run by
# hand, started by the mesh instead, on the credential the mesh sealed to the machine.
#
# One base, named rather than pinned: the mesh answers with the copy it holds (novox/hq issue 044).
ARG RUNTIME_BASE
FROM ${RUNTIME_BASE}
ENTRYPOINT ["node", "dist/mesh.js"]
-38
View File
@@ -1,38 +0,0 @@
# mesh-console
The mesh's tools, on the machine a person sits at, served by a module the mesh assigned there
(novox/hq [ADR 0152](https://git.novox.be/novox/hq), design 34).
Assign it to a machine and an agent on that machine has the mesh's tools at
`http://127.0.0.1:<port>/mcp` — MCP over HTTP, `initialize`, `tools/list`, `tools/call`. A person at
a terminal reaches the same endpoint with `mesh tools --console http://127.0.0.1:<port>` and
`mesh call <module>.<tool> --console …`, with no credential of their own: the console holds it.
## What it is
The tool runtime's own client, `mesh serve`, started by the mesh on the credential it sealed to the
machine for `<node>.mesh-console`. The manifest says three things nothing else in the catalogue says
together:
- `invokes: ["*"]` — it calls every tool on the mesh, and the bus grants exactly that publish side;
- a listener `from: machine` — loopback only, and the filter opens nothing for it;
- no `emits`, no `consumes`, no `tools` — nothing on the bus can address it.
**Loopback is the authority boundary.** Whoever can connect is on the machine, and whoever is on the
machine is the account that owns the mesh there (ADR 0034, ADR 0144). There is no token and no login,
and `mesh serve` refuses to bind anything but a loopback address.
## What it lists
What the running modules answer: every tool runtime serves a `tools` verb for its module, and the
console asks the catalogue which modules the mesh holds and each module what it serves. A module that
did not answer — not assigned, not up, or built before the runtime answered `tools` — is named in the
list's `_meta.notAnswering` and can still be called by `<module>.<tool>`.
The mesh's own verbs (`status`, `push`, `assign`) are the `mesh-controller` seat's tools under
ADR 0132 and are not served on the bus yet; they appear here when they are.
## Port
The manifest declares port 4270 and the mesh assigns the machine port as it does for any listener;
the console binds `127.0.0.1:${port:4270}`. `node show <machine>` says which port a machine was given.
-64
View File
@@ -1,64 +0,0 @@
{
"module": "mesh-console",
"version": "1",
"slug": "console",
"capabilities": [
"container-runtime"
],
"invokes": [
"*"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"listens": [
{
"name": "mcp",
"port": 4270,
"protocol": "tcp",
"from": "machine",
"why": "the mesh's tools for whoever is on this machine, over MCP on loopback; the machine's login is the authority (novox/hq ADR 0152)"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "server",
"type": "container",
"name": "mesh-console",
"network": "host",
"args": [
"serve"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_CONSOLE_LISTEN": "127.0.0.1:${port:4270}"
},
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro"
],
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
+6 -6
View File
@@ -1,9 +1,9 @@
// mesh-vault's events entrypoint, loaded by the per-node tool host (the provisioner runs in the same
// process — ADR 0052). The lifecycle events are EMITTED from the provisioner, where custody
// actually changes (novox/hq ADR 0041/0042):
// module.mesh-vault.secret.provisioned — a consumer was granted a secret
// module.mesh-vault.secret.rotated — that consumer's value changed (`rotate secret`)
// module.mesh-vault.secret.deprovisioned — the consumer went away and its secret was withdrawn
// mesh-vault.provisioned — a consumer was granted a secret
// mesh-vault.rotated — that consumer's value changed (`rotate secret`)
// mesh-vault.deprovisioned — the consumer went away and its secret was withdrawn
// Here the vault reacts to them, keeping a lightweight audit line of who holds what and when it
// moved — the audit an owner of secrets is best placed to log. Fingerprints, never values.
@@ -16,15 +16,15 @@ interface SecretEvent {
rotations?: number;
}
await on<SecretEvent>("secret.provisioned", async (e) => {
await on<SecretEvent>("provisioned", async (e) => {
console.log(`[mesh-vault] secret provisioned for ${e.body.as} on ${e.body.consumer} (${e.body.fingerprint})`);
});
await on<SecretEvent>("secret.rotated", async (e) => {
await on<SecretEvent>("rotated", async (e) => {
console.log(`[mesh-vault] secret rotated for ${e.body.as} — rotation ${e.body.rotations} (${e.body.fingerprint})`);
});
await on<SecretEvent>("secret.deprovisioned", async (e) => {
await on<SecretEvent>("deprovisioned", async (e) => {
console.log(`[mesh-vault] secret withdrawn from ${e.body.as}`);
});
+13 -7
View File
@@ -11,14 +11,14 @@
"container-runtime"
],
"emits": [
"secret.provisioned",
"secret.rotated",
"secret.deprovisioned"
"provisioned",
"rotated",
"deprovisioned"
],
"consumes": [
"mesh-vault.secret.provisioned",
"mesh-vault.secret.rotated",
"mesh-vault.secret.deprovisioned"
"mesh-vault.provisioned",
"mesh-vault.rotated",
"mesh-vault.deprovisioned"
],
"receives": {
"secret": "${dir:grants}/mesh.json"
@@ -98,5 +98,11 @@
"from": "Dockerfile"
}
]
}
},
"claims": [
{
"name": "mesh-vault",
"scope": "mesh"
}
]
}
+1 -1
View File
@@ -43,6 +43,6 @@ runProvisioner("secret", {
async remove(p: { as: string }): Promise<void> {
if (!ledger.withdraw(p.as)) return;
console.log(`[mesh-vault] withdrawn: ${p.as}`);
await announce("secret.deprovisioned", { as: p.as });
await announce("deprovisioned", { as: p.as });
},
});
+13
View File
@@ -20,7 +20,20 @@ COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts provisioner/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
# **sqlcmd, which this module's client drives, has to be here** — it never was, so every tool failed
# with `spawn sqlcmd ENOENT`. go-sqlcmd is one static binary; fetched at a pinned release and checked
# against its digest, so a build that receives anything else stops here.
FROM ${BUILD_BASE} AS sqlcmd
ARG SQLCMD_VERSION=v1.10.0
ARG SQLCMD_SHA256=92516d98c63d99b0994de5b61350c91f6915f9b76f139a59039fbcb225c2e987
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates bzip2 \
&& curl -fsSL -o /tmp/sqlcmd.tar.bz2 \
"https://github.com/microsoft/go-sqlcmd/releases/download/${SQLCMD_VERSION}/sqlcmd-linux-amd64.tar.bz2" \
&& echo "${SQLCMD_SHA256} /tmp/sqlcmd.tar.bz2" | sha256sum -c - \
&& tar -xjf /tmp/sqlcmd.tar.bz2 -C /usr/local/bin sqlcmd
FROM ${RUNTIME_BASE}
COPY --from=sqlcmd /usr/local/bin/sqlcmd /usr/local/bin/sqlcmd
COPY --from=build /app/modules/mssql/dist /app/modules/mssql/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
+109 -12
View File
@@ -29,11 +29,40 @@ export interface MssqlConn {
readonly port: number;
readonly user: string;
readonly password: string;
/**
* The read-only login's password, which the mesh mints for this module (`own-secrets.reader`).
* Absent when the mesh has not delivered it: then a caller's statement is refused, never run as
* the administrator (novox/hq issue 193).
*/
readonly readerPassword?: string;
}
/**
* The login a caller's statement runs as (novox/hq issue 193). It may connect to every database and
* read every table, and holds no other permission. A statement cannot climb out of a login the way it
* could out of a transaction wrapped around it as text, and the administrator — who can run programs
* on the server — never runs a caller's text.
*/
export const READER = "mesh_mssql_reader";
/** Who a sqlcmd invocation logs in as, and whether the text is a caller's rather than the module's. */
interface Invocation {
readonly user: string;
readonly password: string;
/**
* A caller's text: sqlcmd substitutes no `$(NAME)` in it, which would read this process's
* environment — the administrator's password among it. (Its own commands are kept out by the
* caller's text never beginning a line; see readOnlyQuery.)
*/
readonly caller: boolean;
}
export class MssqlClient {
constructor(private readonly conn: MssqlConn) {}
/** The reader is made once per process: idempotent, and repeating it re-sets a rotated password. */
private readerReady?: Promise<void>;
/**
* Build from the module's resolved environment. Reads MESH_MSSQL_* first (the documented
* names), falling back to the MESH_PROVISION_* keys the manifest already sets on the provisioner
@@ -48,7 +77,9 @@ export class MssqlClient {
if (!host || !password) {
throw new Error("mssql host or admin password is not set — mssql's own code cannot reach the server");
}
return new MssqlClient({ host, port, user, password });
const readerPassword = env.MESH_MSSQL_READER_PASSWORD ??
readSecretFile(env.MESH_MSSQL_READER_PASSWORD_FILE);
return new MssqlClient({ host, port, user, password, readerPassword });
}
get host(): string {
@@ -86,7 +117,12 @@ export class MssqlClient {
}
/** The one execution boundary: invoke `sqlcmd` and return its concatenated stdout. */
private async sqlcmd(sql: string, database: string, variables: Record<string, string> = {}): Promise<string> {
private async sqlcmd(
sql: string,
database: string,
variables: Record<string, string> = {},
as: Invocation = { user: this.conn.user, password: this.conn.password, caller: false },
): Promise<string> {
// `-h -1` drops the column-header rule; `-y 0`/`-Y 0` lift the display-width cap so a long
// JSON document is not truncated; `-W` trims trailing whitespace so the JSON chunks rejoin
// cleanly. sqlcmd from the mssql-tools ships in the runtime container, the way `psql` ships
@@ -95,8 +131,9 @@ export class MssqlClient {
"sqlcmd",
[
"-S", `${this.conn.host},${this.conn.port}`,
"-U", this.conn.user,
"-U", as.user,
"-d", database,
...(as.caller ? ["-x"] : []),
"-C",
"-b",
"-h", "-1",
@@ -107,7 +144,7 @@ export class MssqlClient {
],
// `variables` reach sqlcmd as environment variables, which it substitutes as `$(NAME)` scripting
// variables: a value that must not appear on argv, or in the message of a failed command.
{ env: { ...process.env, ...variables, SQLCMDPASSWORD: this.conn.password }, maxBuffer: 16 << 20 },
{ env: { ...process.env, ...variables, SQLCMDPASSWORD: as.password }, maxBuffer: 16 << 20 },
);
return stdout;
}
@@ -225,16 +262,76 @@ export class MssqlClient {
}));
}
/** Run a read-only SELECT against a named database, for the mssql_query tool. */
async readOnlyQuery(database: string, sql: string): Promise<QueryResult> {
// The read-only guarantee is a wrapping transaction that is always rolled back: any write the
// statement attempts is undone. The rows are rendered by FOR JSON inside query().
const rows = await this.query(
`BEGIN TRANSACTION;\n${stripTrailingSemis(sql)}\nFOR JSON PATH, INCLUDE_NULL_VALUES;\nROLLBACK;`,
database,
/**
* Make the read-only login, idempotently, with the password the mesh minted for it: it may connect
* to every database and read every table, and is taken out of the administrators' role should
* anyone have put it there. Run as the administrator, because only it can make a login.
*/
async ensureReader(): Promise<void> {
const password = this.conn.readerPassword;
if (!password) throw readerMissing();
const logins = await this.query(
`SELECT 1 AS ok FROM sys.server_principals WHERE name = ${literal(READER)}`,
);
return { command: sql.trimStart().split(/\s+/)[0]?.toUpperCase() ?? "", rows };
if (logins.length === 0) {
await this.exec(
`CREATE LOGIN ${ident(READER)} WITH PASSWORD = ${literal(password)}, CHECK_POLICY = OFF`,
);
} else {
await this.exec(`ALTER LOGIN ${ident(READER)} WITH PASSWORD = ${literal(password)}`);
await this.exec(`ALTER LOGIN ${ident(READER)} ENABLE`);
}
await this.exec(
`IF IS_SRVROLEMEMBER('sysadmin', ${literal(READER)}) = 1 ` +
`ALTER SERVER ROLE sysadmin DROP MEMBER ${ident(READER)}`,
);
await this.exec(`GRANT CONNECT ANY DATABASE TO ${ident(READER)}`);
await this.exec(`GRANT SELECT ALL USER SECURABLES TO ${ident(READER)}`);
}
/**
* Run a caller's SELECT against a named database as the read-only login, for the mssql_query tool
* (novox/hq issue 193). Read-only by the login, not by a transaction wrapped around the text; the
* rows are rendered by FOR JSON. Never as the administrator: without the reader's password the call
* is refused.
*/
async readOnlyQuery(database: string, sql: string): Promise<QueryResult> {
const password = this.conn.readerPassword;
if (!password) throw readerMissing();
// **One line, refused otherwise.** sqlcmd reads a line that BEGINS with `:` or `!!` as its own
// command rather than SQL, and `:!!` starts a program in this container, which holds the
// administrator's password. Its switch for refusing those (-X) makes it ignore -Q in the
// version shipped here, so instead no line of a caller's text can begin one: the text follows
// this module's own on the first line, and a line break in it is refused. Proven on a throwaway
// server: the same text at the start of a line ran a program; mid-line it is a syntax error.
if (/[\r\n]/.test(sql)) {
throw new Error(
"mssql_query: the statement must be one line — sqlcmd takes a line beginning with ':' or " +
"'!!' as a command of its own, which can start a program (novox/hq issue 193)",
);
}
this.readerReady ??= this.ensureReader().catch((err) => {
this.readerReady = undefined; // asked again next call, not failed for the process's life
throw err;
});
await this.readerReady;
const stdout = await this.sqlcmd(
`SET NOCOUNT ON; ${stripTrailingSemis(sql)}\nFOR JSON PATH, INCLUDE_NULL_VALUES;`,
database,
{},
{ user: READER, password, caller: true },
);
const command = /^\s*([A-Za-z]+)/.exec(sql)?.[1]?.toUpperCase() ?? "";
return { command, rows: parseJsonRows(stdout) };
}
}
function readerMissing(): Error {
return new Error(
"the read-only login's password was not delivered (own-secrets.reader, " +
"MESH_MSSQL_READER_PASSWORD_FILE), so the statement is refused rather than run as the " +
"administrator (novox/hq issue 193)",
);
}
/** Generate a URL-safe password. */
+6 -3
View File
@@ -38,7 +38,8 @@
},
"own-secrets": {
"sa": "${dir:state}/sa.secret",
"broker": "${dir:mesh-state}/broker"
"broker": "${dir:mesh-state}/broker",
"reader": "${dir:state}/reader.secret"
},
"resources": [
{
@@ -101,13 +102,15 @@
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:grants}:/var/lib/mssql/grants:ro",
"${dir:state}/sa.secret:/run/secrets/sa:ro"
"${dir:state}/sa.secret:/run/secrets/sa:ro",
"${dir:state}/reader.secret:/run/secrets/reader:ro"
],
"env": {
"MESH_PROVISION_MSSQL": "mssql://sa@mssql:1433/master",
"MESH_PROVISION_PASSWORD_FILE": "/run/secrets/sa",
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_RECEIVES": "/var/lib/mssql/grants/mesh.json"
"MESH_RECEIVES": "/var/lib/mssql/grants/mesh.json",
"MESH_MSSQL_READER_PASSWORD_FILE": "/run/secrets/reader"
},
"artifact": "runtime"
}
+5 -1
View File
@@ -1,9 +1,13 @@
{
"name": "@novox/module-mssql",
"version": "0.1.0",
"description": "mssql — provides the mesh mssql-database interface. Its client, provisioner, tools and events live here (novox/hq ADR 0039).",
"description": "mssql \u2014 provides the mesh mssql-database interface. Its client, provisioner, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts provisioner/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
},
+96
View File
@@ -0,0 +1,96 @@
// What holds mssql_query to being read-only (novox/hq issue 193): a caller's statement runs as the
// reader login and never as the administrator, with sqlcmd's variable substitution off, on one line
// that follows the module's own — a line break is refused before sqlcmd starts — and with no
// transaction wrapped around it as text. Without the reader's password the statement is refused.
//
// sqlcmd is a fake on PATH that records each call's login, flags and text. That the reader cannot
// write is the server's to enforce and was proven against a real server; this holds the module to
// asking for it. Run against the compiled module (npm test builds first), the way the runtime loads it.
import { test, before, after } from "node:test";
import assert from "node:assert/strict";
import { chmod, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { MssqlClient, READER } from "../dist/client.js";
let dir: string;
let log: string;
const originalPath = process.env.PATH;
before(async () => {
dir = await mkdtemp(join(tmpdir(), "mssql-reader-"));
log = join(dir, "calls.jsonl");
await writeFile(join(dir, "sqlcmd"), `#!/usr/bin/env node
const fs = require("node:fs");
const args = process.argv.slice(2);
const at = (flag) => args[args.indexOf(flag) + 1];
fs.appendFileSync(${JSON.stringify(log)}, JSON.stringify({
user: at("-U"), database: at("-d"), sql: at("-Q"), noVariables: args.includes("-x"),
password: process.env.SQLCMDPASSWORD,
}) + "\\n");
const sql = at("-Q");
if (/FROM sys.server_principals/.test(sql)) process.stdout.write("");
else if (/FOR JSON/.test(sql)) process.stdout.write('[{"name":"alpha","n":1}]\\n');
`);
await chmod(join(dir, "sqlcmd"), 0o755);
process.env.PATH = `${dir}:${originalPath}`;
});
after(async () => {
process.env.PATH = originalPath;
await rm(dir, { recursive: true, force: true });
});
async function calls(): Promise<Record<string, unknown>[]> {
const text = await readFile(log, "utf8").catch(() => "");
await writeFile(log, "");
return text.split("\n").filter(Boolean).map((line) => JSON.parse(line));
}
const conn = { host: "127.0.0.1", port: 1433, user: "sa", password: "admin-secret" };
test("a caller's statement runs as the reader, without variables, on the module's first line", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
const result = await client.readOnlyQuery("inventory", "SELECT '$(SQLCMDPASSWORD)' AS p");
const asked = (await calls()).at(-1)!;
assert.equal(asked.user, READER, "the statement never runs as the administrator");
assert.equal(asked.password, "reader-secret");
assert.equal(asked.noVariables, true, "no $(NAME) is substituted in a caller's text");
const [first] = String(asked.sql).split("\n");
assert.ok(first.startsWith("SET NOCOUNT ON; SELECT '$(SQLCMDPASSWORD)'"), "the caller's text never begins a line");
assert.doesNotMatch(String(asked.sql), /BEGIN TRANSACTION|ROLLBACK/, "no transaction wrapped around it as text");
assert.deepEqual(result.rows, [{ name: "alpha", n: 1 }]);
assert.equal(result.command, "SELECT");
});
test("a line break in a caller's statement is refused before sqlcmd starts", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
for (const sql of ["SELECT 1\n:!! id", "SELECT 1\r\n:!! id", "SELECT 1\r:!! id"]) {
await assert.rejects(client.readOnlyQuery("inventory", sql), /must be one line/);
}
assert.deepEqual(await calls(), []);
});
test("the reader is made as the administrator, kept out of sysadmin, and granted only reading", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
await client.readOnlyQuery("inventory", "SELECT 1 AS x");
await client.readOnlyQuery("inventory", "SELECT 2 AS x");
const made = await calls();
const asAdmin = made.filter((c) => c.user === "sa").map((c) => String(c.sql));
assert.ok(asAdmin.some((s) => s.startsWith(`CREATE LOGIN [${READER}]`)));
assert.ok(asAdmin.some((s) => /ALTER SERVER ROLE sysadmin DROP MEMBER/.test(s)));
assert.ok(asAdmin.includes(`GRANT CONNECT ANY DATABASE TO [${READER}]`));
assert.ok(asAdmin.includes(`GRANT SELECT ALL USER SECURABLES TO [${READER}]`));
assert.equal(asAdmin.filter((s) => s.startsWith("CREATE LOGIN")).length, 1, "made once, not per call");
assert.equal(made.filter((c) => c.user === READER).length, 2);
});
test("without the reader's password the statement is refused, and nothing runs as the administrator", async () => {
const client = new MssqlClient(conn);
await assert.rejects(client.readOnlyQuery("inventory", "SELECT 1"), /refused rather than run as the administrator/);
assert.deepEqual(await calls(), []);
});
+1 -1
View File
@@ -16,7 +16,7 @@ export function getMssqlTools(mssql: MssqlClient): ToolDefinition[] {
},
{
name: "mssql_query",
description: "Run a read-only SELECT against a named database (wrapped in a rolled-back transaction).",
description: "Run a read-only SELECT against a named database, as a login that can read every table and change nothing.",
input: {
database: { type: "string", description: "the database to query" },
sql: { type: "string", description: "a single SELECT statement" },
+2 -2
View File
@@ -26,7 +26,7 @@
},
"accesses": [
{
"path": "/services/media",
"id": "media",
"mode": "read-write"
}
],
@@ -93,7 +93,7 @@
"volumes": [
"${dir:data}:/home/node/.n8n",
"${dir:state}/n8n-database.secret:/run/secrets/database:ro",
"/services/media:/media-library"
"${access:media}:/media-library"
]
},
{
+2 -1
View File
@@ -3,7 +3,8 @@
"version": "1",
"capabilities": [
"package-manager",
"service-manager"
"service-manager",
"uplink-networkmanager"
],
"claims": [
{
+257 -10
View File
@@ -1,22 +1,269 @@
// The firewall's own code, in the module (novox/hq ADR 0039). The mesh computes this node's whole
// rule set from every module's `listens` and writes it to /etc/nftables.conf (novox/hq ADR 0045);
// the module loads it through its own mesh-filter unit, reloaded whenever the rules change, whose
// stop deletes only the mesh's table and never flushes the whole ruleset (novox/hq ADR 0100). This
// code exists only to read back what is actually enforced — the enforcement itself is declarative.
// The packet filter's own code, in the module (novox/hq ADR 0039). The mesh computes this node's
// rule set from every module's `listens` and writes it to the filter file (ADR 0045); the module
// loads it through its own unit. This code reads the filter back as the machine enforces it, reloads
// the mesh's own table, and removes one thing the mesh did not write when the operator names it
// (ADR 0168, ADR 0170) — the seat's three verbs, over the machine's own tools. Root is the module's
// concern (ADR 0175 §4): the runtime loading this bundle runs as the operator's account (to-be 38
// WP4), so the commands go through sudo without a prompt where the account is not root.
import { execFile } from "node:child_process";
import { accessSync, constants } from "node:fs";
import { delimiter, join } from "node:path";
import { promisify } from "node:util";
const run = promisify(execFile);
const execFileP = promisify(execFile);
/** A command runner, so the acts can be tested without a packet filter. */
export type Runner = (cmd: string, args: string[]) => Promise<string>;
/** Where the mesh writes this node's filter: the path the manifest's `filtering.into` names. A
* bundle has no environment of its own (to-be 38 WP4), so the path is said here once, and a test
* holds it to the manifest's. */
export const FILTER_FILE = "/etc/nftables.conf";
/** The command as it is run: as given when this process is root, else through sudo without a
* prompt. The packet filter answers only to root, listing included. */
export function escalated(cmd: string, args: string[], uid: number | undefined = process.getuid?.()): [string, string[]] {
if (uid === 0) return [cmd, args];
return ["sudo", ["-n", cmd, ...args]];
}
/** Whether a tool is on this machine: an executable of that name on the path, or where the
* system keeps its administration. Asked before a tool is run, so "not here" and "refused" are
* never confused — the former is a fact to work around, the latter an error to say. */
export function installed(tool: string, path: string = process.env.PATH ?? ""): boolean {
const dirs = [...path.split(delimiter), "/usr/sbin", "/sbin", "/usr/bin"].filter((d) => d !== "");
return dirs.some((dir) => {
try {
accessSync(join(dir, tool), constants.X_OK);
return true;
} catch {
return false;
}
});
}
export const execRunner: Runner = async (cmd, args) => {
const [program, argv] = escalated(cmd, args);
try {
const { stdout } = await execFileP(program, argv, { maxBuffer: 16 * 1024 * 1024 });
return stdout;
} catch (err) {
// What failed is named by how it failed, not by prose: sudo missing is a spawn error; sudo
// refusing speaks on its own stderr line; anything else is the command's own failure.
const e = err as { code?: string | number; stderr?: string };
if (program === "sudo") {
if (e.code === "ENOENT") {
throw new Error(`${cmd} needs root, and sudo is not installed here for the runtime's account to escalate with`);
}
const stderr = String(e.stderr ?? "").trim();
if (/^sudo: .*command not found/m.test(stderr)) throw new Error(`${cmd} is not installed here`);
if (/^sudo:/m.test(stderr)) {
throw new Error(`${cmd} needs root and the runtime's account may not run it without a prompt: ${stderr}`);
}
}
throw err;
}
};
/** The mesh's own tables, which `remove` never touches. */
const MESH_TABLES = new Set(["inet mesh", "inet mesh_guard"]);
/** The tables iptables-nft manages, spoken through iptables rather than nft. */
const IPTABLES_TABLES = new Set(["filter", "nat", "raw", "mangle", "security"]);
/** The chains the kernel has built in; flushing one is the owner's act, not an operator's removal. */
const BUILT_IN = new Set(["INPUT", "FORWARD", "OUTPUT", "PREROUTING", "POSTROUTING"]);
/** The chain the container runtime leaves for an administrator, which is emptied, never deleted. */
const USER_CHAIN = "DOCKER-USER";
export interface Removal {
where: string;
did: string[];
}
export class FirewallClient {
static fromEnv(_env: NodeJS.ProcessEnv = process.env): FirewallClient {
private readonly run: Runner;
private readonly filterFile: string;
private readonly have: (tool: string) => boolean;
constructor(run: Runner = execRunner, filterFile: string = FILTER_FILE, have: (tool: string) => boolean = installed) {
this.run = run;
this.filterFile = filterFile;
this.have = have;
}
/** The filter as this machine has it: its own tools, the mesh's file. */
static onThisMachine(): FirewallClient {
return new FirewallClient();
}
/** The mesh's live table — exactly what is dropping and accepting on this node right now. */
/** The mesh's live table — exactly what the mesh's own filter is dropping and accepting. */
async ruleset(): Promise<string> {
const { stdout } = await run("nft", ["list", "table", "inet", "mesh"]);
return stdout;
return this.run("nft", ["list", "table", "inet", "mesh"]);
}
/** The packet filter as the machine enforces it: nftables whole or narrowed, and the legacy filter's
* listings where the tools exist. */
async rules(table?: string, chain?: string): Promise<{ nftables: string; legacy: Record<string, string> }> {
let nftables: string;
if (table && chain) {
const [family, name] = splitTable(table);
nftables = await this.run("nft", ["list", "chain", family, name, chain]);
} else if (table) {
const [family, name] = splitTable(table);
nftables = await this.run("nft", ["list", "table", family, name]);
} else {
nftables = await this.run("nft", ["list", "ruleset"]);
}
const legacy: Record<string, string> = {};
if (!table) {
for (const tool of ["iptables-legacy", "ip6tables-legacy"]) {
if (!this.have(tool)) continue; // no legacy tool, nothing to list
try {
const out = await this.run(tool, ["-S"]);
if (out.trim()) legacy[tool] = out;
} catch (err) {
// The tool is here and would not answer: said, not swallowed — a listing that silently
// leaves out a predecessor's rules reads as "none".
legacy[tool] = `error: ${err instanceof Error ? err.message : String(err)}`;
}
}
}
return { nftables, legacy };
}
/** Load the mesh's own filter again from the file the mesh writes, and answer with the table. */
async reload(): Promise<{ loaded: string; table: string }> {
await this.run("nft", ["-f", this.filterFile]);
return { loaded: this.filterFile, table: await this.ruleset() };
}
/** Whether the found front end is in force, whose chains `remove` leaves alone. Absent, it is
* not; present and not answering, nothing is removed on a guess. */
private async ufwActive(): Promise<boolean> {
if (!this.have("ufw")) return false;
let out: string;
try {
out = await this.run("ufw", ["status"]);
} catch (err) {
throw new Error(`cannot tell whether the found firewall is in force, so nothing of its is removed: ${err instanceof Error ? err.message : String(err)}`);
}
return /^Status:\s*active/m.test(out);
}
/** Remove one rule set the mesh did not write, named as the host reports it (ADR 0168). */
async remove(where: string): Promise<Removal> {
const did: string[] = [];
const legacy = /^chain (\S+) \((iptables-legacy|ip6tables-legacy|iptables|ip6tables)\)$/.exec(where.trim());
const nft = /^table (\S+) (\S+), chain (\S+)$/.exec(where.trim());
if (legacy) {
const [, chain, tool] = legacy;
await this.refuseOwned(chain, "ip", "filter");
await this.removeChainWith(tool, undefined, chain, did);
return { where, did };
}
if (nft) {
const [, family, name, chain] = nft;
const table = `${family} ${name}`;
if (MESH_TABLES.has(table)) throw new Error(`${where} is the mesh's own table; it is not removed, it is composed`);
await this.refuseOwned(chain, family, name);
if ((family === "ip" || family === "ip6") && IPTABLES_TABLES.has(name)) {
const tool = family === "ip6" ? "ip6tables" : "iptables";
await this.removeChainWith(tool, name, chain, did);
return { where, did };
}
// A table of the machine's own: a chain of it goes, and the table with it when nothing is left.
const listing = await this.run("nft", ["list", "table", family, name]);
const base = new RegExp(`chain ${escape(chain)} \\{[^}]*type \\S+ hook`).test(listing);
for (const from of chainsJumpingTo(listing, chain)) {
await this.deleteNftRules(family, name, from, chain, did);
}
if (base) {
await this.run("nft", ["flush", "chain", family, name, chain]);
did.push(`nft flush chain ${family} ${name} ${chain}`);
} else {
await this.run("nft", ["delete", "chain", family, name, chain]);
did.push(`nft delete chain ${family} ${name} ${chain}`);
}
return { where, did };
}
throw new Error(`${JSON.stringify(where)} is not a rule set as the host reports one: ` +
"`chain X (iptables-legacy)` or `table <family> <name>, chain X`");
}
private async refuseOwned(chain: string, family: string, table: string): Promise<void> {
if (chain !== USER_CHAIN && chain.startsWith("DOCKER")) {
throw new Error(`chain ${chain} is the container runtime's own; it is left`);
}
if (BUILT_IN.has(chain)) {
throw new Error(`chain ${chain} is built in; its policy is its owner's and it is not flushed`);
}
if (chain.startsWith("ufw") && (await this.ufwActive())) {
throw new Error(`chain ${chain} belongs to the found firewall, which is in force; converge retires it`);
}
void family; void table;
}
/** Through an iptables tool: the user chain is emptied back to its one return; another chain loses
* the jumps into it, is flushed and deleted. */
private async removeChainWith(tool: string, table: string | undefined, chain: string, did: string[]): Promise<void> {
const t = table && table !== "filter" ? ["-t", table] : [];
if (chain === USER_CHAIN) {
await this.run(tool, [...t, "-F", chain]);
await this.run(tool, [...t, "-A", chain, "-j", "RETURN"]);
did.push(`${tool} ${[...t, "-F", chain].join(" ")}`, `${tool} ${[...t, "-A", chain, "-j", "RETURN"].join(" ")}`);
return;
}
const listing = await this.run(tool, [...t, "-S"]);
for (const line of listing.split("\n")) {
const fields = line.trim().split(/\s+/);
if (fields[0] !== "-A") continue;
const j = fields.indexOf("-j");
const g = fields.indexOf("-g");
const target = j >= 0 ? fields[j + 1] : g >= 0 ? fields[g + 1] : "";
if (target !== chain) continue;
const args = [...t, "-D", ...fields.slice(1)];
await this.run(tool, args);
did.push(`${tool} ${args.join(" ")}`);
}
await this.run(tool, [...t, "-F", chain]);
await this.run(tool, [...t, "-X", chain]);
did.push(`${tool} ${[...t, "-F", chain].join(" ")}`, `${tool} ${[...t, "-X", chain].join(" ")}`);
}
private async deleteNftRules(family: string, name: string, from: string, target: string, did: string[]): Promise<void> {
const listing = await this.run("nft", ["-a", "list", "chain", family, name, from]);
for (const line of listing.split("\n")) {
if (!new RegExp(`\\b(jump|goto) ${escape(target)}\\b`).test(line)) continue;
const handle = /# handle (\d+)/.exec(line)?.[1];
if (!handle) continue;
await this.run("nft", ["delete", "rule", family, name, from, "handle", handle]);
did.push(`nft delete rule ${family} ${name} ${from} handle ${handle}`);
}
}
}
function splitTable(table: string): [string, string] {
const parts = table.trim().split(/\s+/);
if (parts.length !== 2) throw new Error(`a table is \`family name\`, not ${JSON.stringify(table)}`);
return [parts[0], parts[1]];
}
/** Which chains of a listed table jump or go to the named one. */
export function chainsJumpingTo(listing: string, target: string): string[] {
const out: string[] = [];
let chain = "";
for (const raw of listing.split("\n")) {
const line = raw.trim();
const head = /^chain (\S+) \{/.exec(line);
if (head) { chain = head[1]; continue; }
if (line === "}") { chain = ""; continue; }
if (chain && chain !== target && new RegExp(`\\b(jump|goto) ${escape(target)}\\b`).test(line) && !out.includes(chain)) {
out.push(chain);
}
}
return out;
}
function escape(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\-]/g, "\\$&");
}
+34 -3
View File
@@ -7,7 +7,12 @@
"claims": [
{
"name": "node-packet-filter",
"scope": "node"
"scope": "node",
"serves": [
"rules",
"reload",
"remove"
]
}
],
"filtering": {
@@ -19,6 +24,11 @@
"type": "package",
"package": "nftables"
},
{
"id": "legacy-tools",
"type": "package",
"package": "iptables"
},
{
"id": "unit",
"type": "file",
@@ -30,7 +40,7 @@
"id": "stock-unit-stop",
"type": "file",
"path": "/etc/systemd/system/nftables.service.d/mesh.conf",
"content": "# The mesh: stopping the stock unit deletes only the mesh's table, never the whole ruleset\n# (novox/hq ADR 0100) \u2014 a flush would take the container runtime's rules and any firewall with it.\n[Service]\nExecStop=\nExecStop=nft delete table inet mesh\n",
"content": "# The mesh: stopping the stock unit deletes only the mesh's table, never the whole ruleset\n# (novox/hq ADR 0100) — a flush would take the container runtime's rules and any firewall with it.\n[Service]\nExecStop=\nExecStop=nft delete table inet mesh\n",
"mode": "0644"
},
{
@@ -46,6 +56,27 @@
"reload-on": [
"filtering"
]
},
{
"id": "front-end",
"type": "package",
"package": "ufw",
"absent": true
}
]
],
"tools": [
"firewall_rules"
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
]
}
]
}
}
+7 -3
View File
@@ -1,11 +1,15 @@
{
"name": "@novox/module-firewall",
"name": "@novox/module-nftables",
"version": "0.1.0",
"description": "firewall — applies the mesh-computed packet filter (ADR 0045). Its diagnostic tool lives here.
"description": "nftables — loads the mesh's packet filter and holds the node-packet-filter seat: its verbs rules, reload and remove (novox/hq ADR 0045, ADR 0170).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . --outDir dist",
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
"@novox/mesh-sdk": "^0.1.1"
},
"devDependencies": {
"@types/node": "^22.0.0",
+108
View File
@@ -0,0 +1,108 @@
// `remove` acts on one rule set the mesh did not write, named as the host reports it (novox/hq ADR
// 0168, 0169), over the shapes two machines of the first mesh reported live: a predecessor's chain in
// the legacy filter, the runtime's user chain in the IPv6 legacy filter, a leftover front-end chain,
// and the same in an iptables-nft table. It refuses what is not the operator's to remove.
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { FILTER_FILE, FirewallClient, chainsJumpingTo, escalated, installed, type Runner } from "../client.ts";
const legacy = [
"-P INPUT ACCEPT", "-P FORWARD DROP", "-P OUTPUT ACCEPT",
"-N DOCKER", "-N DOCKER-USER", "-N HAL-MESH-ONLY",
"-A FORWARD -j DOCKER-USER",
"-A DOCKER-USER -i enp6s0 -p tcp -m conntrack --ctstate NEW -j HAL-MESH-ONLY",
"-A HAL-MESH-ONLY -m conntrack --ctorigdstport 80 -j RETURN",
"-A HAL-MESH-ONLY -m comment --comment \"HAL: not public -> mesh only\" -j DROP",
].join("\n") + "\n";
function fake(ufwActive = false): { run: Runner; asked: string[] } {
const asked: string[] = [];
const run: Runner = async (cmd, args) => {
asked.push([cmd, ...args].join(" "));
if (cmd === "ufw") return ufwActive ? "Status: active\n" : "Status: inactive\n";
if (args.join(" ") === "-S") return legacy;
if (cmd === "nft" && args[0] === "list" && args[1] === "table") {
return "table ip6 own {\n\tchain forward {\n\t\ttype filter hook forward priority filter; policy accept;\n\t\tjump deny\n\t}\n\tchain deny {\n\t\tdrop\n\t}\n}\n";
}
if (cmd === "nft" && args[0] === "-a") {
return "table ip6 own {\n\tchain forward {\n\t\ttype filter hook forward priority filter; policy accept;\n\t\tjump deny # handle 7\n\t}\n}\n";
}
return "";
};
return { run, asked };
}
test("a predecessor's chain in the legacy filter loses its jumps, is flushed and deleted", async () => {
const f = fake();
const out = await new FirewallClient(f.run, undefined, () => true).remove("chain HAL-MESH-ONLY (iptables-legacy)");
assert.deepEqual(out.did, [
"iptables-legacy -D DOCKER-USER -i enp6s0 -p tcp -m conntrack --ctstate NEW -j HAL-MESH-ONLY",
"iptables-legacy -F HAL-MESH-ONLY",
"iptables-legacy -X HAL-MESH-ONLY",
]);
});
test("the runtime's user chain is emptied back to its one return, never deleted", async () => {
const f = fake();
const out = await new FirewallClient(f.run, undefined, () => true).remove("chain DOCKER-USER (ip6tables-legacy)");
assert.deepEqual(out.did, ["ip6tables-legacy -F DOCKER-USER", "ip6tables-legacy -A DOCKER-USER -j RETURN"]);
const nft = await new FirewallClient(fake().run, undefined, () => true).remove("table ip6 filter, chain DOCKER-USER");
assert.deepEqual(nft.did, ["ip6tables -F DOCKER-USER", "ip6tables -A DOCKER-USER -j RETURN"]);
});
test("a chain of the machine's own nftables table goes with the rules that reach it", async () => {
const f = fake();
const out = await new FirewallClient(f.run, undefined, () => true).remove("table ip6 own, chain deny");
assert.deepEqual(out.did, ["nft delete rule ip6 own forward handle 7", "nft delete chain ip6 own deny"]);
});
test("what is not the operator's to remove is refused by name", async () => {
const c = new FirewallClient(fake(true).run, undefined, () => true);
await assert.rejects(c.remove("table inet mesh, chain forward"), /the mesh's own table/);
await assert.rejects(c.remove("chain DOCKER (iptables-legacy)"), /container runtime's own/);
await assert.rejects(c.remove("chain FORWARD (iptables-legacy)"), /built in/);
await assert.rejects(c.remove("chain ufw6-docker-logging-deny (ip6tables-legacy)"), /found firewall, which is in force/);
await assert.rejects(c.remove("something else"), /not a rule set as the host reports one/);
// Retired, a front end's leftover is nobody's and goes.
const retired = await new FirewallClient(fake(false).run, undefined, () => true).remove("chain ufw6-docker-logging-deny (ip6tables-legacy)");
assert.ok(retired.did.includes("ip6tables-legacy -X ufw6-docker-logging-deny"));
});
test("which chains jump to a target is read from a listing", () => {
const listing = "table ip6 own {\n\tchain a {\n\t\tjump deny\n\t}\n\tchain b {\n\t\tgoto deny\n\t}\n\tchain deny {\n\t\tdrop\n\t}\n}\n";
assert.deepEqual(chainsJumpingTo(listing, "deny"), ["a", "b"]);
});
test("the filter's commands run as given by root and through sudo without a prompt by anyone else", () => {
assert.deepEqual(escalated("nft", ["list", "ruleset"], 0), ["nft", ["list", "ruleset"]]);
assert.deepEqual(escalated("nft", ["-f", "/etc/nftables.conf"], 1000), ["sudo", ["-n", "nft", "-f", "/etc/nftables.conf"]]);
assert.deepEqual(escalated("iptables-legacy", ["-S"], undefined), ["sudo", ["-n", "iptables-legacy", "-S"]]);
});
test("the filter file is the one the manifest's filtering names", () => {
const manifest = JSON.parse(readFileSync(new URL("../module.json", import.meta.url), "utf8")) as { filtering: { into: string } };
assert.equal(FILTER_FILE, manifest.filtering.into);
});
test("a tool is installed when an executable of its name is on the path, and not otherwise", () => {
assert.equal(installed("sh"), true);
assert.equal(installed("no-such-tool-of-the-mesh"), false);
});
test("a found firewall that is absent guards nothing; one that will not answer stops the removal", async () => {
// Absent: its leftover chain is nobody's and goes, without asking it.
const absent = fake(true);
const out = await new FirewallClient(absent.run, undefined, () => false).remove("chain ufw6-docker-logging-deny (ip6tables-legacy)");
assert.ok(out.did.includes("ip6tables-legacy -X ufw6-docker-logging-deny"));
assert.ok(!absent.asked.some((a) => a.startsWith("ufw ")));
// Present and failing — refused by sudo, say — nothing is removed on a guess.
const refusing: Runner = async (cmd, args) => {
if (cmd === "ufw") throw new Error("ufw needs root and the runtime's account may not run it without a prompt");
return fake().run(cmd, args);
};
await assert.rejects(
new FirewallClient(refusing, undefined, () => true).remove("chain ufw6-docker-logging-deny (ip6tables-legacy)"),
/cannot tell whether the found firewall is in force/,
);
});
+38 -6
View File
@@ -1,19 +1,51 @@
// firewall's tools — one, and the useful one: what is actually enforced. The rules are the mesh's,
// computed from every module's listens; this reads the live table so a declared scope can be checked
// against what the packet filter is really doing.
// The packet filter's tools: the node-packet-filter seat's three verbs — what the machine enforces,
// reload the mesh's own, remove one thing the mesh did not write — and the module's own reading of
// the mesh's table (novox/hq ADR 0045, ADR 0168, ADR 0170).
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { FirewallClient } from "../client.js";
export function getSeatVerbs(firewall: FirewallClient): ToolDefinition[] {
return [
{
name: "rules",
description:
"The packet filter as this machine enforces it now: the nftables ruleset and, where the tool exists, the legacy filter's listings. Narrowed to one table or chain when asked.",
input: {
table: { type: "string", description: "one nftables table, as `family name` (optional)" },
chain: { type: "string", description: "one chain of that table (optional)" },
},
run: async (args) => firewall.rules(args.table ? String(args.table) : undefined, args.chain ? String(args.chain) : undefined),
},
{
name: "reload",
description: "Load the mesh's own filter again from the file the mesh writes, and answer with the mesh's table as loaded.",
input: {},
run: async () => firewall.reload(),
},
{
name: "remove",
description:
"Remove one rule set the mesh did not write, named exactly as `node show` lists it: `chain X (iptables-legacy)` or `table ip6 filter, chain DOCKER-USER`. " +
"Refuses the mesh's tables, the runtime's own chains, a built-in chain and an active found firewall's chains. An operator's act, by name, never a flush.",
input: { where: { type: "string", description: "the rule set, as `node show` lists it" } },
run: async (args) => firewall.remove(String(args.where ?? "")),
},
];
}
export function getFirewallTools(firewall: FirewallClient): ToolDefinition[] {
return [
{
name: "firewall_rules",
description: "The mesh's live nftables rules on this node — what is actually accepting and dropping.",
description: "The mesh's live nftables table on this node — what the mesh's own filter is accepting and dropping.",
input: {},
run: async () => ({ ruleset: await firewall.ruleset() }),
},
];
}
registerModuleTools("firewall", () => getFirewallTools(FirewallClient.fromEnv()));
const firewall = FirewallClient.onThisMachine();
// The seat's verbs under the seat's name: the runtime serves them on the seat's subjects where this
// module holds it (ADR 0159, 0160). The module's own under its own.
registerModuleTools("node-packet-filter", () => getSeatVerbs(firewall));
registerModuleTools("nftables", () => getFirewallTools(firewall));
+8 -2
View File
@@ -5,8 +5,14 @@
"flows.deployed"
],
"own-secrets": {
"admin": "${dir:mesh-state}/admin",
"api-token": "${dir:mesh-state}/api-token",
"admin": {
"path": "${dir:mesh-state}/admin",
"taken": "at-start"
},
"api-token": {
"path": "${dir:mesh-state}/api-token",
"taken": "at-start"
},
"broker": "${dir:mesh-state}/broker"
},
"capabilities": [
-24
View File
@@ -1,24 +0,0 @@
# nzbget's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/nzbget
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/nzbget/dist /app/modules/nzbget/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/nzbget/dist/index.js,/app/modules/nzbget/dist/tools/index.js
-178
View File
@@ -1,178 +0,0 @@
// The NZBGet API client — nzbget's own code, living in the module (novox/hq ADR 0039). Ported from
// hal's shared nzbget tools, but self-contained: a change to NZBGet's JSON-RPC now rebuilds only
// nzbget and nothing else. Both this module's tools and its events entrypoint import it, and
// nothing outside nzbget does. NZBGet speaks JSON-RPC at /jsonrpc, behind HTTP Basic auth.
import { readFileSync } from "node:fs";
export interface NzbgetStatus {
/** Bytes/sec — NZBGet reports it split across two 32-bit halves, rejoined here. */
speedBytesPerSec: number;
remainingMB: number;
downloadedTodayMB: number;
downloadedMonthMB: number;
freeDiskMB: number;
paused: boolean;
postJobs: number;
uptimeSec: number;
}
export interface NzbgetQueueItem {
/** The NZBID — stable while the item is queued, so events can diff on it. */
id: number;
name: string;
status: string;
category: string;
sizeMB: number;
remainingMB: number;
percent: number;
}
export interface NzbgetHistoryItem {
/** The NZBID — the same id the item carried in the queue. */
id: number;
name: string;
/** NZBGet's own status string, e.g. "SUCCESS/ALL", "FAILURE/PAR", "DELETED/MANUAL". */
status: string;
category: string;
sizeMB: number;
/** A genuine completion (status starts "SUCCESS") vs a failed or deleted entry — the difference
* between something to announce as done and something that merely left the queue. */
success: boolean;
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> {
if (!file) return {};
try { return JSON.parse(readFileSync(file, "utf8")) as Record<string, string>; }
catch { return {}; }
}
/** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
* absent or unreadable yields undefined so callers fall back rather than crash. */
function readSecret(file?: string): string | undefined {
if (!file) return undefined;
try { return readFileSync(file, "utf8").trim(); }
catch { return undefined; }
}
export class NzbgetClient {
readonly rpcUrl: string;
private readonly auth: string;
constructor(url: string, user: string, password: string) {
this.rpcUrl = `${url.replace(/\/$/, "")}/jsonrpc`;
this.auth = Buffer.from(`${user}:${password}`).toString("base64");
}
/**
* Build from the module's resolved environment. URL and password are read from MESH_NZBGET_URL
* and MESH_NZBGET_PASSWORD; both must be present — an unconfigured NZBGet throws rather than
* pretend to be reachable, so the tools/events simply do not load (the harness treats the throw
* as "exposes nothing"). The control username defaults to "nzbget", NZBGet's own default.
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): NzbgetClient {
const cfg = meshConfig(env.MESH_NZBGET_CONFIG_FILE);
const url = cfg.url ?? env.MESH_NZBGET_URL;
const password = cfg.password ?? readSecret(env.MESH_NZBGET_PASSWORD_FILE) ?? env.MESH_NZBGET_PASSWORD;
if (!url || !password) {
throw new Error("NZBGet not configured — set MESH_NZBGET_URL and MESH_NZBGET_PASSWORD");
}
const user = cfg.user ?? env.MESH_NZBGET_USER ?? "nzbget";
return new NzbgetClient(url, user, password);
}
private async rpc<T>(method: string, params: unknown[] = []): Promise<T> {
const res = await fetch(this.rpcUrl, {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: `Basic ${this.auth}` },
body: JSON.stringify({ method, params, id: 1 }),
});
if (!res.ok) throw new Error(`NZBGet API ${method}: ${res.status} ${await res.text()}`);
const data = (await res.json()) as { result?: T; error?: unknown };
if (data.error) throw new Error(`NZBGet RPC ${method}: ${JSON.stringify(data.error)}`);
return data.result as T;
}
async getVersion(): Promise<string> {
return this.rpc<string>("version");
}
async getStatus(): Promise<NzbgetStatus> {
const s = await this.rpc<Record<string, number | boolean>>("status");
const lo = Number(s.DownloadRateLo ?? 0);
const hi = Number(s.DownloadRateHi ?? 0);
return {
speedBytesPerSec: lo + hi * 4294967296,
remainingMB: Number(s.RemainingSizeMB ?? 0),
downloadedTodayMB: Number(s.DaySizeMB ?? 0),
downloadedMonthMB: Number(s.MonthSizeMB ?? 0),
freeDiskMB: Number(s.FreeDiskSpaceMB ?? 0),
paused: Boolean(s.DownloadPaused),
postJobs: Number(s.PostJobCount ?? 0),
uptimeSec: Number(s.UpTimeSec ?? 0),
};
}
async getQueue(): Promise<NzbgetQueueItem[]> {
const groups = await this.rpc<Record<string, any>[]>("listgroups", [0]);
return groups.map((g) => {
const size = Number(g.FileSizeMB ?? 0);
const remaining = Number(g.RemainingSizeMB ?? 0);
return {
id: Number(g.NZBID),
name: String(g.NZBName ?? "Unknown"),
status: String(g.Status ?? "unknown"),
category: String(g.Category ?? ""),
sizeMB: size,
remainingMB: remaining,
percent: size > 0 ? Math.round(((size - remaining) / size) * 100) : 0,
};
});
}
async getHistory(limit = 20): Promise<NzbgetHistoryItem[]> {
const history = await this.rpc<Record<string, any>[]>("history", [false]);
return history.slice(0, limit).map((h) => {
const status = String(h.Status ?? "");
return {
id: Number(h.NZBID),
name: String(h.Name ?? "Unknown"),
status,
category: String(h.Category ?? ""),
sizeMB: Number(h.FileSizeMB ?? 0),
success: status.startsWith("SUCCESS"),
};
});
}
/** Queue an NZB by URL. Returns the new NZBID; a non-positive id means NZBGet refused it. */
async add(url: string, category = "", priority = 0, paused = false): Promise<number> {
const id = await this.rpc<number>("append", [
"", url, category, priority, false, paused, "", 0, "SCORE", false, [],
]);
if (!id || id <= 0) throw new Error("NZBGet refused the NZB (append returned 0)");
return id;
}
async pauseAll(): Promise<void> {
await this.rpc("pausedownload");
}
async resumeAll(): Promise<void> {
await this.rpc("resumedownload");
}
async pauseItem(id: number): Promise<void> {
await this.rpc("editqueue", ["GroupPause", "", [id]]);
}
async resumeItem(id: number): Promise<void> {
await this.rpc("editqueue", ["GroupResume", "", [id]]);
}
/** Delete an item from the queue or from history. */
async delete(id: number, from: "queue" | "history" = "queue"): Promise<void> {
await this.rpc("editqueue", [from === "history" ? "HistoryDelete" : "GroupDelete", "", [id]]);
}
}
-64
View File
@@ -1,64 +0,0 @@
// nzbget's events. The tool runtime imports this once the broker is bound. It watches the download
// queue and the history and turns their comings and goings into mesh events.
//
// Emits (novox/hq ADR 0041/0042):
// module.nzbget.download.added — an NZB entered the queue
// module.nzbget.download.completed — an NZB finished successfully (left the queue, landed in
// history as SUCCESS). This exact routing key is what the
// plex module consumes (module.*.download.completed) to
// rescan, so the new file becomes a visible item.
// Consumes: none.
//
// Two diffs, each primed silently on the first look (like plex's and sonarr's index.ts) so a
// restart mid-download does not re-announce everything already in flight or already finished. The
// queue tells us what was grabbed; history — not the queue's disappearance — tells us what actually
// succeeded, since a failed or deleted download also leaves the queue.
import { emit } from "@novox/mesh-sdk/events";
import { NzbgetClient } from "./client.js";
const nzbget = NzbgetClient.fromEnv();
const inQueue = new Set<number>();
let queuePrimed = false;
async function pollQueue(): Promise<void> {
const items = await nzbget.getQueue();
const now = new Set(items.map((i) => i.id));
if (queuePrimed) {
for (const item of items) {
if (!inQueue.has(item.id)) {
await emit("download.added", { name: item.name, category: item.category, sizeMB: item.sizeMB });
}
}
}
inQueue.clear();
for (const id of now) inQueue.add(id);
queuePrimed = true;
}
const seenHistory = new Set<number>();
let historyPrimed = false;
async function pollHistory(): Promise<void> {
const items = await nzbget.getHistory(50);
for (const item of items) {
if (!seenHistory.has(item.id)) {
// A newly-appeared history entry is a completion only if it actually succeeded; a failure or
// a manual delete lands in history too, and neither is a "download.completed".
if (historyPrimed && item.success) {
await emit("download.completed", { name: item.name, category: item.category, sizeMB: item.sizeMB });
}
seenHistory.add(item.id);
}
}
historyPrimed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[nzbget] ${err}`));
setInterval(run, everyMs);
run();
};
tick(pollQueue, 20_000);
tick(pollHistory, 30_000);
console.log("[nzbget] watching the queue and history, emitting adds and completions");
-117
View File
@@ -1,117 +0,0 @@
{
"module": "nzbget",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"download.added",
"download.completed"
],
"consumes": [],
"own-secrets": {
"broker": "${dir:mesh-state}/broker",
"password": "${dir:mesh-state}/password"
},
"listens": [
{
"name": "web",
"port": 6789,
"protocol": "tcp",
"from": "mesh",
"why": "the download client's pages"
}
],
"accesses": [
{
"id": "downloads",
"path": "/services/media/downloads",
"mode": "read-write"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/nzbget/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "nzbget",
"image": "lscr.io/linuxserver/nzbget@sha256:5f3d3fa71029004156eff2cbf4ef4455ce4ce59517cf13fa7d1d7c8a4cd2c8a4",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"6789"
],
"volumes": [
"/services/nzbget/config:/config",
"${access:downloads}:/downloads"
]
},
{
"id": "runtime-config",
"type": "file",
"path": "${dir:mesh-state}/config.json",
"mode": "0600",
"content": "{}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-nzbget",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/password:/run/secrets/password:ro",
"${dir:mesh-state}/config.json:/run/config/config.json:ro",
"/services/nzbget/config:/var/lib/nzbget/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_NZBGET_URL": "http://127.0.0.1:6789",
"MESH_NZBGET_PASSWORD_FILE": "/run/secrets/password",
"MESH_NZBGET_CONFIG_FILE": "/run/config/config.json",
"MESH_NZBGET_CONFIG_DIR": "/var/lib/nzbget/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-nzbget",
"version": "0.1.0",
"description": "nzbget — Usenet download client. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-88
View File
@@ -1,88 +0,0 @@
// nzbget's tools — ported from the shared hal sdk (novox/hq ADR 0039), importing nzbget's own
// client. They return structured data (not the pre-formatted text hal returned); the mesh serves
// them through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { NzbgetClient } from "../client.js";
export function getNzbgetTools(nzbget: NzbgetClient): ToolDefinition[] {
return [
{
name: "nzbget_status",
description: "NZBGet server status: download speed, queue remaining, disk free, paused state.",
input: {},
run: async () => nzbget.getStatus(),
},
{
name: "nzbget_queue",
description: "List the current NZBGet download queue — what is downloading and how far along.",
input: {},
run: async () => {
const items = await nzbget.getQueue();
return { count: items.length, items };
},
},
{
name: "nzbget_history",
description: "Recent NZBGet download history, newest first — completed, failed and deleted items.",
input: { limit: { type: "number", description: "how many entries (default 20)" } },
run: async (args) => {
const items = await nzbget.getHistory(args.limit ? Number(args.limit) : 20);
return { count: items.length, items };
},
},
{
name: "nzbget_add",
description: "Queue an NZB download by URL, optionally into a category.",
input: {
url: { type: "string", description: "URL to the NZB file" },
category: { type: "string", description: "category name (determines download directory)" },
priority: { type: "number", description: "-100 very low … 0 normal … 100 very high (default 0)" },
paused: { type: "boolean", description: "add in paused state (default false)" },
},
run: async (args) => {
const id = await nzbget.add(
String(args.url),
args.category ? String(args.category) : "",
args.priority ? Number(args.priority) : 0,
args.paused === true || args.paused === "true",
);
return { added: id, category: args.category ? String(args.category) : null };
},
},
{
name: "nzbget_pause",
description: "Pause or resume all NZBGet downloads.",
input: { resume: { type: "boolean", description: "true to resume, false to pause (default false)" } },
run: async (args) => {
const resume = args.resume === true || args.resume === "true";
if (resume) await nzbget.resumeAll();
else await nzbget.pauseAll();
return { paused: !resume };
},
},
{
name: "nzbget_delete",
description: "Delete an NZB from the queue or from history by its NZBID.",
input: {
id: { type: "number", description: "the NZBID to delete" },
from: { type: "string", description: "'queue' (default) or 'history'" },
},
run: async (args) => {
const from = args.from === "history" ? "history" : "queue";
await nzbget.delete(Number(args.id), from);
return { deleted: Number(args.id), from };
},
},
];
}
// The tools exist only when NZBGet is configured; without a URL and password, nzbget contributes
// none rather than failing the whole runtime.
registerModuleTools("nzbget", (env) => {
try {
return getNzbgetTools(NzbgetClient.fromEnv(env));
} catch {
return [];
}
});
-12
View File
@@ -1,12 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
-24
View File
@@ -1,24 +0,0 @@
# ombi's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/ombi
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/ombi/dist /app/modules/ombi/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/ombi/dist/index.js,/app/modules/ombi/dist/tools/index.js
-122
View File
@@ -1,122 +0,0 @@
// The Ombi API client — ombi's own code, living in the module (novox/hq ADR 0039). Ombi is the
// request front-end: viewers ask for movies and shows, and an operator approves them. This client
// talks its /api/v1 REST API (keyed by an ApiKey header); ombi's tools and events import it.
import { readFileSync } from "node:fs";
export interface OmbiRequest {
kind: "movie" | "tv";
id: number;
title: string;
requestedBy?: string;
requestedDate?: string;
approved: boolean;
available: boolean;
denied: boolean;
tmdbId?: number;
}
export interface RequestCounts {
pending: number;
approved: number;
available: number;
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> {
if (!file) return {};
try { return JSON.parse(readFileSync(file, "utf8")) as Record<string, string>; }
catch { return {}; }
}
/** Read a secret the mesh mounted at a file path (an own-secret delivered by `secret accept`);
* absent or unreadable yields undefined so callers fall back rather than crash. */
function readSecret(file?: string): string | undefined {
if (!file) return undefined;
try { return readFileSync(file, "utf8").trim(); }
catch { return undefined; }
}
export class OmbiClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly apiKey: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/** Build from the module's resolved environment. Ombi's API is keyed; without URL and key there
* is nothing to talk to, so this throws rather than run half-configured. */
static fromEnv(env: NodeJS.ProcessEnv = process.env): OmbiClient {
const cfg = meshConfig(env.MESH_OMBI_CONFIG_FILE);
const url = cfg.url ?? env.MESH_OMBI_URL;
const apiKey = cfg.apiKey ?? readSecret(env.MESH_OMBI_API_KEY_FILE) ?? env.MESH_OMBI_API_KEY;
if (!url) throw new Error("no Ombi URL — set MESH_OMBI_URL");
if (!apiKey) throw new Error("no Ombi API key — set MESH_OMBI_API_KEY");
return new OmbiClient(url, apiKey);
}
private async request(method: string, path: string, body?: unknown): Promise<any> {
const res = await fetch(`${this.baseUrl}/api/v1${path}`, {
method,
headers: {
ApiKey: this.apiKey,
Accept: "application/json",
...(body !== undefined ? { "Content-Type": "application/json" } : {}),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
if (!res.ok) throw new Error(`Ombi API ${method} ${path}: ${res.status} ${await res.text()}`);
const text = await res.text();
return text ? JSON.parse(text) : {};
}
/** All requests, movies and TV together — who asked for what, and where each stands. */
async getRequests(): Promise<OmbiRequest[]> {
const [movies, tv] = await Promise.all([
this.request("GET", "/Request/movie"),
this.request("GET", "/Request/tv"),
]);
const films: OmbiRequest[] = (Array.isArray(movies) ? movies : []).map((r: any) => ({
kind: "movie" as const,
id: r.id,
title: r.title ?? "Unknown",
requestedBy: r.requestedUser?.userName ?? r.requestedUser?.userAlias,
requestedDate: r.requestedDate,
approved: Boolean(r.approved),
available: Boolean(r.available),
denied: Boolean(r.denied),
tmdbId: r.theMovieDbId,
}));
// TV requests carry per-season child requests; the top-level record is approved when all its
// children are, which is the grain an operator acts on.
const shows: OmbiRequest[] = (Array.isArray(tv) ? tv : []).map((r: any) => {
const children: any[] = r.childRequests ?? [];
return {
kind: "tv" as const,
id: r.id,
title: r.title ?? "Unknown",
requestedBy: children[0]?.requestedUser?.userName,
requestedDate: children[0]?.requestedDate,
approved: children.length > 0 && children.every((c) => c.approved),
available: children.length > 0 && children.every((c) => c.available),
denied: children.some((c) => c.denied),
tmdbId: r.theMovieDbId,
};
});
return [...films, ...shows];
}
/** Live pending/approved/available counts — a one-line health read without listing everything. */
async getCounts(): Promise<RequestCounts> {
const c = await this.request("GET", "/Request/count");
return { pending: c.pending ?? 0, approved: c.approved ?? 0, available: c.available ?? 0 };
}
/** Approve a request. TV approval fans out to the request's child (per-season) requests. */
async approve(kind: "movie" | "tv", id: number): Promise<void> {
await this.request("POST", `/Request/${kind}/approve`, { id });
}
}
-58
View File
@@ -1,58 +0,0 @@
// ombi's events. The tool runtime imports this once the broker is bound. Ombi's timeline is the
// request lifecycle: a viewer files a request, and later an operator approves it. Both transitions
// are worth announcing — the mesh can notify on a new request, and act on an approval (that is when
// a downloader should start looking).
//
// Emits (novox/hq ADR 0041/0042):
// module.ombi.request.created — a viewer filed a new request
// module.ombi.request.approved — a request was approved
//
// Ombi is the origin of these decisions, not a reactor to the mesh, so it consumes nothing.
//
// Both events are observation-based: poll the request list and diff. Creation is diffed on the set
// of request ids; approval on each request's approved flag flipping true. Primed silently on the
// first look, or a restart would re-announce every existing request and approval.
import { emit } from "@novox/mesh-sdk/events";
import { OmbiClient, type OmbiRequest } from "./client.js";
const ombi = OmbiClient.fromEnv();
// Remember each seen request and whether it was approved last time, keyed by kind+id (ids are only
// unique within a kind).
const approvedState = new Map<string, boolean>();
let primed = false;
const keyOf = (r: OmbiRequest): string => `${r.kind}:${r.id}`;
async function pollRequests(): Promise<void> {
const requests = await ombi.getRequests();
for (const r of requests) {
const key = keyOf(r);
const known = approvedState.has(key);
if (primed && !known) {
await emit("request.created", {
kind: r.kind,
id: r.id,
title: r.title,
requestedBy: r.requestedBy,
tmdbId: r.tmdbId,
});
}
// Approval: the flag went from false to true for a request we already knew about.
if (primed && known && r.approved && approvedState.get(key) === false) {
await emit("request.approved", { kind: r.kind, id: r.id, title: r.title, tmdbId: r.tmdbId });
}
approvedState.set(key, r.approved);
}
primed = true;
}
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[ombi] ${err}`));
setInterval(run, everyMs);
run();
};
tick(pollRequests, 30_000);
console.log("[ombi] watching requests for new filings and approvals");
-120
View File
@@ -1,120 +0,0 @@
{
"module": "ombi",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"request.created",
"request.approved"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker",
"api-key": "${dir:mesh-state}/api-key"
},
"listens": [
{
"name": "web",
"port": 3579,
"protocol": "tcp",
"from": "mesh",
"why": "requests from viewers"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/ombi/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "ombi",
"image": "lscr.io/linuxserver/ombi@sha256:a6f76ac521ba01eee2e9f0c23a3fed22e56630d97a04d5eeaeaa36c1e681640d",
"env": {
"PUID": "1000",
"PGID": "1000",
"TZ": "Etc/UTC"
},
"ports": [
"3579"
],
"volumes": [
"/services/ombi/config:/config"
]
},
{
"id": "runtime-config",
"type": "file",
"path": "${dir:mesh-state}/config.json",
"mode": "0600",
"content": "{}\n",
"merge": "json"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-ombi",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/api-key:/run/secrets/api-key:ro",
"${dir:mesh-state}/config.json:/run/config/config.json:ro",
"/services/ombi/config:/var/lib/ombi/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_OMBI_URL": "http://127.0.0.1:3579",
"MESH_OMBI_API_KEY_FILE": "/run/secrets/api-key",
"MESH_OMBI_CONFIG_FILE": "/run/config/config.json",
"MESH_OMBI_CONFIG_DIR": "/var/lib/ombi/config"
},
"restart-on": [
"runtime-config"
],
"artifact": "runtime"
}
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "ombi",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:mesh-state}/route.json"
},
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-ombi",
"version": "0.1.0",
"description": "ombi — media requests. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-46
View File
@@ -1,46 +0,0 @@
// ombi's tools — its own code (novox/hq ADR 0039), importing ombi's client. They return structured
// data; the mesh serves them through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { OmbiClient } from "../client.js";
export function getOmbiTools(ombi: OmbiClient): ToolDefinition[] {
return [
{
name: "ombi_requests",
description: "List media requests — movies and shows — with who asked and whether each is approved or available.",
input: { pending: { type: "boolean", description: "only requests not yet approved (default false)" } },
run: async (args) => {
let requests = await ombi.getRequests();
if (args.pending) requests = requests.filter((r) => !r.approved && !r.denied);
const counts = await ombi.getCounts();
return { counts, count: requests.length, requests };
},
},
{
name: "ombi_approve",
description: "Approve a media request by its kind and id (from ombi_requests).",
input: {
kind: { type: "string", description: '"movie" or "tv"' },
id: { type: "number", description: "the request id" },
},
run: async (args) => {
const kind = String(args.kind);
if (kind !== "movie" && kind !== "tv") throw new Error('kind must be "movie" or "tv"');
const id = Number(args.id);
await ombi.approve(kind, id);
return { approved: { kind, id } };
},
},
];
}
// Exposed only when Ombi is configured; otherwise ombi contributes no tools rather than failing the
// whole runtime.
registerModuleTools("ombi", (env) => {
try {
return getOmbiTools(OmbiClient.fromEnv(env));
} catch {
return [];
}
});
-12
View File
@@ -1,12 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
-24
View File
@@ -1,24 +0,0 @@
# plex's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
WORKDIR /app/modules/plex
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/plex/dist /app/modules/plex/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled.
ENV MESH_TOOL_MODULES=/app/modules/plex/dist/index.js,/app/modules/plex/dist/tools/index.js
-158
View File
@@ -1,158 +0,0 @@
// The Plex API client — plex's own code, living in the module (novox/hq ADR 0039). Moved out of the
// shared hal sdk, where a change to Plex's API rebuilt everything; here it rebuilds only plex. Both
// this module's tools and its events entrypoint import it, and nothing outside plex does.
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
/** Read a secret the mesh mounted at a file path (an own-secret); absent or unreadable yields
* undefined, so callers can fall back rather than crash. */
function readSecret(path: string | undefined): string | undefined {
if (!path) return undefined;
try {
return readFileSync(path, "utf8").trim();
} catch {
return undefined;
}
}
export interface PlexLibrary {
key: string;
title: string;
type: string;
count?: number;
}
export interface PlexSession {
key: string;
title: string;
user: string;
player: string;
state: string;
type: string;
}
export interface PlexItem {
title: string;
type: string;
year?: number;
summary?: string;
addedAt?: string;
}
export class PlexClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly token: string,
) {
this.baseUrl = url.replace(/\/$/, "");
}
/**
* Build from the module's resolved environment. The token is read from MESH_PLEX_TOKEN, or
* discovered from the server's own Preferences.xml under the data directory — the same file Plex
* writes it to, so a running server needs nothing configured by hand.
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): PlexClient {
const url = env.MESH_PLEX_URL ?? `http://127.0.0.1:${env.PLEX_PORT ?? "32400"}`;
const dataDir = env.MESH_PLEX_DATA_DIR ?? "/var/lib/plex";
// The operator-provided token is an own-secret the mesh mounts at MESH_PLEX_TOKEN_FILE (delivered
// by `secret accept`); prefer it, fall back to a bare env var, then to discovery from the data dir.
const token = readSecret(env.MESH_PLEX_TOKEN_FILE) ?? env.MESH_PLEX_TOKEN ?? PlexClient.detectToken(dataDir);
if (!token) throw new Error("no Plex token — set MESH_PLEX_TOKEN or make the data dir readable");
return new PlexClient(url, token);
}
/** Discover the token from the server's Preferences.xml, falling back to null. */
static detectToken(dataDir: string): string | null {
const prefs = join(dataDir, "config", "Library", "Application Support", "Plex Media Server", "Preferences.xml");
if (existsSync(prefs)) {
const match = readFileSync(prefs, "utf8").match(/PlexOnlineToken="([^"]+)"/);
if (match) return match[1];
}
return null;
}
private async get(path: string): Promise<any> {
const url = `${this.baseUrl}${path}`;
const sep = url.includes("?") ? "&" : "?";
const res = await fetch(`${url}${sep}X-Plex-Token=${this.token}`, { headers: { Accept: "application/json" } });
if (!res.ok) throw new Error(`Plex API ${path}: ${res.status} ${await res.text()}`);
return res.json();
}
async getServerInfo(): Promise<{ name: string; version: string; platform: string }> {
const mc = (await this.get("/")).MediaContainer;
return { name: mc.friendlyName || mc.machineIdentifier, version: mc.version, platform: mc.platform };
}
async getLibraries(): Promise<PlexLibrary[]> {
const dirs = (await this.get("/library/sections")).MediaContainer?.Directory ?? [];
return dirs.map((d: any) => ({ key: d.key, title: d.title, type: d.type, count: d.count }));
}
async getSessions(): Promise<PlexSession[]> {
const sessions = (await this.get("/status/sessions")).MediaContainer?.Metadata ?? [];
return sessions.map((s: any) => ({
key: s.sessionKey ?? s.ratingKey,
title: s.title + (s.grandparentTitle ? ` (${s.grandparentTitle})` : ""),
user: s.User?.title ?? "unknown",
player: s.Player?.title ?? s.Player?.product ?? "unknown",
state: s.Player?.state ?? "unknown",
type: s.type,
}));
}
async search(query: string): Promise<PlexItem[]> {
const hubs = (await this.get(`/hubs/search?query=${encodeURIComponent(query)}&limit=20`)).MediaContainer?.Hub ?? [];
const results: PlexItem[] = [];
for (const hub of hubs) {
for (const m of hub.Metadata ?? []) {
results.push({
title: m.title + (m.grandparentTitle ? ` (${m.grandparentTitle})` : ""),
type: m.type,
year: m.year,
summary: m.summary?.slice(0, 200),
});
}
}
return results;
}
async getRecentlyAdded(limit = 20): Promise<PlexItem[]> {
const items = (await this.get(`/library/recentlyAdded?X-Plex-Container-Size=${limit}`)).MediaContainer?.Metadata ?? [];
return items.map((m: any) => ({
title: m.title + (m.grandparentTitle ? ` (${m.grandparentTitle})` : ""),
type: m.type,
year: m.year,
summary: m.summary?.slice(0, 200),
addedAt: m.addedAt ? new Date(m.addedAt * 1000).toISOString() : undefined,
}));
}
/** Ask Plex to rescan a library section — how a "new media arrived" event becomes a visible item. */
async refreshLibrary(key: string): Promise<void> {
await this.get(`/library/sections/${key}/refresh`);
}
/** Rescan every library, for when what arrived is not known to belong to one. */
async refreshAll(): Promise<void> {
for (const library of await this.getLibraries()) await this.refreshLibrary(library.key);
}
/**
* A health probe that never throws: report whether the Plex server this client is pointed at
* answers, and identify it when it does. Every other call assumes the server is up; this is the
* one that tells the mesh whether it is, so a diagnosis does not start from a stack trace.
*/
async reachable(): Promise<{ reachable: boolean; url: string; server?: { name: string; version: string }; error?: string }> {
try {
const info = await this.getServerInfo();
return { reachable: true, url: this.baseUrl, server: { name: info.name, version: info.version } };
} catch (err) {
return { reachable: false, url: this.baseUrl, error: err instanceof Error ? err.message : String(err) };
}
}
}
-68
View File
@@ -1,68 +0,0 @@
// plex's events. The tool runtime imports this once the broker is bound, and it does two things:
// it watches the server and emits what happened, and it reacts to the mesh's media events.
//
// Emits (novox/hq ADR 0041/0042):
// module.plex.playback.started / .stopped — someone began or ended watching
// module.plex.item.added — a new item appeared in a library
// Consumes:
// module.*.download.completed — a downloader finished; rescan so the file shows up
//
// The polling is deliberately unhurried: Plex is a neighbour on the same node, and an event a few
// seconds late is an event, whereas hammering the server for immediacy nobody asked for is not.
import { emit, on } from "@novox/mesh-sdk/events";
import { PlexClient, type PlexSession } from "./client.js";
const plex = PlexClient.fromEnv();
// Playback, by diffing the set of active sessions. Primed silently on the first look so a server
// that was already streaming when this started does not announce it as freshly begun.
const active = new Map<string, PlexSession>();
let playbackPrimed = false;
async function pollSessions(): Promise<void> {
const sessions = await plex.getSessions();
const now = new Map(sessions.map((s) => [s.key, s]));
if (playbackPrimed) {
for (const [key, s] of now) {
if (!active.has(key)) await emit("playback.started", { title: s.title, user: s.user, player: s.player, kind: s.type });
}
for (const [key, s] of active) {
if (!now.has(key)) await emit("playback.stopped", { title: s.title, user: s.user, player: s.player });
}
}
active.clear();
for (const [key, s] of now) active.set(key, s);
playbackPrimed = true;
}
// New items, by diffing recently-added. Primed silently too, or a restart would re-announce the
// whole recent list as new.
const seen = new Set<string>();
let itemsPrimed = false;
async function pollRecent(): Promise<void> {
const items = await plex.getRecentlyAdded(20);
for (const item of items) {
const id = `${item.title}@${item.addedAt ?? ""}`;
if (!seen.has(id)) {
if (itemsPrimed) await emit("item.added", item);
seen.add(id);
}
}
itemsPrimed = true;
}
// A downloader finished somewhere on the mesh: rescan, so what it fetched becomes a visible item
// rather than a file Plex has not noticed. Idempotent — a rescan too many costs a little disk I/O.
await on("*.download.completed", async () => {
await plex.refreshAll();
});
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const run = (): void => void fn().catch((err) => console.error(`[plex] ${err}`));
setInterval(run, everyMs);
run();
};
tick(pollSessions, 15_000);
tick(pollRecent, 60_000);
console.log("[plex] watching sessions and recently-added, reacting to downloads");
-137
View File
@@ -1,137 +0,0 @@
{
"module": "plex",
"version": "1",
"capabilities": [
"container-runtime"
],
"emits": [
"playback.started",
"playback.stopped",
"item.added"
],
"consumes": [
"*.download.completed"
],
"own-secrets": {
"broker": "${dir:mesh-state}/broker",
"token": "${dir:mesh-state}/token"
},
"listens": [
{
"name": "stream",
"port": 32400,
"protocol": "tcp",
"from": "mesh",
"why": "streaming and the app; reaching it from outside is a route grant later"
}
],
"accesses": [
{
"id": "movies",
"path": "/services/media/movies",
"mode": "read"
},
{
"id": "series",
"path": "/services/media/series",
"mode": "read"
},
{
"id": "anime",
"path": "/services/media/anime",
"mode": "read"
},
{
"id": "music",
"path": "/services/media/music",
"mode": "read"
},
{
"id": "audiobooks",
"path": "/services/media/audiobooks",
"mode": "read"
}
],
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "config",
"type": "directory",
"path": "/services/plex/config",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "transcode",
"type": "directory",
"path": "/services/plex/transcode",
"mode": "0700",
"owner": "1000:1000"
},
{
"id": "server",
"type": "container",
"name": "plex",
"image": "plexinc/pms-docker@sha256:83a425ae9e133b1cb2cc3b809556e01c61cd8ff65c582e41b4374bc2210bac9e",
"network": "host",
"env": {
"PLEX_UID": "1000",
"PLEX_GID": "1000",
"TZ": "Etc/UTC"
},
"volumes": [
"/services/plex/config:/config",
"/services/plex/transcode:/transcode",
"${access:movies}:/movies",
"${access:series}:/series",
"${access:anime}:/anime",
"${access:music}:/music",
"${access:audiobooks}:/audiobooks"
]
},
{
"id": "runtime",
"type": "container",
"name": "mesh-plex",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/token:/run/secrets/token:ro",
"/services/plex/config:/var/lib/plex/config:ro"
],
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_PLEX_URL": "http://127.0.0.1:32400",
"MESH_PLEX_TOKEN_FILE": "/run/secrets/token",
"MESH_PLEX_DATA_DIR": "/var/lib/plex"
},
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
}
]
}
}
-14
View File
@@ -1,14 +0,0 @@
{
"name": "@novox/module-plex",
"version": "0.1.0",
"description": "plex — media server. Its API client, tools and events live here (novox/hq ADR 0039).",
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
-74
View File
@@ -1,74 +0,0 @@
// plex's tools — moved here from the shared sdk (novox/hq ADR 0039), importing plex's own client.
// They return structured data; the mesh serves them through the sdk's tool harness.
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { PlexClient } from "../client.js";
export function getPlexTools(plex: PlexClient): ToolDefinition[] {
return [
{
name: "plex_status",
description: "Plex server status: server info, libraries, active sessions, recently added.",
input: {},
run: async () => {
const [server, libraries, sessions, recent] = await Promise.all([
plex.getServerInfo(),
plex.getLibraries(),
plex.getSessions(),
plex.getRecentlyAdded(10),
]);
return { server, libraries, sessions, recentlyAdded: recent };
},
},
{
name: "plex_reachable",
description: "Health probe: whether the Plex server answers, and which server it is. Never fails.",
input: {},
run: async () => plex.reachable(),
},
{
name: "plex_search",
description: "Search across all Plex libraries — movies, shows, episodes, music.",
input: { query: { type: "string", description: "the search query" } },
run: async (args) => ({ query: String(args.query), results: await plex.search(String(args.query)) }),
},
{
name: "plex_sessions",
description: "Active Plex playback sessions — who is watching what, and where.",
input: {},
run: async () => {
const sessions = await plex.getSessions();
return { count: sessions.length, sessions };
},
},
{
name: "plex_recently_added",
description: "Recently added media in Plex.",
input: { limit: { type: "number", description: "how many items (default 20)" } },
run: async (args) => ({ items: await plex.getRecentlyAdded(args.limit ? Number(args.limit) : 20) }),
},
{
name: "plex_refresh",
description: "Ask Plex to rescan its libraries so new files on disk become visible items.",
input: { library: { type: "string", description: "a library section key; omitted rescans all" } },
run: async (args) => {
if (args.library) {
await plex.refreshLibrary(String(args.library));
return { refreshed: String(args.library) };
}
await plex.refreshAll();
return { refreshed: "all" };
},
},
];
}
// The tools exist only when a token can be found; without one, plex contributes none rather than
// failing the whole runtime.
registerModuleTools("plex", (env) => {
try {
return getPlexTools(PlexClient.fromEnv(env));
} catch {
return [];
}
});
-12
View File
@@ -1,12 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "tools/index.ts"]
}
-30
View File
@@ -1,30 +0,0 @@
# portainer's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the sdk it will run against. The compiler
# is invoked by its real path: node_modules/.bin entries are launcher symlinks the base image
# resolved away.
WORKDIR /app/modules/portainer
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts tools/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/portainer/dist /app/modules/portainer/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled. A container that instead ran only its
# provisioner (`run`) served no tools and emitted no events; a container that named no command
# ran no provisioner at all.
ENV MESH_TOOL_MODULES=/app/modules/portainer/dist/tools/index.js
-107
View File
@@ -1,107 +0,0 @@
// The Portainer API client — portainer's own code, living in the module (novox/hq ADR 0039).
// portainer is tools-only: its "events" would really be the underlying containers' lifecycle,
// which the host owns and emits — so this module reads Portainer's own resources (endpoints,
// stacks, containers) and exposes them, and stops there.
import { readFileSync } from "node:fs";
export interface PortainerEndpoint {
id: number;
name: string;
type: number;
url: string;
status: number;
}
export interface PortainerStack {
id: number;
name: string;
type: number;
endpointId: number;
status: number;
}
export interface PortainerContainer {
id: string;
names: string[];
image: string;
state: string;
status: string;
}
/** The settings-merged config the mesh delivers (novox/hq ADR 0046): { url, apiKey, token, password, user, ... }. */
function meshConfig(file?: string): Record<string, string> {
if (!file) return {};
try { return JSON.parse(readFileSync(file, "utf8")) as Record<string, string>; }
catch { return {}; }
}
export class PortainerClient {
readonly baseUrl: string;
constructor(
url: string,
private readonly token: string,
) {
this.baseUrl = url.replace(/\/+$/, "");
}
/**
* Build from the module's resolved environment. The URL is MESH_PORTAINER_URL (or the local
* dashboard port) and the API token is MESH_PORTAINER_TOKEN — an access token minted in
* Portainer, sent as X-API-Key. Throws when no token is configured, so a misconfigured module
* exposes nothing rather than calling Portainer unauthenticated.
*/
static fromEnv(env: NodeJS.ProcessEnv = process.env): PortainerClient {
const cfg = meshConfig(env.MESH_PORTAINER_CONFIG_FILE);
const url = cfg.url ?? env.MESH_PORTAINER_URL ?? `https://127.0.0.1:${env.PORTAINER_PORT ?? "9443"}`;
const token = cfg.token ?? env.MESH_PORTAINER_TOKEN;
if (!token) throw new Error("no Portainer token — set MESH_PORTAINER_TOKEN");
return new PortainerClient(url, token);
}
private async get<T>(path: string): Promise<T> {
const res = await fetch(`${this.baseUrl}${path}`, { headers: { "X-API-Key": this.token } });
if (!res.ok) throw new Error(`Portainer ${path}: ${res.status} ${await res.text()}`);
return res.json() as Promise<T>;
}
/** The environments (endpoints) Portainer manages — each a Docker host or cluster it talks to. */
async listEndpoints(): Promise<PortainerEndpoint[]> {
const raw = await this.get<any[]>("/api/endpoints");
return (raw ?? []).map((e) => ({
id: e.Id,
name: e.Name,
type: e.Type,
url: e.URL,
status: e.Status,
}));
}
/** The stacks (compose/swarm deployments) Portainer knows about. */
async listStacks(): Promise<PortainerStack[]> {
const raw = await this.get<any[]>("/api/stacks");
return (raw ?? []).map((s) => ({
id: s.Id,
name: s.Name,
type: s.Type,
endpointId: s.EndpointId,
status: s.Status,
}));
}
/**
* The containers on one endpoint, read through Portainer's Docker API proxy. Includes stopped
* containers, so the caller sees the whole picture rather than only what is running.
*/
async listContainers(endpointId: number): Promise<PortainerContainer[]> {
const raw = await this.get<any[]>(`/api/endpoints/${endpointId}/docker/containers/json?all=1`);
return (raw ?? []).map((c) => ({
id: c.Id,
names: c.Names ?? [],
image: c.Image,
state: c.State,
status: c.Status,
}));
}
}

Some files were not shown because too many files have changed in this diff Show More