Waves 2-3: nine modules' code moves into bundles the node's runtime serves (hq ADR 0198, to-be 38 WP4c) #248

2026-10-03T22:51:40Z

mesh-admin commented

2026-10-03 22:51:40 +00:00

Pair: mesh-controller #254 updates the three gitea tests that compose the forge from this catalogue. Merge them together. Rebased onto main after wave 1 (#245) merged; the diff is these nine commits only.

Waves 2 and 3 of hq to-be 38 WP4c, under ADR 0198, 0188, 0192 and 0193. There is one commit per module. Each module's own long-running code leaves its runtime container and becomes one TypeScript bundle named code. The node's runtime (node-tools) launches its loads; steps and schedules become process resources. The container, build.on mesh-tools, the Dockerfile and the broker own-secret are removed. mesh-state is removed only where nothing else names it.

Steps carry their words in the process's env. mesh-controller #250 fills ${port:…} there and mesh-host #85 runs a run-once process as its own oneshot unit, so the 0600 env-file workaround from wave 1 is not used here. Wave 1's mosquitto (bootstrap-env), home-assistant (provisions-env) and nodered (mqtt-env) could now drop theirs: each holds only ${port:} and ${dir:} values.

Per module

Module	Runtime loads	Steps / schedules / packages	Notes
audit-logger	`index.js` (subscribes `#`)		trail at `${dir:trail}/audit.log`
gitea	handlers, tools, provisioner		words are host paths
mesh-vault	handlers, tools, provisioner		`mesh-state` removed
records	handlers, tools	package `git`
mailu	handlers, tools, provisioner		`mailu-admin` now publishes 8080 to `from: machine` (new `listens` entry `admin-api`); `MESH_MAILU_URL=http://127.0.0.1:${port:8080}/api/v1`. automx keeps its image. Mail is still read through `docker exec mailu-imap`
lab	tools	packages git, make, python, file, iproute2, sudo, npm, go, incus	code change: the forge setting is read from `MESH_LAB_ENV_FILE` (lab.env) at each call, because a bundle's words may not carry `${setting:…}`. `mesh-state` removed
route-adapter	none	run-once process `route-adapter` (`node index.js`), restart-on kept
openai-consumer	none	process `openai-consumer-apply`, schedule `/5 * * *`
anthropic-consumer	`usage/index.js`	process `anthropic-consumer-apply`, schedule `/5 * * *`	code change: usage used to spawn the runtime image's `emit` with the module's credential. It now emits through the SDK, so the runtime runs it once at start and then every 5 min. `mesh-state` removed

Left as containers

These three are not touched. Each needs a mesh change first.

mesh-catalog imports pg.
mongodb and mssql shell out to mongosh and sqlcmd, which no machine's system package provides. ADR 0198 says they need a driver in the bundle.

The TypeScript toolchain resolves imports only from the toolchain image's own node_modules, and external only re-copies that directory. So the builder cannot install a module's own npm dependencies today. I checked locally that pg@8, mongodb@6 and mssql@11 each esbuild into one ESM file with no external and run outside any node_modules. Once the builder installs a bundle's declared dependencies before compiling (or the toolchain carries them), they move like the others. mesh-catalog's prepares: true then becomes a run-once process running prepare/index.js, since prepares needs a container of the module's own artifact.

Validation

Builder-style build per module. Each module was compiled with tsc (--rootDir ., NodeNext) against @novox/mesh-sdk@0.1.6 from the Gitea registry. Launchers were written, then every entrypoint and launcher was esbuilt with the builder's flags. All nine bundle cleanly with no external.
Every load launched like node-tools does. Each load was started with its words, and every one answered initialize and tools/list. The expected mesh/subscribe and mesh/publish asks were seen. The route-adapter step ran locally to completion.
Module tests. gitea 13/13, mesh-vault 5/5, records 3/3, route-adapter 11/11, anthropic-consumer 4/4. audit-logger 0/1 fails the same way on main: its in-memory broker matches only # and the test subscribes **.
Controller catalogue tests. go test ./internal/catalogue/... ./cmd/mesh-controller/ was run with this branch as the sibling catalogue (with #254), store-backed. All pass except the known unrelated TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves. Re-run after the rebase onto main: pass.
Throwaway declaration test (deleted after use). Each module was composed beside the node's runtime with every need, secret and setting supplied. Each composes. No container left runs module code; what remains is third-party images. No ${…} is left except ${secret:…}. Every word and step env resolves to host paths and ports. Every load is in MESH_TOOL_MODULES. Files a word names exactly are in the runtime's restart-on.

Needs live proof

gitea's pull.merged / repo.created from the runtime. The builder's merge-driven plans depend on it, so check this first on the build node.
audit-logger receives every event, which proves the runtime's grant for **.
Every provisioner acts: gitea npm, mesh-vault, mailu smtp.
The runtime account can reach:
- the docker socket, for mailu's doveadm and lab;
- the incus socket, for lab;
- the operator-owned state that the composer chowns.
mailu's admin API answers on the published loopback port.
records clones into ${dir:checkout} as the operator account. A checkout left root-owned by the container may need re-cloning.
lab: the system go is not the image's pinned 1.26.8, and incus / npm come from packages.
The scheduled processes fire every 5 min and write the credentials as before.
anthropic usage events reach model-usage.
route-adapter re-runs on a route change and the proxy picks the files up.

Nothing live was touched. Not merged.

**Pair:** mesh-controller #254 updates the three gitea tests that compose the forge from this catalogue. Merge them together. Rebased onto main after wave 1 (#245) merged; the diff is these nine commits only. Waves 2 and 3 of hq to-be 38 WP4c, under ADR 0198, 0188, 0192 and 0193. There is one commit per module. Each module's own long-running code leaves its runtime container and becomes one TypeScript bundle named `code`. The node's runtime (node-tools) launches its `loads`; steps and schedules become `process` resources. The container, `build.on` mesh-tools, the Dockerfile and the `broker` own-secret are removed. `mesh-state` is removed only where nothing else names it. **Steps carry their words in the process's `env`.** mesh-controller #250 fills `${port:…}` there and mesh-host #85 runs a run-once process as its own oneshot unit, so the 0600 env-file workaround from wave 1 is not used here. Wave 1's mosquitto (`bootstrap-env`), home-assistant (`provisions-env`) and nodered (`mqtt-env`) could now drop theirs: each holds only `${port:}` and `${dir:}` values. ## Per module | Module | Runtime loads | Steps / schedules / packages | Notes | |---|---|---|---| | audit-logger | `index.js` (subscribes `#`) | | trail at `${dir:trail}/audit.log` | | gitea | handlers, tools, provisioner | | words are host paths | | mesh-vault | handlers, tools, provisioner | | `mesh-state` removed | | records | handlers, tools | package `git` | | | mailu | handlers, tools, provisioner | | `mailu-admin` now publishes 8080 to `from: machine` (new `listens` entry `admin-api`); `MESH_MAILU_URL=http://127.0.0.1:${port:8080}/api/v1`. automx keeps its image. Mail is still read through `docker exec mailu-imap` | | lab | tools | packages git, make, python, file, iproute2, sudo, npm, go, incus | **code change:** the forge setting is read from `MESH_LAB_ENV_FILE` (lab.env) at each call, because a bundle's words may not carry `${setting:…}`. `mesh-state` removed | | route-adapter | none | run-once process `route-adapter` (`node index.js`), restart-on kept | | | openai-consumer | none | process `openai-consumer-apply`, schedule `*/5 * * * *` | | | anthropic-consumer | `usage/index.js` | process `anthropic-consumer-apply`, schedule `*/5 * * * *` | **code change:** usage used to spawn the runtime image's `emit` with the module's credential. It now emits through the SDK, so the runtime runs it once at start and then every 5 min. `mesh-state` removed | ## Left as containers These three are not touched. Each needs a mesh change first. - **mesh-catalog** imports `pg`. - **mongodb** and **mssql** shell out to `mongosh` and `sqlcmd`, which no machine's system package provides. ADR 0198 says they need a driver in the bundle. The TypeScript toolchain resolves imports only from the toolchain image's own `node_modules`, and `external` only re-copies that directory. So the builder cannot install a module's own npm dependencies today. I checked locally that `pg@8`, `mongodb@6` and `mssql@11` each esbuild into one ESM file with no `external` and run outside any node_modules. Once the builder installs a bundle's declared `dependencies` before compiling (or the toolchain carries them), they move like the others. mesh-catalog's `prepares: true` then becomes a run-once process running `prepare/index.js`, since `prepares` needs a container of the module's own artifact. ## Validation - **Builder-style build per module.** Each module was compiled with tsc (`--rootDir .`, NodeNext) against `@novox/mesh-sdk@0.1.6` from the Gitea registry. Launchers were written, then every entrypoint and launcher was esbuilt with the builder's flags. All nine bundle cleanly with no `external`. - **Every load launched like node-tools does.** Each load was started with its words, and every one answered `initialize` and `tools/list`. The expected `mesh/subscribe` and `mesh/publish` asks were seen. The route-adapter step ran locally to completion. - **Module tests.** gitea 13/13, mesh-vault 5/5, records 3/3, route-adapter 11/11, anthropic-consumer 4/4. audit-logger 0/1 fails the same way on main: its in-memory broker matches only `#` and the test subscribes `**`. - **Controller catalogue tests.** `go test ./internal/catalogue/... ./cmd/mesh-controller/` was run with this branch as the sibling catalogue (with #254), store-backed. All pass except the known unrelated `TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves`. Re-run after the rebase onto main: pass. - **Throwaway declaration test** (deleted after use). Each module was composed beside the node's runtime with every need, secret and setting supplied. Each composes. No container left runs module code; what remains is third-party images. No `${…}` is left except `${secret:…}`. Every word and step env resolves to host paths and ports. Every load is in `MESH_TOOL_MODULES`. Files a word names exactly are in the runtime's restart-on. ## Needs live proof - gitea's `pull.merged` / `repo.created` from the runtime. The builder's merge-driven plans depend on it, so check this first on the build node. - audit-logger receives every event, which proves the runtime's grant for `**`. - Every provisioner acts: gitea npm, mesh-vault, mailu smtp. - The runtime account can reach: - the docker socket, for mailu's doveadm and lab; - the incus socket, for lab; - the operator-owned state that the composer chowns. - mailu's admin API answers on the published loopback port. - records clones into `${dir:checkout}` as the operator account. A checkout left root-owned by the container may need re-cloning. - lab: the system `go` is not the image's pinned 1.26.8, and `incus` / `npm` come from packages. - The scheduled processes fire every 5 min and write the credentials as before. - anthropic usage events reach model-usage. - route-adapter re-runs on a route change and the proxy picks the files up. Nothing live was touched. Not merged.

jschoubben added 9 commits 2026-10-03 22:52:33 +00:00

audit-logger: its handler runs in the node's runtime (hq ADR 0198) dd93cfd613

The mesh-audit-logger container goes with its Dockerfile, build bases and bus credential: its one entrypoint is a load of one bundle, which subscribes to every event through the runtime and writes the trail at the host path the container used to mount.

gitea: its watcher, tools and provisioner run in the node's runtime (hq ADR 0198) 944f086ec7

The mesh-gitea container goes with its Dockerfile, build bases and bus credential; its env becomes the bundle's words with mount targets folded back to host paths: the config file, the admin password and the kept-token state directory are read where the mesh writes them.

mesh-vault: its handlers, tools and provisioner run in the node's runtime (hq ADR 0198) 043ae17fbf

The mesh-vault container goes with its Dockerfile, build bases, bus credential and state directory; its env was already host paths, so it becomes the bundle's words unchanged.

records: its consumer and tools run in the node's runtime (hq ADR 0198) 8877f893e5

The records container goes with its Dockerfile, build bases and bus credential. The checkout, the config file and the origin file are read where the mesh writes them, and git comes from the machine's git package instead of the image's apt layer.

mailu: its handlers, tools and provisioner run in the node's runtime (hq ADR 0198) 3b8164f1c0

The mesh-mailu container goes with its Dockerfile, the mesh-tools build bases and its bus credential; automx keeps its own image. The code reached the admin API by its name on the mailu network, which a process on the machine cannot, so the admin container publishes 8080 to this machine only and the bundle reaches it on loopback at that port. Mail is still read through docker exec into mailu-imap, so the runtime's account needs the docker socket as nextcloud's does.

lab: its tools run in the node's runtime (hq ADR 0198) 59c42b2086

The mesh-lab container goes with its Dockerfile, build bases, bus credential and state directory. What the image installed — git, make, python, file, iproute2, sudo, npm, go and the incus client — are packages of the machine, and docker and incus are reached through their sockets as the runtime's account. The forge is an operator's setting, which reaches a file and never a bundle's words, so the tools read it from the env-file the mesh already fills, at each call; that is the one code change.

route-adapter: its step is a run-once process (hq ADR 0198) 35ef72081f

The mesh-route-adapter container goes with its Dockerfile and build bases. The step runs node on the bundle as a run-once process, reading what the mesh contributed and its config where the mesh writes them and writing the proxy's dynamic directory at the path the container used to mount; it still runs again when a route or its config changes.

openai-consumer: its apply is a scheduled process (hq ADR 0198) 3c6b70845c

The mesh-openai-consumer-apply container goes with its Dockerfile and build bases. The same entrypoint runs every five minutes as a process on the machine, reading the binding and writing the credentials at the host paths the container used to mount.

anthropic-consumer: its usage runs in the node's runtime, and its apply is a scheduled process (hq ADR 0198) 568674fef7

Both containers go with the Dockerfile, build bases, bus credential and state directory. apply needs no bus and runs every five minutes as a process on the machine at the host paths the container mounted. usage emitted by spawning the runtime image's own emit command with the module's credential, which exists nowhere now, so it is loaded by the node's runtime instead: it emits through the SDK as this module and reads on the cadence the schedule gave it, once at start and every five minutes. That is the one code change.

jschoubben force-pushed feat/0198-waves-2-3-module-code-moves from 4aacea5453 to 568674fef7

2026-10-03 22:52:33 +00:00

Compare

mesh-admin merged commit bd2123166f into main

2026-10-03 23:01:18 +00:00

mesh-admin deleted branch feat/0198-waves-2-3-module-code-moves

2026-10-03 23:01:18 +00:00

mesh-admin referenced this pull request

2026-10-03 23:18:30 +00:00

The last three: mesh-catalog, mongodb and mssql code moves into bundles the node's runtime serves (hq ADR 0198, to-be 38 WP4c) #250

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#248