diff --git a/01-RESEARCH/022-where-a-modules-long-running-code-runs/00-overview.md b/01-RESEARCH/022-where-a-modules-long-running-code-runs/00-overview.md new file mode 100644 index 0000000..86b4f32 --- /dev/null +++ b/01-RESEARCH/022-where-a-modules-long-running-code-runs/00-overview.md @@ -0,0 +1,41 @@ +--- +status: graduated +initiated: 2026-10-03 +touches: [the tool runtime, the per-module containers, the SDK, the bus grants, 03-DESIGN/01-to-be/38-building-the-operators-machine.md] +became: [02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md, 03-DESIGN/01-to-be/38-building-the-operators-machine.md] +--- + +# 022 — Where a module's long-running code runs + +## What was investigated + +Twenty-three modules still run their own code in a container built on the runtime's image. Their tools +can move as bundles ([ADR 0192](../../02-DECISIONS/0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md), +[ADR 0193](../../02-DECISIONS/0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md)); +the rest of what those containers run cannot yet. This asks where that code goes and how it reaches +what its container handed it. + +## What that code is, measured 2026-10-03 + +| | modules | +|---|---| +| subscribes to events on the bus | audit-logger (everything), mesh-catalog (two seat events), mesh-vault, records (`gitea.pull.merged`), and postgres, mongodb, mssql, redis, mosquitto logging their own lifecycle | +| provisioners: read the grants the mesh delivered as files, act on the backend, emit | 12 | +| a run-once preparation step | mesh-catalog | +| a command-line client of the backend | psql, mosquitto_ctrl, git (packages on every machine's system); mongosh, sqlcmd (not in its repositories) | +| a service reached by a container name | icecast, mailu-admin, minio, mongodb-server, mssql | +| a main of its own | anthropic-consumer, openai-consumer, route-adapter | + +A provisioner needs nothing a launched bundle lacks: files named by its words, its backend, and an emit +that already travels through the runtime. The one thing missing is **a subscription** — events +delivered to the module's code, acknowledged when it has handled them. + +## Options + +1. **The runtime launches it and is its bus**: the stdio channel gains a subscription; the runtime + binds the module's durable consumer and delivers each event to the child, acknowledging when the + child answers. One bus connection per machine; any language. Chosen. +2. **A process per module with its own bus client and credential.** Every language's SDK would carry + a transport and every module a credential on disk — what ADR 0188 rejected for tools, for the same + reasons. +3. **Keep the containers for this code.** Leaves ADR 0188's rule broken for 23 modules indefinitely. diff --git a/02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md b/02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md new file mode 100644 index 0000000..1344666 --- /dev/null +++ b/02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md @@ -0,0 +1,79 @@ +--- +topic: what runs on it +status: accepted +date: 2026-10-03 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md +--- + +# 198. A module's long-running code is launched by the node's runtime, and reaches the bus through it + +## Context + +[ADR 0193](0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md) made +every tools bundle a child the node's runtime launches, speaking MCP over stdio, and gave the channel +one bus verb: a tool's emit, published by the runtime as the module. Twenty-three modules still run the +rest of their own code — event handlers, provisioners, a preparation step, three mains — in a container +on the runtime's image, because that code needs what a container gave it: a bus connection that can +*subscribe*, and its module's words. Research [022](../01-RESEARCH/022-where-a-modules-long-running-code-runs/00-overview.md) +measured what it uses. [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md) +§1 says a module's own code is never an image. + +## Considered Options + +1. **The runtime launches it and is its bus.** Chosen. +2. A process per module with its own bus client and credential. Rejected for the reasons ADR 0188 + rejected it for tools: a transport in every language's SDK, a credential per module on disk, and a + bus change rebuilding every module. +3. Keep the containers for it. Rejected: ADR 0188's rule stays broken for most of the catalogue. + +## Decision + +**1. A module's long-running code is a bundle the node's runtime launches and supervises,** exactly as +its tools are: an executable entrypoint, given the runtime's words and its module's +([ADR 0192](0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md)), +started at the runtime's start and again when it exits. A bundle may serve tools, run long, or both. + +**2. The runtime is its bus.** The stdio channel carries, beside MCP, the mesh's verbs a module's code +uses: `mesh/publish` (ADR 0193), **`mesh/subscribe`** — the runtime binds that module's durable +consumer, as the module's own runtime did, and delivers each event to the child as a `mesh/event` +request, acknowledging it on the bus only when the child has answered — and **`mesh/ask`**, a tool +call made on the module's behalf. The runtime's account is granted what each module it carries +consumes, and the consumer keeps the module's name, so no event is lost or replayed in the move. + +**3. A preparation step is a run-once process the host runs before the runtime starts the module,** +with its module's words and no bus — what it already was. + +**4. What a container reached by its network is reached on the machine.** A service by its published +port (`${port:…}`) on loopback; a backend's command-line client as a package of the machine's system, +or, where the system has none, the backend's own driver inside the bundle. + +## Consequences + +- The per-module containers go, and with them the runtime image as a way module code runs; ADR 0188's + registration rule can then refuse a module's own image without exception. +- One bus connection per machine carries every module's events; a module's handler is a function of + the events it is handed, in any language, with no bus client of its own. +- What got harder: the runtime holds every carried module's consumer and must not acknowledge an event + before the child has handled it — a child that dies mid-event leaves it unacknowledged, and it is + delivered again. The runtime's grants widen to what its modules consume. +- Two modules need code before they can move: their backends' clients exist on no machine's system, + so they talk to the backend through a driver instead. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A subscribed event reaches the child and is acknowledged only after it answered | the runtime's test over a real bus: a child that answers is acknowledged once; one that dies mid-event is delivered again | +| The module's consumer keeps its name | the composer's test: the durable consumer the runtime binds is the one the module's own runtime bound | +| No module's own code is an image | the catalogue's registration check, without exception, once the last container has moved | +| Live | every moved module's provisioner and handlers act, on their machines, from the node's runtime; `docker ps` shows no runtime-image container on any machine | + +## References + +- [ADR 0188](0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md), + [ADR 0192](0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md), + [ADR 0193](0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md) +- Research [022](../01-RESEARCH/022-where-a-modules-long-running-code-runs/00-overview.md) +- [to-be 38](../03-DESIGN/01-to-be/38-building-the-operators-machine.md) WP4c diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 948c0f6..c645d64 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -297,6 +297,7 @@ python3 00-META/checks/index.py fail if stale - **0193** — [Every bundle the runtime serves is launched, and the runtime knows no language](0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md) - **0195** — [The mesh's tools are found by address, not announced whole](0195-the-meshs-tools-are-found-by-address-not-announced-whole.md) - **0197** — [Every tool announces itself on the bus, in the NATS services protocol](0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md) +- **0198** — [A module's long-running code is launched by the node's runtime, and reaches the bus through it](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md) ### How it is built diff --git a/03-DESIGN/01-to-be/38-building-the-operators-machine.md b/03-DESIGN/01-to-be/38-building-the-operators-machine.md index 2f56b0b..0299ddd 100644 --- a/03-DESIGN/01-to-be/38-building-the-operators-machine.md +++ b/03-DESIGN/01-to-be/38-building-the-operators-machine.md @@ -14,6 +14,7 @@ decisions: - 02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md - 02-DECISIONS/0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md - 02-DECISIONS/0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md + - 02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md --- # 38. Building the operator's machine @@ -297,6 +298,17 @@ consumes with, the words its code reads at import, the packages the image instal client), and the service it reaches by a container network name. That begins with a decision record, after which the twenty-three move and the registration gate refuses the container shape for all. +*Decided 2026-10-03, [ADR 0198](../../02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md):* +the node's runtime launches that code as it launches tools, and is its bus — `mesh/subscribe` and +`mesh/ask` beside `mesh/publish` on the stdio channel, the module's own durable consumer bound by the +runtime and acknowledged only after the child answered. **In order:** the runtime's subscription and +its grants; the SDK's `on` and provisioner bound to the channel; then the modules in three waves — the +provisioners and handlers whose backends are reached on loopback with a system package (postgres, +redis, mosquitto, influxdb, keycloak, umami, cloudflare-dns, grafana, icecast, home-assistant, nodered, +nextcloud, minio), the two whose clients exist on no system (mongodb, mssql: a driver in the bundle), +and last the mesh's own (mesh-catalog, mesh-vault, records, gitea, mailu, audit-logger, lab, and the +three mains). + ## WP5 — The shell, on a server first *mesh-catalog #224, already written. Half a day to assign and prove.*