7.0 KiB
topic, status, date, deciders, reconstructed, extends
| topic | status | date | deciders | reconstructed | extends |
|---|---|---|---|---|---|
| what runs on it | accepted | 2026-10-04 | jochen | false | 02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md |
202. A module keeps its current state in key-value buckets it declares, and reaches them through the runtime
Context
A module's code reaches the bus through the node's runtime: it publishes events, subscribes to them and asks tools (ADR 0198). Events are kept for a week and replayed to a consumer that was away; requests are kept nowhere. What neither gives is the current value of something, seen by every machine, including one that joins after it was written. The first module to need it — the operator's agent on a machine — registers MCP servers for every machine as events, and a machine assigned later never hears of them; and it would replay a week of licence rotations where it needs only the binding that holds now. Research 024 measured the alternatives and the grants against a real server.
Design 32 §4 already names state as one of the mesh's relationships — 1:1, last per subject — and reserves it to the mesh's own declarations. Design 25 §1 expects key-value buckets on the bus.
Considered Options
- Key-value buckets a module declares, created by the controller, reached through the runtime. Chosen.
- State as events on EVENTS, read last-per-subject. Rejected: retention is per stream and EVENTS keeps seven days, so a value unchanged for a week disappears; a second stream over the same subjects is refused by the server (design 32 §3). And events give no get, list or delete.
- A last-per-subject stream per module, written by hand. Rejected: it is what a key-value bucket is on the server, without the client's get, list, delete and watch — the mesh writing NATS's key-value layer again.
- State in a module's own files or database, shared by asking a tool. Rejected for state every machine must see: a machine joining later has to know whom to ask and poll, and an owner that is down answers nothing — the property the bus exists to remove.
Decision
1. A module declares its state by name. state names the buckets it owns, by local name; every
instance of the module may write and read them. reads names another module's bucket as
<module>.<name>, read-only. A bucket's options are its owner's: how many past values a key keeps,
and how long a value lives. A manifest names no bucket, stream or subject (design 32 §1).
2. One bucket per module per name, mesh-wide. A key may name a machine by the module's own convention; the mesh does not scope buckets per machine.
3. The controller creates the buckets, from the catalogue, on every raise — from registration, like a seat's stream, so a reader can watch a bucket whose owner is not yet assigned anywhere. A module never creates one. The runtime's grant on each bucket is the union of what its carried modules may do: an owner's instances write and read, a reader's read.
4. Each assignment is issued its buckets in its membership (ADR 0160),
by the name the module uses for each and whether it may write. The runtime serves mesh/state.get,
put, delete, keys and watch on the bundle's channel from that list, and refuses — with the
reason — a bucket the module was not issued and a write to one it only reads. A watch delivers the
current values first, without deletions, then an end-of-current marker, then every change, each as a
mesh/state request the bundle answers.
5. No secret is stored in a bucket, sealed or not. A bucket is a stream, and design 32 §10 keeps every secret off streams. A value that needs a secret names it; the secret travels on request/reply.
6. The mesh caps size; a bucket outlives its module. One value per key and no expiry unless the owner says otherwise; at most 256 KiB a value and 64 MiB a bucket. Unassigning a module leaves its buckets and what is in them (ADR 0030); a bucket whose declaration is gone from the catalogue is reported, never removed by the mesh.
Consequences
- A machine that joins reads the current state at once, and every machine sees a change as it happens, with no consumer created per reader and nothing replayed.
- The runtime's channel has a sixth verb family, and the SDKs a small state surface over it — a contract, which ADR 0039 admits: it changes when the verbs do, rarely, and every module should be rebuilt when it does.
- What got harder: the runtime must keep each module to its own buckets, because one principal per machine carries all of them and the server enforces only the union. A write the server refuses surfaces to a client as a timeout, not a refusal, so the runtime's own refusal is what a module sees.
- The secrets rule is only partly mechanical. Sealed values cannot be recognised; the runtime refuses a value with a field whose name says it is a credential, which catches the ordinary mistake and not a determined one. For the operator's agent this means an MCP server's authorisation header stays out of its bucket.
- Buckets accumulate as modules come and go; that they are reported rather than removed is the price of not deleting data.
How it is checked
| Rule | Checked by |
|---|---|
| A manifest's state names are local, and a read names a bucket its owner declares | the catalogue's registration check, per manifest; a catalogue test that every reads whose owner is present names a bucket that owner declares |
| Buckets exist for every declared state | the controller's raise asserts them idempotently; its test over a real bus |
| Owners write, readers only read | the composer's test of the grants, per principal kind; the runtime's refusal test over a real bus |
| A watch hands current values first, without deletions, then changes | the runtime's test over a real bus |
| No credential-named field in a value | the runtime's refusal test |
| Live | one module puts on one machine and another machine's watch sees it; a machine assigned afterwards reads it at start |