--- topic: what runs on it status: accepted date: 2026-10-04 deciders: jochen reconstructed: false extends: 02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md --- # 202. A module keeps its current state in key-value buckets it declares, and reaches them through the runtime ## Context A module's code reaches the bus through the node's runtime: it publishes events, subscribes to them and asks tools ([ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)). Events are kept for a week and replayed to a consumer that was away; requests are kept nowhere. What neither gives is **the current value of something**, seen by every machine, including one that joins after it was written. The first module to need it — the operator's agent on a machine — registers MCP servers for every machine as events, and a machine assigned later never hears of them; and it would replay a week of licence rotations where it needs only the binding that holds now. Research [024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md) measured the alternatives and the grants against a real server. [Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 already names *state* as one of the mesh's relationships — 1:1, last per subject — and reserves it to the mesh's own declarations. [Design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 expects key-value buckets on the bus. ## Considered Options 1. **Key-value buckets a module declares, created by the controller, reached through the runtime.** Chosen. 2. **State as events on EVENTS, read last-per-subject.** Rejected: retention is per stream and EVENTS keeps seven days, so a value unchanged for a week disappears; a second stream over the same subjects is refused by the server (design 32 §3). And events give no get, list or delete. 3. **A last-per-subject stream per module, written by hand.** Rejected: it is what a key-value bucket is on the server, without the client's get, list, delete and watch — the mesh writing NATS's key-value layer again. 4. **State in a module's own files or database, shared by asking a tool.** Rejected for state every machine must see: a machine joining later has to know whom to ask and poll, and an owner that is down answers nothing — the property the bus exists to remove. ## Decision **1. A module declares its state by name.** `state` names the buckets it owns, by local name; every instance of the module may write and read them. `reads` names another module's bucket as `.`, read-only. A bucket's options are its owner's: how many past values a key keeps, and how long a value lives. A manifest names no bucket, stream or subject (design 32 §1). **2. One bucket per module per name, mesh-wide.** A key may name a machine by the module's own convention; the mesh does not scope buckets per machine. **3. The controller creates the buckets, from the catalogue, on every raise** — from registration, like a seat's stream, so a reader can watch a bucket whose owner is not yet assigned anywhere. A module never creates one. The runtime's grant on each bucket is the union of what its carried modules may do: an owner's instances write and read, a reader's read. **4. Each assignment is issued its buckets in its membership** ([ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)), by the name the module uses for each and whether it may write. The runtime serves `mesh/state.get`, `put`, `delete`, `keys` and `watch` on the bundle's channel from that list, and refuses — with the reason — a bucket the module was not issued and a write to one it only reads. A watch delivers the current values first, without deletions, then an end-of-current marker, then every change, each as a `mesh/state` request the bundle answers. **5. No secret is stored in a bucket, sealed or not.** A bucket is a stream, and design 32 §10 keeps every secret off streams. A value that needs a secret names it; the secret travels on request/reply. **6. The mesh caps size; a bucket outlives its module.** One value per key and no expiry unless the owner says otherwise; at most 256 KiB a value and 64 MiB a bucket. Unassigning a module leaves its buckets and what is in them ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)); a bucket whose declaration is gone from the catalogue is reported, never removed by the mesh. ## Consequences - A machine that joins reads the current state at once, and every machine sees a change as it happens, with no consumer created per reader and nothing replayed. - The runtime's channel has a sixth verb family, and the SDKs a small state surface over it — a contract, which ADR 0039 admits: it changes when the verbs do, rarely, and every module should be rebuilt when it does. - What got harder: the runtime must keep each module to its own buckets, because one principal per machine carries all of them and the server enforces only the union. A write the server refuses surfaces to a client as a timeout, not a refusal, so the runtime's own refusal is what a module sees. - The secrets rule is only partly mechanical. Sealed values cannot be recognised; the runtime refuses a value with a field whose name says it is a credential, which catches the ordinary mistake and not a determined one. For the operator's agent this means an MCP server's authorisation header stays out of its bucket. - Buckets accumulate as modules come and go; that they are reported rather than removed is the price of not deleting data. ## How it is checked | Rule | Checked by | |---|---| | A manifest's state names are local, and a read names a bucket its owner declares | the catalogue's registration check, per manifest; a catalogue test that every `reads` whose owner is present names a bucket that owner declares | | Buckets exist for every declared state | the controller's raise asserts them idempotently; its test over a real bus | | Owners write, readers only read | the composer's test of the grants, per principal kind; the runtime's refusal test over a real bus | | A watch hands current values first, without deletions, then changes | the runtime's test over a real bus | | No credential-named field in a value | the runtime's refusal test | | Live | one module puts on one machine and another machine's watch sees it; a machine assigned afterwards reads it at start | ## References - Research [024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md) - [Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 and §10, [design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 and §3 - [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0039](0039-what-the-sdk-holds-and-refuses.md), [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md), [ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)