jochen
e1b0bbde91
Events miss a machine that joins after them and replay history where only the latest matters. A module now declares state it owns and reads; the controller creates the buckets, the runtime serves them on the bundle's channel. Designs 32 and 25 amended; grants measured against a running server.
107 lines
7.0 KiB
Markdown
107 lines
7.0 KiB
Markdown
---
|
|
topic: what runs on it
|
|
status: accepted
|
|
date: 2026-10-04
|
|
deciders: jochen
|
|
reconstructed: false
|
|
extends: 02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md
|
|
---
|
|
|
|
# 201. A module keeps its current state in key-value buckets it declares, and reaches them through the runtime
|
|
|
|
## Context
|
|
|
|
A module's code reaches the bus through the node's runtime: it publishes events, subscribes to them
|
|
and asks tools ([ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)).
|
|
Events are kept for a week and replayed to a consumer that was away; requests are kept nowhere. What
|
|
neither gives is **the current value of something**, seen by every machine, including one that joins
|
|
after it was written. The first module to need it — the operator's agent on a machine — registers MCP
|
|
servers for every machine as events, and a machine assigned later never hears of them; and it would
|
|
replay a week of licence rotations where it needs only the binding that holds now. Research
|
|
[024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md) measured the alternatives and
|
|
the grants against a real server.
|
|
|
|
[Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 already names *state* as one of the
|
|
mesh's relationships — 1:1, last per subject — and reserves it to the mesh's own declarations.
|
|
[Design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 expects key-value buckets on the bus.
|
|
|
|
## Considered Options
|
|
|
|
1. **Key-value buckets a module declares, created by the controller, reached through the runtime.**
|
|
Chosen.
|
|
2. **State as events on EVENTS, read last-per-subject.** Rejected: retention is per stream and EVENTS
|
|
keeps seven days, so a value unchanged for a week disappears; a second stream over the same subjects
|
|
is refused by the server (design 32 §3). And events give no get, list or delete.
|
|
3. **A last-per-subject stream per module, written by hand.** Rejected: it is what a key-value bucket
|
|
is on the server, without the client's get, list, delete and watch — the mesh writing NATS's
|
|
key-value layer again.
|
|
4. **State in a module's own files or database, shared by asking a tool.** Rejected for state every
|
|
machine must see: a machine joining later has to know whom to ask and poll, and an owner that is
|
|
down answers nothing — the property the bus exists to remove.
|
|
|
|
## Decision
|
|
|
|
**1. A module declares its state by name.** `state` names the buckets it owns, by local name; every
|
|
instance of the module may write and read them. `reads` names another module's bucket as
|
|
`<module>.<name>`, read-only. A bucket's options are its owner's: how many past values a key keeps,
|
|
and how long a value lives. A manifest names no bucket, stream or subject (design 32 §1).
|
|
|
|
**2. One bucket per module per name, mesh-wide.** A key may name a machine by the module's own
|
|
convention; the mesh does not scope buckets per machine.
|
|
|
|
**3. The controller creates the buckets, from the catalogue, on every raise** — from registration,
|
|
like a seat's stream, so a reader can watch a bucket whose owner is not yet assigned anywhere. A module
|
|
never creates one. The runtime's grant on each bucket is the union of what its carried modules may do:
|
|
an owner's instances write and read, a reader's read.
|
|
|
|
**4. Each assignment is issued its buckets in its membership** ([ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)),
|
|
by the name the module uses for each and whether it may write. The runtime serves `mesh/state.get`,
|
|
`put`, `delete`, `keys` and `watch` on the bundle's channel from that list, and refuses — with the
|
|
reason — a bucket the module was not issued and a write to one it only reads. A watch delivers the
|
|
current values first, without deletions, then an end-of-current marker, then every change, each as a
|
|
`mesh/state` request the bundle answers.
|
|
|
|
**5. No secret is stored in a bucket, sealed or not.** A bucket is a stream, and design 32 §10 keeps
|
|
every secret off streams. A value that needs a secret names it; the secret travels on request/reply.
|
|
|
|
**6. The mesh caps size; a bucket outlives its module.** One value per key and no expiry unless the
|
|
owner says otherwise; at most 256 KiB a value and 64 MiB a bucket. Unassigning a module leaves its
|
|
buckets and what is in them ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)); a bucket
|
|
whose declaration is gone from the catalogue is reported, never removed by the mesh.
|
|
|
|
## Consequences
|
|
|
|
- A machine that joins reads the current state at once, and every machine sees a change as it
|
|
happens, with no consumer created per reader and nothing replayed.
|
|
- The runtime's channel has a sixth verb family, and the SDKs a small state surface over it — a
|
|
contract, which ADR 0039 admits: it changes when the verbs do, rarely, and every module should be
|
|
rebuilt when it does.
|
|
- What got harder: the runtime must keep each module to its own buckets, because one principal per
|
|
machine carries all of them and the server enforces only the union. A write the server refuses
|
|
surfaces to a client as a timeout, not a refusal, so the runtime's own refusal is what a module sees.
|
|
- The secrets rule is only partly mechanical. Sealed values cannot be recognised; the runtime refuses
|
|
a value with a field whose name says it is a credential, which catches the ordinary mistake and not a
|
|
determined one. For the operator's agent this means an MCP server's authorisation header stays out of
|
|
its bucket.
|
|
- Buckets accumulate as modules come and go; that they are reported rather than removed is the price
|
|
of not deleting data.
|
|
|
|
## How it is checked
|
|
|
|
| Rule | Checked by |
|
|
|---|---|
|
|
| A manifest's state names are local, and a read names a bucket its owner declares | the catalogue's registration check, per manifest; a catalogue test that every `reads` whose owner is present names a bucket that owner declares |
|
|
| Buckets exist for every declared state | the controller's raise asserts them idempotently; its test over a real bus |
|
|
| Owners write, readers only read | the composer's test of the grants, per principal kind; the runtime's refusal test over a real bus |
|
|
| A watch hands current values first, without deletions, then changes | the runtime's test over a real bus |
|
|
| No credential-named field in a value | the runtime's refusal test |
|
|
| Live | one module puts on one machine and another machine's watch sees it; a machine assigned afterwards reads it at start |
|
|
|
|
## References
|
|
|
|
- Research [024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md)
|
|
- [Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 and §10, [design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 and §3
|
|
- [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0039](0039-what-the-sdk-holds-and-refuses.md),
|
|
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md),
|
|
[ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)
|