diff --git a/01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md b/01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md new file mode 100644 index 0000000..91bc89e --- /dev/null +++ b/01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md @@ -0,0 +1,152 @@ +--- +status: graduated +initiated: 2026-10-04 +touches: [the bus, what a module declares, the tool runtime, the SDK, the bus grants, 03-DESIGN/01-to-be/25-the-bus-on-nats.md, 03-DESIGN/01-to-be/32-what-a-module-declares.md] +became: [02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md, 03-DESIGN/01-to-be/32-what-a-module-declares.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md] +--- + +# 024 — State a module keeps on the bus + +## What is investigated + +A place on the bus where a module's own code keeps **current state** — not history — that every +machine sees, including a machine that joins after the state was written: put, get, delete, list and +watch, reached through the node's runtime the way a bundle already publishes, asks and subscribes +([ADR 0198](../../02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)). +On NATS that is a key-value bucket. The questions are what a module declares, who creates the +bucket, what the grants are, what the runtime's verbs are, and what may never be stored. + +## Why + +The mesh carries two kinds of module traffic and a third is missing. + +- **Events** land in the EVENTS stream: limits retention, seven days, ten thousand messages per + subject, a durable consumer per consuming module that replays what it missed. Never a secret + ([design 32](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §10). +- **Requests** are core request/reply — tool calls, a bundle's `mesh/ask` — and are kept nowhere. + +Neither is *the current value of something*. Two cases from the first module that needs it, the +operator's agent on a machine ([design 36](../../03-DESIGN/01-to-be/36-the-operators-agent-on-a-machine.md)): + +1. **An MCP server registered for every machine.** Registering emits an event every machine's copy + of the module consumes. A machine the module is assigned to *after* the registration has no + durable consumer yet — the consumer is created at assignment — so it never hears of it. Wanted + instead: one entry per server, for every machine or for one; every machine reads the whole current + set when it starts and watches for changes; unregistering is a delete; any machine can list it. +2. **Which licence a machine is bound to** ([design 39](../../03-DESIGN/01-to-be/39-the-anthropic-licence-manager.md)). + As events, a machine that was off for a day replays every rotation since and asks for a token + after each. It needs only the latest binding and its generation. The token itself stays on + request/reply and is never stored. + +The design already expects this. [Design 25](../../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1: +"conditions and observed state in key-value buckets that anything may watch". +[Research 017](../017-a-mesh-that-heals-itself/01-the-intended-behaviour.md) wants a provisioner's +"what I applied" and a rotation's step kept in one rather than in memory. Nothing implements it. + +## What exists, measured 2026-10-04 + +| | fact | where | +|---|---|---| +| streams | five kinds of mesh stream: CONTROL (work queue), NODES and ASSIGNMENTS (last per subject), EVENTS (limits: 7 days, 10 000 per subject), one work queue per seat that accepts | the controller's broker streams | +| the state relationship | [design 32](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 already names *state* — 1:1, last per subject — and says it is "declared: the mesh's own". Two streams use it, both written by the controller. No module can declare it | design 32, the controller | +| key-value buckets | none, anywhere | all four code repositories | +| the runtime's bus verbs | `mesh/publish`, `mesh/ask`, `mesh/subscribe`; delivery back to the bundle is `mesh/event` | the runtime's launcher | +| the runtime's principal | one bus user per machine carries every assigned module; its grant is the union of theirs. That one module's code does not act as another is the runtime's to keep: it publishes under the module's own name by construction | the controller's grant composition, the runtime's bus | +| what a bundle is issued | a membership per assignment, last per subject, read directly by the runtime: where it serves, where it emits, what it reaches | ADR 0160 | +| who creates bus objects | the controller only — mesh streams on every raise, a seat's stream at registration, a module's consumer at assignment. No module reaches the JetStream API | design 25 §3 | + +### What a key-value bucket needs from a grant, against a real server + +Measured against nats-server 2.10 with the Go client the runtime already uses, a bucket created by +an unrestricted user and used by two users holding only the subjects below (`B` is the bucket): + +| operation | subject published | writer | reader | +|---|---|---|---| +| bind to the bucket | `$JS.API.STREAM.INFO.KV_B` | yes | yes | +| get | `$JS.API.DIRECT.GET.KV_B.>` | yes | yes | +| put, delete | `$KV.B.>` | yes | **refused** | +| list keys, watch | `$JS.API.CONSUMER.CREATE.KV_B.>` — an ordered, ephemeral consumer | yes | yes | +| stop a watch cleanly | `$JS.API.CONSUMER.DELETE.KV_B.>` | yes | yes | +| answers | its own inbox, which every principal already subscribes | — | — | + +*Checked again once built, 2026-10-04:* the grants the controller composes for two machines' runtimes — +one carrying the owner, one only a reader — were loaded into a server as composed, and each operation +was run as each runtime's user. The owner's did all of them; the reader's read, listed and watched, +and its put and delete were refused by the server. + +Three things the measurement showed that reading the documentation would not have: + +1. **A refused put is not an error to the caller; it is a timeout.** The server reports the + permission violation asynchronously, on the connection, and the client waits out its deadline + for an acknowledgement that never comes. So a runtime that relies on the grant alone tells a + bundle "timed out" for "you may not write this" — it must refuse first, from what the module was + issued, with the reason. +2. **A watch's current values include deletions.** A key deleted earlier arrives among the initial + values as a delete marker, before the end-of-current marker. A bundle asking "what is there now" + must not be handed those. +3. **Without the consumer-delete grant, stopping a watch hangs** until its deadline, and the + ephemeral consumer lingers on the server until it times out by itself. + +### Whether the events shape is enough instead + +Honestly compared, because a new primitive is a cost: + +- **EVENTS cannot be made last-per-subject for some subjects.** Retention is per stream, and + JetStream refuses a second stream overlapping the first (verified and recorded in design 32 §3). + A state subject inside `mesh.mod.*.event.>` keeps EVENTS' seven days: a licence binding unchanged + for a week disappears. +- **A separate last-per-subject stream per module** is possible — it is exactly what a key-value + bucket *is* on the server: a stream with one message per subject, a rollup for purge, and direct + reads. Building it by hand gives up the client's get, list, delete and watch, which are the + operations both cases need, and would be the mesh writing NATS's own key-value layer again. +- **Consumers are the wrong reader.** A durable consumer per reading module is created at + assignment and replays from where it is; state wants "everything current, now, then changes", + which an ordered ephemeral consumer from the last value per subject gives and a durable does not. + +So key-value is not a convenience over events; it is the state relationship design 32 already +names, opened to modules. + +## Questions, and what this effort proposes + +1. **What a manifest says.** `state` names the buckets a module owns, by local name — every + instance of the module may write them and read them. `reads` names another module's bucket as + `.`, read-only. Names only, never a bucket or subject (design 32 §1). A bucket's + options — how many past values it keeps, how long a value lives — are the owner's to declare, + the way a seat declares its own retention (design 32 §3). +2. **Scope.** One bucket per module per name, mesh-wide. A key may carry a machine by the module's + own convention (`all.`, `.`). A bucket per machine was considered and + not proposed: "list every server for every machine" becomes a walk over buckets, and the grant + could only narrow writes, which nothing asked for — every instance of the owner already writes. +3. **Who creates the bucket.** The controller, from the catalogue, on every raise — a bucket exists + from registration, like a seat's stream, so a reader can watch before the owner is assigned + anywhere. Never a module. +4. **The runtime's verbs.** `mesh/state.get`, `mesh/state.put`, `mesh/state.delete`, + `mesh/state.keys`, `mesh/state.watch`, each naming the bucket as the module named it. A watch + is answered once the current values are on their way, then each change is delivered to the + bundle as a `mesh/state` request it answers — current values first (no deletions among them), an + end-of-current marker, then changes. A child that restarts watches again, as it subscribes + again. The runtime refuses, with the reason, a bucket the module was not issued, and a write to + one it only reads. +5. **Secrets.** None in a bucket, sealed or not: a bucket is a stream (design 32 §10). Sealed values + are plain base64 and cannot be recognised, so the mechanical check is partial and said to be: the + runtime refuses a value carrying a field whose name says it is a credential (`password`, + `secret`, `token`, `authorization`, …), which catches the ordinary mistake and not a determined + one. For the first consumer this has a concrete consequence: an MCP server registered with an + authorisation header keeps that header out of the bucket. +6. **History, lifetime, size.** One value per key unless the owner says more; no expiry unless it + says one; a value at most 256 KiB and a bucket at most 64 MiB, the mesh's caps rather than a + module's. **A bucket outlives its module's assignment** — what a module stored is data, and data + outlives what declared it ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md)); + unassigning is not cleaning up. A bucket whose declaration is gone is reported, never removed. +7. **Events or state.** State (above). + +## The work, once decided + +1. A decision record, then design 32 (*state* becomes a relationship a module declares) and design + 25 (key-value buckets are part of the bus) amended. +2. The controller: the manifest's two words and their registration check; buckets asserted on every + raise; the grants for owners' and readers' runtimes; the buckets issued in each membership. +3. The runtime: the five verbs, the watch delivery, the refusals; tested against a real server. +4. The SDK, TypeScript and Go: a small state surface over the verbs. +5. Proved on a running mesh with one small module, then handed to the operator's agent, whose + registered servers move from events to a bucket. diff --git a/02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md b/02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md new file mode 100644 index 0000000..67f831a --- /dev/null +++ b/02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md @@ -0,0 +1,106 @@ +--- +topic: what runs on it +status: accepted +date: 2026-10-04 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md +--- + +# 201. A module keeps its current state in key-value buckets it declares, and reaches them through the runtime + +## Context + +A module's code reaches the bus through the node's runtime: it publishes events, subscribes to them +and asks tools ([ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)). +Events are kept for a week and replayed to a consumer that was away; requests are kept nowhere. What +neither gives is **the current value of something**, seen by every machine, including one that joins +after it was written. The first module to need it — the operator's agent on a machine — registers MCP +servers for every machine as events, and a machine assigned later never hears of them; and it would +replay a week of licence rotations where it needs only the binding that holds now. Research +[024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md) measured the alternatives and +the grants against a real server. + +[Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 already names *state* as one of the +mesh's relationships — 1:1, last per subject — and reserves it to the mesh's own declarations. +[Design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 expects key-value buckets on the bus. + +## Considered Options + +1. **Key-value buckets a module declares, created by the controller, reached through the runtime.** + Chosen. +2. **State as events on EVENTS, read last-per-subject.** Rejected: retention is per stream and EVENTS + keeps seven days, so a value unchanged for a week disappears; a second stream over the same subjects + is refused by the server (design 32 §3). And events give no get, list or delete. +3. **A last-per-subject stream per module, written by hand.** Rejected: it is what a key-value bucket + is on the server, without the client's get, list, delete and watch — the mesh writing NATS's + key-value layer again. +4. **State in a module's own files or database, shared by asking a tool.** Rejected for state every + machine must see: a machine joining later has to know whom to ask and poll, and an owner that is + down answers nothing — the property the bus exists to remove. + +## Decision + +**1. A module declares its state by name.** `state` names the buckets it owns, by local name; every +instance of the module may write and read them. `reads` names another module's bucket as +`.`, read-only. A bucket's options are its owner's: how many past values a key keeps, +and how long a value lives. A manifest names no bucket, stream or subject (design 32 §1). + +**2. One bucket per module per name, mesh-wide.** A key may name a machine by the module's own +convention; the mesh does not scope buckets per machine. + +**3. The controller creates the buckets, from the catalogue, on every raise** — from registration, +like a seat's stream, so a reader can watch a bucket whose owner is not yet assigned anywhere. A module +never creates one. The runtime's grant on each bucket is the union of what its carried modules may do: +an owner's instances write and read, a reader's read. + +**4. Each assignment is issued its buckets in its membership** ([ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)), +by the name the module uses for each and whether it may write. The runtime serves `mesh/state.get`, +`put`, `delete`, `keys` and `watch` on the bundle's channel from that list, and refuses — with the +reason — a bucket the module was not issued and a write to one it only reads. A watch delivers the +current values first, without deletions, then an end-of-current marker, then every change, each as a +`mesh/state` request the bundle answers. + +**5. No secret is stored in a bucket, sealed or not.** A bucket is a stream, and design 32 §10 keeps +every secret off streams. A value that needs a secret names it; the secret travels on request/reply. + +**6. The mesh caps size; a bucket outlives its module.** One value per key and no expiry unless the +owner says otherwise; at most 256 KiB a value and 64 MiB a bucket. Unassigning a module leaves its +buckets and what is in them ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)); a bucket +whose declaration is gone from the catalogue is reported, never removed by the mesh. + +## Consequences + +- A machine that joins reads the current state at once, and every machine sees a change as it + happens, with no consumer created per reader and nothing replayed. +- The runtime's channel has a sixth verb family, and the SDKs a small state surface over it — a + contract, which ADR 0039 admits: it changes when the verbs do, rarely, and every module should be + rebuilt when it does. +- What got harder: the runtime must keep each module to its own buckets, because one principal per + machine carries all of them and the server enforces only the union. A write the server refuses + surfaces to a client as a timeout, not a refusal, so the runtime's own refusal is what a module sees. +- The secrets rule is only partly mechanical. Sealed values cannot be recognised; the runtime refuses + a value with a field whose name says it is a credential, which catches the ordinary mistake and not a + determined one. For the operator's agent this means an MCP server's authorisation header stays out of + its bucket. +- Buckets accumulate as modules come and go; that they are reported rather than removed is the price + of not deleting data. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A manifest's state names are local, and a read names a bucket its owner declares | the catalogue's registration check, per manifest; a catalogue test that every `reads` whose owner is present names a bucket that owner declares | +| Buckets exist for every declared state | the controller's raise asserts them idempotently; its test over a real bus | +| Owners write, readers only read | the composer's test of the grants, per principal kind; the runtime's refusal test over a real bus | +| A watch hands current values first, without deletions, then changes | the runtime's test over a real bus | +| No credential-named field in a value | the runtime's refusal test | +| Live | one module puts on one machine and another machine's watch sees it; a machine assigned afterwards reads it at start | + +## References + +- Research [024](../01-RESEARCH/024-state-a-module-keeps-on-the-bus/00-overview.md) +- [Design 32](../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 and §10, [design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 and §3 +- [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0039](0039-what-the-sdk-holds-and-refuses.md), + [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md), + [ADR 0198](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 07994db..1d582bd 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -300,6 +300,7 @@ python3 00-META/checks/index.py fail if stale - **0195** — [The mesh's tools are found by address, not announced whole](0195-the-meshs-tools-are-found-by-address-not-announced-whole.md) - **0197** — [Every tool announces itself on the bus, in the NATS services protocol](0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md) - **0198** — [A module's long-running code is launched by the node's runtime, and reaches the bus through it](0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md) +- **0201** — [A module keeps its current state in key-value buckets it declares, and reaches them through the runtime](0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md) ### How it is built diff --git a/03-DESIGN/01-to-be/25-the-bus-on-nats.md b/03-DESIGN/01-to-be/25-the-bus-on-nats.md index 54a5c14..ad0f387 100644 --- a/03-DESIGN/01-to-be/25-the-bus-on-nats.md +++ b/03-DESIGN/01-to-be/25-the-bus-on-nats.md @@ -7,8 +7,10 @@ code: - mesh-tools src/broker-amqp.ts (to be replaced) - mesh-catalog modules/nats (to be written) - mesh-sdk src (the protocol's NATS binding, step 3) -updated: 2026-10-02 + - mesh-tools node-tools/internal/bus (a module's state, ADR 0201) +updated: 2026-10-04 decisions: + - 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md - 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md - 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md - 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md @@ -48,6 +50,7 @@ mesh's own state lives, and where what a module may say is decided by what it de | **events** — a module says something happened | 1:many | delivered to every consumer that declared it; dead-lettered when it cannot be | | **tools** — a module or a person asks another's tool | request/reply | one answer, from one server, or a timeout | | **work to a role** — a module submits to a capability without knowing who provides it | job | exactly one holder does it; it queues while nobody does | +| **state** — a module's current value of something, every machine reading it | key-value | the newest per key, kept until replaced or deleted; read whole by a machine that joins later | The last two rows are the ones worth dwelling on, because they are not messaging in the sense of carrying bytes from A to B. **A role is addressable**, so a caller names the capability and never @@ -56,7 +59,8 @@ changing ([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) property of the mesh's architecture that happens to be expressed in subjects. And more of the mesh lands here as it is built: conditions and observed state in key-value -buckets that anything may watch, the server's own advisories becoming observations like any other +buckets that anything may watch — the first of them a module's own declared state, *2026-10-04* +([ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md)) — the server's own advisories becoming observations like any other ([research 017](../../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md)), and a person's client speaking the bus directly rather than through a surface built over it (§7). None of that is a message being moved; all of it is the bus being the mesh's centre. @@ -84,8 +88,16 @@ mesh.seat..event. a role's own event (JetStream: EVENT mesh.seat..tool. a role's tool (core request/reply) mesh.ask.. the controller's command api (core request/reply) mesh.assignment.. an assignment's membership (JetStream: ASSIGNMENTS, last-per-subject) +$KV._. a module's state (JetStream: a key-value bucket per declared name) ``` +**Added 2026-10-04** ([ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md)): +the last row is outside `mesh.` on purpose. A key-value bucket is NATS's own construct and lives +under NATS's own prefix, which is what lets the server's key-value layer — direct reads, rollups, +delete markers, watches — do the work instead of the mesh writing it again. The bucket is named for +the module and the local name joined by an underscore, which neither may contain, so two modules can +never derive one bucket. + **Revised 2026-10-01** ([ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)): the rows above for a module's and a seat's tools are the shapes the controller *issues*, not rules a runtime carries. Every assignment is published a membership — what it serves and where, in which queue, its seat verbs, @@ -155,6 +167,7 @@ Core NATS is at-most-once. Everything the mesh must not lose lives in a JetStrea | CONTROL | `mesh.control.>` except `alive` (a build's outcome moved to its seat, ADR 0121) | work queue, one consumer (the controller), explicit ack | the store-window guarantee ([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)): the controller `nak`s with a delay while its store is away and the message is redelivered; nothing is dropped | | NODES | `mesh.node.>` | last per subject | one declaration per node, always the newest | | EVENTS | `mesh.mod.*.event.>` and `mesh.seat.*.event.>` | limits (age, size), durable consumer per subscribing module | a subscriber that was down catches up; after `max-deliver` attempts the advisory feeds `mesh.events.dead` (its own small stream). *2026-10-01:* a build's whole log is here too, as the build-machine seat's `log.` events ([ADR 0157](../../02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md)) — one subject per build, a week of retention, read back by `builds --log ` with a consumer that is gone when the reading is done | +| `KV__` | `$KV._.>` | the newest value per key — as many past values as the owner declared — no age unless the owner declared one; a value at most 256 KiB, a bucket at most 64 MiB | a module's state (ADR 0201): one per name in a manifest's `state`, created from the catalogue on every raise, so it exists before its owner runs anywhere; kept when the module is unassigned, because what it holds is data | Tool calls and heartbeats stay on core NATS: a lost heartbeat is the next heartbeat; a lost tool call is a timeout the caller already handles. @@ -234,6 +247,14 @@ expresses this exactly, per subject, and better than a vhost could: permissions for each consumed event's subject, its tool subjects, and that same inbox prefix. Nothing else. A module that tries to publish outside its emits is refused by the server, not by convention. +- **A module's state** (ADR 0201), for whichever principal carries the module — today the machine's + runtime, whose grant is the union of its modules': binding to the bucket, reading a key directly, + and an ordered consumer for listing and watching, created and deleted on the bucket's own stream + and nothing else's; and, for the owner's instances only, publishing under the bucket's own + `$KV..>`. *Measured 2026-10-04 against a running server:* without the consumer-delete + grant a watch cannot be stopped cleanly, and a write the server refuses reaches the writer as a + timeout rather than a refusal — so the runtime refuses first, from the membership, and the grant + is the second line. - **The controller's user** owns `mesh.control.>`, `mesh.node.>` and the streams, and may submit work to the seats the mesh's own flows use — a build, for one (ADR 0121). **A host's user** may publish its own `mesh.control..>` and subscribe its own diff --git a/03-DESIGN/01-to-be/32-what-a-module-declares.md b/03-DESIGN/01-to-be/32-what-a-module-declares.md index 1cf0358..d8c1850 100644 --- a/03-DESIGN/01-to-be/32-what-a-module-declares.md +++ b/03-DESIGN/01-to-be/32-what-a-module-declares.md @@ -11,8 +11,10 @@ code: - mesh-host internal/apply/apply.go - mesh-tools src/main.ts - mesh-catalog modules/mesh-catalog -updated: 2026-10-02 + - mesh-tools node-tools/internal/runtime (a module's state, ADR 0201) +updated: 2026-10-04 decisions: + - 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md - 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md - 02-DECISIONS/0126-a-module-declares-its-own-seats.md - 02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md @@ -67,6 +69,8 @@ the catalogue, and the mesh would have hundreds of copies of a decision it made | `tools: status` | queue-group subscription on `mesh.mod..tool.status` | | seat `telegram-sender`, `accepts: send` | work-queue consumer on `mesh.seat.telegram-sender.accept.send` | | `uses: telegram-sender` | publish on that seat's `accept` subjects, and nothing else | +| `state: servers` | a key-value bucket for the module, created by the controller; its instances write and read it | +| `reads: billing.orders` | read and watch billing's `orders` bucket, and nothing else of it | **Wildcards, and they are the mesh's rather than a bus's.** *Added 2026-09-27, from [issue 127](../../04-ISSUES/127-a-module-event-derives-a-subject-nothing-publishes/00-report.md).* A @@ -221,7 +225,7 @@ service" versus "one worker per machine". | credential | sealed, per consumer | none | none | none | none | | reply | — | none | none, or an event later | a report | awaited | | retention | — | age and size | work queue, explicit ack | **last per subject** | none | -| declared | `provides`/`requires` | `emits`/`consumes` | seat `accepts` / `uses` | the mesh's own | `serves` | +| declared | `provides`/`requires` | `emits`/`consumes` | seat `accepts` / `uses` | the mesh's own; a module's `state` / `reads` | `serves` | **Job** is the one [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) had no room for. Its table has events at 1:many and provisions at 1:1; a module submitting work to a service @@ -236,6 +240,31 @@ last-per-subject retention, and a node that has seen sequence *n* refuses *n−1 That is the wire-level answer to [issue 107](../../04-ISSUES/107-a-declaration-carries-no-order/00-report.md). +**A module declares state too.** *Added 2026-10-04, +[ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md).* +State was the mesh's alone, and modules had the same need with nowhere to put it: an MCP server +registered for every machine, sent as an event, never reached a machine assigned afterwards — its +consumer did not exist yet when the event passed — and a licence binding sent as events replays a +week of rotations where only the latest matters. So a module names the state it **owns** with +`state`, and another module's it **reads** with `reads: .`. Each is a key-value bucket +the controller creates from the catalogue, mesh-wide, existing from registration so a reader can +watch before the owner runs anywhere ([design 25](25-the-bus-on-nats.md) §3). Every instance of the +owner writes; a reader reads and watches. A key may name a machine by the module's own convention; +the mesh keeps one bucket per name, not one per machine, because "every server, for every machine" +is then one list rather than a walk. + +What a module sees is what it named. Its assignment's membership lists its buckets by those names, +with whether it may write, and the runtime answers `get`, `put`, `delete`, `keys` and `watch` for +them on the bundle's channel — refusing, with the reason, a name it was not issued or a write to a +bucket it only reads. A watch hands the current values first, then every change: a bundle that starts +late, or starts again, has the whole of the state before it has any of the news. + +The owner says how many past values a key keeps and how long a value lives, as a seat says how long +its backlog survives (§3); the mesh caps a value's size and a bucket's. **A bucket outlives its +module's assignment** — what a module stored is data +([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md)) — and one whose +declaration is gone is reported, never removed by the mesh. + ## 5. Seats A module declares a seat with its protocol, and the mesh enforces one holder at its scope @@ -454,6 +483,14 @@ sealing key leaks, that stream is an archive rather than a moment. So: existing discipline — *fetched from it, not carried* — applied to the one payload where carrying it is worst. +**A key-value bucket is a stream, so the same holds for it.** *Added 2026-10-04, +[ADR 0201](../../02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md).* +No secret is put in a module's state, sealed or not: state is exactly what a machine joining a year +later reads in full. A value that needs a secret names it, and the secret travels on request/reply. +Sealed values are plain text to anything inspecting them, so this is checked only partly — the +runtime refuses a value carrying a field whose name says it is a credential, which catches the +ordinary mistake and not a determined one. + **The bootstrap, which is circular and has a precedent.** The vault makes every secret ([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)), including the bus's own passwords. The vault is a module, and a module needs a bus account, whose password the vault @@ -523,3 +560,9 @@ billing existing under that name. on, and exactly those two are rebuilt. - **A stale declaration is refused.** A bed: replay sequence *n−1* after *n*, and the node refuses it rather than applying it. +- **A module reaches only the state it declared.** The composer's test: an owner's runtime may write + its buckets, a reader's may only read, and nothing else is granted; the runtime's test over a real + bus: a name not issued and a reader's write are refused with the reason. +- **State is current at once.** The runtime's test over a real bus: a watch hands the current values + without deletions, then an end-of-current marker, then changes. Live: a machine assigned after a put + reads it at start.