novox/hq
18
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Bus state by template, no readers, and the ledger: designs 25, 32 and 55 and the glossary follow ADR 0595-0597

Browse Source 
Research 082 graduated into three accepted records; the bus module's design (25) and the manifest's (32) are the home of bus state for every module, and the knowledge base (55) becomes its first ledger.
This commit is contained in:
jschoubben
2026-11-03 10:42:58 +01:00
parent 08b968fd80
commit ed67d1f84f
4 changed files with 457 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files
00-META/glossary.md
+33 -3
View File
@@ -213,7 +213,8 @@ every other domain has something to run on. **Recorded by** the controller and t
A module reaches it by requiring `mesh-bus` (ADR 0128);
one that does not require it has no account on it. Held by the `mesh-broker` seat, which is named
after the role rather than the server, so the server can change without the seat doing so. The `nats`
module holds it.
module holds it. The **bus module** is the module holding `mesh-broker`; it also provides bus state (Data,
ADR 0595).
- **the deprecated broker** — the lavinmq module. It was the mesh's bus and is not any more. It keeps
running as an **ordinary provider** of the `amqp` provision, for modules that need a message broker
of their own the way something needs a database
@@ -724,8 +725,8 @@ becomes a message). **Decided by** ADR 0227, 0231, 0240, 0252, 0283, 0564.
**Purpose.** To know every item of data a module holds, how precious it is, and to keep it recoverable.
**Recorded by** the manifests' `data` declarations and every node's `node-backup` seat.
**Upstream of** Delivery and Health and repair. **Decided by** ADR 0232, 0233, 0235, 0571.
**Uses:** module, node, provider and consumer (Provisioning), person.
**Upstream of** Delivery and Health and repair. **Decided by** ADR 0232, 0233, 0235, 0571, 0595, 0596, 0597.
**Uses:** module, node, provider and consumer (Provisioning), person, bus module (Core), tool runner (Core).
- **data class** — how precious an item of data is, ranked by the operator: `irreplaceable`,
`valuable`, `rebuildable`, `cache`; and `none` for a provision that keeps nothing of anybody's
@@ -740,6 +741,25 @@ becomes a message). **Decided by** ADR 0227, 0231, 0240, 0252, 0283, 0564.
self-check read them there, never the controller (ADR 0571 §5).
- **sticky binding** — a consumer's tie to the data a provider keeps for it, which moves only by a
person (ADR 0232).
- **bus state** — what a module keeps on the bus, asked for in `requires` as a provision of the bus module, by a
template and its options; created by the bus module and reached by the owner's account alone. Another module
asks the owner's tools for its data, or hears its declared events (ADR 0595, ADR 0596).
- **storage type** — how the bus keeps bus state: the **stream**, the base, or one built on it, the **key-value
bucket**, the **object store** or the **ledger**, each with calls of its own (ADR 0595 §2).
*Not:* ~~data type~~
- **configuration** of a storage type — the settings it is given: what is kept, for how long, how much, and what
happens when full. Never bare where it could be read as a config value (see [Homonyms](#homonyms)).
- **template** — a storage type with a configuration, predefined and named: **state**, **events**, **work
queue**, **log**, **objects** and **ledger** (ADR 0595 §2). What keeps its data safe, "when full" among it, is
fixed by the template. *Template of bus state* where it could be read as a template file (see
[Homonyms](#homonyms)). The template log is the module log of ADR 0297.
- **option** of a template — one setting a module may change, within its bound, such as state's past values (1 to
64) (ADR 0595 §3).
- **ledger** — a storage type keeping, per instance of a type, a history of every **operation** (one change, kept
in order) and a state derived from that history by each operation's **ledger effect** (one of a fixed set: set
fields, set an entry of a map, append to a list, remove), through the tool runner's `apply` (ADR 0597).
- **snapshot** of a ledger's state — the latest state of one instance, on a subject of its own, never expiring,
from which a rebuild starts; distinct from the stream snapshot.
## Connectivity — how nodes and modules reach one another
@@ -1076,6 +1096,16 @@ renaming one sense, the old sense moves to a *Not:* line and becomes mechanical.
| | an account in a group of the machine | **account group** | Placement |
| domain | an area of the mesh that owns a set of concepts and the one word for each (ADR 0244), the sections of this page; the one sense "domain" keeps bare | **mesh domain** | The record |
| | one of the eight concerns a direction and a design are reviewed against (playbook 01); never written bare | **architectural domain** | The record |
| configuration | the settings of a storage type, which a template fixes or a module gives | **configuration** of a storage type | Data |
| | anything a module reads that someone may change | **config value** | Config |
| snapshot | the latest state of one ledger instance, from which a rebuild starts | **snapshot** of a ledger's state | Data |
| | the bus's own backup of each stream | **stream snapshot** | Data |
| template | a storage type with a configuration, asked for in `requires` | **template** of bus state | Data |
| | a file with placeholders a provisioner fills | **template file** | Config, Provisioning |
| option | a setting of a template a module may change | **option** of a template | Data |
| | one choice of an ask to a person | an ask's **option** | Operator and conversation |
| effect | what an operation does to a ledger's state | **ledger effect** | Data |
| | a tool's mark: reads, acts or dangerous | a tool's **effect** | Identity and access |
| push | the controller giving nodes their declarations, until release 6 | **send** (verb `push`) | Placement |
| | a git push | **git push** | The record |
| | a phone notification | **push notification** | Operator and conversation |
03-DESIGN/01-to-be/25-the-bus-on-nats.md
+217 -4
View File
@@ -7,9 +7,14 @@ code:
- mesh-tools src/broker-amqp.ts (to be replaced)
- mesh-catalog modules/nats (to be written)
- mesh-sdk src (the protocol's NATS binding, step 3)
- mesh-tools node-tools/internal/bus (a module's state, ADR 0201)
updated: 2026-10-10
- mesh-tools node-tools/internal/bus (a module's state, ADR 0201; the new calls and apply, ADR 0595, ADR 0597)
- mesh-sdk (the typed bus-state entry, its binding and the new calls' package, ADR 0595)
- mesh-delivery (the judge's rules and the grants part's binding, ADR 0595)
updated: 2026-11-03
decisions:
- "ADR 0595"
- "ADR 0596"
- "ADR 0597"
- "ADR 0289"
- "ADR 0264"
- "ADR 0235"
@@ -38,6 +43,11 @@ under which subject, with which guarantee, under whose account; how a node joins
reaches a tool; how the mesh moves from the bus it has to this one; and how each claim is checked.
Prose and diagrams only; no configuration is pasted.
**Amended, 2026-11-03** (ADR 0595, ADR 0596, ADR 0597): what a module keeps on the bus is its **bus state**, a
provision of the bus module asked for by template, created by the bus module, read by its owner alone; and the
**ledger** is a storage type of its own. § 12 is the design; the `KV_<module>_<name>` row of § 3 and the module's
state in § 4 hold for a module still on the old sections.
## 1. What the bus is for
**The bus is where the mesh happens.** Not a transport the mesh sends things over — the place a
@@ -179,7 +189,7 @@ Core NATS is at-most-once. Everything the mesh must not lose lives in a JetStrea
| EVENTS | `mesh.mod.*.event.>` and `mesh.seat.*.event.>`; and `mesh.again.>` (ADR 0264) | limits (age, size), durable consumer per subscribing module | a subscriber that was down catches up; after `max-deliver` attempts the advisory feeds `mesh.events.dead` (its own small stream). *2026-10-08, ADR 0264:* that stream is DEAD_LETTERS, below; each consumer also filters `mesh.again.<consumer>.>`, where a message it gave up on is delivered again to it alone. *2026-10-01:* a build's whole log is here too, as the build seat's `log.<build id>` events (ADR 0157) — one subject per build, a week of retention, read back by `builds --log <id>` with a consumer that is gone when the reading is done |
| DEAD_LETTER_NOTICES | the server's maximum-deliveries advisory | work queue, a week; one consumer (the controller), no `max-deliver` | the server's word that a consumer gave up on a message, kept whether or not a controller is listening, until the controller has copied the message (ADR 0264) |
| DEAD_LETTERS | `mesh.events.dead.<stream>.<consumer>` | limits: no age, 256 MiB, refuses new when full; a day's de-duplication | every message a consumer gave up on, with its consumer, subject, deliveries and when, until a person delivers it again or drops it with why (ADR 0264); a `max-deliveries` condition is open per consumer while it holds any |
| `KV_<module>_<name>` | `$KV.<module>_<name>.>` | the newest value per key — as many past values as the owner declared — no age unless the owner declared one; a value at most 256 KiB, a bucket at most 64 MiB | a module's state (ADR 0201): one per name in a manifest's `state`, created from the catalog on every raise, so it exists before its owner runs anywhere; kept when the module is unassigned, because what it holds is data |
| `KV_<module>_<name>` | `$KV.<module>_<name>.>` | the newest value per key — as many past values as the owner declared — no age unless the owner declared one; a value at most 256 KiB, a bucket at most 64 MiB | a module's state (ADR 0201): one per name in a manifest's `state`, created from the catalog on every raise, so it exists before its owner runs anywhere; kept when the module is unassigned, because what it holds is data. *2026-11-03, ADR 0595:* a module's kept data in the new form is created by the bus module from `requires`, by template (§ 12) |
Tool calls and heartbeats stay on core NATS: a lost heartbeat is the next heartbeat; a lost tool
call is a timeout the caller already handles.
@@ -266,7 +276,7 @@ expresses this exactly, per subject, and better than a vhost could:
permissions for each consumed event's subject, its tool subjects, and that same inbox prefix.
Nothing else. A module that tries to publish outside its emits is refused by the server, not by
convention.
- **A module's state** (ADR 0201), for whichever principal carries the module — today the machine's
- **A module's state** (ADR 0201; for the new form, § 12.2: the owner's account alone, no reader, ADR 0596), for whichever principal carries the module — today the machine's
runtime, whose grant is the union of its modules': binding to the bucket, reading a key directly,
and an ordered consumer for listing and watching, created and deleted on the bucket's own stream
and nothing else's; and, for the owner's instances only, publishing under the bucket's own
@@ -617,3 +627,206 @@ find what changed and why.
that needs it? ADR 0039's test —
*does editing it recompile unrelated modules, and does it change often* — probably says no for
one instance; worth asking again if a second module needs the same shape.
## 12. Bus state: what a module keeps on the bus, and the bus module that provides it
*Added 2026-11-03* (ADR 0595, ADR 0596, ADR 0597, from research 082; the operator, 2026-11-02 20:45: "The names
simply represent named configurations or templates"). Until now a manifest declared what a module keeps on the bus
in four sections with four vocabularies (`state`, `logs`, `streams`, `objects`), the grants part created each
entry once and never changed it, and nothing compared what the bus holds with what was asked. This section is the
design of the one form that replaces them; [design 32](32-what-a-module-declares.md) § 4 says how a manifest asks
for it, and [design 55](55-the-knowledge-base.md) § 2.1 is its first ledger.
**Bus state is a provision of the bus module** (ADR 0595 §1). The **bus module** is the module holding the seat
`mesh-broker` (today `nats`). A module asks it for each piece of its **bus state** the way it asks postgres for a
database: an entry in `requires`, by its local name, its storage type, its template and the template's options.
The bus module creates it; the module receives the names on the bus in its membership; nothing else creates it.
### 12.1 Three layers
```
layer 3 templates state events work queue log objects ledger
│ │ │ │ │ │
layer 2 storage types key-value │ │ │ object store ledger
built on the bucket │ │ │ │ (a stream and
stream │ │ │ │ │ key-value buckets)
└──────────┴───────────┴──────────┴──────────┴────────────┘
layer 1 base type stream
```
A **storage type** is how the bus keeps the data: the **stream** is the base, and the **key-value bucket**, the
**object store** and the **ledger** are built on it, each with calls of its own (ADR 0595 §2). A **configuration**
is the settings a storage type is given. A **template** is a storage type with a configuration, predefined and
named: six exist, from the uses counted on main (158 state entries, 6 event streams, 4 logs, 3 work queues). An
**option** is one setting of a template a module may change, within its bound. A module gives a configuration of
its own only where no template fits: the bare stream, or a ledger whose history expires.
**What keeps the data safe is fixed, never an option** (ADR 0595 §3). Each template fixes what happens when it is
full, and the judge refuses a manifest that gives it:
| Template | On | Fixed | Options, their bounds (default) |
|---|---|---|---|
| state | key-value | refuses when full; a value at most 256 KiB; the bucket at most 64 MiB; one copy, on disk | past values 1 to 64 (1), 0 refused; time to live (none); a lifetime per key; one bucket per machine |
| events | stream | drops the oldest when full; the module's own subjects | age 1 to 30 days (7); cap 1 to 8192 MiB (1024); a grant (ADR 0363 §4) |
| work queue | stream | removed once taken; refuses when full; one taker; no grant | deliveries 1 to 10; duplicate detection up to its age; cap; age |
| log | stream | refuses when full; never rewritten; an entry at most 256 KiB; one subject per key | cap 1 to 8192 MiB (1024); age 1 to 3650 days (none) |
| objects | object store | refuses when full; kept for good | cap 1 to 8192 MiB, required; a grant |
| ledger | ledger | as log, plus its type and a snapshot per instance; its state and index buckets as state | cap; keeping days only in a ledger's own configuration |
The template log is the **module log** of ADR 0297. The mesh, never the module, names the bus objects and their
subjects (ADR 0201 §4): `<module>_<local name>`, and a local name holding the separator is refused so no two
modules' names meet.
### 12.2 Who creates it, and how a module receives it
```
indexer's judge ──verdict: accepted, with the module's bus-state entries whole──▶ bus module
│ creates each entry
▼
grants part ◀──────────────── created (module, local name, bus name, template, settings)
│ writes the binding into the module's membership
▼
the tool runner reads the membership ──▶ the module's calls reach its own entries, and only those
```
- **The trigger is the judge's verdict** (ADR 0595 §4), heard as an event that carries the module's entries whole,
so the bus module never reads the indexer's state and never polls. The judge checks every entry once, at the merge
check and at take-in (ADR 0578; ADR 0595 §8).
- **The bus module has a process of its own** beside its tool bundle, with a declared health check. Its credential
creates streams and buckets under modules' names only, and deletes only through `drop`.
- **The binding travels on `created`** (ADR 0595 §1, §7). The grants part, which issues every membership, hears it
and writes the binding (the names on the bus) into the module's membership, so no part writes another's state.
A module whose binding is missing at start fails loudly, naming the entry; it never starts blind. A `refused`
create hands no binding, and a refused or overdue creation raises a condition naming the module and the entry.
- **Only the owner reaches it** (ADR 0596 §2). The membership grants the owner's account its own entries and no
other module's: the new form has no readers. What another module needs of that data it asks through the owner's
tools or seat verbs, and what the owner lets others hear is its declared events, subscribed to through
`consumes` (ADR 0363, unchanged). A part watches only its own state, and hears another's events (ADR 0596 §4).
### 12.3 The seat's verbs, events and fill
The seat `mesh-broker`, until now a seat with no verbs, gains these (ADR 0595 §5, §6). Where the seat's definition
lives, and so the exact form of each verb's arguments, waits for the research effort on how seats are defined,
tracked on issue 1966; the build's third step does not start before it.
| Verb | Does | Who may call it |
|---|---|---|
| `list` | every entry, its template, its settings, its fill against its cap | any caller whose role grants it (ADR 0386) |
| `show` | one entry, the same | the same |
| `check` | compares what the bus holds with what was asked, entry by entry | the same |
| `orphans` | entries the bus holds that no manifest asks for; never one whose manifest the judge refused or outgrew, nor one of the foundation's old sections | the same |
| `create` | creates one entry now | the bus module's own process only |
| `apply` | applies a difference that only grows: a larger cap, a longer age | the bus module's own process only |
| `recreate` | rebuilds an entry whose settings differ in a way that cannot grow, from its data where the storage type allows | the bus module's process, on the operator's warrant |
| `drop` | removes one orphan | the bus module's process, on the operator's warrant |
`create`, `apply`, `recreate` and `drop` are granted to the bus module's own credential alone, so a module calling
them is refused by the bus. A person asks for them through the bus module's tools, and `recreate` and `drop` act
only on the operator's warrant (ADR 0259 §6), asked through the operator channel. Before `recreate` or `drop` acts,
the bus module takes a stream snapshot of the entry. A drifted entry is said, never quietly fixed: only a difference
that grows is applied without a person.
**Events**: `created`, `refused` (with why), `drifted` and `drift-cleared`, `full` and `full-cleared`, `orphaned`;
each carries the module, the local name, the bus name, the template, and the settings asked against those held. The
bus module asks for them with the template events at 30 days, the bound's top, because a drift or an orphan is read
back days later by a person; the conditions hold the live state. This settles where they are kept, which
ADR 0595 §12 left to the design.
**Fill is the bus module's one periodic read** (ADR 0595 §6): every minute, for every entry that refuses when full,
because the bus says nothing as a store fills. The condition `full` is raised from 75 percent of the cap and cleared
below it. The grants part's `mesh-grants.history-full` moves to it: the new condition is raised first, then the old
key cleared.
**Its tools** (tools-are-how-a-module-is-spoken-to): the four reading verbs answer every entry the bus module
created and its fill; `create`, `apply`, `recreate` and `drop` are its tools for a person, the last two asking the
warrant; its health names the process's last pass, the entries it could not create, and the entries over 75 percent.
### 12.4 The ledger
A **ledger** is a storage type whose state is derived from its history (ADR 0597). It is built on a stream and
key-value buckets, and asked for with the template ledger.
```
module ──apply(instance, operation)──▶ tool runner on its own machine
│ 1 checks the operation against the type
│ 2 reads the history's last sequence for the instance; replays if the state is behind
│ 3 appends to the history, that sequence expected ──▶ history (stream, a subject per instance)
│ 4 derives the state by the operation's effect ──▶ state (key-value, updated at the revision read)
│ 5 writes the instance's snapshot ──▶ snapshot subject (latest only, never expires)
▼
answers: taken, with its sequence (and "state behind" when the state bucket is full)
module's process: a watcher keeps each declared index (a second key-value bucket) from the history
```
- **A type** is a schema and the operations allowed on it; an **instance** of a type is named by a path
(`listing/a17`, `decision/0290`); every change is an **operation**, kept in the instance's history.
- **A ledger effect** is what an operation does to the state, one of a fixed set: set fields, set an entry of a
map, append to a list, remove (ADR 0597 §2). An effect copies only the fields its type names, and may add the
operation's sequence and author. Where a type pairs a text field with its digest, `apply` refuses an operation
whose digest does not match. A module's own rules are checked by the module before it calls `apply`, never by the
effects. Only the owning module calls `apply` on its ledger, for every effect, remove among them; a person
removes through the owner's tools. This settles who may call the remove effect, which ADR 0597 §13 left to the
design.
- **One writer wins, and the state never goes back** (§5): the expected sequence makes the bus refuse a racing
writer; the state is written as an update of the revision read, never a plain put; when that update loses to
another writer, `apply` answers success with its operation's sequence and re-reads, never appending again.
- **A full state bucket loses no operation** (§6): `apply` answers that the operation is taken, with its sequence,
and that the state is behind because the bucket is full; the next `apply` after room is made catches it up.
- **The author is proven** (§7): `apply` stamps the owning module's authenticated account and the author it passes
on, the account that called the owner's tool, never words the caller chose.
- **Every instance has a snapshot** (§8): the latest state on a subject of its own, never expiring, written by every
`apply`. It makes a kept ledger's rebuild fast and an expiring ledger's possible; a repair that finds it behind
the state rewrites it.
- **Only the tool runner's ledger credential publishes on a ledger's history** (§9): one per module, held by the
tool runner and never by the module's own code, whose account may read its ledger and not publish on it. Only the
owner's account reads the history, the snapshot and the state.
- **An index** is a second key-value bucket kept by a watcher in the owning module's process, rebuildable from the
history; it raises a condition when it lags behind (§4).
### 12.5 The calls, and where they live
The new calls live in a small mesh-sdk package of their own, served by the tool runner and imported only by the
modules that use them; the module library, which 112 modules import, does not change (ADR 0595 §9). They are the
object store's (put, get, list, delete, info, watch); a stream's read-back and fill; a work queue's size and dead
letters; and the ledger's: `apply`, `get`, `history`, `watch`, `query`, and its repairs `rebuild` (the state from
the history or the snapshot), `reindex`, `snapshot` and `lag` (ADR 0597 §10). mesh-sdk's catalog and membership
packages change once for the typed `requires` entry and the binding, rebuilding about 30 modules.
### 12.6 The move, and the order of the build
**The build, each step undoable on its own** (ADR 0595 §10): (1) mesh-sdk's typed `requires`, binding and calls
package; (2) the judge's rules; (3) the bus module's process, verbs and events, creating only for the new form; (4)
the tool runner's calls, `apply` and its ledger credential among them; (5) modules move, each owner first. Undoing
step 3 leaves what it created in place, and a module moved after it fails loudly on its missing binding. The
placement holds a moved module off a machine whose tool runner does not read the typed binding, and says so.
**A module moves when it is next touched** (ADR 0595 §11), keeping its local names and carrying its old settings
as options, so its bus objects and data stay and no drift is raised. It asks for each entry once: an old section
and the new form are never both used for one local name. During the move the grants part keeps creating the old
sections' entries, so each entry has one creator. The indexer counts the modules on each old section, a tool reads
the count, and the judge refuses a section once its count is 0. The foundation's own kept data (the bus module's,
the grants part's and the indexer's) stays on the old sections for good and is not counted: those modules cannot
wait for a process that needs them first.
**Readers move after their owners** (ADR 0596 §3). Today 195 `reads` entries in 37 modules read another module's
bus state, the indexer's 36 times and the assigner's 30. An owner declares the events and gains the tools its
readers need before any reader moves: the judge refuses a moved module's subscription to an event its owner does
not declare, and the review of a reader's move reads that the owner's tool is on main and delivered. These moves
are tracked by issue 1966.
### 12.7 How it is checked
| Rule | Checked by |
|---|---|
| A manifest's bus state is valid, once | the judge's unit tests, each refusal naming the field: an unknown template, an option outside its bound, past values 0, "when full" given, a fixed setting given, a name used twice across storage types, two modules' bus names alike, a `reads` in the new form, an effect outside the fixed set (ADR 0595 §8, ADR 0596 §2, ADR 0597 §2) |
| The bus module creates, and the binding follows | live, by tool call: a module asking for a state, a log, objects and a ledger in `requires` receives each binding in its membership, written after `created`, never before; `show` gives each entry's template, settings and fill; a module calling `create` is refused |
| Drift, orphans and the warrant | live: a bucket whose settings differ raises `drifted` and its condition; `apply` of a larger cap clears it with `drift-cleared`; a refused create raises its condition and hands no binding; `recreate` and `drop` without the operator's warrant are refused; an orphan is listed, raises `orphaned`, and goes only by `drop`, after a stream snapshot |
| Full | live: each template at its cap refuses, events drop the oldest; `full` raised at 75 percent and cleared below |
| Only the owner reaches its data | live: another module's account reading an entry in the new form, a ledger's history, snapshot or state among them, is refused; a raw publish on a ledger's history is refused |
| The ledger keeps its promises | one integration test against the real bus (ADR 0597's How it is checked): two writers race and one is refused; two interleaved applies leave the state at the higher sequence; a write cut between history and state is repaired; a full state bucket answers taken with its sequence and catches up; an idle instance whose operations expired rebuilds from its snapshot; `rebuild`, `reindex` and `lag` repair by hand; a forged author is refused; every `apply` leaves the snapshot equal to the state; an index behind raises its condition and `reindex` clears it |
| The calls rebuild only their users | a read of the delivery plan: only the importers of the calls package rebuild when it changes |
| The move keeps data | live: a module moved from `state` keeps its bucket and data and raises no drift; a new entry in an old section is still created by the grants part; the count falls by one; at 0 a manifest using the section is refused and the foundation's are still accepted; a moved module is held off a machine whose tool runner does not read the typed binding |
**Acceptance.** The operator asks the bus module's `list` and sees every module's kept data with its template, its
settings and its fill; a module that moved keeps its data; a drift or an orphan reaches the operator as a condition
and is changed only on their word; and the knowledge base keeps its records as a ledger (design 55 § 10).
03-DESIGN/01-to-be/32-what-a-module-declares.md
+41 -2
View File
@@ -12,8 +12,11 @@ code:
- mesh-tools src/main.ts
- mesh-catalog modules/mesh-catalog
- mesh-tools node-tools/internal/runtime (a module's state, ADR 0201)
updated: 2026-10-12
- mesh-sdk (the typed bus-state entry in `requires` and its binding, ADR 0595)
updated: 2026-11-03
decisions:
- "ADR 0595"
- "ADR 0596"
- "ADR 0307"
- "ADR 0240"
- "ADR 0233"
@@ -227,7 +230,7 @@ service" versus "one worker per machine".
| credential | sealed, per consumer | none | none | none | none |
| reply | — | none | none, or an event later | a report | awaited |
| retention | — | age and size | work queue, explicit ack | **last per subject** | none |
| declared | `provides`/`requires` | `emits`/`consumes` | seat `accepts` / `uses` | the mesh's own; a module's `state` / `reads` | `serves` |
| declared | `provides`/`requires` | `emits`/`consumes` | seat `accepts` / `uses` | the mesh's own; a module's bus state in `requires` (ADR 0595), the old `state` / `reads` until it moves | `serves` |
**Job** is the one ADR 0041 had no room
for. Its table has events at 1:many and provisions at 1:1; a module submitting work to a service
@@ -278,6 +281,36 @@ module's assignment** — what a module stored is data
(ADR 0030) — and one whose
declaration is gone is reported, never removed by the mesh.
**A module asks for its bus state in `requires`, and reads no other module's.** *Amended 2026-11-03, ADR 0595
and ADR 0596, from research 082.* The `state` and `reads` of the two paragraphs above, and the sections `logs`,
`streams` and `objects` beside them, are four vocabularies for one need: what a module keeps on the bus. They are
replaced by one form, a **provision** of the bus module, as a database is postgres's:
- **An entry of `requires`, in its typed long form** (ADR 0595 §1): the local name, the storage type, the template
(state, events, work queue, log, objects, ledger) and the template's options, within their bounds; a ledger also
names its type (its schema, its operations each with its ledger effect, its indexes). The templates, their fixed
settings and their options are [design 25](25-the-bus-on-nats.md) § 12.1. mesh-sdk's catalog package carries the
entry's type and its strict decoding; every rule that judges it lives in the indexer's judge (ADR 0595 §8).
- **The binding is what the module receives**: the names on the bus for each local name, written into its
membership by the grants part once the bus module says `created` (design 25 § 12.2). The module still names only
locally (§ 1). A module whose binding is missing at start fails loudly, naming the entry.
- **One name, one form.** A module asks for each entry once: an old section and the new form are never both used for
one local name, and one name serves one storage type only.
- **No readers.** The new form has no `reads`, and the judge refuses one (ADR 0596 §2). Only the owner's account
reaches its bus state. **A module's events are subscribed to; its data is asked for**: what another module needs
of a module's kept data it asks through that module's tools or seat verbs (§ 4's tool relationship, `serves`),
and what a module lets others hear is its declared events, heard through `consumes` (ADR 0363, unchanged; ADR 0596
§1). A bucket's layout is never a contract another module depends on; a tool or a declared event is.
- **The move is a module at a time** (ADR 0595 §11, ADR 0596 §3), each when it is next touched, keeping its local
names and its old settings as options so its data stays. A reader moves after its owner: the owner declares the
event or gains the tool its readers need first, and the judge refuses a moved module's subscription to an event
its owner does not declare. An old section is refused once no module but the foundation's uses it.
What follows for the paragraphs above: for a module that has moved, ADR 0201 §2 (another module's state read by
naming it in `reads`) no longer holds, and the bucket "every instance of the owner writes; a reader reads and
watches" becomes "every instance of the owner writes and reads". The runtime's `get`, `put`, `delete`, `keys`,
`watch` and `history` stay the calls of the template state.
**A module declares the data it holds.** *Added 2026-10-06,
ADR 0233.*
State on the bus is one kind of data; the rest lives on a machine's disks or with a provider, and the
@@ -602,6 +635,12 @@ billing existing under that name.
- **A module reaches only the state it declared.** The composer's test: an owner's runtime may write
its buckets, a reader's may only read, and nothing else is granted; the runtime's test over a real
bus: a name not issued and a reader's write are refused with the reason.
- **Bus state is asked for once, in one form, and read by its owner alone** (ADR 0595 §8, ADR 0596). The judge's
unit tests: a `reads` in the new form, a local name used twice across storage types or in an old section and the
new form, an unknown template and an option outside its bound are each refused, naming the field. Live: a moved
module receives its binding after `created`; another module's account reading its entries is refused; a moved
reader hears its owner's declared event and asks its owner's tool. The rest is [design 25](25-the-bus-on-nats.md)
§ 12.7.
- **State is current at once.** The runtime's test over a real bus: a watch hands the current values
without deletions, then an end-of-current marker, then changes. Live: a machine assigned after a put
reads it at start.
03-DESIGN/01-to-be/55-the-knowledge-base.md
+166 -114
View File
@@ -2,7 +2,7 @@
layer: to-be
status: in-progress
code: [mesh-catalog, hq]
updated: 2026-11-02
updated: 2026-11-03
decisions:
- "ADR 0290"
- "ADR 0321"
@@ -10,6 +10,9 @@ decisions:
- "ADR 0424"
- "ADR 0592"
- "ADR 0593"
- "ADR 0595"
- "ADR 0596"
- "ADR 0597"
---
# 55 — The knowledge base
@@ -23,14 +26,26 @@ written one section at a time in a draft, and reviewed one section at a time. A
never a file. § 1, § 2, § 4, § 5, § 8.1 and the fifth slice of § 9 say how; until it is built, the module keeps its
records in the store as before.
**Amended, 2026-11-03** (ADR 0597, ADR 0595 and ADR 0596, from research 082; the operator, 2026-11-02 21:53: "the
writes are the record"). The knowledge base is the first **ledger**: a record is an instance of a ledger type, every
change to it is an operation kept in the ledger's history, and its head is the ledger's state, derived from those
operations by the tool runner's `apply` and never written by the module. A section's text lives only in the
`write-section` operation that wrote it, with its digest; the head names each section by its title, digest and that
operation's sequence. So the object bucket `sections`, the state `names`, the stream `versions` and the module's own
commit rule fall away (ADR 0597 §12, changing ADR 0592 §1 to §12, §17, §25, §32, §33). Its kept data is asked for in
`requires` by template (ADR 0595), and no other module reads it: another module asks the knowledge base's tools or
hears its events (ADR 0596). The ledger itself, the same for every module, is designed in
[design 25](25-the-bus-on-nats.md) § 12; how a module asks for it, in [design 32](32-what-a-module-declares.md) § 4.
§ 1, § 1.1, § 2.1, § 4, § 8.1, § 9 and § 10 say what changes here.
**The mesh keeps its decision records and research efforts whole in a module of its own, `mesh-knowledge`, one
instance on the control-node, holding the seat `knowledge-base`: each record's number, status, topic, typed links,
its clauses with their own status, and its reasoning in titled sections, with every version kept, all of it on the
bus. A record is written section by section, checked when it is submitted, reviewed section by section by the
knowledge base's own judgement and accepted there; the operator's Approve is asked when a record needs it. Agents
ask what holds, and why, in one bounded call.**
(ADR 0290 and ADR 0592,
from research 045.)
(ADR 0290, ADR 0592 and ADR 0597,
from research 045 and research 082.)
A **clause** is one numbered statement a decision record makes, with its own status. A **section** is one titled
part of a record's reasoning. The **knowledge base** is the module that holds the records; the seat it holds is
@@ -42,7 +57,7 @@ read the reasoning behind each. A session writes a research effort of more than
session on another node takes the draft over and submits it, and every section is judged.
This document designs the first slice in detail, tracked by issue 425, and the fifth, the records on the bus
(ADR 0592). The slices after the first are in
(ADR 0592), kept as a ledger (ADR 0597). The slices after the first are in
[§ 9](#9-the-slices-after-the-first), which also designs the third: designs and the glossary's entries become records,
and the knowledge base works out each design's state (ADR 0423, ADR 0424).
@@ -53,22 +68,27 @@ and the knowledge base works out each design's state (ADR 0423, ADR 0424).
mesh-knowledge (seat knowledge-base, held once for the mesh)
├ its process, launched by the node runtime
├ on the bus, written by it alone (§ 2)
│ state heads one key per record: its fields, clauses, links, sections as title and digest
│ state drafts one key per record being written: its author and its sections as title and digest
├ on the bus, asked for in requires by template (ADR 0595), read by it alone (§ 2)
│ ledger records one instance per record (decision/0290, research/082):
│ history every operation, in order, each section's text in its write-section
│ state the head, derived by apply: fields, clauses, links, sections as
│ title, digest and sequence
│ snapshot the latest head per record, never expiring
│ state drafts one key per record being written: its author and its sections as title,
│ digest and the sequence of their write-section
│ state asked one key per part asked of the judge and not yet answered: ticket, node, when
│ state names per digest: how many versions and drafts name it, and since when none has
│ state findings the whole-set findings standing now, and the told marks (told.<topic>)
│ stream versions one message per version: who, when, why, its mark, the head as it stood
│ stream reviews one message per part reviewed: the digest judged, the verdict, the findings
│ objects sections every section's body, once, named by its sha256 digest
├ in its memory the search index, built from the bus at start and kept by watching heads
│ events reviews one message per part reviewed: the digest judged, the verdict, the findings
├ written through the tool runner's apply on the control-node, the only publisher on the
│ ledger's history (ADR 0597 §3, §9)
├ in its memory the search index, built from the bus at start and kept by watching the
│ ledger's state
├ verbs ask, get, list, holds, links, history, findings, progress,
│ reserve, open, write, remove, order, submit, drop, take-over, propose,
│ withdraw, retract, import, return
│ served by the control-node's tool runner
├ events reserved, opened, written, removed, ordered, submitted, dropped, taken-over, proposed,
│ part-reviewed, reviewed, accepted, changed, swept, moved, checked, formed,
│ part-reviewed, reviewed, accepted, changed, moved, checked, formed,
│ imported, drafted, confirmed, returned
├ consumes the router's warrants: the operator's Approve for a record that needs it (§ 5)
├ consumes claude-code's judged: the judge's answer to a part (§ 5)
@@ -90,32 +110,45 @@ The module is written in Go against the SDK. It holds no checkout of this reposi
the records module, as the issue tracker's import does. Until the move of § 8.1, its records are in a database in
the store; from the move on, it requires no database (ADR 0592 §1, changing ADR 0290 §7).
**A write is whole or refused, and `submit` commits when its version is stored** (ADR 0592 §10). Every change to
`heads`, `drafts`, `asked`, `names` and `findings` is a compare-and-set on the key's revision, retried at most three
times and then refused with the reason. The commit point of `submit` is the version message stored on `versions`: a
failure before it is stored refuses the call, and nothing is taken. Once it is stored, the call answers that the
version is taken; the head's update against its revision and the event are then completed at once, retried, and at
the next start for any head behind its last stored version. So nothing an author was told was refused takes effect
later. A reservation is a head created only if the key is absent, so two sessions reserving at once get two numbers
or one refusal.
**A record is changed by an operation, and the head follows from it** (ADR 0597 §1 to §8). Every change to a record
is one operation the module hands to `apply`: among them `reserve`, `propose`, `write-section`, `submit`,
`take-over`, `review` (a part's verdict), `ask` and `warrant` (the operator's ask and answer), `accept` and
`withdraw`. The module checks its own rules first (an `accept` needs the operator's Approve where the record needs
it, § 5; only the author writes a draft, § 4), then calls `apply`, which checks the operation against the record's
type, appends it to the instance's history with the sequence it expects, derives the head by the operation's effect,
and writes the head and its snapshot. The author is the account the bus proved for the caller of the module's verb,
passed on by the module and stamped by `apply`, never a name the caller gave (ADR 0597 §7).
**A write is whole or refused, and an operation commits when its history takes it** (ADR 0597 §5, §6, changing
ADR 0592 §10). The commit point of every change to a record is the operation appended to the ledger's history: a
failure before it is appended refuses the call, and nothing is taken; a racing writer is refused by the bus, which
expects the history's last sequence for the instance. Once it is appended, the call answers that it is taken, with
its sequence; the head is derived by `apply` as an update of the revision it read, and a head behind its history is
caught up by the next `apply` or read. A full state bucket answers that the operation is taken and the head behind,
never that it was refused. So nothing an author was told was refused takes effect later. A reservation is an
operation `reserve` on an instance with no history, so two sessions reserving at once get two numbers or one
refusal. The module's other state (`drafts`, `asked`, `findings`) keeps the compare-and-set on the key's revision,
retried at most three times and then refused with the reason (ADR 0597 §12).
**The search index** is a full-text index in the module's own memory, written with the pure Go library bleve. At
start the module reads every head and the bodies of every current version into it, then keeps it current by watching
`heads`, noting the last revision of `heads` it took in. `ask` and `holds` answer from it, ranked by title, clauses
start the module reads every head from the ledger's state and the bodies of every current version from their
`write-section` operations into it, then keeps it current by watching the ledger's state, noting the last sequence it
took in (ADR 0597 §12, changing ADR 0592 §11). The index is the module's own, in its memory; the ledger's declared
index (ADR 0597 §4) is not used for it, because a full-text index is not a key-value bucket. `ask` and `holds` answer from it, ranked by title, clauses
and sections as before, and refuse with "index not built" while it is built; `get`, `list` and `propose` answer from
`heads` at once (ADR 0592 §11). The integration test measures the start; the estimate is 20 to 40 seconds and about
the ledger's state at once (ADR 0592 §11). The integration test measures the start; the estimate is 20 to 40 seconds and about
100 MB of memory for today's 736 records. Nothing but the module writes it, and nothing outside it reads it.
### 1.1 Its tools, health and events
Every piece of state has a tool that reads it, and every act the module does by itself can be done by hand
(ADR 0592 §32):
(ADR 0592 §32, changed by ADR 0597 §12: the sweep and the digest check become the ledger's repairs):
| Tool | Reads or does |
|---|---|
| read a head | one key of `heads`, whole |
| read a version | one message of `versions`, with its sections' titles, digests and authors |
| read a section | one object of `sections`, by its digest, checked against it |
| read a head | one instance's state in the ledger, whole, with the sequence of the last operation it holds |
| read a record's history | the instance's operations in order, each with its sequence, author and when; a version is its `submit` |
| read a section | one `write-section` operation, by its sequence, its text checked against its digest |
| read a draft | one key of `drafts`, with its author and since when |
| read a review | the part verdicts of one version, from `reviews` |
| in-flight parts | the keys of `asked`: each part waiting on the judge, its ticket and since when; ask one again |
@@ -123,18 +156,18 @@ Every piece of state has a tool that reads it, and every act the module does by
| fill | each bucket's and stream's size and count against its cap |
| rebuild the index | the search index, built again from the bus |
| review again | asks one part's review again, for a version still proposed |
| sweep | lists the objects no version or draft names and since when; sweeps them now; dry run |
| integrity | checks every named digest has its object; restores a body from given text once its digest matches |
| the ledger's repairs | the tool runner's `rebuild` (a head from the history or the snapshot), `snapshot` and `lag` (how far each head is behind its history), served for the module's own ledger (ADR 0597 §10) |
| integrity | checks every section a head or draft names resolves to a `write-section` whose text hashes to its digest |
| form-only | lists, dry-runs and runs the form-only versions of § 2.2 |
| move, and check the move | § 8.1, each with a dry run |
**Health** (ADR 0592 §33) is unhealthy, naming the cause, when a bucket or stream does not answer, when the index is
behind the latest revision of `heads`, when a digest some version names has no object (counted, each named), and
when a bucket or stream is at its cap. It names the source it serves from (the database or the bus) and the result
behind the ledger's state, when a head lags behind its history (the ledger's lag, ADR 0597 §12, changing
ADR 0592 §33), and when a bucket or stream is at its cap. It names the source it serves from (the database or the bus) and the result
of the move's last check, and counts drafts idle more than a week, records the form-only check refused, and parts
waiting more than 30 minutes. A part waiting more than 30 minutes is asked of the operator once through the operator
channel, and the ask clears when the part is judged; the module stays healthy meanwhile, because a record waiting on
its review is not the module failing (issue 508). The condition at 75 percent of a cap is the grants part's (§ 2.1).
its review is not the module failing (issue 508). The condition at 75 percent of a cap is the bus module's (ADR 0595 §6; [design 25](25-the-bus-on-nats.md) § 12).
**Events** (ADR 0592 §34) are kept 30 days; what a later reader needs is also in the version message. Each carries:
@@ -148,7 +181,6 @@ its review is not the module failing (issue 508). The condition at 75 percent of
| `taken-over` | the record, the previous author, the new author, the reason |
| `part-reviewed` | the record, the version, the part, its digest, the verdict |
| `reviewed` | the record, the version, every part's verdict: one event per version, as before |
| `swept` | the digest removed, since when nothing named it |
| `moved` | the record, its counts on the bus |
| `checked` | the move's check: records compared, every difference |
| `formed` | the record, the version, the names changed |
@@ -164,7 +196,7 @@ None is named `drafted`, which stays the event of a clause the import's judgemen
parts by title; for a design, its `to-be` section and, where something of it runs, its `as-is` section (§ 9);
- for a decision, its **clauses** (§ 3);
- its **links**, typed, each stored once and read from both ends;
- its **versions**: every change is a new version with who made it, when, and why in one line; `history` lists them
- its **versions**: every `submit` is a new version with who made it, when, and why in one line; `history` lists them
and gives the difference between any two. A progressive insight is a version marked as one, dated, saying what stood
before. A form-only version (§ 2.2) is marked `form`.
@@ -180,66 +212,72 @@ None is named `drafted`, which stays the event of a clause the import's judgemen
### 2.1 Where it is kept
A record is kept in three of the bus's own forms, so that a new version costs only what changed (ADR 0592 §1 to §9):
A record is kept as one instance of the knowledge base's ledger `records` (ADR 0597 §11), so that a new version
costs only the sections it changes; the forms of ADR 0592 §1 to §9 that it replaces are named in its §12.
```
heads (state) versions (stream) sections (object bucket)
┌─────────────────────┐ ┌──────────────────────────┐ ┌────────────────────────┐
│ decision.0290 │ │ decision.0290 version 7 │ │ 3f9a… "Context" text │
│ title, status, … │ │ who, when, why, mark │ │ 81c2… "Options" text │
│ clauses, links │ │ the head as it stood: │──────▶ │ 0d77… "Context" text, │
│ current: 8 │ │ Context 3f9a… │ │ as changed in 8 │
│ sections: │ ├──────────────────────────┤ │ … │
│ Context 0d77… │──────▶│ decision.0290 version 8 │──────▶ └────────────────────────┘
│ Options 81c2… │ │ Context 0d77… │ each body once, named by
└─────────────────────┘ │ Options 81c2… │ its sha256 digest
└──────────────────────────┘
records: history (stream, one subject per instance) records: state (key-value, derived)
┌───────────────────────────────────────────────┐ ┌─────────────────────────────────┐
│ 101 decision/0290 propose fields, … │ │ decision/0290 │
│ 102 decision/0290 write-section "Context" │ │ title, status, clauses, links │
│ text, digest 3f9a… │ apply │ current sections: │
│ 103 decision/0290 write-section "Options" │ ───────▶ │ Context 3f9a… 102 │
│ text, digest 81c2… │ │ Options 81c2… 103 │
│ 104 decision/0290 submit sections 102, 103 │ │ pending sections: none │
│ 230 decision/0290 write-section "Context" │ │ after operation 231 │
│ text, digest 0d77… │ └─────────────────────────────────┘
│ 231 decision/0290 submit sections 230, 103 │ records: snapshot (one per instance,
└───────────────────────────────────────────────┘ the latest head, never expiring)
```
- **The head** is one key in the state `heads`, `<kind>.<number>`: the record's fields, its clauses with their status
and source, its links, the operator's ask and warrant, its current and pending version, the latest verdict of each
part's review (§ 5), and its sections in order, each a title and a digest. A key's value stays under 256 KiB
(ADR 0201). A reserved number is a head at `reserved`, with who reserved it, when, for what and when it expires.
- **A version** is one message on the stream `versions`: who wrote it (the caller the bus proved), when, why, its
mark, who wrote each section, the reason of any take-over since the version before, and the head as it stood. It never copies a section's body. The stream keeps its history: every message,
with no age limit, and when it reaches its cap it refuses a new message rather than dropping its oldest, with a
condition raised at 75 percent of the cap. A module's stream may declare that retention; before, a stream was kept at
most 30 days (ADR 0592 §6, changing ADR 0363 §1). The stream `reviews` keeps its history the same way. The
condition at 75 percent is raised by the grants part (mesh-grants), in one place, for every history stream and
object bucket of every module (ADR 0593 §10, changing ADR 0592 §16).
- **A section's body** is one object in the bucket `sections`, named by the sha256 digest of the body: a section
unchanged in a new version costs nothing. The module checks that a body hashes to its name each time it reads one.
The state `names` keeps, per digest, how many versions and drafts name it and since when none has; an object no
version and no draft names is removed a day after the last stopped naming it, each removal the event `swept`. The bucket is storage of the bus, as the object bucket of ADR 0486 is. Any module
may declare an object bucket of its own in its manifest, with a local name and a cap, created as its state buckets
are, kept with no age limit and refusing a new object when full (ADR 0592 §5).
- **The head** is the instance's state, `<kind>/<number>`: the record's fields, its clauses with their status and
source, its links, the operator's ask and warrant, its current and pending version, the latest verdict of each
part's review (§ 5), and its sections in order, each a title, a digest and the sequence of the `write-section`
that wrote it, never the text. A section's title is held to 200 bytes, so an entry is at most about 300 bytes,
and a head with 64 pending and 64 current sections holds at most about 40 KiB of them, within a state value's
256 KiB (ADR 0597 §11). A reserved number is a head at `reserved`, with who reserved it, when, for what and when it
expires. It is derived by `apply` and never written by the module.
- **A version** is the operation `submit`: who wrote it (the caller the bus proved), when, why, its mark, the
reason of any take-over since the version before, and the version's sections (title, digest and sequence each),
which its effect sets as the head's current sections. It never copies a section's text. Who wrote each section is
the author of its `write-section`.
- **A section's text** is in the `write-section` operation that wrote it, with its digest, at most 40 KiB within a
history entry's 256 KiB: a section unchanged in a new version costs nothing, since no operation is written for it.
`apply` refuses a `write-section` whose digest does not match its text, and the module checks the text against
the digest each time it reads one. A dropped draft's operations stay in the history, named by no version
(ADR 0597 §11).
- **The history** is kept for good: the ledger template's history refuses a new operation when full, rather than
dropping its oldest, and the bus module raises its condition at 75 percent of the cap (ADR 0595 §3, §6). Each
instance's snapshot, the latest head on a subject of its own, makes a rebuild fast (ADR 0597 §8).
- **A draft** is a key in the state `drafts` (§ 4). **A part's verdict** is a message on the stream `reviews`, and a
part asked of the judge and not yet answered a key in the state `asked` (§ 5). **The whole-set findings** standing
now are keys in the state `findings`, and what was told to the operator once is a key `told.<topic>` there.
now are keys in the state `findings`, and what was told to the operator once is a key `told.<topic>` there. These
keep their own forms (ADR 0597 §12), asked for in `requires`: the states by the template state; `reviews`, kept for
good and heard only where granted (ADR 0592 §6, §8), as a bare stream with its own configuration (kept for good,
refusing when full, a grant), because no template both keeps for good and takes a grant (ADR 0595 §2).
**Caps** (ADR 0592 §25). Each is set from the move's dry run, within its form's bound: a stream at most 8192 MiB
(ADR 0363), a state at most 64 MiB (ADR 0201), an object bucket at most 8192 MiB (ADR 0592 §5). The starting figures, from
today's 83 MB for 736 records with every version copying every section, are:
**Caps** (ADR 0592 §25, its object bucket's cap lapsing by ADR 0597 §12). Each is set from the move's dry run, within
its template's bound (ADR 0595 §3): a history at most 8192 MiB, a state or a ledger's state at most 64 MiB. The
starting figures, from today's 83 MB for 736 records with every version copying every section, are:
| Bucket or stream | Cap |
|---|---|
| `versions` (stream) | 1 GiB |
| `reviews` (stream) | 512 MiB |
| `sections` (object bucket) | 1 GiB |
| `drafts` (state) | 64 MiB |
| `heads`, `asked`, `names`, `findings` (state) | 64 MiB, ADR 0201's bound |
| Kept data | Template | Cap |
|---|---|---|
| `records` (history) | ledger | 1 GiB |
| `records` (state) | ledger | 64 MiB |
| `reviews` | a bare stream, its own configuration | 512 MiB |
| `drafts` | state | 64 MiB |
| `asked`, `findings` | state | 64 MiB, the template's bound |
A write refused at a cap names the cap, and the module's health turns unhealthy naming it.
**Who may read each** follows the mesh's own forms, read-only (ADR 0592 §8, §9). `versions` and `reviews` are
restricted streams (ADR 0363): a module hears them only through its own `consumes` and a `grant` in mesh-knowledge's
trusted config value `readers`, which names the web page that shows records once it exists and nothing today. The
object bucket `sections` is read the same way, through a `reads` entry and a `grant` of `readers`: ADR 0592 extends
ADR 0363's grant to a module's object bucket, and the grants part composes a read-only grant on that one bucket
(ADR 0593 §10).
`heads` is read by a module that names it in `reads`; `drafts`, `asked`, `names` and `findings` by mesh-knowledge
alone. No secret is kept in any of them. A grant cannot limit a reader to some objects of a bucket
(ADR 0488), which is why reading is announced per bucket and never per record.
**Nobody else reads any of it** (ADR 0596 §2, ADR 0597 §9, changing ADR 0592 §8 and §9). Only the knowledge base's
own account reads its ledger's history, snapshot and state and its other kept data; the ledger's history is published
on only by the tool runner's ledger credential. Another module asks the knowledge base's verbs (`get`, `history`,
`ask`) or hears its events (`reviewed`, `accepted` and the rest); the reads of `heads` and `sections` that ADR 0592
let another module declare lapse once those verbs answer what each reader needed. `reviews` stays a restricted event
family (ADR 0363 §4), heard only by a module granted it in mesh-knowledge's trusted config value `readers`, which
names nothing today. A draft is answered by the verbs to its author only, and a version's bodies as `get` answers
them, so the text in the history is no more open than ADR 0592 kept it. No secret is kept in any of it.
### 2.2 A section is a part with a title
@@ -304,9 +342,11 @@ version (ADR 0592 §17):
```
A draft is a key in the state `drafts`: its author, when it was opened, the version it starts from, the record's
fields and clauses, and its sections in order as title and digest. Each `write` stores the section's body in the
object bucket `sections` and its digest in the draft, so the knowledge base holds the only copy of a draft and a
session on any node can read it. A draft holds at most 64 sections and 1 MiB; one untouched for a week is counted in
fields and clauses, and its sections in order as title, digest and the sequence of the `write-section` that wrote
each. Each `write` appends a `write-section` operation with the section's title, text and digest to the record's
history at once, through `apply`, and names its sequence in the draft (ADR 0597 §11, changing ADR 0592 §17), so an
unsubmitted draft's text is already in the history, the knowledge base holds the only copy of a draft, and a
session on any node can read it through the verbs; `submit` carries the draft's sections into the version. A draft holds at most 64 sections and 1 MiB; one untouched for a week is counted in
the health, and one idle for 14 days is dropped and its author told by `dropped`.
**A draft and a proposal have one author** (ADR 0592 §19). Only the author writes a draft or revises a proposal;
@@ -454,32 +494,38 @@ field by field and section by section, and the result is written on issue 426.
### 8.1 The move off the store
The records leave the store once, and the database is dropped only after a check and the operator's yes
(ADR 0592 §12 to §16):
(ADR 0592 §12 to §16, the forms copied into changed by ADR 0597 §12):
1. **Order.** Two deliveries, in order (ADR 0593 §10, changing ADR 0592 §16): first the mesh-sdk release sdk1
(issue 1585), which carries the manifest's field types for `objects` and `retention: history`, their checks in
the indexer's judge, and in the grants part the creation of the object buckets and history streams, the
read-only grants of `readers` on them and the condition at 75 percent for every such bucket and stream in one
place; then mesh-knowledge. An indexer whose judge predates these keys refuses a manifest that has them, so
mesh-knowledge cannot be delivered before the release. The claude-code module's `judged` event is
1. **Order.** The ledger is built first, in the order of ADR 0595 §10: mesh-sdk's typed `requires`, binding and
package of the new calls; the indexer's judge's rules; the bus module's process, verbs and events; the tool
runner's calls, `apply` among them, with its ledger credential per module; then mesh-knowledge moves, asking for
its ledger, states and stream in `requires`. An indexer whose judge predates the new form refuses a manifest
that uses it, and the placement holds the module off a machine whose tool runner does not read the typed
binding, so mesh-knowledge cannot be delivered before them. The release sdk1 (issue 1585, ADR 0593 §10), which
created object buckets and history streams through the grants part, is no longer needed by the move: the
knowledge base asks for no object bucket and no stream `versions`. The claude-code module's `judged` event is
delivered beside, and is not needed for the move (§ 5).
2. **Copy.** The move copies everything the database holds onto the bus, every record, version, clause, link,
review, finding, ask with its warrant, reservation and told mark: each version's sections into `sections` by
digest, each version as a message on `versions`, each record's head into `heads`, each standing finding into
`findings`, each told mark as `told.<topic>`. A clause keeps its number, status and source as the database holds
review, finding, ask with its warrant, reservation and told mark: each record's versions as operations on its
instance in the ledger, in order (a `write-section` for each section a version changed, then its `submit`, then
the record's review, ask, warrant and acceptance), through `apply`, so each head is derived and never copied;
each standing finding into `findings`, each told mark as `told.<topic>`. A clause keeps its number, status and source as the database holds
them (ADR 0321). It emits `moved` per record, writes nothing twice, and a second run changes nothing and says so.
A dry run lists what it would write and the size each bucket and stream will need, from which the caps are set.
3. **The window.** From the move's final copy through the check to the switch, every writing verb is refused with
its reason ("the knowledge base is moving"). Reading verbs answer throughout.
4. **Check.** A tool compares the database and the bus record by record: for each record the count of each of
those, the digest of every section of every version, computed from the database's text and read from the bus,
that each body hashes to its name, and a digest of every review's, finding's and ask's content. It emits
those, the digest of every section of every version, computed from the database's text and read from its
`write-section` on the bus, that each text hashes to its digest, that each head derived equals the head the
database holds, and a digest of every review's, finding's and ask's content. It emits
`checked` with every difference, and the health shows its last result.
5. **Switch.** Only once the check shows no difference, the module answers from the bus. The switch is a key on the
bus, so a restart never falls back to the database. A difference is told to the operator, and the module stays on
the database.
6. **Rollback.** Until the database is dropped, the module writes each change to the database too, so a rollback of
the delivery reads a current database. A rollback is safe only until the drop.
6. **Rollback.** Until the database is dropped, the module writes each change to the database too, after `apply`
has taken its operation, so a rollback of the delivery reads a current database; a write the database refuses
after the ledger took it is retried and, past its tries, raises a condition naming the record, never answered as
refused. A rollback is safe only until the drop. This settles the rollback ADR 0597 §13 left to the design.
7. **Tell and ask.** Once the check is clean, the operator is told its result and asked, once, whether the database
may be dropped, because dropping it removes data.
8. **Drop.** On the operator's yes, the delivery after it removes the module's requirement of a database, and the
@@ -494,7 +540,7 @@ The records leave the store once, and the database is dropped only after a check
| 2 | the decision records and research efforts leave this repository; path mentions become "ADR NNNN" and "research NNN"; playbooks 01 and 02 changed | this repository holds no decision record, and `get` answers every one | issue 426 |
| 3 | designs and the glossary's entries as records, moved in one delivery, and each design's state worked out by the knowledge base (§ 9.1 to § 9.3; ADR 0423, ADR 0424) | `get` answers design 47 with its sections and links, a word's entry answers from the knowledge base, and a design's state follows its issues' deliveries without anyone writing it | issue 428 |
| 4 | meeting notes, drawings and blog posts | not decided; needs its own decision | issue 429 |
| 5 | the records on the bus, written and reviewed by section, the move off the store and the titles (§ 1, § 2, § 4, § 5, § 8.1; ADR 0592), delivered as the mesh-sdk release sdk1, then mesh-knowledge, with claude-code's `judged` event beside | the acceptance tests of the fifth slice in § 10, live; the knowledge base requires no database | the issues its hand-off opens (playbook 04) |
| 5 | the records on the bus, written and reviewed by section, the move off the store and the titles (§ 1, § 2, § 4, § 5, § 8.1; ADR 0592), kept as the first ledger (ADR 0597), delivered after the ledger's build (ADR 0595 §10), then mesh-knowledge, with claude-code's `judged` event beside | the acceptance tests of the fifth slice in § 10, live; the knowledge base requires no database | the issues its hand-off opens (playbook 04) |
Numbers from the issue tracker, which `reserve` needs, are the tracker's own slice, tracked by issue 424. This
repository keeps no copy of the records, generated or otherwise (issue 435, answered by the operator). Playbooks
@@ -592,10 +638,10 @@ milestones and nothing of how a delivery runs; it only listens for the announcem
| An answer stays within its bound | the module's test over the full import of `main` at a recorded commit: every `ask`, `get` and `holds` of a fixed question list |
| A record written in this repository meanwhile writes its clauses | `records.py`, check `clauses`, on every hq pull request, until the records left this repository (issue 426); since then `records.py`, check `no-records`, refuses any record written here |
| Live | the acceptance sentence, asked from a session outside this repository |
| A record is kept in the bus's forms, a section once by its digest, and read only as announced | one integration test that starts the module whole against a real bus server: a record of three sections and a second version changing one give one head, two version messages and four objects; a module with no `grant` in `readers` is refused the streams and the object bucket |
| A record is kept as a ledger instance, a section's text once in its operation, and read by no other module | one integration test that starts the module whole against a real bus server: a record of three sections and a second version changing one give one head, two `submit` operations and four `write-section` operations, the changed section's text found only in its operation; another module's account reading the ledger's history, snapshot or state, the head among them, is refused (ADR 0597 How it is checked, clauses 9, 11, 12) |
| Search answers from the index as it did from the store | a unit test over a fixed set of records and a fixed question list; the integration test restarts the module and asks again |
| A draft is written by section and has one author, and a take-over tells the previous one | unit tests of each verb of § 4, of a refusal naming the author, and of `take-over` emitting `taken-over`; a take-over refused while the operator's Approve is waited for |
| A stream that keeps its history, and an object bucket of a module's own | in the integration test: a history stream at its cap refuses a new message and keeps its oldest; an object no version or draft names is removed after a day, one still named is kept; unit tests of registration refusing a name already taken |
| A ledger that keeps its history | in the integration test: the history at its cap refuses a new operation and keeps its oldest; a draft's written section is in the history before `submit` and named by no version after `drop`; a section title over 200 bytes is refused |
| A long record is judged whole, one part at a time | unit tests: the frame never passes 16 KiB and says what it cut, with twenty clauses of 2000 bytes and a 40 KiB Context; every question stays under 60 KiB with a 40 KiB section and that frame, and the head part with 40 KiB of clauses; a section over 40 KiB is refused saying split it; an unchanged section keeps its review, a changed frame asks it again |
**The fifth slice's tests, one per failure path** (ADR 0592; the review by architectural domain's testing ruling). Unit tests are
@@ -607,14 +653,14 @@ table-driven; one integration test starts the module whole against the real bus
| two writes race on one draft | unit: the second compare-and-set loses, is retried, and is refused with the reason past its tries |
| `submit` races `take-over` | unit: one wins; the other is refused naming the new author or the new version |
| two reservations race | unit: two numbers, or one refused; never one number twice |
| a write cut between the stream and the head | unit: the call answered that the version is taken; a start completes the head from the stored version and sends the event once |
| a failure before the version is stored | unit: the call is refused, and no head, event or version follows later |
| a write cut between the history and the head | the ledger's integration test (design 25 § 12): the call answered that the operation is taken; the next `apply` or read derives the head; the module sends the event once |
| a failure before the operation is appended | unit: the call is refused, and no head, event or version follows later |
| a change through the bus during the move | unit: read back from the database, so a rollback of the delivery reads a current database (ADR 0592 §14) |
| a section changed after the operator's Approve | unit: the record is asked again; a version with every section unchanged is not (ADR 0592 §21) |
| a named digest has no object | unit: health unhealthy, naming it; the integrity tool restores it from given text |
| a body does not hash to its name | unit: the read refuses it, naming the digest |
| a full object bucket | integration: a write refused naming the cap; health unhealthy |
| a full history stream | integration: `submit` refused with the head unchanged |
| a head behind its history | unit: health unhealthy naming the lag; the ledger's `rebuild` clears it |
| a text does not hash to its digest | unit: `apply` refuses the `write-section`, and a read refuses it, naming the digest |
| a full ledger state | integration: `submit` answers taken with the head behind; health unhealthy naming the cap |
| a full ledger history | integration: `submit` refused with the head unchanged |
| a judge timeout for one part | unit: that part stays asked and the version proposed, saying which; the others keep their verdicts |
| a restart with parts asked | integration: the parts in `asked` are collected again and judged |
| `reviewed` once per version | unit: one `reviewed` after the last `part-reviewed`, carrying every verdict |
@@ -622,12 +668,13 @@ table-driven; one integration test starts the module whole against the real bus
| a second run of the move | unit: changes nothing and says so |
| the move's check | unit: a changed review, a missing finding and a missing ask, each named |
| writes during the switch | unit: every writing verb refused with its reason; reading verbs answer |
| an older judge | the indexer's judge test (release sdk1): a manifest with `objects` or `retention: history` refused by a judge that does not know them, accepted by one that does |
| the grants part's forms | the grants part's test (release sdk1), against a real bus server: an object bucket and a history stream created for their module; a read-only grant on that one bucket for a module named in `readers`, removed at the next composition once withdrawn; the condition at 75 percent raised and cleared below |
| an older judge | the indexer's judge test (ADR 0595 §8): a manifest asking for a ledger in `requires` refused by a judge that does not know the template, accepted by one that does |
| the bus module's forms | the bus module's test (design 25 § 12), against a real bus server: the ledger, states and stream created for their module, `reviews` heard by a module granted it in `readers` and by no other; the condition at 75 percent raised and cleared below |
| a reference | unit: "record › title › row" resolves to the section; a nested title is refused |
| a form-only version | unit: accepted when only names and links changed and each target resolves; refused otherwise |
| an unproven caller | unit: `open`, `write`, `submit`, `propose` and `take-over` refuse it |
| the index behind | unit: health unhealthy while the index's revision is behind `heads`; `ask` refuses "index not built" |
| the index behind | unit: health unhealthy while the index's sequence is behind the ledger's state; `ask` refuses "index not built" |
| the move as operations | unit: each record's versions copied as `write-section` and `submit` operations in order; the derived head equals the database's; a second run appends nothing |
| the start | integration: the time to a built index measured and reported |
| A section is a part with a title | a table test, one case per refused name and text of § 2.2; a form-only version whose text changed beyond its links is refused |
| Every bucket has a tool, and health says the index | the manifest test compares tools and events with the code; a unit test of the health before and after the index is built |
@@ -669,12 +716,17 @@ done:
6. After the titles' form-only versions, no record holds a section named like a file, `history` shows each as a
change of form, and `get` of "research 062 › Direction" answers that section.
7. The fill tool shows each bucket and stream against its cap; the second version of a record with one changed
section adds one object, not one per section.
section adds one `write-section` and one `submit`, not one operation per section.
8. During the move's window a `write` is refused saying the knowledge base is moving; after the switch a restart
still serves from the bus; the health names the source and the check's result.
9. A part's review waiting on the judge survives a restart of the module, and one past 30 minutes reaches the
operator once.
10. Every test of the table above passes in the one check of the pull requests, within the test budget.
11. As a ledger (ADR 0597 How it is checked, clauses 11 and 12): `get` answers the same digest for every record
before and after the move; `ask` answers from bodies read from their operations; `mesh-knowledge.reviewed` is
emitted once per review; the module's manifest asks for no stream `versions`, no object bucket `sections` and
no state `names`, and the bus holds none for it once the move's check is clean; its tools and health name the
ledger's repairs and lag, and no digest store; the consumers of `reviews` still hear it.
## What it leaves out