ADR 0235: back the bus up by the server's own snapshot of each stream
Resolves ADR 0233's flagged item (the bus's streams copied as live files) as a progressive insight pointing here. The bus module's own account is granted the snapshot API and nothing else; restore builds a new store beside the live one for a person to swap in. To-be 43 gains the bus's section and its restore steps; design 25 and to-be 45 point to it.
This commit is contained in:
@@ -8,8 +8,9 @@ code:
|
||||
- mesh-catalog modules/nats (to be written)
|
||||
- mesh-sdk src (the protocol's NATS binding, step 3)
|
||||
- mesh-tools node-tools/internal/bus (a module's state, ADR 0201)
|
||||
updated: 2026-10-04
|
||||
updated: 2026-10-06
|
||||
decisions:
|
||||
- 02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md
|
||||
- 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
@@ -175,6 +176,13 @@ call is a timeout the caller already handles.
|
||||
Streams and consumers are objects the controller creates at genesis and asserts on start; a module
|
||||
declares nothing about them. The controller is the only writer of stream definitions.
|
||||
|
||||
*Added 2026-10-06, [ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md):*
|
||||
one other user reads the streams whole, and writes none of them. The module that is the bus — the one
|
||||
holding `mesh-broker` — has an account of its own whose only grant is the snapshot API: the stream
|
||||
names, a stream's information, the snapshot request and its acknowledgements, and its own inbox. The
|
||||
night's backup takes every stream through it ([to-be 43](43-backups-against-mistakes.md)). The
|
||||
controller stays the only writer of stream definitions; the writers table holds that at composition.
|
||||
|
||||
### The store window, and what moving it into the server changes
|
||||
|
||||
The guarantee ([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)) is
|
||||
|
||||
@@ -5,8 +5,11 @@ code:
|
||||
- mesh-controller: internal/catalogue/seats.go (node-backup), internal/catalogue/seat_contributions.go (BackupSeat, CheckBackup, a contribution's directories)
|
||||
- mesh-catalog: modules/restic (the holder; it measures the declared data and deletes a retired item, ADR 0233), every module's `data` section
|
||||
- mesh-controller: internal/catalogue/data.go (the lines derived from a module's data, ADR 0233)
|
||||
- mesh-catalog: modules/nats/snapshot (the bus's snapshot and its restore, ADR 0235)
|
||||
- mesh-controller: internal/broker (the bus module's snapshot-only account, ADR 0235)
|
||||
updated: 2026-10-06
|
||||
decisions:
|
||||
- 02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md
|
||||
- 02-DECISIONS/0233-a-module-declares-the-data-it-holds-and-the-mesh-protects-and-watches-it-from-that.md
|
||||
- 02-DECISIONS/0214-backups-guard-against-mistakes-and-stay-on-the-machine.md
|
||||
- 02-DECISIONS/0053-a-step-that-runs-on-a-schedule.md
|
||||
@@ -72,6 +75,37 @@ has not seen, so the object store's first night costs its full size and later ni
|
||||
changed. Then the rotation prunes to 14 daily, 8 weekly and 6 monthly snapshots. A dump that fails
|
||||
fails the night for that module only; the others are still taken.
|
||||
|
||||
## The bus: the server's own snapshot, never its files
|
||||
|
||||
*Added 2026-10-06, [ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md).*
|
||||
The bus keeps everything it holds in streams — its key-value buckets are streams too — and its store
|
||||
is written all the time, so a copy of the store's files may not restore. Its streams are protected by a
|
||||
dump like a database's: in the bus's own container, a program built into the bus's image asks the
|
||||
server for each stream through the server's snapshot API, one at a time, acknowledging each chunk so
|
||||
the server's flow control keeps moving; the server goes on taking writes, and no stream is paused or
|
||||
reconfigured. It runs under the bus module's own account, which may take snapshots and do nothing
|
||||
else. It writes one archive into the module's snapshots directory, which the night copies: a manifest
|
||||
— when, how long, and per stream its messages, bytes, sequences, consumers and the checksum of each
|
||||
file — then each stream's configuration and the server's archive of it. The live store is not copied.
|
||||
|
||||
Every message up to the last sequence the manifest gives for a stream is in the archive, exactly; a
|
||||
stream written during its snapshot may carry part of what arrived while its blocks were read, after
|
||||
that sequence. A failure fails the bus's night, keeps the previous archive, and is said as
|
||||
`backup-stale` like any other.
|
||||
|
||||
**Restoring the bus** is done beside it, never over it — a stream is restored only where it does not
|
||||
exist, and on the live bus every one exists:
|
||||
|
||||
1. `restore` the bus module's snapshots from a named night, beside the live ones;
|
||||
2. check the archive against its manifest (`mesh-nats-snapshot verify`);
|
||||
3. build a new store from it with the bus's own image: the program starts the bus's server on loopback
|
||||
with the bus's one account, restores every stream, holds each to the manifest and stops it;
|
||||
4. swap the new store in for the live one with the bus stopped — a person's act and a planned bus step
|
||||
(to-be 45), in one command line, because the machine's node-engine starts a stopped container again
|
||||
at its next reconcile. The store kept aside is removed by a person once the bus is seen whole.
|
||||
|
||||
The module's README has the commands.
|
||||
|
||||
## The verbs
|
||||
|
||||
On the seat, for a person or an agent:
|
||||
@@ -92,6 +126,9 @@ a manifest. Not off-site: a lost machine loses its backups with its data, by the
|
||||
## Proving it
|
||||
|
||||
- A night that fails, or does not run, reaches the operator's output channel, naming the module.
|
||||
- The bus's snapshot is proven by restoring it, in tests against the bus's own release: a server
|
||||
written to throughout its snapshot, restored into a fresh server and into a new store served by a
|
||||
third, compared message by message (ADR 0235).
|
||||
- Weekly, the holder checks the repository's integrity and restores the newest dump of one database,
|
||||
in rotation, into a throwaway instance with no network, comparing table row counts with the live
|
||||
database.
|
||||
|
||||
@@ -408,7 +408,11 @@ controller: the streams are snapshotted; a `bus-maintenance` condition is open f
|
||||
the bus is replaced; afterwards D6, D7 and the round trip must pass, or the step is reported failed and
|
||||
the snapshot is the way back. A step whose new version cannot be reverted (the bus's 2.10 → 2.11 is one)
|
||||
says so before it starts and runs only on a person's explicit word, recorded as a hand act. Whether the
|
||||
bus becomes a cluster that can be upgraded live is left to its own effort.
|
||||
bus becomes a cluster that can be upgraded live is left to its own effort. *2026-10-06:* the snapshot
|
||||
and the way back from it exist — the bus image's own snapshot program, the same one the night's backup
|
||||
runs, and a restore that builds a new store beside the live one for a person to swap in
|
||||
([ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md),
|
||||
[to-be 43](43-backups-against-mistakes.md)).
|
||||
|
||||
## 9. Before merge: facts and replays (rule 9)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user