ADR 0233: how data is measured, and what the first night costs

Nothing large is walked; the first night's size on the home server is measured and stated, the previews are left out, the platform database is dumped, and the bus's live copy is flagged.
2026-10-06 17:06:18 +02:00
parent 7188d443bc
commit 082693fb25
3 changed files with 37 additions and 18 deletions
@@ -118,10 +118,22 @@ anything irreplaceable depends on the machine's `node-backup` holder (ADR 0207)
 up or watches its array — so it cannot be assigned where nothing would. Valuable and rebuildable data
 is backed up where a holder is and refuses no machine for want of one.

-**5. The holder measures; the controller judges.** Once an hour, and after every night, the holder
-measures every item that is not a cache — the size of its files and its newest change, in one walk as
-root — and reads the redundant storage it is on: a ZFS pool's health and its own verdict, an md array's
-members, a btrfs filesystem's error counters. It answers this, with each item's newest good backup, in
+**5. The holder measures; the controller judges — and nothing large is ever walked.** The holder
+measures every item that is not a cache by the item's own `measure`:
+
+- `walk` (the default, for small items): every file summed and the newest change found, in one walk as
+  root — **at most once a day**, after the night, and **stopped after ten minutes or two million
+  files**, said as "measured partially": a partial size is a lower bound, shown and never compared;
+- `dataset`: the ZFS dataset holding the path — its size from the filesystem's own counters — and the
+  newest change among the path's top-level entries; hourly, nothing walked. Items on one dataset share
+  its size, so a shrink of it is one condition naming them all;
+- `shallow`: the newest change among the top-level entries only, and no size; hourly.
+
+A large item never says `walk`: the media library is `dataset`, Plex's metadata and previews, the
+artifact store and the models are `shallow`. A file walk over a pool of that size every hour is load on
+the array that protects the library, and competes with the server reading it. The holder also reads the
+redundant storage each item is on: a ZFS pool's health and its own verdict, an md array's members, a
+btrfs filesystem's error counters. It answers this, with each item's newest good backup, in
 `node-backup.backed-up`; it decides nothing. **The holder is the measurer**, not the node-engine: it
 already reads every declared path as root, it is a module that can be replaced, and the core gains no
 work. The controller's self-check asks every holder on every run.
@@ -144,7 +156,7 @@ class kept in it.

 | Condition | Raised when |
 |---|---|
-| `data-shrank` | an item holds less than half of its largest size in seven days, and at least 16 MiB less |
+| `data-shrank` | an item holds less than half of its largest size in seven days, and at least 16 MiB less; one condition per dataset for items measured from a dataset's counters; never from a partial size |
 | `data-missing` | an item's path is gone |
 | `empty-replacement` | a copy in use is less than half the size (and 16 MiB less) of a copy of the same thing kept elsewhere: an item retired on another machine, or a consumer's data at another provider — issue 273's shape |
 | `data-held-twice` (warning) | a consumer has active data at two providers whose sizes cannot be compared |
@@ -169,7 +181,7 @@ the media library and the photo sites' storage are irreplaceable; everything peo

 | Module | Own data | Its consumers' | Kept with a provider |
 |---|---|---|---|
-| plex | the media library, eight operator paths: **irreplaceable**, by **redundancy**; metadata, previews, working data: rebuildable (backed up); transcode: cache | | |
+| plex | the media library, eight operator paths: **irreplaceable**, by **redundancy**, measured from its dataset; metadata: rebuildable, backed up, measured shallow; **previews: rebuildable, not backed up** (Plex makes them again; too large to copy nightly); working data: rebuildable; transcode: cache | | |
 | photos | | | object store and document store: **irreplaceable** |
 | postgres, mssql, mongodb | the store: valuable, by dump; the dumps: rebuildable | valuable | |
 | minio, influxdb | data (and influx's config): valuable | valuable | |
@@ -184,28 +196,33 @@ the media library and the photo sites' storage are irreplaceable; everything peo
 | home-assistant | its configuration and history: valuable, written all the time (a day) | | |
 | nats | the bus's streams and buckets: valuable, written all the time (a day) | | |
 | n8n | data: valuable; scratch: cache | | |
-| supabase | database, storage, functions: valuable; its seeded configuration: rebuildable | | |
+| supabase | database: valuable, **by `pg_dumpall`** inside its container; storage, functions: valuable; the dump and its seeded configuration: rebuildable | | |
 | claude-code | the operator's agent's home: valuable | | |
 | ssh-client | the operator's keys: valuable | | |
 | radarr, sonarr, lidarr, bazarr | the database: valuable, by sqlite's own backup; settings: valuable; covers and the night's copy: rebuildable; lidarr's start-up scripts: cache | | |
 | bookshelf, jackett, kometa, nzbget, ombi, qbittorrent, tautulli | configuration: valuable | | |
-| distribution, ollama | the artifact store, the models: rebuildable, **not backed up** (too large; made again) | | |
+| distribution, ollama | the artifact store, the models: rebuildable, **not backed up** (too large; made again), measured shallow | | |
 | only-office | its working data, database, fonts: rebuildable; logs, queue, cache: cache | | |
 | records, route-proxy | the record's clone, issued certificates: rebuildable; the authority's roots: cache | | |
 | restic | the repository: rebuildable, not backed up — it is the copy, never the only copy of anything; its state: cache | | |
 | build-agent, icecast, lab, model-usage, searxng | cache | | |

-**Uncertain, for the operator:** supabase's database is copied as live files, which may not restore — a
-dump is wanted; the bus's streams are copied live; mailu's queue is valuable though transient; Plex's
-previews may be large to copy nightly; the agent's home and the operator's keys are valuable, and are
-backed up only on machines that hold `node-backup` (today, neither workstation does).
+**Uncertain, for the operator:** the bus's streams are copied as live files — the bus's image carries
+no tool to snapshot a stream safely, so a consistent copy needs the bus's own backup command run from
+somewhere that has it, which is not built; mailu's queue is valuable though transient; the agent's home
+and the operator's keys are valuable, and are backed up only on machines that hold `node-backup`
+(today, neither workstation does).

 ## Consequences

 - **The first run of the self-check records what every machine holds**, and the first night backs up
-  what was not backed up before: on the home server, every application's own data and Plex's metadata
-  and previews; on the control node, the bus's streams and the certificate authority. The first night
-  there costs that data's full size, later nights what changed.
+  what was not backed up before: on the home server, every application's own data and Plex's metadata;
+  on the control node, the bus's streams and the certificate authority. Measured read-only on
+  2026-10-06, the home server's new nightly data is about 11 GB of application data (the chat server's
+  database 5.6 GB, the database platform's 2.1 GB before its dump, the network controller 1.1 GB, the
+  viewing history 0.9 GB, dashboards 0.5 GB, the rest under 0.2 GB each) and Plex's metadata, at most
+  the 136 GB its disk holds — under 150 GB against 33.6 TB free where the repository is: under half a
+  percent. The previews, on the media pool, are left out. Later nights cost what changed.
 - **The media library is watched, not copied.** Its array is read every hour; a degraded pool is
  urgent. A pool that loses a vdev still loses the library — the operator's accepted risk, now said
  rather than silent. SMART pre-failure warnings are not read: the pool's own device error counters are,
@@ -273,7 +273,7 @@ mesh protects only what it is told about. One section, `data`, says all of it:

 | `data.` | says |
 |---|---|
-| `own` | each item the module keeps: `id`, `path` (`${dir:<id>}` or `${access:<id>}`, or beneath one), `class`, and its protection — `backup` (`copy`, `none`, or `{dump, into}`) or `redundancy` with why that is enough — with optional `within`, `active`, `why` |
+| `own` | each item the module keeps: `id`, `path` (`${dir:<id>}` or `${access:<id>}`, or beneath one), `class`, and its protection — `backup` (`copy`, `none`, or `{dump, into}`) or `redundancy` with why that is enough — with optional `measure` (`walk`, `dataset`, `shallow`), `within`, `active`, `why` |
 | `consumers` | per provision it grants: the `class` of what it keeps for each consumer, and `in` — the own item, or the required provision, holding it |
 | `kept-by` | per provision it requires: the `class` of what of its own the provider keeps |

@@ -50,8 +50,10 @@ bucket, runs a bad migration — and not for the day a disk dies (ADR 0214).
 *Added 2026-10-06, [ADR 0233](../../02-DECISIONS/0233-a-module-declares-the-data-it-holds-and-the-mesh-protects-and-watches-it-from-that.md).*
 The mesh composes a second file for the holder: every data item every module on the machine declares,
 with its class, its path, the path that covers it and its protection. Once an hour and after every
-night the holder measures each item that is not a cache — the size of its files and its newest change,
-in one walk as root — and reads the redundant storage it is on (a ZFS pool's health, an md array's
+night the holder measures each item that is not a cache, by the item's `measure`: a bounded walk at
+most daily for a small item (stopped after ten minutes or two million files and said as partial), a ZFS
+dataset's own counters hourly, or the top-level entries only — never an hourly walk of anything large —
+and reads the redundant storage it is on (a ZFS pool's health, an md array's
 members, a btrfs filesystem's error counters). `backed-up` says each item with that measurement and its
 newest good backup. The holder judges nothing: the controller's self-check (to-be 45, D13) keeps the
 readings and raises what they show.