novox/hq
9
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Design 42 zfs-owns-zfs: schedule the run in list form, so nothing waits on the calendar fix

Browse Source 
Today's node-engine writes a cron step as an invalid calendar but a
minute list as a valid one, so the scheduled run declares
0,15,30,45 and the milestone no longer waits on another session's fix.
This commit is contained in:
jschoubben
2026-11-12 23:02:13 +01:00
parent 0ea81e0e96
commit f26c8afc20
3 changed files with 68 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files
00-META/glossary.md
+1 -1
View File
@@ -1023,7 +1023,7 @@ becomes a message). **Decided by** ADR 0227, 0231, 0240, 0252, 0283, 0564, 0668.
- **ZFS snapshot** — a read-only point in time of one ZFS dataset on a machine, kept inside that dataset's pool, so - **ZFS snapshot** — a read-only point in time of one ZFS dataset on a machine, kept inside that dataset's pool, so
it guards against a mistake and never against the pool's loss; taken by a person with `zfs_snapshot`, or by the zfs it guards against a mistake and never against the pool's loss; taken by a person with `zfs_snapshot`, or by the zfs
module's scheduled run, which names its own `mesh-<kind>-<time>`, marks them and removes only those past their keep module's scheduled run, which names its own `mesh-<kind>-<time>`, marks them and removes only those past their keep
count (ADR 0679 §7; design 42 zfs-owns-zfs §19, §20). Never bare, beside a ledger's snapshot and the stream snapshot count (ADR 0679 §7; design 42 zfs-owns-zfs §18, §19). Never bare, beside a ledger's snapshot and the stream snapshot
(see [Homonyms](#homonyms)); not a backup or a restore point, which are kept elsewhere. (see [Homonyms](#homonyms)); not a backup or a restore point, which are kept elsewhere.
*Not:* ~~pool snapshot~~, ~~dataset snapshot~~ *Not:* ~~pool snapshot~~, ~~dataset snapshot~~
- **disk readings** — the size of each data item a node's modules declare, measured by its `measure` and kept - **disk readings** — the size of each data item a node's modules declare, measured by its `measure` and kept
03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md
+66 -68
View File
@@ -27,29 +27,21 @@ main file's, [*The zfs module*](../42-the-machines-modules-in-order.md#the-zfs-m
the milestone builds, cited as "design 42 zfs-owns-zfs §n". the milestone builds, cited as "design 42 zfs-owns-zfs §n".
**Where.** Built under issue 1979. Three repositories, one delivery group with one check: `mesh-host` (the **Where.** Built under issue 1979. Three repositories, one delivery group with one check: `mesh-host` (the
node-engine: §2, §3, §4, its three subtasks of issue 1979), mesh-catalog's checks (§5) and mesh-catalog's node-engine: §1, §2, §3, its three subtasks of issue 1979), mesh-catalog's checks (§4) and mesh-catalog's
`modules/zfs` (§6 to §27). **Ordering edges:** the zfs module's new version needs the node-engine of §2, §3 and §4 `modules/zfs` (§5 to §26). **Ordering edge:** the zfs module's new version needs the node-engine of §1, §2 and §3
on its machine (a node-engine capability edge the delivery plan derives, §6), and its scheduled run needs the on its machine (a node-engine capability edge the delivery plan derives, §5); §4 has no edge, it gates the catalog's
node-engine's cron-to-calendar fix of §1, built outside this milestone; §5 has no edge, it gates the catalog's own own check. Nothing waits on the node-engine's cron-to-calendar fix: the scheduled run's schedule is written in a form
check. today's node-engine already turns into a valid calendar (§18).
## The prerequisites ## The prerequisites
1. **The scheduled run waits on the node-engine's valid calendar for a cron step** (ADR 0679 §14). The node-engine 1. **The node-engine's apply of a module stops at its first failed resource.** When one resource of a module fails,
writes a schedule's field `*/N` as `0/N` for the minute and the hour and as `1/N` for the day of the month and the
month; today `*/15 * * * *` becomes `*-*-* *:*/15:00`, which the service manager refuses, so the zfs module's scrub
timer fails. That fix is not built in this milestone: the operator gave it to another session, which tracks it in
an issue of its own. It is an ordering edge: the module's new version, whose scheduled run (§19) keeps its
`*/15 * * * *` schedule, is delivered to the home server only once a node-engine with that fix runs there.
Checked by the delivery plan naming the edge, and by `<node>/node-service-manager.status` of the module's timer
showing it active (acceptance test 3).
2. **The node-engine's apply of a module stops at its first failed resource.** When one resource of a module fails,
the node-engine applies none of that module's later resources in the declared order; the report names the the node-engine applies none of that module's later resources in the declared order; the report names the
failed resource with its error and each one left unapplied, and the module's phase is `failed`. Resources already failed resource with its error and each one left unapplied, and the module's phase is `failed`. Resources already
applied stay as applied. Other modules on the node go on as today. Today the apply goes on, and stopped the applied stay as applied. Other modules on the node go on as today. Today the apply goes on, and stopped the
distribution's scrub timer after the module's own timer failed (ADR 0679 §14). Checked by a unit test: a module of distribution's scrub timer after the module's own timer failed (ADR 0679 §14). Checked by a unit test: a module of
three resources whose second fails leaves the third untouched and reports it as left. three resources whose second fails leaves the third untouched and reports it as left.
3. **The node-engine declares a resource only while a config value holds** (lead's answer, 2026-11-12, to this 2. **The node-engine declares a resource only while a config value holds** (lead's answer, 2026-11-12, to this
part's design question: build it in the node-engine, design-it-as-it-should-be; ADR 0679 §3, §8). A resource of part's design question: build it in the node-engine, design-it-as-it-should-be; ADR 0679 §3, §8). A resource of
any type may carry the manifest field `if-config`: a list of conditions, each `<key>` (the module's config value is any type may carry the manifest field `if-config`: a list of conditions, each `<key>` (the module's config value is
set and is neither `false`, `off`, `none` nor an empty list) or `<key>=<value>` (the value equals it; for a list set and is neither `false`, `off`, `none` nor an empty list) or `<key>=<value>` (the value equals it; for a list
@@ -59,46 +51,46 @@ check.
refuses an `if-config` key the module does not declare under `config`. Checked by the node-engine's table test (a refuses an `if-config` key the module does not declare under `config`. Checked by the node-engine's table test (a
value unset, `off`, set; a list with and without the entry; a value going from set to unset removes the resource value unset, `off`, set; a list with and without the entry; a value going from set to unset removes the resource
and puts the original back) and the catalog check's own test. and puts the original back) and the catalog check's own test.
4. **A template writes one line per entry** (lead's answer, 2026-11-12, as §3; ADR 0679 §4). A template file may 3. **A template writes one line per entry** (lead's answer, 2026-11-12, as §2; ADR 0679 §4). A template file may
hold `${config-lines:<key>:<prefix>}`: for a list value one line per entry, `<prefix><entry>`, in the list's order; hold `${config-lines:<key>:<prefix>}`: for a list value one line per entry, `<prefix><entry>`, in the list's order;
for a set scalar value one line `<prefix><value>`; nothing for an unset value or an empty list. With a fourth part, for a set scalar value one line `<prefix><value>`; nothing for an unset value or an empty list. With a fourth part,
`${config-lines:<key>:<prefix>:mib-bytes}`, a number given in MiB is written in bytes. The prefix is literal text `${config-lines:<key>:<prefix>:mib-bytes}`, a number given in MiB is written in bytes. The prefix is literal text
with no `}`, `$` or newline, refused by the catalog check; an entry holding a newline is refused before the file is with no `}`, `$` or newline, refused by the catalog check; an entry holding a newline is refused before the file is
written, and the file is left as it was. Checked by the node-engine's table test (a list of two, an empty list, an written, and the file is left as it was. Checked by the node-engine's table test (a list of two, an empty list, an
unset and a set scalar, `mib-bytes`, a newline in an entry) and the catalog check's own test. unset and a set scalar, `mib-bytes`, a newline in an entry) and the catalog check's own test.
5. **The catalog refuses a second writer of one file.** mesh-catalog's checks refuse a catalog in which two 4. **The catalog refuses a second writer of one file.** mesh-catalog's checks refuse a catalog in which two
modules' manifests declare a `file` resource at one path, naming both modules and the path (ADR 0679 §14; ADR 0051 modules' manifests declare a `file` resource at one path, naming both modules and the path (ADR 0679 §14; ADR 0051
§9, rival ownership). A file only read is not a writer. Checked by the check's own test over two manifests that §9, rival ownership). A file only read is not a writer. Checked by the check's own test over two manifests that
declare one path (refused) and two that declare different paths (passed); it runs in the catalog's check on declare one path (refused) and two that declare different paths (passed); it runs in the catalog's check on
every pull request. every pull request.
6. **The order on the machine.** The zfs module's new version declares its own timer resource (the scheduled run of 5. **The order on the machine.** The zfs module's new version declares its own timer resource (the scheduled run of
§19) before the resources that stop the distribution's scrub timers, so by §2 those are stopped only once the §18) before the resources that stop the distribution's scrub timers, so by §1 those are stopped only once the
module's own timer is active (ADR 0679 §14). The delivery plan places the node-engine of §2, §3 and §4 on the home server before the module's own timer is active (ADR 0679 §14). The delivery plan places the node-engine of §1, §2 and §3 on the home server before the
module (the capability edge above), and §1's edge holds the module until the calendar fix runs there. Checked by the module's manifest test, which pins the module (the capability edge above). Checked by the module's manifest test, which pins the resource order, and by
resource order, and by acceptance test 3 of the main file. acceptance test 3 of the main file.
## What the module owns ## What the module owns
7. **The packages.** The manifest declares `zfs-utils` and `zfs-dkms` present (ADR 0679 §1; ADR 0303 §2: the 6. **The packages.** The manifest declares `zfs-utils` and `zfs-dkms` present (ADR 0679 §1; ADR 0303 §2: the
node-engine installs a missing one, upgrades none, removes none). `zfs-utils-debug` is not declared and is left as node-engine installs a missing one, upgrades none, removes none). `zfs-utils-debug` is not declared and is left as
found. Fate: the README's "It installs nothing" goes; it says the two packages and why the debug package is not found. Fate: the README's "It installs nothing" goes; it says the two packages and why the debug package is not
the module's. Read by `pacman_packages` as declared by `zfs`. Checked by the manifest test. the module's. Read by `pacman_packages` as declared by `zfs`. Checked by the manifest test.
8. **One pool, renamed `pool`.** The config value `scrub-pool` becomes `pool` (ADR 0679 §1): type string, trusted, 7. **One pool, renamed `pool`.** The config value `scrub-pool` becomes `pool` (ADR 0679 §1): type string, trusted,
required, same description but for the rename. It is moved by mesh-config's rename window (mesh-config answers both required, same description but for the rename. It is moved by mesh-config's rename window (mesh-config answers both
names meanwhile; its seat verb `renames` says the window), and every template and the module's code read `pool`. names meanwhile; its seat verb `renames` says the window), and every template and the module's code read `pool`.
One pool per assignment; the manifest has no per-pool form and none is added. The module reads, never writes, One pool per assignment; the manifest has no per-pool form and none is added. The module reads, never writes,
`/etc/zfs/zpool.cache` and `/etc/zfs/zfs-list.cache`. Checked by the manifest test (no `scrub-pool` left) and the `/etc/zfs/zpool.cache` and `/etc/zfs/zfs-list.cache`. Checked by the manifest test (no `scrub-pool` left) and the
rename's live read in `zfs_config`. rename's live read in `zfs_config`.
9. **What is not its own.** `sharenfs` and `sharesmb` are on no list of §15; a setting naming them is refused 8. **What is not its own.** `sharenfs` and `sharesmb` are on no list of §14; a setting naming them is refused
(§10), and `zfs_config` never reports either as drift (ADR 0679 §2). The pool's files are the operator's (ADR (§9), and `zfs_config` never reports either as drift (ADR 0679 §2). The pool's files are the operator's (ADR
0051). Checked by §26's refused-forms test. 0051). Checked by §25's refused-forms test.
## The config values and their forms ## The config values and their forms
10. **Every new value is "not managed" until set, and has a fixed form** (ADR 0679 §3, §4). Each value below has no 9. **Every new value is "not managed" until set, and has a fixed form** (ADR 0679 §3, §4). Each value below has no
default in the manifest; absent means not managed: the module writes nothing, converges nothing and reports no default in the manifest; absent means not managed: the module writes nothing, converges nothing and reports no
drift for it. A value of another form is refused by the module's own parse, before any file is written or command drift for it. A value of another form is refused by the module's own parse, before any file is written or command
run, emitting `config.refused` (§22). Trusted values (every list and object below, and `hostid`) are set on the run, emitting `config.refused` (§21). Trusted values (every list and object below, and `hostid`) are set on the
operator's Approve (ADR 0348 §4, §8). The module never reads a value from the machine into config. operator's Approve (ADR 0348 §4, §8). The module never reads a value from the machine into config.
- `arc-max`, `arc-min` (number): whole MiB, `arc-min` ≤ `arc-max`, `arc-max` at least 64 and below the machine's - `arc-max`, `arc-min` (number): whole MiB, `arc-min` ≤ `arc-max`, `arc-max` at least 64 and below the machine's
memory read from `/proc/meminfo`. ZFS's read cache in memory (the ARC) is bounded by them. memory read from `/proc/meminfo`. ZFS's read cache in memory (the ARC) is bounded by them.
@@ -119,53 +111,53 @@ check.
- `pool-properties` (object, trusted): `autotrim`, `autoexpand`, `autoreplace` (`on`, `off`, `default`) and - `pool-properties` (object, trusted): `autotrim`, `autoexpand`, `autoreplace` (`on`, `off`, `default`) and
`failmode` (`wait`, `continue`, `panic`, `default`). `failmode` (`wait`, `continue`, `panic`, `default`).
- `dataset-properties` (object, trusted): a dataset of the pool (the pool's name or one beginning `<pool>/`) to - `dataset-properties` (object, trusted): a dataset of the pool (the pool's name or one beginning `<pool>/`) to
properties of §15's list with their forms. properties of §14's list with their forms.
- `snapshots` (object, trusted): `datasets` (list of datasets of the pool; absent means every filesystem of the - `snapshots` (object, trusted): `datasets` (list of datasets of the pool; absent means every filesystem of the
pool), and keep counts `hourly` (0 to 48), `daily` (0 to 62), `weekly` (0 to 104), `monthly` (0 to 120); absent pool), and keep counts `hourly` (0 to 48), `daily` (0 to 62), `weekly` (0 to 104), `monthly` (0 to 120); absent
or all zero means none. or all zero means none.
- `hostid` (string, trusted): `generate`, written once (§14). - `hostid` (string, trusted): `generate`, written once (§13).
Checked by §26's refused-forms tests, one table per value. Checked by §25's refused-forms tests, one table per value.
## Files, units and the event daemon ## Files, units and the event daemon
11. **The node-engine places the module's files** (ADR 0679 §1, §4, §8), by §3 and §4. 10. **The node-engine places the module's files** (ADR 0679 §1, §4, §8), by §2 and §3.
`/etc/modprobe.d/zfs.conf` carries `if-config: [arc-max, arc-min, module-options]` and holds, by §4, a header `/etc/modprobe.d/zfs.conf` carries `if-config: [arc-max, arc-min, module-options]` and holds, by §3, a header
comment, then `options zfs zfs_arc_max=` and `options zfs zfs_arc_min=` with `arc-max` and `arc-min` in bytes comment, then `options zfs zfs_arc_max=` and `options zfs zfs_arc_min=` with `arc-max` and `arc-min` in bytes
(`mib-bytes`), then `options zfs ` with each `module-options` entry; with none of the three set the file is not (`mib-bytes`), then `options zfs ` with each `module-options` entry; with none of the three set the file is not
placed. The module's own `zed.rc` is placed in its folder (§13), not over the package's: it holds the package's placed. The module's own `zed.rc` is placed in its folder (§12), not over the package's: it holds the package's
variables but every `ZED_NOTIFY_*` and mail line, and `ZED_SCRUB_AFTER_RESILVER=1`, which only the handler variables but every `ZED_NOTIFY_*` and mail line, and `ZED_SCRUB_AFTER_RESILVER=1`, which only the handler
`resilver_finish-start-scrub` reads, so it acts only where that handler is placed. The package's `resilver_finish-start-scrub` reads, so it acts only where that handler is placed. The package's
`/etc/zfs/zed.d/zed.rc` and `/etc/default/zfs` are left as the package wrote them: no value of ADR 0679 changes `/etc/zfs/zed.d/zed.rc` and `/etc/default/zfs` are left as the package wrote them: no value of ADR 0679 changes
either, so writing them would only rewrite what ADR 0679 §3 leaves alone; `zfs_config` shows both as the either, so writing them would only rewrite what ADR 0679 §3 leaves alone; `zfs_config` shows both as the
package's, and a `.pacnew` beside either as a warning. Every line is from a parsed value, never from free text, and package's, and a `.pacnew` beside either as a warning. Every line is from a parsed value, never from free text, and
a test refuses a newline, a quote, `$`, a backtick and a semicolon in every field (§26). Checked by the manifest a test refuses a newline, a quote, `$`, a backtick and a semicolon in every field (§25). Checked by the manifest
test and §26. test and §25.
12. **The units.** The node-engine declares `zfs-import-cache.service`, `zfs-mount.service`, `zfs.target` and 11. **The units.** The node-engine declares `zfs-import-cache.service`, `zfs-mount.service`, `zfs.target` and
`zfs-import.target` running and enabled at boot, as they are on the home server today, so declaring them changes `zfs-import.target` running and enabled at boot, as they are on the home server today, so declaring them changes
nothing; `zfs-import-scan.service`, `zfs-share.service` and `zfs-volume-wait.service` are declared with no state nothing; `zfs-import-scan.service`, `zfs-share.service` and `zfs-volume-wait.service` are declared with no state
(ADR 0117), owned and left as found. `zfs-zed.service` is declared twice by §3: running and enabled with (ADR 0117), owned and left as found. `zfs-zed.service` is declared twice by §2: running and enabled with
`if-config: [zed=on]`, stopped and disabled with `if-config: [zed=off]`, and so not declared, and left as found, `if-config: [zed=on]`, stopped and disabled with `if-config: [zed=off]`, and so not declared, and left as found,
while `zed` is not managed. The distribution's `zfs-trim-weekly@<pool>.timer` and `zfs-trim-monthly@<pool>.timer` while `zed` is not managed. The distribution's `zfs-trim-weekly@<pool>.timer` and `zfs-trim-monthly@<pool>.timer`
are stopped and disabled with `if-config: [trim-schedule]` (the module's run trims, §19). The distribution's scrub are stopped and disabled with `if-config: [trim-schedule]` (the module's run trims, §18). The distribution's scrub
timers are stopped and disabled after the module's own (§6). Read by `<node>/node-service-manager.status`. timers are stopped and disabled after the module's own (§5). Read by `<node>/node-service-manager.status`.
Checked by the manifest test (each unit, its state and its condition). Checked by the manifest test (each unit, its state and its condition).
13. **ZED's handlers without links** (ADR 0679 §11). The module's folder `/etc/zfs/mesh-zed.d` (root's, 0755), its 12. **ZED's handlers without links** (ADR 0679 §11). The module's folder `/etc/zfs/mesh-zed.d` (root's, 0755), its
`zed.rc` (§11) and a drop-in for `zfs-zed.service` whose `ExecStart` adds `-d /etc/zfs/mesh-zed.d` carry `zed.rc` (§10) and a drop-in for `zfs-zed.service` whose `ExecStart` adds `-d /etc/zfs/mesh-zed.d` carry
`if-config: [zed=on]`. Each handler is a file of its own in that folder, a copy of the package's script by its `if-config: [zed=on]`. Each handler is a file of its own in that folder, a copy of the package's script by its
content kept in the module's files archive, mode 0755, with `if-config: [zed-handlers=<its name>]`, so only the content kept in the module's files archive, mode 0755, with `if-config: [zed-handlers=<its name>]`, so only the
handlers set are placed. The package's links in `/etc/zfs/zed.d` stay as the package left them and are not read handlers set are placed. The package's links in `/etc/zfs/zed.d` stay as the package left them and are not read
while the drop-in holds. The mesh creates no symlinks. The node-engine restarts `zfs-zed.service` on a change of while the drop-in holds. The mesh creates no symlinks. The node-engine restarts `zfs-zed.service` on a change of
the drop-in, a handler or `zed.rc` (`restart-on`). Checked by the manifest test (no link resource, each handler's the drop-in, a handler or `zed.rc` (`restart-on`). Checked by the manifest test (no link resource, each handler's
condition) and a live `<node>/node-files.list` of the folder. condition) and a live `<node>/node-files.list` of the folder.
14. **The machine's ZFS id.** `/etc/hostid` is written only when `hostid` is `generate` and the file is absent, by the apply 13. **The machine's ZFS id.** `/etc/hostid` is written only when `hostid` is `generate` and the file is absent, by the apply
of §17 with `zgenhostid`; never rewritten. Before the Approve, the value's description and `zfs_config` say that of §16 with `zgenhostid`; never rewritten. Before the Approve, the value's description and `zfs_config` say that
the pool must then be imported once by hand after the next reboot to check it imports under the new id; the the pool must then be imported once by hand after the next reboot to check it imports under the new id; the
module never reboots. Emits `config.applied` with target `file`. Checked by a unit test (absent file: one module never reboots. Emits `config.applied` with target `file`. Checked by a unit test (absent file: one
`zgenhostid`; present file: none). `zgenhostid`; present file: none).
## The pool ## The pool
15. **Pool and dataset properties** (ADR 0679 §5). For each set pool property the apply runs 14. **Pool and dataset properties** (ADR 0679 §5). For each set pool property the apply runs
`zpool set <property>=<value> <pool>`, and for `default` `zpool set <property>=<ZFS's default> <pool>`. For each `zpool set <property>=<value> <pool>`, and for `default` `zpool set <property>=<ZFS's default> <pool>`. For each
set dataset property it runs `zfs set <property>=<value> <dataset>`, and for `default` set dataset property it runs `zfs set <property>=<value> <dataset>`, and for `default`
`zfs inherit <property> <dataset>`, so its source stays default. The dataset list is `compression` (`off`, `lz4`, `zfs inherit <property> <dataset>`, so its source stays default. The dataset list is `compression` (`off`, `lz4`,
@@ -173,8 +165,8 @@ check.
(a power of two from `512` to `16M`, with `K` or `M`), `primarycache` and `secondarycache` (`all`, `metadata`, (a power of two from `512` to `16M`, with `K` or `M`), `primarycache` and `secondarycache` (`all`, `metadata`,
`none`), `sync` (`standard`, `always`, `disabled`), `logbias` (`latency`, `throughput`), `quota` and `none`), `sync` (`standard`, `always`, `disabled`), `logbias` (`latency`, `throughput`), `quota` and
`reservation` (`none` or digits with `K`, `M`, `G` or `T`). A property already at its value runs nothing. Each `reservation` (`none` or digits with `K`, `M`, `G` or `T`). A property already at its value runs nothing. Each
change is read back with `zpool get -Hp` or `zfs get -Hp` and emitted (§22). Checked by §26's convergence tests. change is read back with `zpool get -Hp` or `zfs get -Hp` and emitted (§21). Checked by §25's convergence tests.
16. **Cache and log devices** (ADR 0679 §5). From `zpool status -P -L <pool>` the apply knows each device's kernel 15. **Cache and log devices** (ADR 0679 §5). From `zpool status -P -L <pool>` the apply knows each device's kernel
path, and from the link under `/dev/disk/by-id` each declared name's kernel path. Then, in this order: path, and from the link under `/dev/disk/by-id` each declared name's kernel path. Then, in this order:
- an entry of the pool whose kernel path is a declared device's, but named by its kernel name, is removed - an entry of the pool whose kernel path is a declared device's, but named by its kernel name, is removed
(`zpool remove <pool> <entry>`) and added again by its stable name; (`zpool remove <pool> <entry>`) and added again by its stable name;
@@ -186,29 +178,35 @@ check.
- a log device not declared, or one to rename, is removed only while the pool is ONLINE with no read, write or - a log device not declared, or one to rename, is removed only while the pool is ONLINE with no read, write or
checksum errors and no scrub or resilver runs; otherwise it is a wait, `config.waiting` {key, waits-on, since}. checksum errors and no scrub or resilver runs; otherwise it is a wait, `config.waiting` {key, waits-on, since}.
A declared device that is not present is a wait. Each change is read back from `zpool status -P` and emitted. A declared device that is not present is a wait. Each change is read back from `zpool status -P` and emitted.
Checked by §26's convergence tests and acceptance test 2. Checked by §25's convergence tests and acceptance test 2.
17. **Who applies, and when** (ADR 0679 §8). A `process` resource `zfs-config-apply`, `run: ./zfs-tools apply`, as 16. **Who applies, and when** (ADR 0679 §8). A `process` resource `zfs-config-apply`, `run: ./zfs-tools apply`, as
root, oneshot, ordered after `zfs-import.target` and enabled at boot; restarted by the node-engine on a change of root, oneshot, ordered after `zfs-import.target` and enabled at boot; restarted by the node-engine on a change of
the module's config file (`restart-on`), which is how a set value reaches it from mesh-config (ADR 0348 §5, §6); the module's config file (`restart-on`), which is how a set value reaches it from mesh-config (ADR 0348 §5, §6);
started by hand through `zfs_config_apply` (§23). It holds an exclusive lock on `/run/zfs-tools/apply.lock`; a started by hand through `zfs_config_apply` (§22). It holds an exclusive lock on `/run/zfs-tools/apply.lock`; a
second start waits for it. It runs nothing while a scrub or resilver runs on the pool or before the pool is second start waits for it. It runs nothing while a scrub or resilver runs on the pool or before the pool is
imported: it emits `config.waiting` and the next start applies. It never runs on a timer. While the bus or the imported: it emits `config.waiting` and the next start applies. It never runs on a timer. While the bus or the
control-node is down it applies the values its config file last held. It writes `arc-max` and `arc-min` live to control-node is down it applies the values its config file last held. It writes `arc-max` and `arc-min` live to
`/sys/module/zfs/parameters/zfs_arc_max` and `zfs_arc_min` as well as to §11's file and reads them back; each `/sys/module/zfs/parameters/zfs_arc_max` and `zfs_arc_min` as well as to §10's file and reads them back; each
`module-options` entry whose live value differs is a warning naming it, "takes effect at the next module load", `module-options` entry whose live value differs is a warning naming it, "takes effect at the next module load",
never success and never unhealthy. never success and never unhealthy.
18. **Never converged, always asked** (ADR 0679 §6). No setting adds, replaces, attaches, detaches or removes a data 17. **Never converged, always asked** (ADR 0679 §6). No setting adds, replaces, attaches, detaches or removes a data
device, a spare, a special or a dedup device; none upgrades the pool, destroys a dataset, rolls back, renames or device, a spare, a special or a dedup device; none upgrades the pool, destroys a dataset, rolls back, renames or
clears. The call-site test (`TestTheBundleRunsOnlyReadingVerbsSnapshotAndScrub`, renamed for what it now holds) clears. The call-site test (`TestTheBundleRunsOnlyReadingVerbsSnapshotAndScrub`, renamed for what it now holds)
allows `zpool add` only with `cache` or `log`, `zpool remove` only of a cache or log entry, `zpool set`, `zfs set` allows `zpool add` only with `cache` or `log`, `zpool remove` only of a cache or log entry, `zpool set`, `zfs set`
and `zfs inherit` only for the lists of §10 and §15, `zfs destroy` only as §20 builds it, `zpool trim` only from and `zfs inherit` only for the lists of §9 and §14, `zfs destroy` only as §19 builds it, `zpool trim` only from
the scheduled run, and refuses `upgrade`, `replace`, `attach`, `detach`, `clear`, `rollback`, `rename`, `-f`, `-r`, the scheduled run, and refuses `upgrade`, `replace`, `attach`, `detach`, `clear`, `rollback`, `rename`, `-f`, `-r`,
`-R` and `-d` everywhere. `-R` and `-d` everywhere.
## ZFS snapshots on a schedule ## ZFS snapshots on a schedule
19. **The scheduled run** (ADR 0679 §7). The existing `process` resource `zfs-mesh-scrub` (`./zfs-tools scrub-run`, 18. **The scheduled run** (ADR 0679 §7, §14). The existing `process` resource `zfs-mesh-scrub` (`./zfs-tools
every 15 minutes) becomes `zfs-mesh-run` (`./zfs-tools run`), root as today, and does three things each time, in scrub-run`) becomes `zfs-mesh-run` (`./zfs-tools run`), root as today, and its `schedule` changes from
`*/15 * * * *` to the list form `0,15,30,45 * * * *`. Reason: today's node-engine writes a step `*/15` as
`*:*/15`, which the service manager refuses, so the timer failed; it writes the list form as
`*-*-* *:0,15,30,45:00`, which is valid, so the module's timer is active without the cron-to-calendar fix (lead's
answer, 2026-11-12). Checked by the module's manifest test (the schedule string) and by `systemd-analyze calendar`
of the generated timer's `OnCalendar` on the home server passing with a next run within 15 minutes (acceptance
test 3). It and does three things each time, in
order: the scrub when due (as today), a trim when `trim-schedule` is due (`zpool trim <pool>`, by `trim-day` and order: the scrub when due (as today), a trim when `trim-schedule` is due (`zpool trim <pool>`, by `trim-day` and
`trim-time` as the scrub reads its own), and the ZFS snapshots when `snapshots` is set. Reason: one schedule `trim-time` as the scrub reads its own), and the ZFS snapshots when `snapshots` is set. Reason: one schedule
resource the node-engine already runs, and no second timer. For the snapshots: at the first run of each hour it resource the node-engine already runs, and no second timer. For the snapshots: at the first run of each hour it
@@ -217,19 +215,19 @@ check.
per dataset, never recursive, and sets on each `zfs set mesh:schedule=<kind>`. No ZFS snapshot is taken while the per dataset, never recursive, and sets on each `zfs set mesh:schedule=<kind>`. No ZFS snapshot is taken while the
pool's capacity (`zpool list -Hp -o capacity`) is above 80 %: a warning says so, and `snapshot.skipped` pool's capacity (`zpool list -Hp -o capacity`) is above 80 %: a warning says so, and `snapshot.skipped`
{pool, capacity} is emitted. {pool, capacity} is emitted.
20. **Removed only by their own record.** After taking, per dataset and kind, the run removes the oldest past its keep 19. **Removed only by their own record.** After taking, per dataset and kind, the run removes the oldest past its keep
count, only when it carries `mesh:schedule=<kind>`, is in the ledger `snapshots` (§22), has no hold count, only when it carries `mesh:schedule=<kind>`, is in the ledger `snapshots` (§21), has no hold
(`zfs holds`) and no clone (`clones` empty): `zfs destroy -- <dataset>@<snapshot>`, never `-r`, `-R` or `-d`. A (`zfs holds`) and no clone (`clones` empty): `zfs destroy -- <dataset>@<snapshot>`, never `-r`, `-R` or `-d`. A
ZFS snapshot taken by hand is never removed. Each removal emits `snapshot.removed` {snapshot, kind, space freed} ZFS snapshot taken by hand is never removed. Each removal emits `snapshot.removed` {snapshot, kind, space freed}
and records the ledger's `removed` operation. and records the ledger's `removed` operation.
21. **A person's ZFS snapshot stays apart.** `zfs_snapshot` refuses a name beginning `mesh-`, and emits 20. **A person's ZFS snapshot stays apart.** `zfs_snapshot` refuses a name beginning `mesh-`, and emits
`snapshot.taken` with `by: person`; the run emits it with `by: schedule`. A waiter on a person's ZFS snapshot waits on `snapshot.taken` with `by: person`; the run emits it with `by: schedule`. A waiter on a person's ZFS snapshot waits on
`snapshot.taken` with that snapshot's name, which no scheduled one carries. Fate: the README's "No tool of the `snapshot.taken` with that snapshot's name, which no scheduled one carries. Fate: the README's "No tool of the
mesh destroys a snapshot" becomes §20's rule. mesh destroys a snapshot" becomes §19's rule.
## What it says, keeps and shows ## What it says, keeps and shows
22. **Events and kept data** (ADR 0679 §9; ADR 0363 §1; ADR 0595). The module declares a stream of its own, local 21. **Events and kept data** (ADR 0679 §9; ADR 0363 §1; ADR 0595). The module declares a stream of its own, local
name `zfs`, kept 30 days, carrying `config.applied` {target (`pool`, `dataset`, `file` or `unit`), key, before, name `zfs`, kept 30 days, carrying `config.applied` {target (`pool`, `dataset`, `file` or `unit`), key, before,
after, read-back}, `config.refused` {key, value, reason}, `config.waiting` {key, waits-on, since}, after, read-back}, `config.refused` {key, value, reason}, `config.waiting` {key, waits-on, since},
`snapshot.taken` {snapshot, by}, `snapshot.removed`, `snapshot.skipped`, and today's `scrub.requested`; today's `snapshot.taken` {snapshot, by}, `snapshot.removed`, `snapshot.skipped`, and today's `scrub.requested`; today's
@@ -238,15 +236,15 @@ check.
managed (after its last `config.applied`); a ledger `snapshots`, one instance per ZFS snapshot the run took, managed (after its last `config.applied`); a ledger `snapshots`, one instance per ZFS snapshot the run took,
operations `taken` and `removed`, kept by the ledger's template. The apply process and the run declare the bus operations `taken` and `removed`, kept by the ledger's template. The apply process and the run declare the bus
account (`bus`) that writes them; the tools read them. account (`bus`) that writes them; the tools read them.
23. **Tools** (ADR 0679 §12). `zfs_config`: every owned value with its setting, what runs on the machine, its state 22. **Tools** (ADR 0679 §12). `zfs_config`: every owned value with its setting, what runs on the machine, its state
(`not managed`, `as set`, `waiting` with on what and since, `refused` with why, `takes effect at the next module (`not managed`, `as set`, `waiting` with on what and since, `refused` with why, `takes effect at the next module
load`), and a `.pacnew` beside an owned file as a warning. `zfs_config_apply`: starts §17's unit through load`), and a `.pacnew` beside an owned file as a warning. `zfs_config_apply`: starts §16's unit through
`<node>/node-service-manager.start`, with `why`, and answers from the state `applied` afterwards. `<node>/node-service-manager.start`, with `why`, and answers from the state `applied` afterwards.
`zfs_snapshots_schedule`: the schedule, its next run, what it would remove next and the ledger. `zfs_snapshots_schedule`: the schedule, its next run, what it would remove next and the ledger.
`zfs_snapshots_run_now` and `zfs_snapshots_prune_now`: a run's snapshot or removal step by hand, with `why`. `zfs_snapshots_run_now` and `zfs_snapshots_prune_now`: a run's snapshot or removal step by hand, with `why`.
`zfs_snapshots_pause` and `zfs_snapshots_resume`: a key `paused` in the state `applied`, read by the run, said in `zfs_snapshots_pause` and `zfs_snapshots_resume`: a key `paused` in the state `applied`, read by the run, said in
the health while it holds. The existing read tools stay. Each is in `replaces` where it replaces a command. the health while it holds. The existing read tools stay. Each is in `replaces` where it replaces a command.
24. **Health** (ADR 0679 §10). `zfs_health` is unhealthy for a `refused` or a failed command in the state `applied`, 23. **Health** (ADR 0679 §10). `zfs_health` is unhealthy for a `refused` or a failed command in the state `applied`,
naming the key and why; warnings: a wait (naming on what and since when), the last scheduled ZFS snapshot older naming the key and why; warnings: a wait (naming on what and since when), the last scheduled ZFS snapshot older
than its cadence, a failed run, `autoreplace` on or a spare in the pool while `zed` is not `on`, the pool above than its cadence, a failed run, `autoreplace` on or a spare in the pool while `zed` is not `on`, the pool above
80 % with snapshots set, a parameter awaiting a module load. Pool faults reach the operator once, never from a ZED 80 % with snapshots set, a parameter awaiting a module load. Pool faults reach the operator once, never from a ZED
@@ -254,7 +252,7 @@ check.
bucket `findings` for a pool or device that leaves ONLINE, saying each as an event, and removing it when the state bucket `findings` for a pool or device that leaves ONLINE, saying each as an event, and removing it when the state
is ONLINE again; the condition keeper raises and clears the condition from it (ADR 0564 §2; design 47 G §383). Its is ONLINE again; the condition keeper raises and clears the condition from it (ADR 0564 §2; design 47 G §383). Its
process declares the bus account that writes them; today it writes the journal only. process declares the bus account that writes them; today it writes the journal only.
25. **When the module leaves** (ADR 0679 §13; ADR 0303 §2; ADR 0269). The packages stay. The import, mount and target 24. **When the module leaves** (ADR 0679 §13; ADR 0303 §2; ADR 0269). The packages stay. The import, mount and target
units stay enabled and running. `/etc/default/zfs` and `zed.rc` get back their found content (ADR 0118); units stay enabled and running. `/etc/default/zfs` and `zed.rc` get back their found content (ADR 0118);
`zfs.conf`, the folder `/etc/zfs/mesh-zed.d` and the drop-in are removed, and before leaving `zfs_config` says that `zfs.conf`, the folder `/etc/zfs/mesh-zed.d` and the drop-in are removed, and before leaving `zfs_config` says that
the next module load uses ZFS's defaults. The distribution's scrub timer is given back as found (ADR 0269). Pool the next module load uses ZFS's defaults. The distribution's scrub timer is given back as found (ADR 0269). Pool
@@ -262,13 +260,13 @@ check.
## How it is checked ## How it is checked
26. **Unit tests**, table-driven, one test per area, within the module's 500: no settings gives no `zpool` or `zfs` 25. **Unit tests**, table-driven, one test per area, within the module's 500: no settings gives no `zpool` or `zfs`
command that writes; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel name command that writes; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel name
to its stable name, a removal, a log removal refused while not healthy, a wait during a scrub); every refused form to its stable name, a removal, a log removal refused while not healthy, a wait during a scrub); every refused form
(a newline, quote, `$`, backtick or semicolon in each field; a parameter not on the list; a property not on the (a newline, quote, `$`, backtick or semicolon in each field; a parameter not on the list; a property not on the
lists; a `sharenfs` setting; `-f`; a device with a signature or another pool's label); the call-site test of §18; lists; a `sharenfs` setting; `-f`; a device with a signature or another pool's label); the call-site test of §17;
pruning never removes a `mesh-` ZFS snapshot without the property or not in the ledger, nor one with a hold or pruning never removes a `mesh-` ZFS snapshot without the property or not in the ledger, nor one with a hold or
clone; `zfs_snapshot` refuses a `mesh-` name; a parameter awaiting a module load is a warning; each change emits clone; `zfs_snapshot` refuses a `mesh-` name; a parameter awaiting a module load is a warning; each change emits
its event with before and after; a scheduled `snapshot.taken` does not match a person's waiter. The module has no its event with before and after; a scheduled `snapshot.taken` does not match a person's waiter. The module has no
integration test; the live checks stand for it, said in the pull request. integration test; the live checks stand for it, said in the pull request.
27. **Live, on the home server**: the main file's acceptance tests 1 to 3, in that order. 26. **Live, on the home server**: the main file's acceptance tests 1 to 3, in that order.
03-DESIGN/01-to-be/42-the-machines-modules-in-order.md
+1 -1
View File
@@ -212,7 +212,7 @@ flowchart LR
drives' kernel names, both stay ONLINE (`zfs_pool_layout`). drives' kernel names, both stay ONLINE (`zfs_pool_layout`).
3. The module's own scrub timer is active before the distribution's scrub timer is stopped, read from 3. The module's own scrub timer is active before the distribution's scrub timer is stopped, read from
`<node>/node-service-manager.status` of both. `<node>/node-service-manager.status` of both.
4. The unit tests of the milestone part's line 26 pass, within the module's budget of 500 unit tests. 4. The unit tests of the milestone part's line 25 pass, within the module's budget of 500 unit tests.
## Milestones ## Milestones