diff --git a/03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md b/03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md index 0a39f100..471c3d78 100644 --- a/03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md +++ b/03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md @@ -160,13 +160,20 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614 `/etc/zfs/zed.d` is not read while the drop-in holds. The mesh creates no symlinks. The node-engine restarts `zfs-zed.service` on a change of the drop-in, a handler or `zed.rc` (`restart-on`). Checked by the manifest test (no link resource, each handler's condition). -15. **The node's ZFS id** (ADR 0679 §1, §3, §4). `/etc/hostid` holds four bytes, which no template writes, so the apply - (§18) writes it with ZFS's own writer, `zgenhostid -f `, only while `hostid` is set, and only when the pool's - label (`zdb -C `, its `hostid`) is 0 or equals the value: then the next boot imports without `zpool import - -f`. Otherwise it writes nothing and emits `config.refused` naming why ("the pool's import would need -f"). Before - the Approve, the value's description and `zfs_config` say the change needs a check of the pool's import at the next - boot; the module never reboots. After the reboot, `zfs_pools` shows the pool imported. Emits `config.applied` with - target `file`. Checked by §31 (unset: no `zgenhostid`; a label of another id: refused; 0 or equal: one run). +15. **The node's ZFS id** (ADR 0679 §1, §3, §4, §17). `/etc/hostid` holds four bytes, binary, which no template writes, + so it is no resource of the manifest and the apply (§18) writes it with ZFS's own writer, `zgenhostid -f `, + only while `hostid` is set, and only when the pool's label (`zdb -C `, its `hostid`) is 0 or equals the value: + then the next boot imports without `zpool import -f`. When the label shows another value, or cannot be read, it + writes nothing and emits `config.refused` naming why ("the pool's import would need -f", "the pool's label cannot be + read"). Before the Approve, the value's description and `zfs_config` say the change needs a check of the pool's + import at the next boot; the module never reboots. After the reboot, `zfs_pools` shows the pool imported. Emits + `config.applied` with target `file`. The apply reads `/etc/hostid` back at every start and after every write: while + `hostid` is set, a value there other than the set one emits `config.refused` naming it, because the node-engine's + composer cannot see a second writer of a file that is no resource. Each refusal of this line makes `zfs_health` + unhealthy (§29), naming the key and why, until an apply finds the label and the file agreeing with `hostid`, or + `hostid` is unset. No resource or template of the manifest writes `/etc/hostid`. Checked by §31 (unset: no + `zgenhostid`; a label of another id, or one that cannot be read: refused; 0 or equal: one run; a file holding + another value: refused; unhealthy, then healthy again; no manifest resource for the file). ## The pool @@ -194,7 +201,7 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614 waits-on, since}. A declared device that is not present is a wait. Each change is read back from `zpool status -P` and emitted. Checked by §31's convergence tests (the empty list among them) and acceptance test 2. -18. **The apply: who, when, and what wakes a wait** (ADR 0679 §9). A `process` resource `zfs-config-apply`, `run: +18. **The apply: who, when, and what wakes a wait** (ADR 0679 §9, §17). A `process` resource `zfs-config-apply`, `run: ./zfs-tools apply`, as root, oneshot, ordered after `zfs-import.target`, enabled at boot, with `restart-on` the config file (§9); it converges §15 to §17 and writes the ARC values live. It holds an exclusive lock on `/run/zfs-tools/apply.lock`; a second start waits for it. It converges nothing while a scrub or resilver runs on @@ -207,7 +214,7 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614 to `/sys/module/zfs/parameters/zfs_arc_max` and `zfs_arc_min` as well as to §12's file, and reads both back; each `module-options` entry whose live value differs is a warning naming it, "takes effect at the next load of the kernel module", never success and never unhealthy. -19. **The call-site test** (ADR 0679 §7). Every command the bundle runs is built by one of its argument functions, and +19. **The call-site test** (ADR 0679 §7, §17). Every command the bundle runs is built by one of its argument functions, and the test (`TestTheBundleRunsOnlyReadingVerbsSnapshotAndScrub`, renamed `TestTheBundleRunsOnlyTheFormsOfADR0679`) allows only: `zpool add cache|log`, `zpool remove` of a cache or log device, `zpool set`, `zfs set` and `zfs inherit` of a listed property, `zfs snapshot`, `zfs set mesh:schedule=` on a ZFS snapshot named `mesh-`, @@ -217,7 +224,9 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614 data vdevs, spares, special and dedup devices, `upgrade`, a dataset's destroy, `rollback`, `rename` and `clear` are never run (each an act of its own, asked of the operator). The same test pins the module's other root forms: `wipefs --no-act `, `zdb -l `, `zdb -C `, `zgenhostid -f `, `udevadm monitor --udev - --subsystem-match=block` and `systemctl show`. Fate: the README's list of allowed forms is replaced by this one. + --subsystem-match=block` and `systemctl show`, and refuses every other `zgenhostid` or `zdb` form: `zgenhostid` + only as `-f` with the value §11 validated, `zdb` only as these two reads. Fate: the README's list of allowed forms + is replaced by this one. ## The scheduled run and ZFS snapshots @@ -302,22 +311,27 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614 `zfs.conf`, `/etc/zfs/mesh-zed.d` and the drop-in are removed, and before leaving `zfs_config` says that the next load of the kernel module uses ZFS's defaults. The distribution's scrub and trim timers are put back as the module's declaration found them, started and enabled again where it stopped or disabled them. Pool and dataset - properties and devices stay as the last apply set them, also on a rollback (§8). + properties and devices stay as the last apply set them, also on a rollback (§8). `/etc/hostid` stays, and the + module says so before it leaves (`zfs_config`): it is no resource the node-engine removes, and removing it would + change the node's hostid, so that the pool's next import could need `zpool import -f` (ADR 0679 §15). ## How it is checked -31. **Unit tests**, table-driven, one test per area, within the module's 500: no values gives no `zpool` or `zfs` - write but the due scrub; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel - name to its stable name, a removal, an empty list removing nothing, a log removal refused while not healthy, a - `log-mirror` change, a wait during a scrub); every refused form (a newline, quote, `$`, backtick or semicolon in - each field; a parameter or property not on its list; a `sharenfs` value; `-f`; a device with a signature or another - pool's label; a hostid of the wrong form or a label of another id); the call-site test of §19; the lock (a second - apply waits, a run with the lock held does nothing); the apply started only by `config-changed`, never a timer; - one scheduled resource; both ARC writes read back; applying from the last config file with the bus down; the - three templates (config file, options file, `zed.rc`); pruning never removes a ZFS snapshot whose name, property - or ledger entry does not fit, nor one with a hold or clone, and removes every surplus in one run; no ZFS snapshot - taken without a ledger record; `snapshots` back to `none` removes nothing; `zfs_snapshot` refuses a `mesh-` +31. **Unit tests**, table-driven, one test per area, within the module's 500: no values gives no `zpool` or `zfs` write + but the due scrub; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel name to + its stable name, a removal, an empty list removing nothing, a log removal refused while not healthy, a `log-mirror` + change, a wait during a scrub); every refused form (a newline, quote, `$`, backtick or semicolon in each field; a + parameter or property not on its list; a `sharenfs` value; `-f`; a device with a signature or another pool's label; + a hostid of the wrong form, a label of another id or one that cannot be read, and a hostid file holding another + value than the set one, each unhealthy and healthy again once the label and the file agree or `hostid` is unset); + the call-site test of §19, refusing any other `zgenhostid` or `zdb` form and a `zgenhostid` value not validated; the + lock (a second apply waits, a run with the lock held does nothing); the apply started only by `config-changed`, + never a timer; one scheduled resource; both ARC writes read back; applying from the last config file with the bus + down; the three templates (config file, options file, `zed.rc`); pruning never removes a ZFS snapshot whose name, + property or ledger entry does not fit, nor one with a hold or clone, and removes every surplus in one run; no ZFS + snapshot taken without a ledger record; `snapshots` back to `none` removes nothing; `zfs_snapshot` refuses a `mesh-` name; a scheduled `snapshot.taken.schedule` does not wake a person's waiter; a parameter awaiting a load is a - warning; the `.pacnew` warning; each change emits its event with before and after; one test per tool. The module - adds no integration test; §5 is the milestone's, and the main file's live tests stand for the module whole, said - in the pull request. + warning; the `.pacnew` warning; each change emits its event with before and after; one test per tool; no resource or + template of the manifest for `/etc/hostid`; the hostid converged at the apply's start after `zfs-import.target`; the + leave keeps `/etc/hostid` and says so. The module adds no integration test; §5 is the milestone's, and the main + file's live tests stand for the module whole, said in the pull request. diff --git a/03-DESIGN/01-to-be/42-the-machines-modules-in-order.md b/03-DESIGN/01-to-be/42-the-machines-modules-in-order.md index 5e832053..9aece7a6 100644 --- a/03-DESIGN/01-to-be/42-the-machines-modules-in-order.md +++ b/03-DESIGN/01-to-be/42-the-machines-modules-in-order.md @@ -170,7 +170,8 @@ named ZFS snapshot, a scrub's start, pause and stop. the states of ZFS's import, mount, share, volume and target units and of the distribution's scrub and trim timers; and, for the one pool its config value `pool` names, that pool's cache and log devices, its properties and its datasets' properties from fixed lists. The node-engine's composer refuses a second module owning one of its paths - (ADR 0593 §7, ADR 0051 §9). + (ADR 0593 §7, ADR 0051 §9), all but `/etc/hostid`, which is no resource: the module's apply reads it back instead + (ADR 0679 §1, §17). - **What it never writes** (ADR 0679 §1, §2): the package's own `/etc/default/zfs` and `/etc/zfs/zed.d` with its `zed.rc`, and what ZFS writes itself; it reads and shows them. A dataset's `sharenfs` is the seat `node-nfs-server`'s, `sharesmb` the SMB module's; the pool's files are the operator's (ADR 0051); the weekly @@ -178,10 +179,11 @@ named ZFS snapshot, a scrub's start, pause and stop. - **An unset or empty value changes nothing** (ADR 0679 §3): every new config value is "not managed" until the operator sets it through mesh-config, on the Approve for a trusted value (ADR 0348 §4, §8). Placing the new version on a node runs no `zpool` or `zfs` write but the scrub its schedule already runs, and places none of its new files. -- **Who applies** (ADR 0679 §9): the node-engine places the packages, files and units as the module's resources, - some only while a value holds; the module's root process `zfs-tools apply` converges the pool's devices and - properties at its start and on `config-changed`, never on a timer; the module's one scheduled run, under the same - lock, starts the scrub and the trim when due and takes and removes the scheduled ZFS snapshots (ADR 0679 §8). +- **Who applies** (ADR 0679 §9, §17): the node-engine places the packages, files and units as the module's resources, + some only while a value holds, all but `/etc/hostid`; the module's root process `zfs-tools apply` converges the pool's + devices and properties at its start and on `config-changed`, never on a timer, and writes `/etc/hostid` with ZFS's own + writer only while `hostid` is set and the pool's label agrees; the module's one scheduled run, under the same lock, + starts the scrub and the trim when due and takes and removes the scheduled ZFS snapshots (ADR 0679 §8). - **What it says** (ADR 0679 §10, §11, §12): every change, refusal and wait is an event, each kind on a subject of its own, on an events stream asked for by template; the last change per value, the schedule's pause and every scheduled ZFS snapshot are kept on the bus in its own states and ledger (ADR 0595). Its health names a refusal or a @@ -190,7 +192,8 @@ named ZFS snapshot, a scrub's start, pause and stop. - **What it never does** (ADR 0679 §7): it converges no data device, spare, special or dedup device, upgrades no pool, destroys no dataset, rolls back, renames and clears nothing; each is an act of its own, decided separately. - **When it leaves** (ADR 0679 §15): ZFS keeps running (the import, mount and ZFS target are never stopped), the - packages stay, its own files are removed and the distribution's timers are put back as found. + packages stay, its own files are removed but `/etc/hostid`, which stays and is said, and the distribution's timers + are put back as found. - **Its prerequisites** (ADR 0679 §16): three node-engine capabilities, built first in the same milestone under issue 1979, with their readers in mesh-sdk and the manifest validator. @@ -200,7 +203,7 @@ flowchart LR config -->|the module's config file| engine[the node-engine] engine -->|packages, files, units| node[ZFS on the node] engine -->|restarts on its config file| apply[zfs-tools apply, root] - apply -->|zpool add, remove, set; zfs set, inherit| node + apply -->|zpool add, remove, set; zfs set, inherit; zgenhostid| node sched[the scheduled run] -->|scrub, trim, ZFS snapshots and their removal| node follower[the event follower] -->|scrub ended, device appeared: start| apply apply -->|config.applied, refused, waiting| stream[(events stream)]