novox/hq
9
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Design 42 zfs-owns-zfs: the apply writes /etc/hostid and reads it back, as ADR 0679 version 17 says

Browse Source 
No template writes the four binary bytes of /etc/hostid, so it is no resource
and the composer cannot see a second writer: the apply guards it by the pool's
label, reads it back, is unhealthy on a mismatch, and keeps the file on leave.
This commit is contained in:
jschoubben
2026-11-12 23:59:15 +01:00
parent e97647cd18
commit 04de5c3b30
2 changed files with 49 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
03-DESIGN/01-to-be/42-milestones/zfs-owns-zfs.md
+39 -25
View File
@@ -160,13 +160,20 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614
`/etc/zfs/zed.d` is not read while the drop-in holds. The mesh creates no symlinks. The node-engine restarts
`zfs-zed.service` on a change of the drop-in, a handler or `zed.rc` (`restart-on`). Checked by the manifest test (no
link resource, each handler's condition).
15. **The node's ZFS id** (ADR 0679 §1, §3, §4). `/etc/hostid` holds four bytes, which no template writes, so the apply
(§18) writes it with ZFS's own writer, `zgenhostid -f <value>`, only while `hostid` is set, and only when the pool's
label (`zdb -C <pool>`, its `hostid`) is 0 or equals the value: then the next boot imports without `zpool import
-f`. Otherwise it writes nothing and emits `config.refused` naming why ("the pool's import would need -f"). Before
the Approve, the value's description and `zfs_config` say the change needs a check of the pool's import at the next
boot; the module never reboots. After the reboot, `zfs_pools` shows the pool imported. Emits `config.applied` with
target `file`. Checked by §31 (unset: no `zgenhostid`; a label of another id: refused; 0 or equal: one run).
15. **The node's ZFS id** (ADR 0679 §1, §3, §4, §17). `/etc/hostid` holds four bytes, binary, which no template writes,
so it is no resource of the manifest and the apply (§18) writes it with ZFS's own writer, `zgenhostid -f <value>`,
only while `hostid` is set, and only when the pool's label (`zdb -C <pool>`, its `hostid`) is 0 or equals the value:
then the next boot imports without `zpool import -f`. When the label shows another value, or cannot be read, it
writes nothing and emits `config.refused` naming why ("the pool's import would need -f", "the pool's label cannot be
read"). Before the Approve, the value's description and `zfs_config` say the change needs a check of the pool's
import at the next boot; the module never reboots. After the reboot, `zfs_pools` shows the pool imported. Emits
`config.applied` with target `file`. The apply reads `/etc/hostid` back at every start and after every write: while
`hostid` is set, a value there other than the set one emits `config.refused` naming it, because the node-engine's
composer cannot see a second writer of a file that is no resource. Each refusal of this line makes `zfs_health`
unhealthy (§29), naming the key and why, until an apply finds the label and the file agreeing with `hostid`, or
`hostid` is unset. No resource or template of the manifest writes `/etc/hostid`. Checked by §31 (unset: no
`zgenhostid`; a label of another id, or one that cannot be read: refused; 0 or equal: one run; a file holding
another value: refused; unhealthy, then healthy again; no manifest resource for the file).
## The pool
@@ -194,7 +201,7 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614
waits-on, since}.
A declared device that is not present is a wait. Each change is read back from `zpool status -P` and emitted.
Checked by §31's convergence tests (the empty list among them) and acceptance test 2.
18. **The apply: who, when, and what wakes a wait** (ADR 0679 §9). A `process` resource `zfs-config-apply`, `run:
18. **The apply: who, when, and what wakes a wait** (ADR 0679 §9, §17). A `process` resource `zfs-config-apply`, `run:
./zfs-tools apply`, as root, oneshot, ordered after `zfs-import.target`, enabled at boot, with `restart-on` the
config file (§9); it converges §15 to §17 and writes the ARC values live. It holds an exclusive lock on
`/run/zfs-tools/apply.lock`; a second start waits for it. It converges nothing while a scrub or resilver runs on
@@ -207,7 +214,7 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614
to `/sys/module/zfs/parameters/zfs_arc_max` and `zfs_arc_min` as well as to §12's file, and reads both back; each
`module-options` entry whose live value differs is a warning naming it, "takes effect at the next load of the
kernel module", never success and never unhealthy.
19. **The call-site test** (ADR 0679 §7). Every command the bundle runs is built by one of its argument functions, and
19. **The call-site test** (ADR 0679 §7, §17). Every command the bundle runs is built by one of its argument functions, and
the test (`TestTheBundleRunsOnlyReadingVerbsSnapshotAndScrub`, renamed `TestTheBundleRunsOnlyTheFormsOfADR0679`)
allows only: `zpool add <pool> cache|log`, `zpool remove` of a cache or log device, `zpool set`, `zfs set` and `zfs
inherit` of a listed property, `zfs snapshot`, `zfs set mesh:schedule=<kind>` on a ZFS snapshot named `mesh-`,
@@ -217,7 +224,9 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614
data vdevs, spares, special and dedup devices, `upgrade`, a dataset's destroy, `rollback`, `rename` and `clear` are
never run (each an act of its own, asked of the operator). The same test pins the module's other root forms:
`wipefs --no-act <device>`, `zdb -l <device>`, `zdb -C <pool>`, `zgenhostid -f <value>`, `udevadm monitor --udev
--subsystem-match=block` and `systemctl show`. Fate: the README's list of allowed forms is replaced by this one.
--subsystem-match=block` and `systemctl show`, and refuses every other `zgenhostid` or `zdb` form: `zgenhostid`
only as `-f` with the value §11 validated, `zdb` only as these two reads. Fate: the README's list of allowed forms
is replaced by this one.
## The scheduled run and ZFS snapshots
@@ -302,22 +311,27 @@ capabilities of §4 (a reader's capability the delivery plan derives, ADR 0614
`zfs.conf`, `/etc/zfs/mesh-zed.d` and the drop-in are removed, and before leaving `zfs_config` says that the next
load of the kernel module uses ZFS's defaults. The distribution's scrub and trim timers are put back as the
module's declaration found them, started and enabled again where it stopped or disabled them. Pool and dataset
properties and devices stay as the last apply set them, also on a rollback (§8).
properties and devices stay as the last apply set them, also on a rollback (§8). `/etc/hostid` stays, and the
module says so before it leaves (`zfs_config`): it is no resource the node-engine removes, and removing it would
change the node's hostid, so that the pool's next import could need `zpool import -f` (ADR 0679 §15).
## How it is checked
31. **Unit tests**, table-driven, one test per area, within the module's 500: no values gives no `zpool` or `zfs`
write but the due scrub; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel
name to its stable name, a removal, an empty list removing nothing, a log removal refused while not healthy, a
`log-mirror` change, a wait during a scrub); every refused form (a newline, quote, `$`, backtick or semicolon in
each field; a parameter or property not on its list; a `sharenfs` value; `-f`; a device with a signature or another
pool's label; a hostid of the wrong form or a label of another id); the call-site test of §19; the lock (a second
apply waits, a run with the lock held does nothing); the apply started only by `config-changed`, never a timer;
one scheduled resource; both ARC writes read back; applying from the last config file with the bus down; the
three templates (config file, options file, `zed.rc`); pruning never removes a ZFS snapshot whose name, property
or ledger entry does not fit, nor one with a hold or clone, and removes every surplus in one run; no ZFS snapshot
taken without a ledger record; `snapshots` back to `none` removes nothing; `zfs_snapshot` refuses a `mesh-`
31. **Unit tests**, table-driven, one test per area, within the module's 500: no values gives no `zpool` or `zfs` write
but the due scrub; each convergence plan from parsed `zpool status` and `zfs get` output (an add, a kernel name to
its stable name, a removal, an empty list removing nothing, a log removal refused while not healthy, a `log-mirror`
change, a wait during a scrub); every refused form (a newline, quote, `$`, backtick or semicolon in each field; a
parameter or property not on its list; a `sharenfs` value; `-f`; a device with a signature or another pool's label;
a hostid of the wrong form, a label of another id or one that cannot be read, and a hostid file holding another
value than the set one, each unhealthy and healthy again once the label and the file agree or `hostid` is unset);
the call-site test of §19, refusing any other `zgenhostid` or `zdb` form and a `zgenhostid` value not validated; the
lock (a second apply waits, a run with the lock held does nothing); the apply started only by `config-changed`,
never a timer; one scheduled resource; both ARC writes read back; applying from the last config file with the bus
down; the three templates (config file, options file, `zed.rc`); pruning never removes a ZFS snapshot whose name,
property or ledger entry does not fit, nor one with a hold or clone, and removes every surplus in one run; no ZFS
snapshot taken without a ledger record; `snapshots` back to `none` removes nothing; `zfs_snapshot` refuses a `mesh-`
name; a scheduled `snapshot.taken.schedule` does not wake a person's waiter; a parameter awaiting a load is a
warning; the `.pacnew` warning; each change emits its event with before and after; one test per tool. The module
adds no integration test; §5 is the milestone's, and the main file's live tests stand for the module whole, said
in the pull request.
warning; the `.pacnew` warning; each change emits its event with before and after; one test per tool; no resource or
template of the manifest for `/etc/hostid`; the hostid converged at the apply's start after `zfs-import.target`; the
leave keeps `/etc/hostid` and says so. The module adds no integration test; §5 is the milestone's, and the main
file's live tests stand for the module whole, said in the pull request.
03-DESIGN/01-to-be/42-the-machines-modules-in-order.md
+10 -7
View File
@@ -170,7 +170,8 @@ named ZFS snapshot, a scrub's start, pause and stop.
the states of ZFS's import, mount, share, volume and target units and of the distribution's scrub and trim timers;
and, for the one pool its config value `pool` names, that pool's cache and log devices, its properties and its
datasets' properties from fixed lists. The node-engine's composer refuses a second module owning one of its paths
(ADR 0593 §7, ADR 0051 §9).
(ADR 0593 §7, ADR 0051 §9), all but `/etc/hostid`, which is no resource: the module's apply reads it back instead
(ADR 0679 §1, §17).
- **What it never writes** (ADR 0679 §1, §2): the package's own `/etc/default/zfs` and `/etc/zfs/zed.d` with its
`zed.rc`, and what ZFS writes itself; it reads and shows them. A dataset's `sharenfs` is the seat
`node-nfs-server`'s, `sharesmb` the SMB module's; the pool's files are the operator's (ADR 0051); the weekly
@@ -178,10 +179,11 @@ named ZFS snapshot, a scrub's start, pause and stop.
- **An unset or empty value changes nothing** (ADR 0679 §3): every new config value is "not managed" until the
operator sets it through mesh-config, on the Approve for a trusted value (ADR 0348 §4, §8). Placing the new version
on a node runs no `zpool` or `zfs` write but the scrub its schedule already runs, and places none of its new files.
- **Who applies** (ADR 0679 §9): the node-engine places the packages, files and units as the module's resources,
some only while a value holds; the module's root process `zfs-tools apply` converges the pool's devices and
properties at its start and on `config-changed`, never on a timer; the module's one scheduled run, under the same
lock, starts the scrub and the trim when due and takes and removes the scheduled ZFS snapshots (ADR 0679 §8).
- **Who applies** (ADR 0679 §9, §17): the node-engine places the packages, files and units as the module's resources,
some only while a value holds, all but `/etc/hostid`; the module's root process `zfs-tools apply` converges the pool's
devices and properties at its start and on `config-changed`, never on a timer, and writes `/etc/hostid` with ZFS's own
writer only while `hostid` is set and the pool's label agrees; the module's one scheduled run, under the same lock,
starts the scrub and the trim when due and takes and removes the scheduled ZFS snapshots (ADR 0679 §8).
- **What it says** (ADR 0679 §10, §11, §12): every change, refusal and wait is an event, each kind on a subject of
its own, on an events stream asked for by template; the last change per value, the schedule's pause and every
scheduled ZFS snapshot are kept on the bus in its own states and ledger (ADR 0595). Its health names a refusal or a
@@ -190,7 +192,8 @@ named ZFS snapshot, a scrub's start, pause and stop.
- **What it never does** (ADR 0679 §7): it converges no data device, spare, special or dedup device, upgrades no
pool, destroys no dataset, rolls back, renames and clears nothing; each is an act of its own, decided separately.
- **When it leaves** (ADR 0679 §15): ZFS keeps running (the import, mount and ZFS target are never stopped), the
packages stay, its own files are removed and the distribution's timers are put back as found.
packages stay, its own files are removed but `/etc/hostid`, which stays and is said, and the distribution's timers
are put back as found.
- **Its prerequisites** (ADR 0679 §16): three node-engine capabilities, built first in the same milestone under issue
1979, with their readers in mesh-sdk and the manifest validator.
@@ -200,7 +203,7 @@ flowchart LR
config -->|the module's config file| engine[the node-engine]
engine -->|packages, files, units| node[ZFS on the node]
engine -->|restarts on its config file| apply[zfs-tools apply, root]
apply -->|zpool add, remove, set; zfs set, inherit| node
apply -->|zpool add, remove, set; zfs set, inherit; zgenhostid| node
sched[the scheduled run] -->|scrub, trim, ZFS snapshots and their removal| node
follower[the event follower] -->|scrub ended, device appeared: start| apply
apply -->|config.applied, refused, waiting| stream[(events stream)]