mesh-catalog/modules/zfs/README.md

zfs

The mesh's view of ZFS on a machine: every pool's health, layout, space, scrubs, I/O, cache and events, every dataset with its space and snapshots, and one act, a named snapshot of one dataset. It is for the home server, whose pool holds the operator's media library, which the mesh does not own (novox/hq ADR 0051). So the module reads, and the one thing it does only adds.

What it does not do

There is no destroy, rollback, set, send, receive, rename, clone or any other change. Each of those changes or removes the operator's data and needs a decision of its own first. zfs_snapshot is the only tool that changes anything, and it only adds a snapshot.

This is checked by TestTheBundleRunsOnlyReadingVerbsAndSnapshot. Every command the bundle runs is built by one of its argument functions, and each function is held to its verb: list, get, status, iostat, events, history or snapshot. A call site that builds its own argument list fails the test.

It installs nothing. ZFS belongs to the machine, and its kernel module is built against that machine's kernel. On a machine without ZFS, every tool answers applicable: false with the reason, and the health check is unhealthy and says to unassign the module, because a module put where it does nothing must not look healthy.

It holds no seat. A new node-* seat is the controller's to define (novox/hq ADR 0121, ADR 0246). This is a module with tools, like disk-load and sensors. If the mesh later gives a machine's storage a seat, these tools become its verbs.

Tools

The tools run as the operator account. What ZFS lets any account read runs as that account. What ZFS keeps for root (events, history and the snapshot itself) goes through sudo -n with a fixed argument list, never a shell. Sizes are whole bytes, read under -p, and each answer also says its findings in lines under said.

tool	what	replaces
zfs_datasets	Every filesystem and volume: mountpoint, used, available, referenced, compression ratio, sharenfs, and the number of snapshots with the space they hold together (usedbysnapshots)	zfs list
zfs_snapshots	The snapshots of one dataset (dataset, optional), oldest first: creation time, used (the space only that snapshot holds) and referenced	zfs list -t snapshot
zfs_snapshot	Takes dataset@name (see below)	zfs snapshot
zfs_pools	Each pool's state with status's explanation and action, space, capacity and fragmentation %, device error count, data errors and last scan	zpool list, zpool status -x
zfs_space	Each pool's space, then every dataset, biggest first: usedbydataset, usedbysnapshots, usedbychildren and usedbyrefreservation, quotas and reservations where set, compression, logical size against physical size, and the savings	zfs list -o space, zpool list -v
zfs_pool_layout	Vdevs and disks by the names the pool knows them by, with state, read, write and checksum errors and status's note; log, cache, special and dedup devices; spares	zpool status, zpool status -v
zfs_scrub	The last scan (scrub or resilver): when, how long, what it repaired, the errors, or its progress while it runs; the days since the last finished scrub, with a warning past 35. With history: true, it also reads the pool's history (as root) for the scrub and resilver that status no longer shows	zpool history -i
zfs_io	A sample of seconds (default 3, from 1 to 10): operations and bytes per second per pool, vdev and disk	zpool iostat -v
zfs_arc	ARC size, target, minimum and maximum, hit ratio since boot, demand hit ratio, and L2ARC where present	arcstat, arc_summary
zfs_events	Recent events, newest first, as root. By default it shows only errors and changes of state, and all: true shows every event (limit, default 50)	zpool events -v
zfs_properties	Every property of one dataset with its source, the usual ones in lines first. The value of keylocation is withheld, and ZFS never gives a key itself	zfs get all
zfs_health	The health check (below)

The snapshot

zfs_snapshot {dataset, name, why, recursive?} refuses all of these before anything runs:

• a dataset that zfs list does not show on this machine;
• a name with anything but letters, digits, ., _ and -;
• a whole name longer than ZFS's 255 characters;
• a snapshot that already exists (with recursive, on any child too);
• a missing why, or a why longer than one line.

It then runs sudo -n zfs snapshot -- <dataset>@<name>, with -r only when recursive is true. It answers from zfs list afterwards, not from what it ran: the snapshot made and its creation time, and any other snapshot that came or went during the call (another schedule's, for example). A snapshot the list does not show is an error, whatever the command said.

The act is recorded as the event snapshot.taken (the dataset, the name, why, whether it was done, and when), and ZFS keeps the command in the pool's own history. An event the bus does not accept is said in the answer as NOT RECORDED.

Health

The module runs two long-running processes of its own binary, and neither writes a file or changes anything:

• The watch (zfs-tools watch) carries the module's health check, zfs_health (novox/hq ADR 0240). Each minute it reads the pools and writes a line in its journal whenever their health changes.
• The event follower (zfs-tools events) runs as root and follows zpool events -f. It writes each error and change of state to its journal as ZFS emits it. ZFS keeps its events in memory only, so they are lost at a reboot, and the journal keeps them past one. The node-engine judges it by its unit. That is the check the module does not run itself, which a tool check needs beside it (ADR 0240 rule 2).

zfs_health is unhealthy when:

• a pool is not ONLINE (the answer gives status's reason and action);
• a device has read, write or checksum errors, or is not online;
• a pool reports data errors;
• a pool is more than 90 % full;
• no pool is imported;
• ZFS is not on the machine.

These are warnings, which leave the module healthy:

• a pool is more than 80 % full;
• a pool's last scrub finished more than 35 days ago, or the pool was never scrubbed.

The health check judges only the last scan that status shows. If that scan was a resilver, the last scrub's age is in zfs_scrub with history: true.

Tests

go test ./... runs against OpenZFS 2.4 output in cmd/zfs-tools/testdata. Each file has the exact form the command prints, with documentation names in place of the real pools and disks. The tests cover:

• parsing every command's output;
• the snapshot's argument list, every refusal (none runs a command), the read-back, a snapshot the list does not show, a refused sudo, an event the bus does not accept, and another snapshot taken meanwhile;
• the health verdicts;
• a machine without ZFS;
• the manifest against the tools the bundle serves.