From bffd0f90d06ca7d16256c4d56c8d5bb57fe0a647 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 8 Oct 2026 20:21:48 +0200 Subject: [PATCH] ADR 0263 and design 52 after review: what each export promises, per-node grants, refusals, not-built checks, and a read-only-first rollout --- ...n-demand-and-only-nodes-that-ask-get-it.md | 38 +++++--- 03-DESIGN/01-to-be/52-shares-and-mounts.md | 86 ++++++++++++++----- 2 files changed, 91 insertions(+), 33 deletions(-) diff --git a/02-DECISIONS/0263-a-share-is-a-provision-a-node-mounts-it-on-demand-and-only-nodes-that-ask-get-it.md b/02-DECISIONS/0263-a-share-is-a-provision-a-node-mounts-it-on-demand-and-only-nodes-that-ask-get-it.md index 3f351c5f..a197ae5b 100644 --- a/02-DECISIONS/0263-a-share-is-a-provision-a-node-mounts-it-on-demand-and-only-nodes-that-ask-get-it.md +++ b/02-DECISIONS/0263-a-share-is-a-provision-a-node-mounts-it-on-demand-and-only-nodes-that-ask-get-it.md @@ -72,22 +72,32 @@ Rejected: path, and read-only or read-write. The path is the operator's data (ADR 0051): the module never creates, chowns, chmods or removes anything under it. It writes its own file in `/etc/exports.d`, never `/etc/exports`, and listens on 2049/tcp from the private network. It serves NFS version 4 only, so no other port opens. -2. **Every export squashes every client to the share's owner.** `all_squash`, with `anonuid` and - `anongid` read by the module from the exported path's owner on the server, never written as - numbers. The clients are `${machine:mesh-range}` and nothing else. A file written from any node lands - owned by the library's owner, and no client acts as any other account on the server. +2. **Every export maps every client to the share's owner.** `all_squash`, with `anonuid` and `anongid` + read by the module from the exported folder's owner on the server (after following any symbolic link + to the real folder), never written as numbers. A share owned by root is refused, so no client is ever + mapped to root. A file written from any node lands owned by the folder's owner. **What this does not + promise:** a client acts as that owner, so it can do whatever the owner may do with what it reaches. + Where a share is a folder inside a larger filesystem, NFS cannot fully confine a client that forges + file handles to that folder; it can reach the rest of that filesystem as the owner. That is why + rule 3 refuses a share at or under a backup repository, or with one below it, and why the export + carries `mp`, so it is served only while its filesystem is mounted. 3. **One share per folder, never a pool's root** (the operator's choice, 2026-10-08): on the home server, the library and the downloads. The backup repository and the archive are not exported. The module - refuses a share that is `/` or that holds a backup repository at its path or one level below, so a - setting cannot export one by mistake. + refuses a share that is `/`, is owned by root, is inside a backup repository, or holds one within four + levels below (refusing when it cannot finish looking), all judged on the real folder after any + symbolic link, so a setting cannot export one by mistake. 4. **A node mounts through one module.** The module `mounts` holds the node seat `node-mounts`. For each share it writes a `.mount` unit and an `.automount` unit, never an fstab line. Only the automount is enabled, so nothing mounts at boot and nothing can fail the boot. Each mount has an idle timeout (default 10 minutes) and a mount timeout (default 30 s; 10 s for an occasional source). A read-write NFS share is mounted `hard`. A mount point is always an autofs mount, so nothing writes into the empty directory under it (issue 169's data-loss case). -5. **Only the nodes that ask get a share.** A node mounts what its `mounts` assignment's setting `shares` - lists. The list may be set once at the mesh-wide settings layer, which is how "every node" is said, +5. **Only the nodes that ask, and that the server grants, get a share.** A share is exported to one + node's address only when the server's setting `grants` names that node for that share and the node's + `mounts` assignment asks for it; the access is the lower of the two, decided on the server. Addresses + come from the controller's list of machines, never from what a client says. A share nobody is granted + is exported to nobody: the control-node, like any node, gets nothing unless both name it. A node mounts + what its `mounts` assignment's setting `shares` lists. The list may be set once at the mesh-wide settings layer, which is how "every node" is said, and a node's own layer can leave a share out. Which nodes are assigned, and whether the laptop mounts read-only, is decided at assignment, which is the operator's; the recommendation is the workstation and the laptop (read-only), not the control-node. @@ -137,11 +147,13 @@ Rejected: - The `mounts` module's `mounts_list` shows every mount point's source, its unit pair, its state and which settings layer asked for it. `mounts_exports` shows the server's export table. A test in the module renders the unit pair from a binding and asserts the options above. -- The lab composes a server and two clients and asserts: the export names only the private network's - range with `all_squash`; a file written from a client is owned by the share's owner; the client's - boot completes with the server stopped; and an occasional source that is absent leaves no failed unit. -- The controller's self-check counts fstab lines of type `nfs` or `cifs` on every node through - `mounts_list`, and reports any that are not adopted. +- **Not built yet:** a lab scenario composing a server and two clients, asserting that the export names + only the granted nodes' addresses with `all_squash`, that a file written from a client is owned by the + share's owner, that a client's boot completes with the server stopped, and that an absent occasional + source leaves no failed unit. +- **Not built yet:** a probe in the controller's self-check that counts fstab lines of type `nfs` or + `cifs` on every node through `mounts_list`, and reports any that are not adopted. +- Until both exist, the modules' own tests and the safe rollout in design 52 are the check. ## References diff --git a/03-DESIGN/01-to-be/52-shares-and-mounts.md b/03-DESIGN/01-to-be/52-shares-and-mounts.md index 1fd51752..d9eb11ba 100644 --- a/03-DESIGN/01-to-be/52-shares-and-mounts.md +++ b/03-DESIGN/01-to-be/52-shares-and-mounts.md @@ -61,14 +61,26 @@ The glossary holds the three words. - **Its setting `shares`.** A list of `name=path`, each optionally `:ro`, separated by spaces: on the home server the library and the downloads. A share's path is the operator's data (ADR 0051); the module never creates, chowns, chmods or removes anything under it. -- **What it refuses.** A path that does not exist or is not a directory, `/`, and a path that holds a - backup repository at its root or one level below. A refused share is said by the tools and makes the - module unhealthy; the others are exported. +- **Its setting `grants`.** Which node may mount which share, and how: `library=:rw,:ro`. + `none` (the default) grants nothing. A share is exported to a node only when `grants` names it and the + node's `mounts` asks for it (its bucket `shares`, field `wants`); the access is the lower of the two. + The node's address comes from the controller's list of machines, rendered into the module's own file, + and must be inside the private network's range; never from what the node says. +- **What it refuses.** A path that does not exist or is not a directory, `/`, a folder owned by root + (uid or gid 0), and a folder that is, is inside, or holds within four levels a backup repository; a look + below that is cut short by its bound refuses rather than exports. A clean look is kept for an hour. + Every check runs on the real folder after any symbolic link. A refused share is said by the tools and + makes the module unhealthy; the others are exported. - **Its file.** `/etc/exports.d/mesh.exports`, written by the module's own process, which runs as root - and looks every 30 s. One line per share: the path, to `${machine:mesh-range}` and nothing else, with - `rw` or `ro`, `sync`, `secure`, `no_subtree_check`, `all_squash`, and `anonuid` and `anongid` read from - the path's owner. `exportfs -ra` runs when the file changes or a share is missing from the kernel's + and looks every 30 s. One line per share: the real path (after any symbolic link), to the addresses of + the nodes the server grants it to and that ask for it, and to nobody else, with + `rw` or `ro`, `sync`, `secure`, `no_subtree_check`, `all_squash`, `anonuid` and `anongid` read from + the folder's owner, and `mp` (or `mp=`), so nothing is served from an + unmounted dataset's empty folder. `exportfs -ra` runs when the file changes or a share is missing from the kernel's table. The filesystem's own id is kept; none is forced. +- **Unassigned.** The process is stopped after its configuration is removed (the node-engine removes a + module's resources in reverse order); finding it gone, it removes its export file and has the kernel + drop the exports. A restart or a push keeps them. - **The service.** `nfs-server.service`, running and enabled. NFS versions 2, 3 and 4.0 are turned off in the module's drop-in in `/etc/nfs.conf.d`, so only 2049/tcp is needed, declared in `listens` with `from: mesh`. @@ -93,6 +105,13 @@ The glossary holds the three words. `password=`, issued outside the mesh and given with `secret accept mounts smb-credentials --from `. The process writes one credentials file per source, readable by root only. A source with no section is said with that remedy and is not mounted. +- **What it refuses.** A mount point at, under or above any data the mesh keeps on this machine (the + controller's `data` verb, asked over the bus; until it is read nothing new is set up), a system path, + a mount point where something else is mounted or a folder that is not empty, and a share this machine + serves itself. It never stops a mount whose mounted filesystem is not its own source, and a setting + entry it cannot read removes nothing. +- **What it asks for.** Its bucket `shares` carries `wants`: each share this machine asks of which server, + read-write or read-only. The server decides; the mount takes the access the server gave this machine. - **Reading the server.** The module reads the server's bucket `exports` (its manifest's `reads`) for each share's path and access: its tools, which reach the bus, write what they read to a file of the module's, and the root process takes only a name, an absolute path and the access from it. A share @@ -147,22 +166,49 @@ Each says the share, the node and since when: *"The home server's share `library from the workstation since 19:21."* An absent occasional source raises nothing; `mounts_list` says *"the share `retro-games` is not present since 18:02"*. -## 5. Migration +## 5. The safe rollout -1. Build and deliver the two seats, `nfs-server` and `mounts`. -2. Assign `nfs-server` to the home server with the library and the downloads. Check `exports` and - `test` from the workstation. -3. `adopt` on the home server clears the library's `sharenfs` property. The underlay range leaves the - export table; the private network's range is in it. -4. Assign `mounts` to the workstation. Its unit pair for the library takes precedence over the unit - generated from fstab. Check `mounts_list` and a file written from the workstation. -5. `adopt` on the workstation comments out the library's fstab line, keeping a copy of the file. -6. Accept the gaming device's credential into the vault with `secret accept`, add the gaming device as an - occasional source, and `adopt` its fstab line. -7. Assign `mounts` to the laptop, read-only. +Read-only first, one node first, and every step checked before the next. Each step names how it is +undone. Every `adopt` is a person's act. Nothing here touches the library's files, owners or modes, or +the media modules' local bind mounts on the home server. -Every `adopt` is a person's act. Nothing in this migration touches the library's files, owners or -modes, or the media modules' bind mounts. +**Before anything:** the seats are merged and the controller that knows them is running; the two +modules' pull request is green against it. On the home server, `node-backup.backed-up` shows a good +night for Plex's own data, and `nfs-server`'s dry runs are read: `exports` lists only the intended +folders, none refused for a reason not understood. + +1. **Assign `nfs-server` to the home server, read-only.** Its `shares` names the library only, and its + `grants` names the workstation read-only (`library=:ro`). Check `exports`: one line, the + workstation's address only, `ro`, `all_squash` to the library's owner, `mp`. The ZFS property's own + export to the home network stays as it is (the packet filter already blocks it). + *Undo:* unassign `nfs-server`; its export file goes with it. +2. **Assign `mounts` to the workstation, read-only.** Its `shares` is `library=:ro`. Check + `node-mounts.list`: the pair is armed, the share reads `armed` then `mounted` after a listing, and the + files are there. Its unit pair takes the place of the unit generated from fstab. + *Undo:* unassign `mounts`; its unit pairs go, and the fstab line is still there. +3. **Watch for a day.** `mounts_health` and `nfs_health` stay healthy; a reboot of the workstation + completes with the home server's share reachable and with it stopped (the share reads `unreachable` + and the boot does not wait). +4. **Read-write.** Change the grant to `rw` and the workstation's setting to drop `:ro`. Write one test + file from the workstation, check its owner on the home server is the library's owner, and remove it. + *Undo:* set both back to `ro`. +5. **Adopt the workstation's library line**: `node-mounts.adopt` dry run, read it, then with + `confirm` and a reason. *Undo:* the dated copy of fstab beside it. +6. **Adopt the ZFS property**: `node-nfs-server.adopt` dry run, then confirmed. The home network's + export goes; the mesh's stays. *Undo:* `zfs set sharenfs=` by a person, recorded as a + hand-act. +7. **The downloads**, the same way: read-only first, then read-write. +8. **The gaming device**: the operator gives `smb-credentials` with `secret accept`; the source is added + to `sources`; `node-mounts.list` says it `absent` while the device is off, and it mounts on access + when it is on. Then its fstab line is adopted. +9. **The laptop**, read-only, granted read-only on the home server. + +**Root.** Both modules' tools act as root through `sudo -n`, and the `sudo` module grants the operator +account every command, so the modules' own checks are the only line before root. Narrowing sudo is a +separate decision. + +**Stop at once** when a share reads `failed` for a reason not understood, when a dry run would touch a +line or a dataset other than the one asked for, or when anything under the library changes owner. ## Not in this design