ADR 0263 and design 52 after review: what each export promises, per-node grants, refusals, not-built checks, and a read-only-first rollout
This commit is contained in:
+25
-13
@@ -72,22 +72,32 @@ Rejected:
|
||||
path, and read-only or read-write. The path is the operator's data (ADR 0051): the module never
|
||||
creates, chowns, chmods or removes anything under it. It writes its own file in `/etc/exports.d`, never `/etc/exports`, and listens on 2049/tcp
|
||||
from the private network. It serves NFS version 4 only, so no other port opens.
|
||||
2. **Every export squashes every client to the share's owner.** `all_squash`, with `anonuid` and
|
||||
`anongid` read by the module from the exported path's owner on the server, never written as
|
||||
numbers. The clients are `${machine:mesh-range}` and nothing else. A file written from any node lands
|
||||
owned by the library's owner, and no client acts as any other account on the server.
|
||||
2. **Every export maps every client to the share's owner.** `all_squash`, with `anonuid` and `anongid`
|
||||
read by the module from the exported folder's owner on the server (after following any symbolic link
|
||||
to the real folder), never written as numbers. A share owned by root is refused, so no client is ever
|
||||
mapped to root. A file written from any node lands owned by the folder's owner. **What this does not
|
||||
promise:** a client acts as that owner, so it can do whatever the owner may do with what it reaches.
|
||||
Where a share is a folder inside a larger filesystem, NFS cannot fully confine a client that forges
|
||||
file handles to that folder; it can reach the rest of that filesystem as the owner. That is why
|
||||
rule 3 refuses a share at or under a backup repository, or with one below it, and why the export
|
||||
carries `mp`, so it is served only while its filesystem is mounted.
|
||||
3. **One share per folder, never a pool's root** (the operator's choice, 2026-10-08): on the home server,
|
||||
the library and the downloads. The backup repository and the archive are not exported. The module
|
||||
refuses a share that is `/` or that holds a backup repository at its path or one level below, so a
|
||||
setting cannot export one by mistake.
|
||||
refuses a share that is `/`, is owned by root, is inside a backup repository, or holds one within four
|
||||
levels below (refusing when it cannot finish looking), all judged on the real folder after any
|
||||
symbolic link, so a setting cannot export one by mistake.
|
||||
4. **A node mounts through one module.** The module `mounts` holds the node seat `node-mounts`. For each
|
||||
share it writes a `.mount` unit and an `.automount` unit, never an fstab line. Only the automount is
|
||||
enabled, so nothing mounts at boot and nothing can fail the boot. Each mount has an idle timeout
|
||||
(default 10 minutes) and a mount timeout (default 30 s; 10 s for an occasional source). A read-write
|
||||
NFS share is mounted `hard`. A mount point is always an autofs mount, so nothing writes into the
|
||||
empty directory under it (issue 169's data-loss case).
|
||||
5. **Only the nodes that ask get a share.** A node mounts what its `mounts` assignment's setting `shares`
|
||||
lists. The list may be set once at the mesh-wide settings layer, which is how "every node" is said,
|
||||
5. **Only the nodes that ask, and that the server grants, get a share.** A share is exported to one
|
||||
node's address only when the server's setting `grants` names that node for that share and the node's
|
||||
`mounts` assignment asks for it; the access is the lower of the two, decided on the server. Addresses
|
||||
come from the controller's list of machines, never from what a client says. A share nobody is granted
|
||||
is exported to nobody: the control-node, like any node, gets nothing unless both name it. A node mounts
|
||||
what its `mounts` assignment's setting `shares` lists. The list may be set once at the mesh-wide settings layer, which is how "every node" is said,
|
||||
and a node's own layer can leave a share out. Which nodes are assigned, and whether the laptop mounts
|
||||
read-only, is decided at assignment, which is the operator's; the recommendation is the workstation
|
||||
and the laptop (read-only), not the control-node.
|
||||
@@ -137,11 +147,13 @@ Rejected:
|
||||
- The `mounts` module's `mounts_list` shows every mount point's source, its unit pair, its state and
|
||||
which settings layer asked for it. `mounts_exports` shows the server's export table. A test in the
|
||||
module renders the unit pair from a binding and asserts the options above.
|
||||
- The lab composes a server and two clients and asserts: the export names only the private network's
|
||||
range with `all_squash`; a file written from a client is owned by the share's owner; the client's
|
||||
boot completes with the server stopped; and an occasional source that is absent leaves no failed unit.
|
||||
- The controller's self-check counts fstab lines of type `nfs` or `cifs` on every node through
|
||||
`mounts_list`, and reports any that are not adopted.
|
||||
- **Not built yet:** a lab scenario composing a server and two clients, asserting that the export names
|
||||
only the granted nodes' addresses with `all_squash`, that a file written from a client is owned by the
|
||||
share's owner, that a client's boot completes with the server stopped, and that an absent occasional
|
||||
source leaves no failed unit.
|
||||
- **Not built yet:** a probe in the controller's self-check that counts fstab lines of type `nfs` or
|
||||
`cifs` on every node through `mounts_list`, and reports any that are not adopted.
|
||||
- Until both exist, the modules' own tests and the safe rollout in design 52 are the check.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -61,14 +61,26 @@ The glossary holds the three words.
|
||||
- **Its setting `shares`.** A list of `name=path`, each optionally `:ro`, separated by spaces: on the
|
||||
home server the library and the downloads. A share's path is the operator's data (ADR 0051); the
|
||||
module never creates, chowns, chmods or removes anything under it.
|
||||
- **What it refuses.** A path that does not exist or is not a directory, `/`, and a path that holds a
|
||||
backup repository at its root or one level below. A refused share is said by the tools and makes the
|
||||
module unhealthy; the others are exported.
|
||||
- **Its setting `grants`.** Which node may mount which share, and how: `library=<workstation>:rw,<laptop>:ro`.
|
||||
`none` (the default) grants nothing. A share is exported to a node only when `grants` names it and the
|
||||
node's `mounts` asks for it (its bucket `shares`, field `wants`); the access is the lower of the two.
|
||||
The node's address comes from the controller's list of machines, rendered into the module's own file,
|
||||
and must be inside the private network's range; never from what the node says.
|
||||
- **What it refuses.** A path that does not exist or is not a directory, `/`, a folder owned by root
|
||||
(uid or gid 0), and a folder that is, is inside, or holds within four levels a backup repository; a look
|
||||
below that is cut short by its bound refuses rather than exports. A clean look is kept for an hour.
|
||||
Every check runs on the real folder after any symbolic link. A refused share is said by the tools and
|
||||
makes the module unhealthy; the others are exported.
|
||||
- **Its file.** `/etc/exports.d/mesh.exports`, written by the module's own process, which runs as root
|
||||
and looks every 30 s. One line per share: the path, to `${machine:mesh-range}` and nothing else, with
|
||||
`rw` or `ro`, `sync`, `secure`, `no_subtree_check`, `all_squash`, and `anonuid` and `anongid` read from
|
||||
the path's owner. `exportfs -ra` runs when the file changes or a share is missing from the kernel's
|
||||
and looks every 30 s. One line per share: the real path (after any symbolic link), to the addresses of
|
||||
the nodes the server grants it to and that ask for it, and to nobody else, with
|
||||
`rw` or `ro`, `sync`, `secure`, `no_subtree_check`, `all_squash`, `anonuid` and `anongid` read from
|
||||
the folder's owner, and `mp` (or `mp=<its filesystem's mount point>`), so nothing is served from an
|
||||
unmounted dataset's empty folder. `exportfs -ra` runs when the file changes or a share is missing from the kernel's
|
||||
table. The filesystem's own id is kept; none is forced.
|
||||
- **Unassigned.** The process is stopped after its configuration is removed (the node-engine removes a
|
||||
module's resources in reverse order); finding it gone, it removes its export file and has the kernel
|
||||
drop the exports. A restart or a push keeps them.
|
||||
- **The service.** `nfs-server.service`, running and enabled. NFS versions 2, 3 and 4.0 are turned off
|
||||
in the module's drop-in in `/etc/nfs.conf.d`, so only 2049/tcp is needed, declared in `listens` with
|
||||
`from: mesh`.
|
||||
@@ -93,6 +105,13 @@ The glossary holds the three words.
|
||||
`password=`, issued outside the mesh and given with `secret accept <node> mounts smb-credentials
|
||||
--from <file>`. The process writes one credentials file per source, readable by root only. A source
|
||||
with no section is said with that remedy and is not mounted.
|
||||
- **What it refuses.** A mount point at, under or above any data the mesh keeps on this machine (the
|
||||
controller's `data` verb, asked over the bus; until it is read nothing new is set up), a system path,
|
||||
a mount point where something else is mounted or a folder that is not empty, and a share this machine
|
||||
serves itself. It never stops a mount whose mounted filesystem is not its own source, and a setting
|
||||
entry it cannot read removes nothing.
|
||||
- **What it asks for.** Its bucket `shares` carries `wants`: each share this machine asks of which server,
|
||||
read-write or read-only. The server decides; the mount takes the access the server gave this machine.
|
||||
- **Reading the server.** The module reads the server's bucket `exports` (its manifest's `reads`) for
|
||||
each share's path and access: its tools, which reach the bus, write what they read to a file of the
|
||||
module's, and the root process takes only a name, an absolute path and the access from it. A share
|
||||
@@ -147,22 +166,49 @@ Each says the share, the node and since when: *"The home server's share `library
|
||||
from the workstation since 19:21."* An absent occasional source raises nothing; `mounts_list` says
|
||||
*"the share `retro-games` is not present since 18:02"*.
|
||||
|
||||
## 5. Migration
|
||||
## 5. The safe rollout
|
||||
|
||||
1. Build and deliver the two seats, `nfs-server` and `mounts`.
|
||||
2. Assign `nfs-server` to the home server with the library and the downloads. Check `exports` and
|
||||
`test` from the workstation.
|
||||
3. `adopt` on the home server clears the library's `sharenfs` property. The underlay range leaves the
|
||||
export table; the private network's range is in it.
|
||||
4. Assign `mounts` to the workstation. Its unit pair for the library takes precedence over the unit
|
||||
generated from fstab. Check `mounts_list` and a file written from the workstation.
|
||||
5. `adopt` on the workstation comments out the library's fstab line, keeping a copy of the file.
|
||||
6. Accept the gaming device's credential into the vault with `secret accept`, add the gaming device as an
|
||||
occasional source, and `adopt` its fstab line.
|
||||
7. Assign `mounts` to the laptop, read-only.
|
||||
Read-only first, one node first, and every step checked before the next. Each step names how it is
|
||||
undone. Every `adopt` is a person's act. Nothing here touches the library's files, owners or modes, or
|
||||
the media modules' local bind mounts on the home server.
|
||||
|
||||
Every `adopt` is a person's act. Nothing in this migration touches the library's files, owners or
|
||||
modes, or the media modules' bind mounts.
|
||||
**Before anything:** the seats are merged and the controller that knows them is running; the two
|
||||
modules' pull request is green against it. On the home server, `node-backup.backed-up` shows a good
|
||||
night for Plex's own data, and `nfs-server`'s dry runs are read: `exports` lists only the intended
|
||||
folders, none refused for a reason not understood.
|
||||
|
||||
1. **Assign `nfs-server` to the home server, read-only.** Its `shares` names the library only, and its
|
||||
`grants` names the workstation read-only (`library=<workstation>:ro`). Check `exports`: one line, the
|
||||
workstation's address only, `ro`, `all_squash` to the library's owner, `mp`. The ZFS property's own
|
||||
export to the home network stays as it is (the packet filter already blocks it).
|
||||
*Undo:* unassign `nfs-server`; its export file goes with it.
|
||||
2. **Assign `mounts` to the workstation, read-only.** Its `shares` is `library=<mount point>:ro`. Check
|
||||
`node-mounts.list`: the pair is armed, the share reads `armed` then `mounted` after a listing, and the
|
||||
files are there. Its unit pair takes the place of the unit generated from fstab.
|
||||
*Undo:* unassign `mounts`; its unit pairs go, and the fstab line is still there.
|
||||
3. **Watch for a day.** `mounts_health` and `nfs_health` stay healthy; a reboot of the workstation
|
||||
completes with the home server's share reachable and with it stopped (the share reads `unreachable`
|
||||
and the boot does not wait).
|
||||
4. **Read-write.** Change the grant to `rw` and the workstation's setting to drop `:ro`. Write one test
|
||||
file from the workstation, check its owner on the home server is the library's owner, and remove it.
|
||||
*Undo:* set both back to `ro`.
|
||||
5. **Adopt the workstation's library line**: `node-mounts.adopt` dry run, read it, then with
|
||||
`confirm` and a reason. *Undo:* the dated copy of fstab beside it.
|
||||
6. **Adopt the ZFS property**: `node-nfs-server.adopt` dry run, then confirmed. The home network's
|
||||
export goes; the mesh's stays. *Undo:* `zfs set sharenfs=<the old value>` by a person, recorded as a
|
||||
hand-act.
|
||||
7. **The downloads**, the same way: read-only first, then read-write.
|
||||
8. **The gaming device**: the operator gives `smb-credentials` with `secret accept`; the source is added
|
||||
to `sources`; `node-mounts.list` says it `absent` while the device is off, and it mounts on access
|
||||
when it is on. Then its fstab line is adopted.
|
||||
9. **The laptop**, read-only, granted read-only on the home server.
|
||||
|
||||
**Root.** Both modules' tools act as root through `sudo -n`, and the `sudo` module grants the operator
|
||||
account every command, so the modules' own checks are the only line before root. Narrowing sudo is a
|
||||
separate decision.
|
||||
|
||||
**Stop at once** when a share reads `failed` for a reason not understood, when a dry run would touch a
|
||||
line or a dataset other than the one asked for, or when anything under the library changes owner.
|
||||
|
||||
## Not in this design
|
||||
|
||||
|
||||
Reference in New Issue
Block a user