120 lines
8.8 KiB
Markdown
120 lines
8.8 KiB
Markdown
# nfs-server
|
|
|
|
A node shares a folder with the mesh's machines, as a module (novox/hq ADR 0263, to-be 52). It holds
|
|
the node seat `node-nfs-server` and provides the provision `nfs-share`, which the `mounts` module on
|
|
another node requires.
|
|
|
|
## The rules it keeps
|
|
|
|
- **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or
|
|
`name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is
|
|
the operator's value (ADR 0112, ADR 0262).
|
|
- **To the nodes that ask, as far as this machine allows.** A share is exported to a node only when
|
|
this machine's `grants` setting names that node for it (`share=node:rw|ro[,node:rw|ro]`, separated by
|
|
spaces; `none`, the default, grants nothing) **and** that node's `mounts` module wants it from this
|
|
machine (its state `shares` on the bus). The access is the lower of the grant and the want, and of the
|
|
share's own `:ro`. The address is the node's private address from the mesh's roster
|
|
(`/etc/nfs-server/machines`, rendered by the mesh), never one a consumer gives, and it must lie inside
|
|
the private network's range. The control node, or any node, gets a share only when both name it. A
|
|
want that is not granted, and a grant nobody wants, are said by `exports`. Port 2049 is opened to the
|
|
mesh only; the home network is never named.
|
|
- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one
|
|
id names different accounts on different machines. Every line is `all_squash` with `anonuid` and
|
|
`anongid` read from the folder's owner on this machine, so a file written from any node lands owned by
|
|
the folder's owner, and root is squashed with everyone else.
|
|
- **Judged where it really is.** A share's path is resolved through every symlink, and every check runs
|
|
on the real folder, which is what the export line names.
|
|
- **Never root's.** A folder owned by root, or by root's group, is refused: squashing every client to
|
|
root would make every client root there.
|
|
- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, is inside a
|
|
backup (restic) repository (any parent up to `/`), or holds one within four levels below it is refused,
|
|
said by the tools, and makes the module unhealthy. The look below is bounded (200 000 folders); when it
|
|
is cut short, the share is refused rather than exported unchecked. A clean look is believed for an hour.
|
|
- **Never where the backup seat keeps its data, checked on every pass.** So that the hour cannot hide a
|
|
repository started inside it, the tools ask the controller over the bus, every five minutes, which module
|
|
holds `node-backup` on this machine (`seat:mesh-controller.seats`) and every path that module keeps data
|
|
in here (`seat:mesh-controller.data`; on the home server the restic repository and its state). They
|
|
leave it in `/var/lib/nfs-server/from-bus/backup.json`, and every pass refuses a share at, inside or
|
|
holding one of those paths, with nothing kept. A failed ask keeps the last paths and says why. Until a
|
|
list has been learnt, what is exported stays exported and the status says the list is not known: a slow
|
|
bus is no reason to take the library away, and the look below still guards.
|
|
- **Only while its filesystem is mounted.** A share that is a filesystem of its own is exported with `mp`,
|
|
a folder inside one with `mp=<its mount point>`, so an unmounted dataset never exports the empty folder
|
|
under it. A folder of the root filesystem carries neither.
|
|
- **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It
|
|
never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its
|
|
own and nothing else, and a test holds it to that.
|
|
- **NFS version 4 only**, so 2049 is the only port a client needs: rpcbind and mountd stay closed.
|
|
|
|
## What it owns
|
|
|
|
- the `nfs-utils` package and `nfs-server.service`, running and enabled;
|
|
- `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on;
|
|
- `/etc/nfs-server/shares.conf`: the settings `shares` and `grants` and the range, rendered by the mesh;
|
|
- `/etc/nfs-server/machines`: every machine of the mesh and its private address, rendered by the mesh;
|
|
- `/var/lib/nfs-server/from-bus/wants.json`: what the nodes want of this machine, written by the tools
|
|
from the bus (the directory belongs to the operator account the tools run as);
|
|
- `/etc/exports.d/mesh.exports`: written by its process, with a marker header;
|
|
- `/run/nfs-server/status.json`: the process's last look, for the tools;
|
|
- the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module
|
|
and for a view (ADR 0201).
|
|
|
|
## How it runs
|
|
|
|
- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, the grants,
|
|
the roster and the wants, judges each folder, writes the export file when it changed, and runs
|
|
`exportfs -ra` when the file changed or a share is missing from the kernel's table. Then it writes its
|
|
status. With no wants taken from the bus yet, it exports nothing.
|
|
- **Its tools** also take every node's `mounts` state from the bus every 30 seconds and write what is
|
|
asked of this machine to the wants file; when the bus does not answer, the last wants stay.
|
|
- **Its tools** run as the operator account. They read the status and the kernel's table, and run what
|
|
only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools.
|
|
- **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS
|
|
server is not active, the process has not looked for five minutes, a share is refused or missing from
|
|
the kernel's table, or the kernel could not read the files again.
|
|
|
|
## Tools
|
|
|
|
| tool | | answers |
|
|
|---|---|---|
|
|
| `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) |
|
|
| `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 |
|
|
| `node-nfs-server.test` | r | whether one share is exported now and the server is active; with an address, whether it is inside the range and the share is exported to it |
|
|
| `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them |
|
|
| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live to at least one node; a dry run unless `confirm` is `true` (or "true"); a person's act, with why |
|
|
| `nfs_health` | r | the module's health, one sentence per finding |
|
|
|
|
`exports` answers, for the `mounts` module:
|
|
|
|
```
|
|
{"shares": [{"name", "path", "access", "uid", "gid", "clients", "to": [{"node", "address", "access"}],
|
|
"mp", "line", "live"}], "unmatched": [...],
|
|
"refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}],
|
|
"service": "active", "range", "looked", "said", "notes"}
|
|
```
|
|
|
|
## When it is unassigned
|
|
|
|
The process writes `/etc/exports.d/mesh.exports`, not the node-engine, because each line carries the
|
|
folder's owner and the nodes' addresses as this machine reads them. The node-engine removes a module's
|
|
resources in reverse order of declaration (mesh-host `internal/apply`), and the process is declared before
|
|
its config file, so on unassignment the config file is gone when the process is stopped. The process then
|
|
removes its export file — the one removal its code makes — and runs `exportfs -ra`, so the kernel lets the
|
|
shares go. A plain restart, a push or a shutdown leaves the config file in place, and the exports stay.
|
|
|
|
## What it leaves
|
|
- **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not
|
|
a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could
|
|
reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so
|
|
a folder that owner cannot read stays closed. A share that must be strictly fenced is better given its
|
|
own dataset.
|
|
- **Exports that are not the mesh's** (`/etc/exports`, a ZFS dataset's `sharenfs`) are left as found and
|
|
shown, until `adopt` takes one over on a person's word.
|
|
|
|
## Root acts and sudo
|
|
|
|
The tools run as the operator account and act as root through `sudo -n`. The `sudo` module grants that
|
|
account every command, so **this module's code is the only line** between a tool call and root: each act
|
|
checks again what it may touch before it runs a command, and a test holds each check. Narrowing what sudo
|
|
grants the account to the commands these modules need is a separate decision, not made here.
|