jochen
48fba44471
mesh/merge-gate fail: builds new: modules/mounts, modules/nfs-server, sent nowhere; no bus step; a manifest the change touches fails the module check: modu…
mesh/repo-check fail: its merge-check.sh failed: long-running resources without health: 70
mesh/delivery-group group feat/mounts-module rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
80 lines
5.1 KiB
Markdown
80 lines
5.1 KiB
Markdown
# nfs-server
|
|
|
|
A node shares a folder with the mesh's machines, as a module (novox/hq ADR 0263, to-be 52). It holds
|
|
the node seat `node-nfs-server` and provides the provision `nfs-share`, which the `mounts` module on
|
|
another node requires.
|
|
|
|
## The rules it keeps
|
|
|
|
- **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or
|
|
`name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is
|
|
the operator's value (ADR 0112, ADR 0262).
|
|
- **To the mesh's machines only.** Every export line names the private network's range
|
|
(`${machine:mesh-range}`) and nothing else, and the module opens port 2049 to the mesh only. The home
|
|
network is never named.
|
|
- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one
|
|
id names different accounts on different machines. Every line is `all_squash` with `anonuid` and
|
|
`anongid` read from the folder's owner on this machine, so a file written from any node lands owned by
|
|
the folder's owner, and root is squashed with everyone else.
|
|
- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, or holds a
|
|
backup (restic) repository at its path or one level below is refused, said by the tools, and makes the
|
|
module unhealthy.
|
|
- **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It
|
|
never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its
|
|
own and nothing else, and a test holds it to that.
|
|
- **NFS version 4 only**, so 2049 is the only port a client needs: rpcbind and mountd stay closed.
|
|
|
|
## What it owns
|
|
|
|
- the `nfs-utils` package and `nfs-server.service`, running and enabled;
|
|
- `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on;
|
|
- `/etc/nfs-server/shares.conf`: the setting and the range, rendered by the mesh;
|
|
- `/etc/exports.d/mesh.exports`: written by its process, with a marker header;
|
|
- `/run/nfs-server/status.json`: the process's last look, for the tools;
|
|
- the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module
|
|
and for a view (ADR 0201).
|
|
|
|
## How it runs
|
|
|
|
- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, judges each
|
|
folder, writes the export file when it changed, and runs `exportfs -ra` when the file changed or a share
|
|
is missing from the kernel's table. Then it writes its status.
|
|
- **Its tools** run as the operator account. They read the status and the kernel's table, and run what
|
|
only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools.
|
|
- **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS
|
|
server is not active, the process has not looked for five minutes, a share is refused or missing from
|
|
the kernel's table, or the kernel could not read the files again.
|
|
|
|
## Tools
|
|
|
|
| tool | | answers |
|
|
|---|---|---|
|
|
| `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) |
|
|
| `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 |
|
|
| `node-nfs-server.test` | r | whether one share is exported to the range now, the server is active, and an address is inside the range |
|
|
| `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them |
|
|
| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live; a dry run without `confirm`; a person's act, with why |
|
|
| `nfs_health` | r | the module's health, one sentence per finding |
|
|
|
|
`exports` answers, for the `mounts` module:
|
|
|
|
```
|
|
{"shares": [{"name", "path", "access", "uid", "gid", "clients", "line", "live"}],
|
|
"refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}],
|
|
"service": "active", "range", "looked", "said", "notes"}
|
|
```
|
|
|
|
## What it leaves
|
|
|
|
- **The export file after unassignment.** The process writes `/etc/exports.d/mesh.exports`, not the
|
|
node-engine, because each line carries the folder's owner as this machine reads it. When the module is
|
|
unassigned, the process stops and the file stays until removed. `exports` on any later holder, and the
|
|
`mounts` module's `mounts_exports`, show it.
|
|
- **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not
|
|
a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could
|
|
reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so
|
|
a folder that owner cannot read stays closed. A share that must be strictly fenced is better given its
|
|
own dataset.
|
|
- **Exports that are not the mesh's** (`/etc/exports`, a ZFS dataset's `sharenfs`) are left as found and
|
|
shown, until `adopt` takes one over on a person's word.
|