From 8738c02d004380236b2e8c14647087329c2c67e1 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 8 Oct 2026 20:18:15 +0200 Subject: [PATCH] nfs-server: export a share only to the nodes granted and wanting it, and never root's, a symlinked or nested backup, or an unmounted dataset The review of #136 found a share could squash every client to root, be judged at a symlink's name rather than its folder, sit inside or above a backup repository deeper than one level, export an unmounted dataset's empty folder, go to every address in the private network's range, and stay exported after the module left. Each is now refused or closed, with a test that reproduced it first: - a folder owned by uid or gid 0 is refused; - the path is resolved through symlinks and every check runs on the real folder, which the export line names; - a share inside a backup repository (any parent) or holding one within four levels is refused, and a look cut short by its bound refuses; - every line carries mp, or mp=; - a share goes only to the nodes this machine's grants setting names and whose mounts module wants it on the bus, at the lower access, at the roster's address inside the range; nothing without wants; - the process is declared before its config, so on unassignment it finds the config gone, removes its export file and lets the shares go; - confirm accepts the seat's string "true". --- modules/nfs-server/README.md | 61 +++- modules/nfs-server/cmd/nfs-server/machine.go | 84 ++++- modules/nfs-server/cmd/nfs-server/main.go | 91 ++++- .../cmd/nfs-server/manifest_test.go | 46 +++ modules/nfs-server/cmd/nfs-server/nfs_test.go | 63 +++- modules/nfs-server/cmd/nfs-server/process.go | 85 ++++- .../nfs-server/cmd/nfs-server/review_test.go | 313 ++++++++++++++++ modules/nfs-server/cmd/nfs-server/shares.go | 338 ++++++++++++++++-- modules/nfs-server/cmd/nfs-server/tools.go | 107 ++++-- modules/nfs-server/module.json | 55 ++- 10 files changed, 1127 insertions(+), 116 deletions(-) create mode 100644 modules/nfs-server/cmd/nfs-server/review_test.go diff --git a/modules/nfs-server/README.md b/modules/nfs-server/README.md index f6d0b0c5..62559bb0 100644 --- a/modules/nfs-server/README.md +++ b/modules/nfs-server/README.md @@ -9,16 +9,30 @@ another node requires. - **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or `name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is the operator's value (ADR 0112, ADR 0262). -- **To the mesh's machines only.** Every export line names the private network's range - (`${machine:mesh-range}`) and nothing else, and the module opens port 2049 to the mesh only. The home - network is never named. +- **To the nodes that ask, as far as this machine allows.** A share is exported to a node only when + this machine's `grants` setting names that node for it (`share=node:rw|ro[,node:rw|ro]`, separated by + spaces; `none`, the default, grants nothing) **and** that node's `mounts` module wants it from this + machine (its state `shares` on the bus). The access is the lower of the grant and the want, and of the + share's own `:ro`. The address is the node's private address from the mesh's roster + (`/etc/nfs-server/machines`, rendered by the mesh), never one a consumer gives, and it must lie inside + the private network's range. The control node, or any node, gets a share only when both name it. A + want that is not granted, and a grant nobody wants, are said by `exports`. Port 2049 is opened to the + mesh only; the home network is never named. - **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one id names different accounts on different machines. Every line is `all_squash` with `anonuid` and `anongid` read from the folder's owner on this machine, so a file written from any node lands owned by the folder's owner, and root is squashed with everyone else. -- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, or holds a - backup (restic) repository at its path or one level below is refused, said by the tools, and makes the - module unhealthy. +- **Judged where it really is.** A share's path is resolved through every symlink, and every check runs + on the real folder, which is what the export line names. +- **Never root's.** A folder owned by root, or by root's group, is refused: squashing every client to + root would make every client root there. +- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, is inside a + backup (restic) repository (any parent up to `/`), or holds one within four levels below it is refused, + said by the tools, and makes the module unhealthy. The look below is bounded (200 000 folders); when it + is cut short, the share is refused rather than exported unchecked. +- **Only while its filesystem is mounted.** A share that is a filesystem of its own is exported with `mp`, + a folder inside one with `mp=`, so an unmounted dataset never exports the empty folder + under it. A folder of the root filesystem carries neither. - **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its own and nothing else, and a test holds it to that. @@ -28,7 +42,10 @@ another node requires. - the `nfs-utils` package and `nfs-server.service`, running and enabled; - `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on; -- `/etc/nfs-server/shares.conf`: the setting and the range, rendered by the mesh; +- `/etc/nfs-server/shares.conf`: the settings `shares` and `grants` and the range, rendered by the mesh; +- `/etc/nfs-server/machines`: every machine of the mesh and its private address, rendered by the mesh; +- `/var/lib/nfs-server/from-bus/wants.json`: what the nodes want of this machine, written by the tools + from the bus (the directory belongs to the operator account the tools run as); - `/etc/exports.d/mesh.exports`: written by its process, with a marker header; - `/run/nfs-server/status.json`: the process's last look, for the tools; - the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module @@ -36,9 +53,12 @@ another node requires. ## How it runs -- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, judges each - folder, writes the export file when it changed, and runs `exportfs -ra` when the file changed or a share - is missing from the kernel's table. Then it writes its status. +- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, the grants, + the roster and the wants, judges each folder, writes the export file when it changed, and runs + `exportfs -ra` when the file changed or a share is missing from the kernel's table. Then it writes its + status. With no wants taken from the bus yet, it exports nothing. +- **Its tools** also take every node's `mounts` state from the bus every 30 seconds and write what is + asked of this machine to the wants file; when the bus does not answer, the last wants stay. - **Its tools** run as the operator account. They read the status and the kernel's table, and run what only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools. - **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS @@ -51,25 +71,30 @@ another node requires. |---|---|---| | `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) | | `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 | -| `node-nfs-server.test` | r | whether one share is exported to the range now, the server is active, and an address is inside the range | +| `node-nfs-server.test` | r | whether one share is exported now and the server is active; with an address, whether it is inside the range and the share is exported to it | | `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them | -| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live; a dry run without `confirm`; a person's act, with why | +| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live to at least one node; a dry run unless `confirm` is `true` (or "true"); a person's act, with why | | `nfs_health` | r | the module's health, one sentence per finding | `exports` answers, for the `mounts` module: ``` -{"shares": [{"name", "path", "access", "uid", "gid", "clients", "line", "live"}], +{"shares": [{"name", "path", "access", "uid", "gid", "clients", "to": [{"node", "address", "access"}], + "mp", "line", "live"}], "unmatched": [...], "refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}], "service": "active", "range", "looked", "said", "notes"} ``` -## What it leaves +## When it is unassigned -- **The export file after unassignment.** The process writes `/etc/exports.d/mesh.exports`, not the - node-engine, because each line carries the folder's owner as this machine reads it. When the module is - unassigned, the process stops and the file stays until removed. `exports` on any later holder, and the - `mounts` module's `mounts_exports`, show it. +The process writes `/etc/exports.d/mesh.exports`, not the node-engine, because each line carries the +folder's owner and the nodes' addresses as this machine reads them. The node-engine removes a module's +resources in reverse order of declaration (mesh-host `internal/apply`), and the process is declared before +its config file, so on unassignment the config file is gone when the process is stopped. The process then +removes its export file — the one removal its code makes — and runs `exportfs -ra`, so the kernel lets the +shares go. A plain restart, a push or a shutdown leaves the config file in place, and the exports stay. + +## What it leaves - **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so diff --git a/modules/nfs-server/cmd/nfs-server/machine.go b/modules/nfs-server/cmd/nfs-server/machine.go index eb96cc60..08196cd0 100644 --- a/modules/nfs-server/cmd/nfs-server/machine.go +++ b/modules/nfs-server/cmd/nfs-server/machine.go @@ -39,6 +39,14 @@ const ( Etab = "/var/lib/nfs/etab" // Unit is the NFS server's unit. Unit = "nfs-server.service" + // RosterFile is the mesh's machines, name and private address, rendered by the mesh (a fact). + RosterFile = "/etc/nfs-server/machines" + // WantsDir is where the tools, as the operator account, write what they read on the bus. + WantsDir = "/var/lib/nfs-server/from-bus" + // WantsFile is every node's wants of this machine, from the `mounts` modules' state on the bus. + WantsFile = WantsDir + "/wants.json" + // MountInfo is the kernel's mount table, read for `mp`. + MountInfo = "/proc/self/mountinfo" ) // CommandTimeout bounds every command. None touches a shared path's contents. @@ -90,6 +98,9 @@ type Machine struct { Run Runner UID int Now func() time.Time + // WalkLimit bounds the look for a backup repository below a share. + WalkLimit int + owners map[string]Owner } // Here is the machine this process runs on. @@ -113,6 +124,9 @@ func (m *Machine) Exists(path string) bool { // Stat is a path's owner. It reads the inode and nothing else. func (m *Machine) Stat(path string) (Owner, error) { + if o, ok := m.owners[path]; ok { + return o, nil + } info, err := os.Stat(m.at(path)) if err != nil { return Owner{}, err @@ -124,6 +138,60 @@ func (m *Machine) Stat(path string) (Owner, error) { return o, nil } +// Real is a path with every symlink resolved, as the machine names it. A path that resolves outside the +// machine's root is refused. +func (m *Machine) Real(p string) (string, error) { + root, err := filepath.EvalSymlinks(m.Root) + if err != nil { + return "", err + } + full, err := filepath.EvalSymlinks(m.at(p)) + if err != nil { + return "", err + } + rel, err := filepath.Rel(root, full) + if err != nil || rel == ".." || strings.HasPrefix(rel, "../") { + return "", fmt.Errorf("%s resolves outside the machine (%s)", p, full) + } + return filepath.Clean("/" + rel), nil +} + +// MountPoints are the kernel's mount points, from its mount table; "/" alone when it cannot be read. +func (m *Machine) MountPoints() []string { + text, err := m.Read(MountInfo) + if err != nil { + return []string{"/"} + } + var out []string + for _, line := range strings.Split(text, "\n") { + f := strings.Fields(line) + if len(f) >= 5 { + out = append(out, unescapeMount(f[4])) + } + } + return out +} + +// unescapeMount undoes the kernel's octal escapes in a mount point (\040 for a space). +func unescapeMount(s string) string { + if !strings.Contains(s, "\\") { + return s + } + var b strings.Builder + for i := 0; i < len(s); i++ { + if s[i] == '\\' && i+3 < len(s) { + var v int + if _, err := fmt.Sscanf(s[i+1:i+4], "%o", &v); err == nil { + b.WriteByte(byte(v)) + i += 3 + continue + } + } + b.WriteByte(s[i]) + } + return b.String() +} + // Children are the names of a directory's subdirectories. Used only to look for a backup repository one // level below a share; it lists names and opens nothing. func (m *Machine) Children(path string) []string { @@ -155,13 +223,14 @@ func (m *Machine) Glob(pattern string) []string { } // written are the only paths Write may touch. -var written = map[string]bool{ExportFile: true, StatusFile: true} +var written = map[string]bool{ExportFile: true, StatusFile: true, WantsFile: true} // Write replaces one of the module's own files, atomically. Every other path is refused: the module owns -// these two files and nothing else on the machine. +// these three files (the export file and the status, written as root; the wants, written by the tools) +// and nothing else on the machine. func (m *Machine) Write(path, content string) error { if !written[path] { - return fmt.Errorf("the module writes only %s and %s, not %s", ExportFile, StatusFile, path) + return fmt.Errorf("the module writes only %s, %s and %s, not %s", ExportFile, StatusFile, WantsFile, path) } full := m.at(path) if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil && !errors.Is(err, fs.ErrExist) { @@ -174,6 +243,15 @@ func (m *Machine) Write(path, content string) error { return os.Rename(tmp, full) } +// unexport removes the module's own export file, and nothing else: the one removal the module makes, +// when it was unassigned (Stopped). +func (m *Machine) unexport() error { + if err := os.Remove(m.at(ExportFile)); err != nil && !errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("the export file %s could not be removed: %v", ExportFile, err) + } + return nil +} + // Escalated is a command as it is run: as given when this process is root, else through sudo without a // prompt. func (m *Machine) Escalated(name string, args ...string) (string, []string) { diff --git a/modules/nfs-server/cmd/nfs-server/main.go b/modules/nfs-server/cmd/nfs-server/main.go index 7dab8055..12315c53 100644 --- a/modules/nfs-server/cmd/nfs-server/main.go +++ b/modules/nfs-server/cmd/nfs-server/main.go @@ -11,17 +11,23 @@ package main import ( "context" + "encoding/json" "fmt" "os" "os/signal" + "sort" "syscall" "time" stdio "git.novox.be/novox/mesh-sdk/go" ) -// ExportsState is the state the exports are kept in on the bus. -const ExportsState = "exports" +// ExportsState is the state the exports are kept in on the bus; MountsState the `mounts` modules' state, +// which says what each node wants of which machine. +const ( + ExportsState = "exports" + MountsState = "mounts.shares" +) func main() { if len(os.Args) > 1 { @@ -55,6 +61,7 @@ func ShareExports(m *Machine) { } last := "" put := func() { + m.TakeWants(machine) _, err := stdio.State(ExportsState).Put(machine, m.Exports()) said := "put" if err != nil { @@ -72,3 +79,83 @@ func ShareExports(m *Machine) { put() } } + +// TakeWants reads every node's `mounts` state from the bus and writes what is asked of this machine to +// the wants file, for the root process. When the bus does not answer, the last wants stay. +func (m *Machine) TakeWants(machine string) { + state := stdio.State(MountsState) + keys, err := state.Keys() + if err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the mounts modules' state could not be listed:", err) + return + } + raw := map[string][]byte{} + for _, k := range keys { + e, err := state.Get(k) + if err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the mounts state of", k, "could not be read:", err) + return + } + if e != nil { + raw[k] = e.Value + } + } + wants, notes := WantsFromStates(machine, raw) + for _, n := range notes { + fmt.Fprintln(os.Stderr, "nfs-server:", n) + } + text, _ := json.MarshalIndent(WantsFromBus{Looked: m.Now().UTC().Format(time.RFC3339), Wants: wants}, "", " ") + have, _ := m.Read(WantsFile) + if wantsOf(have) == wantsOf(string(text)) { + return + } + if err := m.Write(WantsFile, string(text)+"\n"); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the wants could not be written:", err) + } +} + +// wantsOf is a wants file without its time, to compare two. +func wantsOf(text string) string { + var w WantsFromBus + if json.Unmarshal([]byte(text), &w) != nil { + return text + } + raw, _ := json.Marshal(w.Wants) + return string(raw) +} + +// WantsFromStates is what each node's `mounts` state asks of this machine: its wants whose server is this +// machine, keyed by the node. A state that cannot be read is said and skipped. +func WantsFromStates(machine string, states map[string][]byte) (map[string][]Want, []string) { + out := map[string][]Want{} + var notes []string + keys := make([]string, 0, len(states)) + for k := range states { + keys = append(keys, k) + } + sort.Strings(keys) + for _, node := range keys { + var v struct { + Wants []struct { + Server string `json:"server"` + Share string `json:"share"` + Access string `json:"access"` + } `json:"wants"` + } + if err := json.Unmarshal(states[node], &v); err != nil { + notes = append(notes, fmt.Sprintf("the mounts state of %s is not one this module reads: %v", node, err)) + continue + } + for _, w := range v.Wants { + if w.Server != machine || !validName(w.Share) { + continue + } + access := "rw" + if w.Access == "ro" { + access = "ro" + } + out[node] = append(out[node], Want{Share: w.Share, Access: access}) + } + } + return out, notes +} diff --git a/modules/nfs-server/cmd/nfs-server/manifest_test.go b/modules/nfs-server/cmd/nfs-server/manifest_test.go index 7df914fa..548371d3 100644 --- a/modules/nfs-server/cmd/nfs-server/manifest_test.go +++ b/modules/nfs-server/cmd/nfs-server/manifest_test.go @@ -116,3 +116,49 @@ func TestTheDefinitionNamesNoFolderNoMachineAndGivesTheSharesNoDefault(t *testin } } } + +// Review points 5 and 7: the module reads the nodes' wants, renders the mesh's roster, has grants that +// default to none, and its process is declared before its config, so the node-engine — removing a +// module's resources in reverse order — takes the config away before it stops the process, which then +// lets the shares go. +func TestTheManifestCarriesGrantsTheRosterTheWantsAndTheRemovalOrder(t *testing.T) { + m, raw := readManifest(t) + var full struct { + Reads []string `json:"reads"` + Facts map[string]map[string]string `json:"facts"` + Settings map[string]map[string]any `json:"settings"` + } + if err := json.Unmarshal([]byte(raw), &full); err != nil { + t.Fatal(err) + } + if len(full.Reads) != 1 || full.Reads[0] != MountsState { + t.Errorf("reads: %v", full.Reads) + } + roster := full.Facts["machines"] + if roster["path"] != RosterFile || !strings.Contains(roster["template"], "{{range .Machines}}{{.Name}} {{.Address}}") { + t.Errorf("the roster fact: %v", roster) + } + if g := full.Settings["grants"]; g["kind"] != "preference" || g["default"] != "none" { + t.Errorf("grants: %v", g) + } + order := map[string]int{} + var config string + for i, r := range m.Resources { + order[r["id"].(string)] = i + if r["id"] == "config" { + config, _ = r["content"].(string) + } + if r["path"] == WantsDir && r["owner"] != "${machine:account}" { + t.Errorf("the tools, as the operator account, cannot write the wants: %v", r) + } + } + if !strings.Contains(config, "grants=${setting:grants}") { + t.Errorf("the config lacks the grants: %q", config) + } + if _, ok := order["wants-dir"]; !ok { + t.Errorf("no directory for the wants") + } + if !(order["exports"] < order["config"] && order["exports"] < order["config-dir"]) { + t.Errorf("the process must be declared before its config, so unassignment removes the config first: %v", order) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/nfs_test.go b/modules/nfs-server/cmd/nfs-server/nfs_test.go index b2f32a87..b81f81af 100644 --- a/modules/nfs-server/cmd/nfs-server/nfs_test.go +++ b/modules/nfs-server/cmd/nfs-server/nfs_test.go @@ -19,6 +19,7 @@ type fake struct { ran []string now time.Time uid int + owners map[string]Owner // onExportfs, when set, writes the kernel's table as exportfs -ra would. onExportfs func() } @@ -31,7 +32,7 @@ func newFake(t *testing.T) *fake { } func (f *fake) machine() *Machine { - return &Machine{Root: f.root, UID: f.uid, Now: func() time.Time { return f.now }, Run: func(_ context.Context, name string, args ...string) Ran { + return &Machine{Root: f.root, UID: f.uid, owners: f.owners, Now: func() time.Time { return f.now }, Run: func(_ context.Context, name string, args ...string) Ran { line := strings.Join(append([]string{name}, args...), " ") f.ran = append(f.ran, line) if (line == "exportfs -ra" || line == "sudo -n exportfs -ra") && f.onExportfs != nil { @@ -82,8 +83,31 @@ func (f *fake) kernelReads() { const meshRange = "10.0.0.0/24" +// ws is the node every older test serves: granted every share and wanting it. +const wsAddress = "10.0.0.3" + +// config is a config granting every share to ws; serve writes the roster and ws's wants for those shares. func config(shares string) string { - return "# test\nshares=" + shares + "\nrange=" + meshRange + "\n" + parsed, _ := ParseShares(shares) + var grants []string + for _, s := range parsed { + grants = append(grants, s.Name+"=ws:rw") + } + g := strings.Join(grants, " ") + if g == "" { + g = "none" + } + return "# test\nshares=" + shares + "\ngrants=" + g + "\nrange=" + meshRange + "\n" +} + +// serve has ws want these shares of this machine, at the mesh's address in the roster. +func (f *fake) serve(names ...string) { + f.roster("ws " + wsAddress) + var w []Want + for _, n := range names { + w = append(w, Want{Share: n, Access: "rw"}) + } + f.wants(map[string][]Want{"ws": w}) } func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *testing.T) { @@ -91,6 +115,7 @@ func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *t f.dir("/srv/library") f.dir("/srv/downloads") f.file(ConfigFile, config("library=/srv/library downloads=/srv/downloads:ro")) + f.serve("library", "downloads") f.kernelReads() s := f.machine().Pass(context.Background(), nil) if len(s.Shares) != 2 || len(s.Refused) != 0 { @@ -99,16 +124,16 @@ func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *t uid, gid := os.Getuid(), os.Getgid() text := f.read(ExportFile) for _, want := range []string{ - fmt.Sprintf("/srv/library %s(rw,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid), - fmt.Sprintf("/srv/downloads %s(ro,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid), + fmt.Sprintf("/srv/library %s(rw,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", wsAddress, uid, gid), + fmt.Sprintf("/srv/downloads %s(ro,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", wsAddress, uid, gid), } { if !strings.Contains(text, want+"\n") { t.Errorf("the export file lacks %q:\n%s", want, text) } } for _, e := range ParseExports(text, ExportFile) { - if e.Client != meshRange { - t.Errorf("a client other than the private network's range: %+v", e) + if e.Client != wsAddress { + t.Errorf("a client other than the granted and wanting node: %+v", e) } if !strings.Contains(e.Options, "all_squash") || strings.Contains(e.Options, "no_root_squash") { t.Errorf("every client is squashed: %+v", e) @@ -137,6 +162,7 @@ func TestTheRootABackupRepositoryAndAMissingFolderAreRefusedAndNothingIsCreated( f.restic("/srv/backups") f.file("/srv/a-file", "x") f.file(ConfigFile, config("root=/ pool=/srv/pool backups=/srv/backups missing=/srv/nowhere file=/srv/a-file rel=srv/x media=/srv/pool/media")) + f.serve("root", "pool", "backups", "missing", "file", "rel", "media") f.kernelReads() s := f.machine().Pass(context.Background(), nil) refused := map[string]string{} @@ -180,8 +206,19 @@ func TestTheModuleWritesOnlyItsTwoFilesAndHasNoWayToChangeAnOwnerOrMode(t *testi continue } raw, _ := os.ReadFile(file) + text := string(raw) + // The one removal: unexport takes the module's own export file away on unassignment. + if file == "machine.go" { + start := strings.Index(text, "func (m *Machine) unexport() error {") + end := strings.Index(text[start:], "\n}\n") + body := text[start : start+end] + if strings.Count(body, "os.Remove(") != 1 || !strings.Contains(body, "os.Remove(m.at(ExportFile))") { + t.Errorf("unexport removes something other than the export file:\n%s", body) + } + text = text[:start] + text[start+end:] + } for _, word := range []string{"Chown", "Lchown", "Chmod", "os.Remove", "RemoveAll", `"chown"`, `"chmod"`, `"rm"`, "Mkdir(", "MkdirAll(filepath.Join(m.Root"} { - if strings.Contains(string(raw), word) { + if strings.Contains(text, word) { t.Errorf("%s says %s", file, word) } } @@ -224,6 +261,7 @@ func TestAdoptIsADryRunWithoutConfirmAndRefusesWhileTheMeshExportIsNotLive(t *te f.dir("/srv/library") zfsWith(f, "rw=@192.0.2.0/24") f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") // The kernel has not read the mesh's file: no exportfs effect. f.machine().Pass(context.Background(), nil) if _, err := f.machine().Adopt("/srv/library", true, "take it over"); err == nil || !strings.Contains(err.Error(), "not live") { @@ -282,9 +320,10 @@ func TestHealthSaysAStoppedServerAStaleProcessAndAShareNotInTheTable(t *testing. f := newFake(t) f.dir("/srv/library") f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") f.kernelReads() f.machine().Pass(context.Background(), nil) - if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "exports library to 10.0.0.0/24") { + if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "exports library to ws (10.0.0.3, rw)") { t.Errorf("healthy: %+v", h) } f.file(Etab, "") @@ -307,6 +346,7 @@ func TestExportsAnswersTheSharesAndWhatIsNotTheMeshs(t *testing.T) { f.file(ConfigFile, config("library=/srv/library")) f.file("/etc/exports.d/zfs.exports", "# !!! DO NOT EDIT\n/srv/library 192.0.2.0/24(sec=sys,rw,no_subtree_check)\n") f.file("/etc/exports", "# only comments\n") + f.serve("library") zfsWith(f, "rw=@192.0.2.0/24") f.kernelReads() f.machine().Pass(context.Background(), nil) @@ -324,7 +364,7 @@ func TestExportsAnswersTheSharesAndWhatIsNotTheMeshs(t *testing.T) { if err := json.Unmarshal(raw, &shape); err != nil { t.Fatal(err) } - if len(shape.Shares) != 1 || shape.Shares[0].Name != "library" || !shape.Shares[0].Live || shape.Shares[0].Clients != meshRange || shape.Service != "active" { + if len(shape.Shares) != 1 || shape.Shares[0].Name != "library" || !shape.Shares[0].Live || shape.Shares[0].Clients != wsAddress || shape.Service != "active" { t.Errorf("%s", raw) } if len(shape.Foreign) != 2 { @@ -341,6 +381,7 @@ func TestTestSaysWhetherAShareIsServedAndWhetherAnAddressIsInRange(t *testing.T) f := newFake(t) f.dir("/srv/library") f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") f.kernelReads() f.machine().Pass(context.Background(), nil) ok, _ := f.machine().Test("library", "10.0.0.3") @@ -351,6 +392,10 @@ func TestTestSaysWhetherAShareIsServedAndWhetherAnAddressIsInRange(t *testing.T) if out.OK || !strings.Contains(out.Why, "not inside the private network's range") { t.Errorf("%+v", out) } + other, _ := f.machine().Test("library", "10.0.0.9") + if other.OK || !strings.Contains(other.Why, "is not exported to 10.0.0.9") { + t.Errorf("an address in the range the share is not exported to: %+v", other) + } none, _ := f.machine().Test("photos", "") if none.OK || !strings.Contains(none.Why, "no share named photos") { t.Errorf("%+v", none) diff --git a/modules/nfs-server/cmd/nfs-server/process.go b/modules/nfs-server/cmd/nfs-server/process.go index d416029b..b28b933d 100644 --- a/modules/nfs-server/cmd/nfs-server/process.go +++ b/modules/nfs-server/cmd/nfs-server/process.go @@ -4,6 +4,8 @@ import ( "context" "encoding/json" "fmt" + "os" + "strings" "time" ) @@ -21,6 +23,9 @@ type Status struct { Shares []Exported `json:"shares"` Refused []Refused `json:"refused,omitempty"` Problems []string `json:"problems,omitempty"` + // Said is what does not match between the grants and the wants: a node wanting a share it is not + // granted, a grant nobody wants, a share exported to no node. + Said []string `json:"said,omitempty"` // Reloaded is when the process last had the kernel read the export files again, and why. Reloaded *time.Time `json:"reloaded,omitempty"` ReloadWhy string `json:"reload_why,omitempty"` @@ -40,8 +45,17 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status { c, problems := ParseConfig(text) s.Problems = append(s.Problems, problems...) s.Range = c.Range - exports, refused := m.Judge(c) + roster, note := m.Roster() + if note != "" { + s.Problems = append(s.Problems, note) + } + wants, note := m.Wants() + if note != "" { + s.Said = append(s.Said, note) + } + exports, refused, said := m.Judge(c, roster, wants) s.Refused = refused + s.Said = append(s.Said, said...) want := ExportFileText(exports) have, _ := m.Read(ExportFile) @@ -56,7 +70,7 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status { live, note := m.Live() if reload == "" { for _, e := range exports { - if !IsLive(live, e.Path, e.Clients) { + if !SharedLive(live, e) { reload = fmt.Sprintf("the share %s was not in the kernel's table", e.Name) break } @@ -75,7 +89,7 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status { s.Problems = append(s.Problems, note) } for i := range exports { - exports[i].Live = IsLive(live, exports[i].Path, exports[i].Clients) + exports[i].Live = SharedLive(live, exports[i]) } s.Shares = exports raw, _ := json.MarshalIndent(s, "", " ") @@ -85,7 +99,7 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status { return s } -// RunExports is the process: a pass now and every Every until the context ends. +// RunExports is the process: a pass now and every Every until the context ends, then Stopped. func (m *Machine) RunExports(ctx context.Context) error { last := m.Pass(ctx, nil) t := time.NewTicker(Every) @@ -93,6 +107,7 @@ func (m *Machine) RunExports(ctx context.Context) error { for { select { case <-ctx.Done(): + m.Stopped() return nil case <-t.C: last = m.Pass(ctx, &last) @@ -112,3 +127,65 @@ func (m *Machine) LastStatus() (*Status, error) { } return &s, nil } + +// Roster is the mesh's machines by name, with their private addresses, as the mesh rendered them. +func (m *Machine) Roster() (map[string]string, string) { + text, err := m.Read(RosterFile) + if err != nil { + return map[string]string{}, fmt.Sprintf("the mesh's roster (%s) could not be read: %v; no node's address is known, so nothing is exported", RosterFile, err) + } + out := map[string]string{} + for _, line := range strings.Split(text, "\n") { + f := strings.Fields(line) + if len(f) != 2 || strings.HasPrefix(f[0], "#") { + continue + } + out[f[0]] = f[1] + } + return out, "" +} + +// Wants is every node's wants of this machine, as the tools last took them from the bus. With none taken +// yet, nothing is wanted, and nothing is exported. +func (m *Machine) Wants() (map[string][]Want, string) { + text, err := m.Read(WantsFile) + if err != nil { + return map[string][]Want{}, "no node's wants have been taken from the bus yet, so nothing is exported" + } + var w WantsFromBus + if err := json.Unmarshal([]byte(text), &w); err != nil { + return map[string][]Want{}, fmt.Sprintf("the wants taken from the bus (%s) could not be read: %v; nothing is exported", WantsFile, err) + } + if w.Wants == nil { + w.Wants = map[string][]Want{} + } + return w.Wants, "" +} + +// SharedLive says whether the kernel exports a share to every node it is meant for. +func SharedLive(live []Entry, e Exported) bool { + for _, g := range e.To { + if !IsLive(live, e.Path, g.Address) { + return false + } + } + return true +} + +// Stopped is the process ending. When the module was unassigned — the node-engine removes the module's +// files in reverse declaration order, so the config file, declared after this process, is already gone +// when the process is stopped (mesh-host internal/apply: orphans removed in reverse order) — the +// module's export file goes and the kernel lets the shares go. A plain restart, a push or the machine +// shutting down leaves the config file, and the exports stay. +func (m *Machine) Stopped() { + if _, err := m.Read(ConfigFile); err == nil || !os.IsNotExist(err) { + return + } + if err := m.unexport(); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server:", err) + return + } + if _, err := m.AsRoot("exportfs", "-ra"); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the export file is gone, and exportfs -ra failed:", err) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/review_test.go b/modules/nfs-server/cmd/nfs-server/review_test.go new file mode 100644 index 00000000..80a71ebf --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/review_test.go @@ -0,0 +1,313 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" +) + +// The review of 2026-10-08 (mesh-catalog #136): each finding, reproduced. + +// roster writes the machines the mesh renders, name and address. +func (f *fake) roster(lines ...string) { + f.file(RosterFile, "# test\n"+strings.Join(lines, "\n")+"\n") +} + +// wants writes what the nodes' mounts modules want of this machine, as the tools take it from the bus. +func (f *fake) wants(w map[string][]Want) { + raw, _ := json.Marshal(WantsFromBus{Wants: w}) + f.file(WantsFile, string(raw)) +} + +// mountinfo writes the kernel's mount table with these mount points. +func (f *fake) mountinfo(points ...string) { + var b strings.Builder + for i, p := range points { + fmt.Fprintf(&b, "%d 1 0:%d / %s rw,relatime shared:1 - zfs pool%d rw\n", 20+i, 40+i, p, i) + } + f.file(MountInfo, b.String()) +} + +func (f *fake) symlink(target, at string) { + f.dir(filepath.Dir(at)) + if err := os.Symlink(target, filepath.Join(f.root, at)); err != nil { + f.t.Fatal(err) + } +} + +// grantsConfig is a config with grants. +func grantsConfig(shares, grants string) string { + return "# test\nshares=" + shares + "\ngrants=" + grants + "\nrange=" + meshRange + "\n" +} + +func refusedWhy(s Status, name string) string { + for _, r := range s.Refused { + if r.Name == name { + return r.Why + } + } + return "" +} + +// Finding 1: a share owned by root would squash every client to root. +func TestAShareWhoseOwnerIsRootIsRefused(t *testing.T) { + f := newFake(t) + f.dir("/srv/rootowned") + f.dir("/srv/rootgroup") + f.mountinfo("/") + f.owners = map[string]Owner{"/srv/rootowned": {UID: 0, GID: 2000, Dir: true}, "/srv/rootgroup": {UID: 1001, GID: 0, Dir: true}} + f.file(ConfigFile, grantsConfig("a=/srv/rootowned b=/srv/rootgroup", "a=ws:rw b=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "a", Access: "rw"}, {Share: "b", Access: "rw"}}}) + s := f.machine().Pass(context.Background(), nil) + for _, name := range []string{"a", "b"} { + if !strings.Contains(refusedWhy(s, name), "root") { + t.Errorf("%s is not refused for being root's: %+v", name, s) + } + } + if strings.Contains(f.read(ExportFile), "anonuid=0") || strings.Contains(f.read(ExportFile), "anongid=0") { + t.Errorf("an export squashes to root:\n%s", f.read(ExportFile)) + } +} + +// Finding 2: a symlink is judged where it points, and the export names the real folder. +func TestASymlinkIsJudgedAndExportedAtItsRealPath(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.restic("/srv/backups") + f.dir("/srv/real-library") + f.symlink("backups", "/srv/innocent") + f.symlink("real-library", "/srv/library") + f.file(ConfigFile, grantsConfig("innocent=/srv/innocent library=/srv/library", "innocent=ws:rw library=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "innocent", Access: "rw"}, {Share: "library", Access: "rw"}}}) + s := f.machine().Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "innocent"), "backup") { + t.Errorf("a symlink to the backups is not refused: %+v", s) + } + text := f.read(ExportFile) + if strings.Contains(text, "/srv/innocent") || strings.Contains(text, "/srv/backups") { + t.Errorf("the backups are exported through a symlink:\n%s", text) + } + if !strings.Contains(text, "\n/srv/real-library ") || strings.Contains(text, "\n/srv/library ") { + t.Errorf("the export does not name the real folder:\n%s", text) + } +} + +// Finding 3: a folder inside a backup repository, or one holding a repository deeper than one level, +// is refused; a walk cut short refuses rather than exports. +func TestABackupRepositoryAboveOrDeepBelowIsRefusedAndAnUnfinishedLookRefuses(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.restic("/srv/repo") + f.dir("/srv/repo/data/ab") + f.restic("/srv/media/films/old/backup") + f.dir("/srv/wide/a") + f.dir("/srv/wide/b") + f.dir("/srv/wide/c") + f.dir("/srv/clean/films/one") + f.file(ConfigFile, grantsConfig("inside=/srv/repo/data/ab deep=/srv/media wide=/srv/wide clean=/srv/clean", "inside=ws:rw deep=ws:rw wide=ws:rw clean=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "inside", Access: "rw"}, {Share: "deep", Access: "rw"}, {Share: "wide", Access: "rw"}, {Share: "clean", Access: "rw"}}}) + m := f.machine() + m.WalkLimit = 4 + s := m.Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "inside"), "inside the backup repository /srv/repo") { + t.Errorf("a folder inside a backup repository: %q", refusedWhy(s, "inside")) + } + if !strings.Contains(refusedWhy(s, "deep"), "/srv/media/films/old/backup") { + t.Errorf("a repository three levels down: %q", refusedWhy(s, "deep")) + } + m2 := f.machine() + m2.WalkLimit = 2 + s = m2.Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "wide"), "could not make sure") { + t.Errorf("a look cut short by its bound did not refuse: %q", refusedWhy(s, "wide")) + } + m3 := f.machine() + s = m3.Pass(context.Background(), nil) + if refusedWhy(s, "clean") != "" || refusedWhy(s, "wide") != "" { + t.Errorf("a clean folder is refused: %+v", s.Refused) + } +} + +// Finding 5: a share is exported to the nodes the server grants it to and whose mounts want it, at the +// lower of the two accesses, at the roster's address only; the control node and strangers get nothing. +func TestAShareGoesOnlyToTheNodesGrantedAndWantingItAtTheLowerAccess(t *testing.T) { + f := newFake(t) + f.mountinfo("/", "/srv/library", "/srv/pool") + f.dir("/srv/library") + f.dir("/srv/pool/downloads") + f.dir("/srv/pool/unwanted") + f.file(ConfigFile, grantsConfig("library=/srv/library downloads=/srv/pool/downloads unwanted=/srv/pool/unwanted", + "library=ws:rw,laptop:ro downloads=ws:rw unwanted=ws:rw")) + f.roster("ws 10.0.0.3", "laptop 10.0.0.4", "control 10.0.0.1", "outside 192.0.2.9") + f.wants(map[string][]Want{ + "ws": {{Share: "library", Access: "ro"}, {Share: "downloads", Access: "rw"}}, + "laptop": {{Share: "library", Access: "rw"}}, + "control": {{Share: "library", Access: "rw"}}, + "stranger": {{Share: "library", Access: "rw"}}, + "outside": {{Share: "downloads", Access: "rw"}}, + }) + f.kernelReads() + s := f.machine().Pass(context.Background(), nil) + text := f.read(ExportFile) + var lib, dl string + for _, line := range strings.Split(text, "\n") { + if strings.HasPrefix(line, "/srv/library ") { + lib = line + } + if strings.HasPrefix(line, "/srv/pool/downloads ") { + dl = line + } + } + if !strings.Contains(lib, " 10.0.0.3(ro,") || !strings.Contains(lib, " 10.0.0.4(ro,") { + t.Errorf("the library goes ro to the workstation (it wants ro) and ro to the laptop (granted ro): %q", lib) + } + if strings.Contains(text, "10.0.0.1") || strings.Contains(text, meshRange) || strings.Contains(text, "192.0.2.9") { + t.Errorf("the control node, the whole range or an address outside the range is exported to:\n%s", text) + } + if !strings.Contains(dl, " 10.0.0.3(rw,") || strings.Count(dl, "(") != 1 { + t.Errorf("downloads goes rw to the workstation only: %q", dl) + } + if strings.Contains(text, "/srv/pool/unwanted") { + t.Errorf("a share no node wants is exported:\n%s", text) + } + var said = strings.Join(s.Said, "\n") + for _, want := range []string{"control wants library", "stranger", "unwanted is granted to ws, which does not want it"} { + if !strings.Contains(said, want) { + t.Errorf("the status does not say %q:\n%s", want, said) + } + } + for _, e := range s.Shares { + if e.Name == "library" && (len(e.To) != 2 || !e.Live) { + t.Errorf("the library's clients: %+v", e) + } + } + if !strings.Contains(f.read(StatusFile), `"to"`) { + t.Errorf("the status lacks each share's clients") + } +} + +// With no wants file (the bus never answered), nothing is exported: closed, not open. +func TestWithNoWantsNothingIsExported(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.dir("/srv/library") + f.file(ConfigFile, grantsConfig("library=/srv/library", "library=ws:rw")) + f.roster("ws 10.0.0.3") + f.machine().Pass(context.Background(), nil) + if strings.Contains(f.read(ExportFile), "/srv/library") { + t.Errorf("exported with no node asking:\n%s", f.read(ExportFile)) + } +} + +// Finding 6: `mp` keeps a share unexported while its filesystem is not mounted. +func TestTheExportCarriesMpForItsFilesystem(t *testing.T) { + f := newFake(t) + f.mountinfo("/", "/srv/library", "/srv/pool") + f.dir("/srv/library") + f.dir("/srv/pool/downloads") + f.dir("/srv/plain") + f.file(ConfigFile, grantsConfig("library=/srv/library downloads=/srv/pool/downloads plain=/srv/plain", "library=ws:rw downloads=ws:rw plain=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "library", Access: "rw"}, {Share: "downloads", Access: "rw"}, {Share: "plain", Access: "rw"}}}) + f.machine().Pass(context.Background(), nil) + text := f.read(ExportFile) + if !strings.Contains(text, "/srv/library 10.0.0.3(rw,mp,") { + t.Errorf("a dataset share lacks mp:\n%s", text) + } + if !strings.Contains(text, "/srv/pool/downloads 10.0.0.3(rw,mp=/srv/pool,") { + t.Errorf("a folder in a dataset lacks mp=:\n%s", text) + } + if strings.Contains(text, "/srv/plain 10.0.0.3(rw,mp") { + t.Errorf("a folder of the root filesystem names mp:\n%s", text) + } +} + +// Finding 7: the module's export file goes when the module is unassigned, and stays on a restart. +func TestTheExportFileGoesOnUnassignmentAndStaysOnARestart(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.dir("/srv/library") + f.file(ConfigFile, grantsConfig("library=/srv/library", "library=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "library", Access: "rw"}}}) + f.kernelReads() + m := f.machine() + m.Pass(context.Background(), nil) + f.ran = nil + m.Stopped() + if !strings.Contains(f.read(ExportFile), "/srv/library") { + t.Errorf("a restart (config still there) removed the exports") + } + for _, r := range f.ran { + if strings.Contains(r, "exportfs") { + t.Errorf("a restart reloaded the exports: %v", f.ran) + } + } + if err := os.Remove(filepath.Join(f.root, ConfigFile)); err != nil { + t.Fatal(err) + } + m.Stopped() + if _, err := os.Stat(filepath.Join(f.root, ExportFile)); err == nil { + t.Errorf("the export file stays after the module's config was removed") + } + if strings.Contains(f.read(Etab), "/srv/library") { + t.Errorf("the kernel still exports the folder after unassignment: %q", f.read(Etab)) + } +} + +// Finding 9: the seat's schema passes `confirm` as a string. +func TestConfirmIsTrueAsABooleanOrTheStringTrue(t *testing.T) { + for _, c := range []struct { + v any + want bool + }{{true, true}, {"true", true}, {"TRUE", true}, {"false", false}, {"yes", false}, {nil, false}, {false, false}} { + if got := confirmed(map[string]any{"confirm": c.v}); got != c.want { + t.Errorf("confirm %#v read as %v", c.v, got) + } + } +} + +// Grants are read plainly, and "none" grants nothing. +func TestParseGrants(t *testing.T) { + g, problems := ParseGrants("library=ws:rw,laptop:ro downloads=ws:rw") + if len(problems) != 0 || g["library"]["laptop"] != "ro" || g["downloads"]["ws"] != "rw" { + t.Errorf("%v %v", g, problems) + } + g, problems = ParseGrants("none") + if len(g) != 0 || len(problems) != 0 { + t.Errorf("none: %v %v", g, problems) + } + _, problems = ParseGrants("library=ws library=x:rw bad=ws:rx") + if len(problems) != 3 { + t.Errorf("bad grants are said: %v", problems) + } +} + +// The wants are taken from each node's `mounts` state: only what is asked of this machine, keyed by the +// node that asked. +func TestWantsFromTheMountsStateKeepOnlyWhatIsAskedOfThisMachine(t *testing.T) { + states := map[string][]byte{ + "ws": []byte(`{"wants":[{"server":"home","share":"library","access":"rw"},{"server":"other","share":"x","access":"rw"}],"shares":[]}`), + "laptop": []byte(`{"wants":[{"server":"home","share":"library","access":"ro"}]}`), + "broken": []byte(`not json`), + "none": []byte(`{"shares":[]}`), + } + w, notes := WantsFromStates("home", states) + if len(w) != 2 || len(w["ws"]) != 1 || w["ws"][0].Share != "library" || w["laptop"][0].Access != "ro" { + t.Errorf("%+v", w) + } + if len(notes) != 1 || !strings.Contains(notes[0], "broken") { + t.Errorf("a state that cannot be read is said: %v", notes) + } + m := newFake(t).machine() + if err := m.Write(WantsFile, "{}"); err != nil { + t.Errorf("the tools may write the wants file: %v", err) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/shares.go b/modules/nfs-server/cmd/nfs-server/shares.go index 59016fc4..60e64913 100644 --- a/modules/nfs-server/cmd/nfs-server/shares.go +++ b/modules/nfs-server/cmd/nfs-server/shares.go @@ -21,11 +21,23 @@ import ( // written from any node lands owned by the folder's owner, and no client acts as any other account here. // Root is squashed with everyone else. // -// **To the mesh's machines only.** The clients are the private network's range and nothing else: every -// address there is a node. The home network is never named. +// **To the nodes that ask, as far as this machine allows** (review of mesh-catalog #136, point 5). A share +// is exported to a node's address only when this machine's `grants` setting names that node for it, and +// that node's `mounts` module wants it from this machine (its state `shares` on the bus, which the tools +// write to WantsFile). The access is the lower of the two: a grant of ro is never widened by a want. The +// address is the mesh's own, from the roster the mesh renders (RosterFile), never one a consumer gave, +// and it must lie in the private network's range. A share nobody is granted and wants is exported to +// nobody. The home network is never named. // -// **Never the backups.** A share that is the machine's root, that does not exist, that is not a -// directory, or that holds a backup repository at its path or one level below is refused, and said. +// **Never the backups, never root's.** A share is judged at its real path (symlinks resolved). It is +// refused when it is the machine's root, is not there, is not a folder, is owned by root or root's group +// (squashing to root would make every client root there), lies inside a backup repository, or holds one +// within WalkDepth levels below — and when that look is cut short by its bound, it is refused rather +// than exported. +// +// **Only while its filesystem is mounted.** Every line carries `mp`: a share that is a filesystem of its +// own is exported only while it is mounted, and a folder inside one only while that filesystem is +// (`mp=`). Without it, an unmounted dataset would export the empty folder under it. // Share is one share the operator asked for. type Share struct { @@ -35,16 +47,29 @@ type Share struct { Access string `json:"access"` } +// Grantee is one node a share is exported to: its name, its address on the private network, and the +// access it is given. +type Grantee struct { + Node string `json:"node"` + Address string `json:"address"` + Access string `json:"access"` +} + // Exported is a share as this machine exports it. type Exported struct { - Name string `json:"name"` - Path string `json:"path"` - Access string `json:"access"` - UID int `json:"uid"` - GID int `json:"gid"` - Clients string `json:"clients"` - Line string `json:"line"` - Live bool `json:"live"` + Name string `json:"name"` + Path string `json:"path"` + Access string `json:"access"` + UID int `json:"uid"` + GID int `json:"gid"` + // Clients is the addresses it is exported to, separated by commas; empty when no node is both granted + // it and wants it. To says who each is. + Clients string `json:"clients"` + To []Grantee `json:"to"` + // Mp is the export's mp option: "mp", "mp=", or empty for a folder of the root filesystem. + Mp string `json:"mp,omitempty"` + Line string `json:"line"` + Live bool `json:"live"` } // Refused is a share that is not exported, and why. @@ -57,10 +82,25 @@ type Refused struct { // Config is what the mesh rendered from the assignment. type Config struct { Shares []Share + // Grants is, per share, the nodes this machine lets mount it and with which access. + Grants map[string]map[string]string Range string } -// ParseConfig reads the config file: `shares=` and `range=` lines, comments and blanks ignored. +// Want is one share a node's mounts module wants of this machine, as it put it on the bus. +type Want struct { + Share string `json:"share"` + Access string `json:"access"` +} + +// WantsFromBus is the file the tools write from the bus: every node's wants of this machine. +type WantsFromBus struct { + Looked string `json:"looked,omitempty"` + Wants map[string][]Want `json:"wants"` +} + +// ParseConfig reads the config file: `shares=`, `grants=` and `range=` lines, comments and blanks +// ignored. func ParseConfig(text string) (Config, []string) { var c Config var problems []string @@ -79,6 +119,10 @@ func ParseConfig(text string) (Config, []string) { shares, bad := ParseShares(value) c.Shares = shares problems = append(problems, bad...) + case "grants": + grants, bad := ParseGrants(value) + c.Grants = grants + problems = append(problems, bad...) case "range": c.Range = strings.TrimSpace(value) default: @@ -126,6 +170,45 @@ func ParseShares(value string) ([]Share, []string) { return shares, problems } +// ParseGrants reads the `grants` setting: `share=node:rw|ro[,node:rw|ro]`, entries separated by spaces; +// `none` grants nothing. +func ParseGrants(value string) (map[string]map[string]string, []string) { + out := map[string]map[string]string{} + seen := map[string]bool{} + var problems []string + value = strings.TrimSpace(value) + if value == "" || value == "none" { + return out, nil + } + for _, item := range strings.Fields(value) { + share, list, ok := strings.Cut(item, "=") + if !ok || !validName(share) || list == "" { + problems = append(problems, fmt.Sprintf("the grant %q is not share=node:rw|ro[,node:rw|ro]", item)) + continue + } + if seen[share] { + problems = append(problems, fmt.Sprintf("the share %q is granted twice; one entry per share", share)) + continue + } + seen[share] = true + nodes := map[string]string{} + good := true + for _, g := range strings.Split(list, ",") { + node, access, ok := strings.Cut(g, ":") + if !ok || !validName(node) || (access != "rw" && access != "ro") { + problems = append(problems, fmt.Sprintf("the grant %q of share %s is not node:rw or node:ro", g, share)) + good = false + break + } + nodes[node] = access + } + if good { + out[share] = nodes + } + } + return out, problems +} + func validName(n string) bool { if n == "" || n[0] == '-' { return false @@ -141,7 +224,7 @@ func validName(n string) bool { // IsBackupRepository says whether a directory is a restic repository: a `config` file beside `keys`, // `snapshots` and `data`. It looks at names only. func (m *Machine) IsBackupRepository(dir string) bool { - if !m.Exists(path.Join(dir, "config")) { + if o, err := m.Stat(path.Join(dir, "config")); err != nil || o.Dir { return false } for _, sub := range []string{"keys", "snapshots", "data"} { @@ -152,10 +235,83 @@ func (m *Machine) IsBackupRepository(dir string) bool { return true } -// Judge turns the asked-for shares into export lines, or refusals. -func (m *Machine) Judge(c Config) ([]Exported, []Refused) { +// WalkDepth is how many levels below a share are looked through for a backup repository; WalkLimit how +// many folders at most, after which the share is refused rather than exported unchecked. +const ( + WalkDepth = 4 + WalkLimit = 200000 +) + +// repositoryBelow finds a backup repository within WalkDepth levels below dir. complete is false when +// the look stopped at its bound before it saw everything. +func (m *Machine) repositoryBelow(dir string) (found string, complete bool) { + limit := m.WalkLimit + if limit <= 0 { + limit = WalkLimit + } + seen := 0 + level := []string{dir} + for depth := 1; depth <= WalkDepth && len(level) > 0; depth++ { + var next []string + for _, d := range level { + for _, child := range m.Children(d) { + seen++ + if seen > limit { + return "", false + } + p := path.Join(d, child) + if m.IsBackupRepository(p) { + return p, true + } + next = append(next, p) + } + } + level = next + } + return "", true +} + +// mpFor is the export's mp option for a real path: `mp` when it is a mount point of its own, `mp=` when that is not the root filesystem, else nothing. +func mpFor(points []string, real string) string { + best := "" + for _, p := range points { + if p == real { + return "mp" + } + if p != "/" && strings.HasPrefix(real, p+"/") && len(p) > len(best) { + best = p + } + } + if best == "" { + return "" + } + return "mp=" + best +} + +// lower is the narrower of two accesses: ro wins. +func lower(a, b string) string { + if a == "ro" || b == "ro" { + return "ro" + } + return "rw" +} + +// Judge turns the asked-for shares into export lines, or refusals, for the nodes granted and wanting +// each; what does not match is said. +func (m *Machine) Judge(c Config, roster map[string]string, wants map[string][]Want) ([]Exported, []Refused, []string) { var out []Exported var refused []Refused + var said []string + var prefix netip.Prefix + if c.Range != "" { + prefix, _ = netip.ParsePrefix(c.Range) + } + points := m.MountPoints() + shares := map[string]bool{} + for _, s := range c.Shares { + shares[s.Name] = true + } for _, s := range c.Shares { no := func(why string) { refused = append(refused, Refused{Name: s.Name, Path: s.Path, Why: why}) } clean := path.Clean(s.Path) @@ -163,62 +319,170 @@ func (m *Machine) Judge(c Config) ([]Exported, []Refused) { case !path.IsAbs(s.Path): no(fmt.Sprintf("The share %s names %q, which is not an absolute path.", s.Name, s.Path)) continue - case clean == "/": - no(fmt.Sprintf("The share %s is the machine's root, which is never exported.", s.Name)) - continue case strings.ContainsAny(s.Path, " \t\"\\#"): no(fmt.Sprintf("The share %s's path %q has a character an export line cannot carry.", s.Name, s.Path)) continue } - o, err := m.Stat(clean) + real, err := m.Real(clean) if err != nil { no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, clean, err)) continue } + via := "" + if real != clean { + via = fmt.Sprintf(" (%s is a symlink to %s)", clean, real) + } + if real == "/" { + no(fmt.Sprintf("The share %s is the machine's root%s, which is never exported.", s.Name, via)) + continue + } + if strings.ContainsAny(real, " \t\"\\#") { + no(fmt.Sprintf("The share %s's real path %q has a character an export line cannot carry.", s.Name, real)) + continue + } + o, err := m.Stat(real) + if err != nil { + no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, real, err)) + continue + } if !o.Dir { - no(fmt.Sprintf("The share %s's path %s is not a folder.", s.Name, clean)) + no(fmt.Sprintf("The share %s's path %s%s is not a folder.", s.Name, real, via)) continue } - if m.IsBackupRepository(clean) { - no(fmt.Sprintf("The share %s's folder %s is a backup repository, and backups are never exported.", s.Name, clean)) + if o.UID == 0 || o.GID == 0 { + no(fmt.Sprintf("The share %s's folder %s%s is owned by root (%d:%d), and every client would act as root there: give it an owner of its own.", s.Name, real, via, o.UID, o.GID)) continue } - holds := "" - for _, child := range m.Children(clean) { - if m.IsBackupRepository(path.Join(clean, child)) { - holds = path.Join(clean, child) + if m.IsBackupRepository(real) { + no(fmt.Sprintf("The share %s's folder %s%s is a backup repository, and backups are never exported.", s.Name, real, via)) + continue + } + inside := "" + for p := path.Dir(real); ; p = path.Dir(p) { + if m.IsBackupRepository(p) { + inside = p + break + } + if p == "/" { break } } + if inside != "" { + no(fmt.Sprintf("The share %s's folder %s%s is inside the backup repository %s, and backups are never exported.", s.Name, real, via, inside)) + continue + } + holds, complete := m.repositoryBelow(real) if holds != "" { - no(fmt.Sprintf("The share %s's folder %s holds a backup repository (%s), and backups are never exported: share the folders beside it instead.", s.Name, clean, holds)) + no(fmt.Sprintf("The share %s's folder %s%s holds a backup repository (%s), and backups are never exported: share the folders beside it instead.", s.Name, real, via, holds)) + continue + } + if !complete { + no(fmt.Sprintf("The share %s's folder %s%s holds more folders than the module looks through (%d levels, at most %d folders), so it could not make sure no backup repository is under it: share smaller folders.", s.Name, real, via, WalkDepth, m.limit())) continue } if c.Range == "" { no(fmt.Sprintf("The share %s is not exported: the private network's range is not known.", s.Name)) continue } - e := Exported{Name: s.Name, Path: clean, Access: s.Access, UID: o.UID, GID: o.GID, Clients: c.Range} + e := Exported{Name: s.Name, Path: real, Access: s.Access, UID: o.UID, GID: o.GID, Mp: mpFor(points, real), To: []Grantee{}} + granted := c.Grants[s.Name] + nodes := make([]string, 0, len(wants)) + for n := range wants { + nodes = append(nodes, n) + } + sort.Strings(nodes) + wanting := map[string]bool{} + for _, node := range nodes { + for _, w := range wants[node] { + if w.Share != s.Name { + continue + } + wanting[node] = true + g, ok := granted[node] + if !ok { + said = append(said, fmt.Sprintf("%s wants %s, and this machine's grants do not name it, so it is not exported to it", node, s.Name)) + continue + } + addr, known := roster[node] + if !known { + said = append(said, fmt.Sprintf("%s wants %s, and the mesh's roster names no such machine, so it is not exported to it", node, s.Name)) + continue + } + a, err := netip.ParseAddr(addr) + if err != nil || !prefix.Contains(a) { + said = append(said, fmt.Sprintf("%s's address %s is not inside the private network's range %s, so %s is not exported to it", node, addr, c.Range, s.Name)) + continue + } + access := lower(lower(g, w.Access), s.Access) + e.To = append(e.To, Grantee{Node: node, Address: addr, Access: access}) + } + } + grantedNodes := make([]string, 0, len(granted)) + for n := range granted { + if !wanting[n] { + grantedNodes = append(grantedNodes, n) + } + } + sort.Strings(grantedNodes) + for _, n := range grantedNodes { + said = append(said, fmt.Sprintf("%s is granted to %s, which does not want it", s.Name, n)) + } + var addrs []string + for _, g := range e.To { + addrs = append(addrs, g.Address) + } + e.Clients = strings.Join(addrs, ",") e.Line = ExportLine(e) + if len(e.To) == 0 { + said = append(said, fmt.Sprintf("%s is exported to no node: none is both granted it and wants it", s.Name)) + } out = append(out, e) } - return out, refused + for share := range c.Grants { + if !shares[share] { + said = append(said, fmt.Sprintf("the grants name %s, which is not one of this machine's shares", share)) + } + } + sort.Strings(said) + return out, refused, said } -// ExportLine is one share's line in the export file. +func (m *Machine) limit() int { + if m.WalkLimit > 0 { + return m.WalkLimit + } + return WalkLimit +} + +// ExportLine is one share's line in the export file, or empty when it goes to no node. func ExportLine(e Exported) string { - return fmt.Sprintf("%s %s(%s,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", - e.Path, e.Clients, e.Access, e.UID, e.GID) + if len(e.To) == 0 { + return "" + } + var b strings.Builder + b.WriteString(e.Path) + for _, g := range e.To { + opts := g.Access + if e.Mp != "" { + opts += "," + e.Mp + } + fmt.Fprintf(&b, " %s(%s,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", g.Address, opts, e.UID, e.GID) + } + return b.String() } // ExportFileText is the whole export file. func ExportFileText(exports []Exported) string { var b strings.Builder - b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered from its assignment's\n") - b.WriteString("# `shares` setting by the module's own process; edit the setting, never this file.\n") - b.WriteString("# One share per folder, to the private network's range only, every client squashed to the\n") - b.WriteString("# folder's owner.\n") + b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered by the module's own process\n") + b.WriteString("# from its assignment's `shares` and `grants` and the nodes that want each share; edit the settings,\n") + b.WriteString("# never this file. One share per folder, to each granted and wanting node's private address only,\n") + b.WriteString("# every client squashed to the folder's owner.\n") for _, e := range exports { + if e.Line == "" { + fmt.Fprintf(&b, "# share %s: exported to no node\n", e.Name) + continue + } fmt.Fprintf(&b, "# share %s\n%s\n", e.Name, e.Line) } return b.String() diff --git a/modules/nfs-server/cmd/nfs-server/tools.go b/modules/nfs-server/cmd/nfs-server/tools.go index ba6e33f9..114fce40 100644 --- a/modules/nfs-server/cmd/nfs-server/tools.go +++ b/modules/nfs-server/cmd/nfs-server/tools.go @@ -34,16 +34,26 @@ func arg(a map[string]any, k string) string { } func flag(a map[string]any, k string) bool { - v, _ := a[k].(bool) - return v + switch v := a[k].(type) { + case bool: + return v + case string: + return strings.EqualFold(strings.TrimSpace(v), "true") + } + return false } +// confirmed says whether an act was confirmed: the boolean true, or the string "true" the seat's schema +// passes. +func confirmed(a map[string]any) bool { return flag(a, "confirm") } + // Tools are the seat's verbs and the module's health. func Tools(m *Machine) []stdio.Tool { return []stdio.Tool{ verb("exports", "Every share this machine exports for the mesh: its name, folder, rw or ro, the owner every "+ - "client is mapped to (uid, gid), the clients (the private network's range, nothing else) and whether the "+ - "kernel holds it now; the shares the assignment asked for that were refused, and why; and every export "+ + "client is mapped to (uid, gid), the nodes it is exported to (each granted by this machine and wanting it, "+ + "at its private address, with its access) and whether the kernel holds it now; what does not match "+ + "between the grants and the wants; the shares the assignment asked for that were refused, and why; and every export "+ "on the machine that is not the mesh's (/etc/exports, the other files in /etc/exports.d, a ZFS dataset's "+ "sharenfs). Replaces exportfs -v, cat /etc/exports and zfs get sharenfs. (r)", map[string]any{}, func(map[string]any) (any, error) { return m.Exports(), nil }), @@ -52,8 +62,8 @@ func Tools(m *Machine) []stdio.Tool { "to port 2049. Replaces cat /proc/fs/nfsd/clients/*/info and ss -tn sport = :2049. (r)", map[string]any{}, func(map[string]any) (any, error) { return m.Clients(), nil }), verb("test", "Whether this machine exports one share for the mesh now: the share is asked for and not "+ - "refused, the kernel holds it for the private network's range, and the NFS server is active; with an "+ - "address, also whether that address is inside the range. Replaces showmount -e. (r)", + "refused, the kernel holds it for every node it is exported to, and the NFS server is active; with an "+ + "address, also whether that address is inside the range and the share is exported to it. Replaces showmount -e. (r)", map[string]any{"share": str("the share's name"), "address": str("a client's address on the private network (optional)")}, func(a map[string]any) (any, error) { return m.Test(arg(a, "share"), arg(a, "address")) }), verb("reload", "Have the kernel read the export files again now (exportfs -ra), and answer the mesh's shares "+ @@ -62,15 +72,15 @@ func Tools(m *Machine) []stdio.Tool { map[string]any{}, func(map[string]any) (any, error) { return m.Reload() }), verb("adopt", "Take over the export a ZFS dataset's sharenfs property makes for a folder the mesh now "+ "exports: sets sharenfs=off on that one dataset and nothing else, then exportfs -ra. Only when the "+ - "mesh's own export of the same folder is live in the kernel, so the folder is never unexported; "+ - "without confirm, a dry run that says what it would do. A person's act, with why. Replaces zfs set "+ + "mesh's own export of the same folder is live in the kernel to at least one node, so the folder is never "+ + "unexported for everyone; unless confirm is true, a dry run that says what it would do. A person's act, with why. Replaces zfs set "+ "sharenfs=off. (a)", map[string]any{ "path": str("the shared folder, as the share names it"), - "confirm": map[string]any{"type": "boolean", "description": "do it; without, a dry run"}, + "confirm": str("\"true\" to do it; anything else, a dry run"), "why": str("why, in a sentence (required with confirm)"), }, - func(a map[string]any) (any, error) { return m.Adopt(arg(a, "path"), flag(a, "confirm"), arg(a, "why")) }), + func(a map[string]any) (any, error) { return m.Adopt(arg(a, "path"), confirmed(a), arg(a, "why")) }), { Name: "nfs_health", Description: "Whether this machine's shares are served, as the module's health check (novox/hq ADR 0240, " + @@ -89,9 +99,11 @@ type ExportsAnswer struct { Foreign []Entry `json:"foreign"` Service string `json:"service"` Range string `json:"range,omitempty"` - Looked string `json:"looked,omitempty"` - Said []string `json:"said"` - Notes []string `json:"notes,omitempty"` + // Unmatched is what does not match between this machine's grants and the nodes' wants. + Unmatched []string `json:"unmatched,omitempty"` + Looked string `json:"looked,omitempty"` + Said []string `json:"said"` + Notes []string `json:"notes,omitempty"` } // Exports is the machine's exports now. @@ -109,11 +121,12 @@ func (m *Machine) Exports() ExportsAnswer { } for _, e := range st.Shares { if note == "" { - e.Live = IsLive(live, e.Path, e.Clients) + e.Live = SharedLive(live, e) } a.Shares = append(a.Shares, e) } a.Notes = append(a.Notes, st.Problems...) + a.Unmatched = st.Said } entries, sets, notes := m.Foreign() a.Notes = append(a.Notes, notes...) @@ -122,12 +135,17 @@ func (m *Machine) Exports() ExportsAnswer { a.Foreign = append(a.Foreign, Entry{Path: d.Mountpoint, Client: d.ShareNFS, From: "zfs sharenfs on " + d.Name}) } for _, e := range a.Shares { + if len(e.To) == 0 { + a.Said = append(a.Said, fmt.Sprintf("%s: %s, exported to no node", e.Name, e.Path)) + continue + } state := "live" if !e.Live { state = "NOT in the kernel's table" } - a.Said = append(a.Said, fmt.Sprintf("%s: %s %s to %s, every client as %d:%d — %s", e.Name, e.Path, e.Access, e.Clients, e.UID, e.GID, state)) + a.Said = append(a.Said, fmt.Sprintf("%s: %s to %s, every client as %d:%d — %s", e.Name, e.Path, grantees(e.To), e.UID, e.GID, state)) } + a.Said = append(a.Said, a.Unmatched...) for _, r := range a.Refused { a.Said = append(a.Said, "refused: "+r.Why) } @@ -258,26 +276,41 @@ func (m *Machine) Test(share, address string) (TestAnswer, error) { } a.Path, a.Access = found.Path, found.Access live, note := m.Live() - a.Live = note == "" && IsLive(live, found.Path, found.Clients) + a.Live = note == "" && SharedLive(live, *found) var why []string if address != "" { addr, err1 := netip.ParseAddr(address) - prefix, err2 := netip.ParsePrefix(found.Clients) + prefix, err2 := netip.ParsePrefix(st.Range) in := err1 == nil && err2 == nil && prefix.Contains(addr) a.InRange = &in if !in { - why = append(why, fmt.Sprintf("%s is not inside the private network's range %s.", address, found.Clients)) + why = append(why, fmt.Sprintf("%s is not inside the private network's range %s.", address, st.Range)) + } else { + to := false + for _, g := range found.To { + if g.Address == address { + to = true + a.Access = g.Access + } + } + if !to { + why = append(why, fmt.Sprintf("The share %s is not exported to %s: no node there is both granted it and wants it.", share, address)) + } else if note != "" || !IsLive(live, found.Path, address) { + why = append(why, fmt.Sprintf("The kernel does not export %s to %s now.", found.Path, address)) + } } } - if !a.Live { - why = append(why, fmt.Sprintf("The kernel does not export %s to %s now.", found.Path, found.Clients)) + if len(found.To) == 0 { + why = append(why, fmt.Sprintf("The share %s is exported to no node.", share)) + } else if !a.Live { + why = append(why, fmt.Sprintf("The kernel does not export %s to every node it is meant for (%s) now.", found.Path, grantees(found.To))) } if a.Service != "active" { why = append(why, "The NFS server is "+a.Service+".") } a.OK = len(why) == 0 if a.OK { - a.Why = fmt.Sprintf("The share %s (%s, %s) is exported to %s and the NFS server is active.", share, found.Path, found.Access, found.Clients) + a.Why = fmt.Sprintf("The share %s (%s) is exported to %s and the NFS server is active.", share, found.Path, grantees(found.To)) } else { a.Why = strings.Join(why, " ") } @@ -339,12 +372,15 @@ func (m *Machine) Adopt(p string, confirm bool, why string) (AdoptAnswer, error) if mine == nil { return a, fmt.Errorf("the mesh does not export %s: add it to this machine's nfs-server `shares` setting first, so the folder is never unexported", p) } + if len(mine.To) == 0 { + return a, fmt.Errorf("the mesh exports %s to no node yet (no node is both granted it and wants it), so taking over would unexport it for everyone", p) + } live, lnote := m.Live() - if lnote != "" || !IsLive(live, p, mine.Clients) { - return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, mine.Clients) + if lnote != "" || !SharedLive(live, *mine) { + return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, grantees(mine.To)) } plan := fmt.Sprintf("set sharenfs=off on the dataset %s (now %q) and run exportfs -ra; %s stays exported as the share %s to %s, every client as %d:%d", - ds.Name, ds.ShareNFS, p, mine.Name, mine.Clients, mine.UID, mine.GID) + ds.Name, ds.ShareNFS, p, mine.Name, grantees(mine.To), mine.UID, mine.GID) if !confirm { a.Said = "Dry run: would " + plan + "." return a, nil @@ -396,7 +432,7 @@ func (m *Machine) Health() Health { } live, note := m.Live() for _, e := range st.Shares { - if note == "" && !IsLive(live, e.Path, e.Clients) { + if note == "" && !SharedLive(live, e) { found = append(found, fmt.Sprintf("The share %s (%s) is not in the kernel's export table.", e.Name, e.Path)) } } @@ -404,14 +440,16 @@ func (m *Machine) Health() Health { if len(found) > 0 { return Health{Healthy: false, Why: strings.Join(found, " "), Found: found} } - names := make([]string, 0, len(st.Shares)) + var parts []string for _, e := range st.Shares { - names = append(names, e.Name) + if len(e.To) > 0 { + parts = append(parts, fmt.Sprintf("%s to %s", e.Name, grantees(e.To))) + } } - if len(names) == 0 { - return Health{Healthy: true, Why: "The NFS server is active, and the mesh exports nothing here."} + if len(parts) == 0 { + return Health{Healthy: true, Why: "The NFS server is active, and the mesh exports nothing to any node here."} } - return Health{Healthy: true, Why: fmt.Sprintf("The NFS server is active and exports %s to %s.", strings.Join(names, ", "), st.Range)} + return Health{Healthy: true, Why: fmt.Sprintf("The NFS server is active and exports %s.", strings.Join(parts, "; "))} } func sentence(s string) string { @@ -425,3 +463,12 @@ func sentence(s string) string { } return s } + +// grantees says who a share goes to: `ws (10.0.0.3, rw), laptop (10.0.0.4, ro)`. +func grantees(to []Grantee) string { + var out []string + for _, g := range to { + out = append(out, fmt.Sprintf("%s (%s, %s)", g.Node, g.Address, g.Access)) + } + return strings.Join(out, ", ") +} diff --git a/modules/nfs-server/module.json b/modules/nfs-server/module.json index 780e09ba..f36e133c 100644 --- a/modules/nfs-server/module.json +++ b/modules/nfs-server/module.json @@ -44,6 +44,22 @@ "per-machine": true } ], + "reads": [ + "mounts.shares" + ], + "settings": { + "grants": { + "kind": "preference", + "default": "none", + "why": "which nodes may mount which share, and how, is this machine's to say (hq ADR 0263, review of mesh-catalog #136): share=node:rw|ro[,node:rw|ro], entries separated by spaces; none exports to no node, whatever a node wants" + } + }, + "facts": { + "machines": { + "path": "/etc/nfs-server/machines", + "template": "# Written by the mesh (module nfs-server, novox/hq ADR 0263): every machine of the mesh and its private\n# address, from which the module takes a node's address. Replaced on every push.\n{{range .Machines}}{{.Name}} {{.Address}}\n{{end}}" + } + }, "tools": [ "nfs_health" ], @@ -70,19 +86,6 @@ "mode": "0644", "content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263). Replaced on every push; a drop-in of\n# the operator's that sorts after this one overrides it, and is theirs.\n#\n# NFS version 4 only: a client needs port 2049 and nothing else, so the module opens nothing more\n# than that, to the private network. Version 3 needs rpcbind and mountd, on ports the mesh does not open.\n[nfsd]\nvers2=n\nvers3=n\nvers4=y\nvers4.0=n\nvers4.1=y\nvers4.2=y\n" }, - { - "id": "config-dir", - "type": "directory", - "path": "/etc/nfs-server", - "mode": "0755" - }, - { - "id": "config", - "type": "file", - "path": "/etc/nfs-server/shares.conf", - "mode": "0644", - "content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The module's process exports each\n# to the private network's range below, every client mapped to the folder's owner.\nshares=${setting:shares}\nrange=${machine:mesh-range}\n" - }, { "id": "run-dir", "type": "directory", @@ -102,6 +105,19 @@ "kind": "unit" } }, + { + "id": "wants-dir-parent", + "type": "directory", + "path": "/var/lib/nfs-server", + "mode": "0755" + }, + { + "id": "wants-dir", + "type": "directory", + "path": "/var/lib/nfs-server/from-bus", + "mode": "0755", + "owner": "${machine:account}" + }, { "id": "exports", "type": "process", @@ -122,6 +138,19 @@ "looks": 2, "grace": "90s" } + }, + { + "id": "config-dir", + "type": "directory", + "path": "/etc/nfs-server", + "mode": "0755" + }, + { + "id": "config", + "type": "file", + "path": "/etc/nfs-server/shares.conf", + "mode": "0644", + "content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The grants: share=node:rw|ro[,…],\n# which nodes may mount each and how. The module's process exports a share to a node's private address\n# only when it is granted here and that node's mounts module wants it, every client mapped to the\n# folder's owner, and only to addresses inside the range below.\nshares=${setting:shares}\ngrants=${setting:grants}\nrange=${machine:mesh-range}\n" } ], "build": {