diff --git a/modules/mounts/README.md b/modules/mounts/README.md new file mode 100644 index 00000000..0e1a1911 --- /dev/null +++ b/modules/mounts/README.md @@ -0,0 +1,176 @@ +# mounts + +The shares and the occasional sources a machine mounts, and what it mounts and shares, read as tools +(novox/hq ADR 0263, to-be design 52 "Shares and mounts", from research 038). It holds the node seat +`node-mounts`, one holder per machine. + +A machine mounts only what its assignment names (ADR 0263 rule 5): nothing by default, and "every +machine" is the same setting given once at the mesh-wide layer. A share of another machine comes from the +provision `nfs-share`, which `nfs-server` provides on the machine that shares. + +## What it owns + +- The packages `nfs-utils` and `cifs-utils`. +- `/etc/mesh-mounts/mounts.conf`, placed by the mesh from the assignment: the settings `shares` and + `sources`, the binding of `nfs-share` (`from`, the sharing machine, and `at`, where it is reached on + the private network), the operator account, and where the module's secret is. +- One `.mount` and one `.automount` unit per share or source, in `/etc/systemd/system`, named as + `systemd-escape --path` names the mount point and opened by the marker + `# Written by the mesh (module mounts, novox/hq ADR 0263).` A unit file without that marker is not the + module's: it is never written over, stopped or removed, and the share whose mount point it holds is + said as not set up. +- The SMB logins, one file per source in the module's state directory, readable by root only. +- `/var/lib/mesh-mounts/from-bus` (the bundle's: what the sharing machine exports) and `/run/mesh-mounts` + (the processes': what was applied and what was found). + +What it never does: chown or chmod a mount point (it makes one that is missing, and nothing else), touch a +mounted filesystem to look at it, or remove a pair because the sharing machine did not answer. A pair is +removed only when the settings stop naming its mount point. The files a share serves are the operator's +(ADR 0051): the module mounts them and owns nothing in them. + +## The unit pair + +Only the automount is enabled (under `remote-fs.target`; a disk's under `local-fs.target`). The mount +is never enabled: it is made at the first use of its mount point and released after ten idle minutes +(`TimeoutIdleSec=600`). So nothing mounts at boot and nothing can fail a boot, and a mount point is always +an autofs mount, so nothing writes into the empty directory under it. + +| kind | What | Options | TimeoutSec | +|---|---|---|---| +| a node's share, read-write | `:`, `Type=nfs4` | `vers=4.2,hard,timeo=150,retrans=3,_netdev` | 30 | +| a node's share, read-only | `:`, `Type=nfs4` | `ro,vers=4.2,soft,timeo=150,retrans=3,_netdev` | 30 | +| an SMB source | `///`, `Type=cifs` | `credentials=,iocharset=utf8,_netdev,nofail,uid=…,gid=…` | 10 | +| a disk | `/dev/disk/by-uuid/`; the mount `BindsTo=` the disk's device | `nofail` | 10 | + +A share is read-only when the setting says `:ro` or the sharing machine exports it read-only. A +read-write share is mounted hard, so a write is never dropped while the server is away. + +## Settings + +Both are preferences whose default, `none`, mounts nothing (ADR 0262 allows no empty default). + +- `shares`: space-separated `name=mountpoint`, each optionally `:ro`. The name is the share's on the + sharing machine; its path there comes from that machine over the bus, never from this setting. + For example `library=/srv/library downloads=/srv/downloads:ro`. +- `sources`: space-separated occasional sources, each optionally `:ro`: + `name=smb:///@` or `name=disk:@`. + For example `games=smb://192.0.2.76/Share@/mnt/games`. + +A mount point is an absolute, plainly written path, not a directory of the system's own (`/etc`, `/usr`, +`/run`, …). An entry that cannot be read is said, and only that entry is left out. + +## The secret + +`smb-credentials` is the module's own secret, issued by an outside party (the device's owner), so the mesh +never makes one in its place. The operator gives it with +`secret accept mounts smb-credentials --from `; the file holds one section per SMB source: + +``` +[games] +username=… +password=… +domain=… (optional) +``` + +A source whose section is missing is not mounted, and is said with that remedy. No tool answers a +password: the options, fstab lines and mountinfo the tools read are redacted. + +## The processes and the bus + +- `apply` (root, judged by its unit) writes and removes the unit pairs and the logins, makes a missing + mount point, reloads the service manager when anything changed and arms every automount, every 30 s. +- `watch` (root, judged by `mounts_health`) tries each source without touching its mount point — TCP 2049 + for a node's share, 445 for an SMB source, the device for a disk — and says each one's state. +- The bundle (the operator account) reads what the sharing machine exports from `nfs-server`'s state + `exports`, keyed by that machine, and leaves it for `apply`; a failed read keeps the last answer. It puts + what `watch` found in the module's state `shares`, keyed by this machine, with `wants`: each share this + machine asks of which server and whether read-write or read-only (`[{"server", "share", "access"}]`). + The server exports a share only to the machines that want it and that it grants, and decides the access + itself: `apply` mounts a share at the access the server's answer gives this machine (`to[]`), read-only + when either the share or this machine's grant is, and not at all when the server does not export it + here (said as failed, naming the grant that is missing). Every five minutes the bundle also asks the controller's `data` verb where the mesh keeps data on + this machine, and leaves the paths for `apply`; a failed ask keeps the last paths. No module reads + another's files (ADR 0260). + +## What it refuses + +Each refusal is said as the share's state `failed`, with why, and nothing is written for it: + +- a mount point at, under or above a path where the mesh keeps data on this machine (the controller's + `data` verb: Plex's library and configuration, a backup repository, an agent's home). Until that list + has been read once, nothing new is set up; a pair the module already has stays; +- a mount point the system owns (`/`, `/etc`, `/usr`, `/boot`, `/proc`, `/sys`, `/dev`, `/run` and what + is under them, and `/var`, `/home`, `/root`, `/tmp` themselves); +- a mount point that something else is mounted at, or an automount that is not the module's; +- a folder that holds files: a mount would hide them. Only a folder not in the kernel's mount table is + opened, so a server that is away cannot hold the look; +- a share that this machine serves itself. + +And what it never does: + +- remove a pair while an entry of `shares` or `sources` cannot be read: nobody can tell which mount point + the mistyped entry meant, so every pair stays until the settings read again; +- stop or remove a pair whose mount point has a filesystem mounted that is not the pair's own source; the + verb `unmount` refuses the same. + +The states: `armed` (the automount waits), `mounted`, `absent` (an occasional source that is not there — +a state, not a fault), `unreachable` (should be there and is not), `failed` (tried and refused, or not +set up; `why` says which). + +## Health + +Unhealthy when a share that should always be there has been unreachable or failed for five minutes ("The +share library of is not reachable from this machine since 19:21."), when an automount is not +armed, when something the settings name cannot be set up (a missing login, a unit file that is not the +module's), or when `watch` has not looked for two minutes. An occasional source that is absent is +healthy, and said as not present since when. + +## Taking over a hand-written line + +The operator's own fstab lines for a share are taken over only through `adopt`, on their word, and only +once the module's own pair for the same mount point is written and its automount armed: the replacement +works first, then the line goes. Without `confirm` it is a dry run. With it (and `why`), the line is +commented out as `# adopted by the mesh (module mounts) on : `, +the file as it was is kept as `/etc/fstab.mesh-adopted-` readable by root only, and the +service manager reloads. The module never takes a line over on its own. + +## What it reads + +- `/etc/fstab`, every line but swap. A credential in the options (`password=`, `pass=`, `passwd=`) is + answered as `(redacted)`; a `credentials=` file is named and never opened. +- The service manager's mount and automount units, with `systemctl list-units` and `systemctl show`: + state, result, timeout, and whether each was generated from fstab or comes from a unit file. The + kernel's own filesystems (proc, tmpfs, mqueue and the like) are left out. +- `/proc/self/mountinfo`, for what is mounted now. A mount point is never opened or statted, so a + network filesystem whose server is away cannot hang a call. +- `/etc/exports`, `/etc/exports.d/*.exports` and the kernel's live export table `/var/lib/nfs/etab`. +- The ZFS `sharenfs` and `sharesmb` properties, where `zfs` is installed. +- The share sections of `/etc/samba/smb.conf`: path, valid users, read only or writable, force user and + the like. The `[global]` section and every other setting are left out, so no password or password + backend is answered. + +A file that cannot be read, and a command that does not answer, is said under `notes`, never answered +as empty. + +## Tools + +What only root may do (`mount`, `unmount`, `adopt`) goes through `sudo -n`, as the packet filter's and the +service manager's tools do; a refusal by sudo is said as one. + +| tool | | answers | +|---|---|---| +| `mounts_list` | r | every mount point, its fstab line, its mount and automount units and whether it is mounted now; network filesystems marked; a network mount without automount or nofail said. Narrowed with `network` and `match` | +| `mounts_exports` | r | the NFS exports declared and live, the ZFS share properties, the Samba shares, and whether nfs-server and smb are active; an export declared and not live, or live and declared nowhere, said | +| `mounts_health` | r | the health check: healthy or not, with why, one sentence per finding | +| `node-mounts.list` | r | the machine's mount points, and every share and source the module is asked to mount with its state, since when and why | +| `node-mounts.test` | r | whether one share's or source's server answers now, tried without touching its mount point | +| `node-mounts.mount` | a | mount one share or source now | +| `node-mounts.unmount` | a | release one now; its automount stays armed | +| `node-mounts.adopt` | a | take over one hand-written fstab line, as above | + +## Root acts and sudo + +The tools run as the operator account and act as root through `sudo -n`. The `sudo` module grants that +account every command, so **this module's code is the only line** between a tool call and root: each act +checks again what it may touch before it runs a command, and a test holds each check. Narrowing what sudo +grants the account to the commands these modules need is a separate decision, not made here. diff --git a/modules/mounts/cmd/mounts/adopt.go b/modules/mounts/cmd/mounts/adopt.go new file mode 100644 index 00000000..b0ec8e37 --- /dev/null +++ b/modules/mounts/cmd/mounts/adopt.go @@ -0,0 +1,201 @@ +package main + +// Taking over a hand-written fstab line (novox/hq ADR 0263 rule 10, to-be 52 §5). +// +// The line is the operator's, so it is taken over only on a person's word (`confirm`, with why) and only +// once the module's own unit pair for the same mount point exists and its automount is armed: the +// replacement works first, then the line goes. It is commented out, not deleted, with a dated marker; a +// password in it is redacted in the commented line, and the whole file as it was is kept beside it, +// readable by root only, because it holds that password. + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "regexp" + "strings" + "time" +) + +// AdoptedPrefix opens a line the module took over. +const AdoptedPrefix = "# adopted by the mesh (module mounts) on " + +// A password's value runs to the next single comma or blank: mount.cifs writes a comma inside it as two. +var passwordValue = regexp.MustCompile(`(?i)\b(password2?|pass|passwd)=(?:,,+[^,\s]*|[^,\s])*`) + +// userPassword is user=name%password, where mount.cifs takes the password after the percent sign. +var userPassword = regexp.MustCompile(`(?i)\b(user(?:name)?=[^,\s%]*)%(?:,,+[^,\s]*|[^,\s])*`) + +// RedactLine replaces every credential's value in a raw fstab line. +func RedactLine(line string) string { + line = passwordValue.ReplaceAllString(line, "${1}="+Redacted) + return userPassword.ReplaceAllString(line, "${1}%"+Redacted) +} + +// Adoption is what taking a line over does, or would do. +type Adoption struct { + Mountpoint string `json:"mountpoint"` + Line int `json:"line"` + Was string `json:"was"` + Becomes string `json:"becomes"` + Copy string `json:"copy,omitempty"` + Done bool `json:"done"` + Said string `json:"said"` +} + +// PlanAdoption finds the one uncommented fstab line for a mount point and what it becomes. `was` is +// redacted; the new text keeps every other line byte for byte. +func PlanAdoption(fstab, mountpoint string, now time.Time) (string, Adoption, error) { + lines := strings.Split(fstab, "\n") + hit := -1 + for i, raw := range lines { + t := strings.TrimSpace(raw) + if t == "" || strings.HasPrefix(t, "#") { + continue + } + f := strings.Fields(t) + if len(f) >= 2 && unescape(f[1]) == mountpoint { + if hit >= 0 { + return "", Adoption{}, fmt.Errorf("/etc/fstab has more than one line for %s (lines %d and %d); take it over by hand", mountpoint, hit+1, i+1) + } + hit = i + } + } + if hit < 0 { + return "", Adoption{}, fmt.Errorf("/etc/fstab has no line for %s", mountpoint) + } + becomes := AdoptedPrefix + now.Format("2006-01-02") + ": " + RedactLine(lines[hit]) + a := Adoption{Mountpoint: mountpoint, Line: hit + 1, Was: RedactLine(lines[hit]), Becomes: becomes} + lines[hit] = becomes + return strings.Join(lines, "\n"), a, nil +} + +// replacementArmed says whether the module's own pair for a mount point is written and its automount +// armed, with why not. +func (m *Machine) replacementArmed(ctx context.Context, mountpoint string) error { + base := EscapePath(mountpoint) + for _, unit := range []string{base + ".mount", base + ".automount"} { + if !IsOwnUnit(m, filepath.Join(UnitDir, unit)) { + return fmt.Errorf("the module has no unit pair for %s yet (%s is not its own): it takes a line over only once its replacement is in place", mountpoint, unit) + } + } + shown, err := m.Run(ctx, "systemctl", "show", "-p", "ActiveState", "--value", base+".automount") + if err != nil { + return fmt.Errorf("the service manager did not say whether %s.automount is armed: %v", base, err) + } + if strings.TrimSpace(shown) != "active" { + return fmt.Errorf("%s.automount is not armed (%s): the replacement does not work yet, so the line stays", base, strings.TrimSpace(shown)) + } + return nil +} + +// AdoptDryRun is what taking the line over would do, changing nothing. +func (m *Machine) AdoptDryRun(ctx context.Context, mountpoint string, now time.Time) (Adoption, error) { + if err := CheckMountpoint(mountpoint); err != nil { + return Adoption{}, err + } + fstab, err := m.read("/etc/fstab") + if err != nil { + return Adoption{}, errors.New(unreadable("/etc/fstab", err)) + } + _, a, err := PlanAdoption(fstab, mountpoint, now) + if err != nil { + return Adoption{}, err + } + if err := m.replacementArmed(ctx, mountpoint); err != nil { + a.Said = "Would be refused now: " + err.Error() + return a, nil + } + a.Said = fmt.Sprintf("A dry run: line %d of /etc/fstab would be commented out and a copy of the file kept beside it. Nothing changed; call again with confirm and why.", a.Line) + return a, nil +} + +// AdoptAsRoot takes the line over: the process's half, run as root through sudo by the tool. It checks +// everything again, since the tool's look was the operator account's. +func (m *Machine) AdoptAsRoot(ctx context.Context, mountpoint string, now time.Time) (Adoption, error) { + if err := CheckMountpoint(mountpoint); err != nil { + return Adoption{}, err + } + if err := m.replacementArmed(ctx, mountpoint); err != nil { + return Adoption{}, err + } + full := filepath.Join(m.Root, "etc/fstab") + raw, err := os.ReadFile(full) + if err != nil { + return Adoption{}, errors.New(unreadable("/etc/fstab", err)) + } + info, err := os.Stat(full) + if err != nil { + return Adoption{}, err + } + next, a, err := PlanAdoption(string(raw), mountpoint, now) + if err != nil { + return Adoption{}, err + } + a.Copy = "/etc/fstab.mesh-adopted-" + now.Format("20060102-150405") + if err := os.WriteFile(filepath.Join(m.Root, a.Copy), raw, 0o600); err != nil { + return Adoption{}, fmt.Errorf("the copy of /etc/fstab was not kept, so nothing changed: %w", err) + } + tmp := full + ".mesh-new" + if err := os.WriteFile(tmp, []byte(next), info.Mode().Perm()); err != nil { + return Adoption{}, fmt.Errorf("/etc/fstab was not written, so nothing changed: %w", err) + } + if err := os.Rename(tmp, full); err != nil { + os.Remove(tmp) + return Adoption{}, fmt.Errorf("/etc/fstab was not written, so nothing changed: %w", err) + } + a.Done = true + a.Said = fmt.Sprintf("Line %d of /etc/fstab, for %s, is commented out; the file as it was is kept at %s (root only, as it may hold a password).", a.Line, mountpoint, a.Copy) + if _, err := m.Run(ctx, "systemctl", "daemon-reload"); err != nil { + a.Said += " The service manager did not reload: " + err.Error() + } + return a, nil +} + +// Adopt is the verb: a dry run without confirm; with it, the root half through sudo. +func (m *Machine) Adopt(ctx context.Context, mountpoint string, confirm bool, why string, now time.Time) (Adoption, error) { + if !confirm { + return m.AdoptDryRun(ctx, mountpoint, now) + } + if strings.TrimSpace(why) == "" { + return Adoption{}, errors.New("say why the line is taken over (`why`): it is the operator's line, and the act is recorded") + } + self, err := os.Executable() + if err != nil { + return Adoption{}, err + } + out, err := m.Escalate(ctx, self, "adopt", mountpoint) + var a Adoption + if jerr := json.Unmarshal([]byte(strings.TrimSpace(out)), &a); jerr != nil { + if err != nil { + return Adoption{}, err + } + return Adoption{}, fmt.Errorf("the root half answered no adoption: %v", jerr) + } + if !a.Done && a.Said != "" { + return a, errors.New(a.Said) + } + a.Said += " Why: " + strings.TrimSpace(why) + "." + return a, nil +} + +// RunAdopt is `mounts adopt `, as root: the answer on stdout as JSON, also when refused. +func RunAdopt(args []string) int { + if len(args) != 1 { + fmt.Fprintln(os.Stderr, "usage: mounts adopt ") + return 2 + } + a, err := Here().AdoptAsRoot(context.Background(), args[0], time.Now()) + if err != nil { + a = Adoption{Mountpoint: args[0], Said: err.Error()} + } + raw, _ := json.Marshal(a) + fmt.Println(string(raw)) + if err != nil { + return 1 + } + return 0 +} diff --git a/modules/mounts/cmd/mounts/apply.go b/modules/mounts/cmd/mounts/apply.go new file mode 100644 index 00000000..c2df2630 --- /dev/null +++ b/modules/mounts/cmd/mounts/apply.go @@ -0,0 +1,585 @@ +package main + +// The module's two long-running processes, run by the node-engine as root (novox/hq ADR 0263, to-be 52 §2), +// over one pass of the same code every 30 seconds: +// +// - `mounts apply` acts: it writes or removes the module's own unit pairs and the SMB logins, makes a +// missing mount point and arms the automounts, and leaves what it could not do in AppliedFile. The +// service manager judges it by its unit; +// - `mounts watch` judges: it tries each source without touching a mount point and leaves what it found +// in StatusFile, which the health check `mounts_health`, the tools and the bundle read. +// +// Neither reaches the bus: the bundle reads what the sharing machine exports and leaves it in ExportsFile, +// and puts StatusFile in the module's state. +// +// What it never does: write, stop or remove a unit file without the module's marker; chown or chmod a +// mount point (it makes one that is missing, and nothing else); remove a pair because the sharing machine +// did not answer — a pair goes only when the settings stop naming it. + +import ( + "context" + "encoding/json" + "fmt" + "net" + "os" + "os/user" + "path/filepath" + "sort" + "strconv" + "strings" + "time" +) + +// LookEvery is how often the process looks. +const LookEvery = 30 * time.Second + +// ProbeTimeout bounds one try of a source. +const ProbeTimeout = 2 * time.Second + +// The states of a share (to-be 52 §2). +const ( + StateArmed = "armed" // the automount waits for the first use + StateMounted = "mounted" // mounted now + StateAbsent = "absent" // an occasional source that is not there: a state, not a fault + StateUnreachable = "unreachable" // should be there and is not + StateFailed = "failed" // tried and refused, or not set up; why says which +) + +// ExportsAnswer is what the bundle left from the sharing machine's `node-nfs-server.exports`. +type ExportsAnswer struct { + Server string `json:"server"` + At string `json:"at,omitempty"` + Asked string `json:"asked"` + Shares []ServerShare `json:"shares"` + // Error is why the last ask got no answer; the shares are then the last answer's. + Error string `json:"error,omitempty"` + Answered string `json:"answered,omitempty"` +} + +// ShareStatus is one share or source as the process found it. +type ShareStatus struct { + Name string `json:"name"` + Kind string `json:"kind"` + Occasional bool `json:"occasional"` + Mountpoint string `json:"mountpoint"` + What string `json:"what,omitempty"` + Probe string `json:"probe,omitempty"` + State string `json:"state"` + Since string `json:"since"` + Why string `json:"why,omitempty"` + MountUnit string `json:"mount_unit,omitempty"` + AutoUnit string `json:"automount_unit,omitempty"` +} + +// Status is what one look found. +type Status struct { + Looked string `json:"looked"` + Server string `json:"server,omitempty"` + Shares []ShareStatus `json:"shares"` + Problems []string `json:"problems,omitempty"` + Changed []string `json:"changed,omitempty"` +} + +// Applier is the process, over one machine. Act writes and arms; Judge tries and says. A test does both. +type Applier struct { + M *Machine + Act bool + Judge bool + // Dial tries a host:port; Now is the clock; Lookup finds an account's uid and gid. + Dial func(addr string, timeout time.Duration) error + Now func() time.Time + Lookup func(account string) (int, int, error) +} + +// NewApplier is the process on this machine. +func NewApplier() *Applier { + return &Applier{ + M: Here(), + Dial: func(addr string, timeout time.Duration) error { + c, err := net.DialTimeout("tcp", addr, timeout) + if err == nil { + c.Close() + } + return err + }, + Now: time.Now, + Lookup: func(account string) (int, int, error) { + u, err := user.Lookup(account) + if err != nil { + return -1, -1, err + } + uid, _ := strconv.Atoi(u.Uid) + gid, _ := strconv.Atoi(u.Gid) + return uid, gid, nil + }, + } +} + +func (a *Applier) path(p string) string { return filepath.Join(a.M.Root, p) } + +// writeIfChanged writes a file through a temporary beside it, only when its content differs. +func (a *Applier) writeIfChanged(p, content string, mode os.FileMode) (bool, error) { + full := a.path(p) + if old, err := os.ReadFile(full); err == nil && string(old) == content { + if info, err := os.Stat(full); err == nil && info.Mode().Perm() == mode { + return false, nil + } + } + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + return false, err + } + tmp := full + ".mesh-new" + if err := os.WriteFile(tmp, []byte(content), mode); err != nil { + return false, err + } + if err := os.Chmod(tmp, mode); err != nil { + os.Remove(tmp) + return false, err + } + return true, os.Rename(tmp, full) +} + +// ownUnits are the unit files in UnitDir that carry the module's marker, by name. +func (a *Applier) ownUnits() map[string]bool { + out := map[string]bool{} + for _, pattern := range []string{"*.mount", "*.automount"} { + for _, p := range a.M.glob(filepath.Join(UnitDir, pattern)) { + if IsOwnUnit(a.M, p) { + out[filepath.Base(p)] = true + } + } + } + return out +} + +// IsOwnUnit says whether a unit file opens with the module's marker. +func IsOwnUnit(m *Machine, p string) bool { + text, err := m.read(p) + return err == nil && strings.HasPrefix(text, Marker) +} + +// unitStates are the service manager's states for some units: ActiveState, SubState, Result and +// UnitFileState, by unit. +func (a *Applier) unitStates(ctx context.Context, units []string) map[string]map[string]string { + out := map[string]map[string]string{} + if len(units) == 0 { + return out + } + args := append([]string{"show", "-p", "Id,ActiveState,SubState,Result,UnitFileState", "--no-pager"}, units...) + shown, _ := a.M.Run(ctx, "systemctl", args...) + for _, block := range strings.Split(shown, "\n\n") { + kv := map[string]string{} + for _, line := range strings.Split(block, "\n") { + if k, v, ok := strings.Cut(line, "="); ok { + kv[k] = v + } + } + if kv["Id"] != "" { + out[kv["Id"]] = kv + } + } + return out +} + +func readJSON[T any](m *Machine, p string) (T, error) { + var v T + text, err := m.read(p) + if err != nil { + return v, err + } + return v, json.Unmarshal([]byte(text), &v) +} + +// Look is one pass: the units written as asked, the automounts armed, and each share found. +func (a *Applier) Look(ctx context.Context) Status { + now := a.Now() + st := Status{Looked: now.Format(time.RFC3339), Shares: []ShareStatus{}} + previous, _ := readJSON[Status](a.M, StatusFile) + since := map[string]ShareStatus{} + for _, s := range previous.Shares { + since[s.Name] = s + } + + text, err := a.M.read(ConfigFile) + if err != nil { + st.Problems = append(st.Problems, unreadable(ConfigFile, err)+"; nothing is written or removed until the mesh places it") + a.finish(&st, nil) + return st + } + cfg := ParseConfig(text) + st.Server = cfg.ServerNode() + st.Problems = append(st.Problems, cfg.Problems...) + + // Where the mesh keeps data on this machine (finding 4d). Without it nothing new is set up: a pair the + // module already has stays, and one it does not have waits until the list is read. + data, derr := readJSON[DataPaths](a.M, DataFile) + haveData := derr == nil && data.Paths != nil + if !haveData { + why := "it has not been left by the module's tools yet" + if derr == nil && data.Error != "" { + why = data.Error + } + st.Problems = append(st.Problems, "where the mesh keeps data on this machine is not known ("+why+"); nothing new is set up until it is") + } + + exports, eerr := readJSON[ExportsAnswer](a.M, ExportsFile) + byName := map[string]ServerShare{} + for _, e := range exports.Shares { + byName[e.Name] = e + } + + var pairs []Pair + notSet := map[string]ShareStatus{} // a share named but not set up, with why + for _, w := range cfg.Shares { + e, ok := byName[w.Name] + switch { + case cfg.At == "": + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateFailed, + Why: "no machine shares nfs-share for this one: the binding gives no address"} + case cfg.Node != "" && st.Server == cfg.Node: + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateFailed, + Why: "this machine shares it itself (" + cfg.Node + "): a share is never mounted onto the machine that serves it"} + case !ok && eerr != nil: + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateUnreachable, + Why: "what the sharing machine " + st.Server + " exports has not been read from the bus yet"} + case !ok: + why := "the sharing machine " + exports.Server + " does not export a share named " + w.Name + if exports.Error != "" { + why = "what the sharing machine " + st.Server + " exports could not be read from the bus: " + exports.Error + } + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateFailed, Why: why} + case !strings.HasPrefix(e.Path, "/") || strings.ContainsAny(e.Path, " \t\n\\\"'"): + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateFailed, + Why: fmt.Sprintf("the sharing machine gave the path %q, which the module does not mount", e.Path)} + default: + mine, exported := e.ForNode(cfg.Node) + if !exported { + notSet[w.Name] = ShareStatus{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, State: StateFailed, + Why: "the sharing machine " + st.Server + " does not export " + w.Name + " to this machine: its grants do not name " + cfg.Node} + continue + } + pairs = append(pairs, NFSPair(w, cfg.At, mine)) + } + } + + var creds map[string]SMBCredential + needCreds := false + for _, s := range cfg.Sources { + needCreds = needCreds || s.Kind == "smb" + } + if needCreds { + secret, err := a.M.read(cfg.Secret) + if cfg.Secret == "" || err != nil { + st.Problems = append(st.Problems, "the module's secret smb-credentials is not on this machine") + } else { + var p []string + creds, p = ParseSMBCredentials(secret) + for _, x := range p { + st.Problems = append(st.Problems, "smb-credentials: "+x) + } + } + } + uid, gid := -1, -1 + if cfg.Account != "" && a.Lookup != nil { + if u, g, err := a.Lookup(cfg.Account); err == nil { + uid, gid = u, g + } + } + for _, s := range cfg.Sources { + switch s.Kind { + case "disk": + pairs = append(pairs, DiskPair(s)) + case "smb": + c, ok := creds[s.Name] + if !ok || cfg.State == "" { + notSet[s.Name] = ShareStatus{Name: s.Name, Kind: "smb", Occasional: true, Mountpoint: s.Mountpoint, State: StateFailed, + Why: fmt.Sprintf("its login is not given: put a [%s] section in the module's secret with `secret accept mounts smb-credentials --from `", s.Name)} + continue + } + file := filepath.Join(cfg.State, "smb-"+s.Name+".credentials") + changed, err := false, error(nil) + if a.Act { + changed, err = a.writeIfChanged(file, c.CredentialsFile(), 0o600) + } + if err != nil { + notSet[s.Name] = ShareStatus{Name: s.Name, Kind: "smb", Occasional: true, Mountpoint: s.Mountpoint, State: StateFailed, + Why: "its credentials file could not be written: " + err.Error()} + continue + } + if changed { + st.Changed = append(st.Changed, "the login of "+s.Name) + } + pairs = append(pairs, SMBPair(s, file, uid, gid)) + } + } + + // Every mount point the settings still name, set up or not: a pair is removed only when its mount + // point is no longer named, never because an answer is missing. + named := map[string]bool{} + for _, w := range cfg.Shares { + named[EscapePath(w.Mountpoint)] = true + } + for _, s := range cfg.Sources { + named[EscapePath(s.Mountpoint)] = true + } + + own := a.ownUnits() + mounted := ParseMountinfo(func() string { t, _ := a.M.read("/proc/self/mountinfo"); return t }()) + changed := false + var written []Pair + for _, p := range pairs { + ours := own[p.MountUnit] && own[p.AutoUnit] + refuse := func(why string) { + notSet[p.Name] = ShareStatus{Name: p.Name, Kind: p.Kind, Occasional: p.Occasional, Mountpoint: p.Mountpoint, What: p.What, + State: StateFailed, Why: why} + } + if haveData { + if err := nearData(p.Mountpoint, data.Paths); err != nil { + refuse(err.Error()) + continue + } + } else if !ours { + refuse("where the mesh keeps data on this machine is not known yet (the controller's data verb), so a new mount point is not set up") + continue + } + if err := a.M.mountpointFree(p, own, mounted); err != nil { + refuse(err.Error()) + continue + } + if err := a.foreign(p, own); err != nil { + notSet[p.Name] = ShareStatus{Name: p.Name, Kind: p.Kind, Occasional: p.Occasional, Mountpoint: p.Mountpoint, What: p.What, + State: StateFailed, Why: err.Error()} + continue + } + if !a.Act { + written = append(written, p) + continue + } + c1, err1 := a.writeIfChanged(filepath.Join(UnitDir, p.MountUnit), p.Mount, 0o644) + c2, err2 := a.writeIfChanged(filepath.Join(UnitDir, p.AutoUnit), p.Automount, 0o644) + if err1 != nil || err2 != nil { + notSet[p.Name] = ShareStatus{Name: p.Name, Kind: p.Kind, Occasional: p.Occasional, Mountpoint: p.Mountpoint, What: p.What, + State: StateFailed, Why: fmt.Sprintf("its units could not be written: %v %v", err1, err2)} + continue + } + if c1 || c2 { + changed = true + st.Changed = append(st.Changed, "the units of "+p.Name) + } + // A missing mount point is made; one that is there is left exactly as it is. An automount point + // is in mountinfo and is never looked at, so a server that is away cannot hold this look. + if _, there := mounted[p.Mountpoint]; !there { + if _, err := os.Lstat(a.path(p.Mountpoint)); os.IsNotExist(err) { + if err := os.MkdirAll(a.path(p.Mountpoint), 0o755); err == nil { + st.Changed = append(st.Changed, "made the mount point "+p.Mountpoint) + } + } + } + written = append(written, p) + } + + // The module's own pairs whose mount point nothing names any more: stopped, disabled and removed — + // unless an entry of the settings could not be read, when nobody can tell which mount point it named + // (finding 8), or what is mounted there is not the pair's own source (finding 4c). + var gone []string + if len(cfg.Unreadable) > 0 { + if a.Act { + st.Problems = append(st.Problems, "nothing is removed while an entry of shares or sources cannot be read: "+strings.Join(cfg.Unreadable, "; ")) + } + } else { + kept := map[string]bool{} + for unit := range own { + base := strings.TrimSuffix(strings.TrimSuffix(unit, ".automount"), ".mount") + if named[base] || !a.Act || kept[base] { + continue + } + text, _ := a.M.read(filepath.Join(UnitDir, base+".mount")) + what, where := unitField(text, "What"), unitField(text, "Where") + if where != "" { + if err := a.M.notOwnSource(kindOf(what, unitField(text, "Type")), what, where, mounted); err != nil { + kept[base] = true + st.Problems = append(st.Problems, fmt.Sprintf("the pair %s is no longer named and is kept, and nothing is stopped: %v", base, err)) + continue + } + } + gone = append(gone, unit) + } + var still []string + for _, unit := range gone { + if !kept[strings.TrimSuffix(strings.TrimSuffix(unit, ".automount"), ".mount")] { + still = append(still, unit) + } + } + gone = still + } + sort.Strings(gone) + for _, unit := range gone { + if strings.HasSuffix(unit, ".automount") { + a.M.Run(ctx, "systemctl", "disable", "--now", unit) + } else { + a.M.Run(ctx, "systemctl", "stop", unit) + } + } + for _, unit := range gone { + if err := os.Remove(a.path(filepath.Join(UnitDir, unit))); err == nil { + changed = true + st.Changed = append(st.Changed, "removed "+unit+", which nothing names any more") + } + } + if changed { + if _, err := a.M.Run(ctx, "systemctl", "daemon-reload"); err != nil { + st.Problems = append(st.Problems, "the service manager did not reload: "+err.Error()) + } + } + + // Arm every automount that is not. + var units []string + for _, p := range written { + units = append(units, p.MountUnit, p.AutoUnit) + } + states := a.unitStates(ctx, units) + for _, p := range written { + auto := states[p.AutoUnit] + if a.Act && (auto["UnitFileState"] != "enabled" || auto["ActiveState"] != "active") { + if _, err := a.M.Run(ctx, "systemctl", "enable", "--now", p.AutoUnit); err != nil { + st.Problems = append(st.Problems, fmt.Sprintf("%s was not armed: %v", p.AutoUnit, err)) + } else { + st.Changed = append(st.Changed, "armed "+p.AutoUnit) + } + } + } + if !a.Judge { + a.finish(&st, notSet) + return st + } + if len(written) > 0 { + states = a.unitStates(ctx, units) + } + // What the acting process could not do, which only it knows: a unit or a login it could not write. + if applied, err := readJSON[Applied](a.M, AppliedFile); err == nil && !a.Act { + st.Problems = append(st.Problems, applied.Problems...) + for _, s := range applied.NotSet { + if _, said := notSet[s.Name]; !said { + notSet[s.Name] = s + } + } + } + + for _, p := range written { + if _, failed := notSet[p.Name]; failed { + continue + } + s := ShareStatus{Name: p.Name, Kind: p.Kind, Occasional: p.Occasional, Mountpoint: p.Mountpoint, What: p.What, + Probe: p.Probe, MountUnit: p.MountUnit, AutoUnit: p.AutoUnit} + s.State, s.Why = a.judge(p, states, mounted) + st.Shares = append(st.Shares, s) + } + for _, s := range notSet { + st.Shares = append(st.Shares, s) + } + sort.Slice(st.Shares, func(i, j int) bool { return st.Shares[i].Name < st.Shares[j].Name }) + for i := range st.Shares { + s := &st.Shares[i] + if prev, ok := since[s.Name]; ok && prev.State == s.State && prev.Since != "" { + s.Since = prev.Since + } else { + s.Since = st.Looked + } + } + a.finish(&st, notSet) + return st +} + +// AppliedFile is what the acting process could not do, for the judging one. +const AppliedFile = RunDir + "/applied.json" + +// Applied is the acting process's last pass. +type Applied struct { + Looked string `json:"looked"` + Problems []string `json:"problems,omitempty"` + NotSet []ShareStatus `json:"not_set,omitempty"` + Changed []string `json:"changed,omitempty"` +} + +// finish leaves what this pass found: the acting process's account, the judging one's status. +func (a *Applier) finish(st *Status, notSet map[string]ShareStatus) { + if a.Act && !a.Judge { + ap := Applied{Looked: st.Looked, Problems: st.Problems, Changed: st.Changed} + for _, s := range notSet { + ap.NotSet = append(ap.NotSet, s) + } + sort.Slice(ap.NotSet, func(i, j int) bool { return ap.NotSet[i].Name < ap.NotSet[j].Name }) + raw, _ := json.MarshalIndent(ap, "", " ") + if _, err := a.writeIfChanged(AppliedFile, string(raw)+"\n", 0o644); err != nil { + fmt.Fprintln(os.Stderr, "mounts: what was applied was not written:", err) + } + } + if a.Judge { + a.writeStatus(st) + } +} + +// foreign refuses a pair whose unit file exists and is not the module's. +func (a *Applier) foreign(p Pair, own map[string]bool) error { + for _, unit := range []string{p.MountUnit, p.AutoUnit} { + if own[unit] { + continue + } + if _, err := os.Lstat(a.path(filepath.Join(UnitDir, unit))); err == nil { + return fmt.Errorf("%s already has a unit file that is not the module's (%s); it is left as it is, and this share is not mounted here", p.Mountpoint, filepath.Join(UnitDir, unit)) + } + } + return nil +} + +// judge is one pair's state. The source is tried first, without touching the mount point. +func (a *Applier) judge(p Pair, states map[string]map[string]string, mounted map[string]Mounted) (string, string) { + there := true + var tried error + if p.Kind == "disk" { + if !a.M.present(p.Probe) { + there, tried = false, fmt.Errorf("the disk %s is not plugged in", p.Probe) + } + } else if err := a.Dial(p.Probe, ProbeTimeout); err != nil { + there, tried = false, err + } + mnt, auto := states[p.MountUnit], states[p.AutoUnit] + cur, isMounted := mounted[p.Mountpoint] + isMounted = isMounted && cur.Type != "autofs" + switch { + case !there && p.Occasional: + return StateAbsent, "not present: " + tried.Error() + case !there: + return StateUnreachable, fmt.Sprintf("%s does not answer: %v", p.Probe, tried) + case isMounted: + return StateMounted, "" + case mnt["ActiveState"] == "failed": + return StateFailed, fmt.Sprintf("its last mount failed (%s); see the journal of %s", orDefault(mnt["Result"], "failed"), p.MountUnit) + case auto["ActiveState"] == "active": + return StateArmed, "" + default: + return StateFailed, fmt.Sprintf("its automount %s is not armed (%s)", p.AutoUnit, orDefault(auto["ActiveState"], "unknown")) + } +} + +func (a *Applier) writeStatus(st *Status) { + raw, _ := json.MarshalIndent(st, "", " ") + if _, err := a.writeIfChanged(StatusFile, string(raw)+"\n", 0o644); err != nil { + fmt.Fprintln(os.Stderr, "mounts: the status was not written:", err) + } +} + +// RunApply looks every LookEvery until the context ends, saying on stderr what it changed. +func RunApply(ctx context.Context, a *Applier) { + for { + st := a.Look(ctx) + for _, c := range st.Changed { + fmt.Fprintln(os.Stderr, "mounts:", c) + } + select { + case <-ctx.Done(): + return + case <-time.After(LookEvery): + } + } +} diff --git a/modules/mounts/cmd/mounts/bus.go b/modules/mounts/cmd/mounts/bus.go new file mode 100644 index 00000000..ce3d0057 --- /dev/null +++ b/modules/mounts/cmd/mounts/bus.go @@ -0,0 +1,156 @@ +package main + +// The bundle's half on the bus (novox/hq ADR 0263 rule 9, ADR 0201). The bundle runs as the operator +// account and reaches the bus through the runtime; the process runs as root and does not. So every 30 +// seconds the bundle: +// +// - reads what the sharing machine exports from nfs-server's state `exports`, keyed by that machine +// (ADR 0201: a module reads another's state, never its files), and leaves it in ExportsFile for the +// process. A failed read keeps the last answer's shares, so a bus that is away for a minute removes no +// mount; +// - puts what the process last found in the state `shares`, keyed by this machine, so a view or the +// controller watches it without reading this machine's files. + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "time" +) + +// SharesState is the state the module keeps on the bus, one key per machine. +const SharesState = "shares" + +// ExportsState is nfs-server's state this module reads: what each sharing machine exports, by machine. +const ExportsState = "nfs-server.exports" + +// Getter reads one key of a state, nil when it has none; Putter puts one key of a state. +type ( + Getter func(key string) (json.RawMessage, error) + Putter func(key string, value any) error +) + +// AskExports reads what the sharing machine exports, once, and answers what to leave for the process. +func AskExports(cfg Config, previous ExportsAnswer, get Getter, now time.Time) ExportsAnswer { + out := ExportsAnswer{Server: cfg.ServerNode(), At: cfg.At, Asked: now.Format(time.RFC3339)} + if out.Server == "" { + out.Error = "no machine shares nfs-share for this one" + return out + } + raw, err := get(out.Server) + var answer struct { + Shares []ServerShare `json:"shares"` + } + if err == nil && len(raw) == 0 { + err = fmt.Errorf("%s keeps nothing in nfs-server's state exports yet", out.Server) + } + if err == nil { + err = json.Unmarshal(raw, &answer) + if err == nil && answer.Shares == nil { + err = fmt.Errorf("the answer names no shares") + } + } + if err != nil { + out.Error = err.Error() + if previous.Server == out.Server { + out.Shares, out.Answered = previous.Shares, previous.Answered + } + return out + } + out.Shares, out.Answered = answer.Shares, out.Asked + return out +} + +// SharesValue is what the module keeps on the bus for one machine: what its process last found, and what +// the machine asks of which server, so that the server exports a share to the machines that ask for it +// and to nobody else (the review of 2026-10-08, finding 5). The server decides the access it gives. +type SharesValue struct { + Status + Wants []Want `json:"wants"` +} + +// Share is one pass of the bundle's work over a machine. +func (m *Machine) Share(node string, get Getter, put Putter, last *string, now time.Time) []string { + var said []string + wants := []Want{} + if text, err := m.read(ConfigFile); err == nil { + cfg := ParseConfig(text) + wants = cfg.Wants() + if len(cfg.Shares) > 0 { + previous, _ := readJSON[ExportsAnswer](m, ExportsFile) + answer := AskExports(cfg, previous, get, now) + if answer.Error != "" { + said = append(said, "what the sharing machine exports was not read: "+answer.Error) + } + if err := m.leave(ExportsFile, answer); err != nil { + said = append(said, "the answer was not left for the process: "+err.Error()) + } + } + } + if node != "" && put != nil { + value := SharesValue{Wants: wants} + if text, err := m.read(StatusFile); err == nil { + json.Unmarshal([]byte(text), &value.Status) + } + if value.Status.Shares == nil { + value.Status.Shares = []ShareStatus{} + } + raw, _ := json.Marshal(value) + if string(raw) != *last { + if err := put(node, value); err != nil { + said = append(said, "the state was not put: "+err.Error()) + } else { + *last = string(raw) + } + } + } + return said +} + +// leave writes a file of the bundle's for the process, through a temporary beside it. +func (m *Machine) leave(p string, v any) error { + raw, _ := json.MarshalIndent(v, "", " ") + full := filepath.Join(m.Root, p) + tmp := full + ".new" + if err := os.WriteFile(tmp, append(raw, '\n'), 0o644); err != nil { + return err + } + return os.Rename(tmp, full) +} + +// Asker calls a tool on the mesh as this module. +type Asker func(key string, body any) (json.RawMessage, error) + +// DataVerb is the controller's verb that says where the mesh keeps data on a machine. +const DataVerb = "seat:mesh-controller.data" + +// LeaveData asks the controller where the mesh keeps data on this machine and leaves the paths for the +// process (finding 4d). A failed ask keeps the last paths, with why; with none, the process sets up +// nothing new. +func (m *Machine) LeaveData(node string, ask Asker, now time.Time) []string { + if node == "" || ask == nil { + return nil + } + previous, _ := readJSON[DataPaths](m, DataFile) + out := DataPaths{Machine: node, Asked: now.Format(time.RFC3339)} + raw, err := ask(DataVerb, map[string]any{"machine": node}) + var paths []string + if err == nil { + paths, err = DataFrom(raw) + } + var said []string + if err != nil { + out.Error = err.Error() + if previous.Machine == node { + out.Paths, out.Answered = previous.Paths, previous.Answered + } + said = append(said, "where the mesh keeps data here was not read: "+err.Error()) + } else { + out.Paths, out.Answered = paths, out.Asked + } + if err := m.leave(DataFile, out); err != nil { + said = append(said, "the data list was not left for the process: "+err.Error()) + } + return said +} diff --git a/modules/mounts/cmd/mounts/config.go b/modules/mounts/cmd/mounts/config.go new file mode 100644 index 00000000..b0a9deb9 --- /dev/null +++ b/modules/mounts/cmd/mounts/config.go @@ -0,0 +1,337 @@ +package main + +// What this machine is asked to mount (novox/hq ADR 0263, to-be 52 §2), and where the module keeps +// what it learns. +// +// The mesh renders ConfigFile from the assignment: the settings `shares` and `sources`, and the binding +// of the provision `nfs-share` (`from`, the machine that shares, and `at`, where it is reached). The +// long-running process reads it as root and writes the mount units; the bundle reads it as the operator +// account to read what the sharing machine exports from the bus. Nothing in it is secret: the SMB credentials are the +// module's own secret, in a file only root reads. + +import ( + "fmt" + "path/filepath" + "regexp" + "strings" +) + +// Where the module keeps its things. ConfigFile is placed by the mesh; the rest by the module's code. +const ( + ConfigFile = "/etc/mesh-mounts/mounts.conf" + // BusDir is written by the bundle, which runs as the operator account and reaches the bus; the + // process, which runs as root and does not, reads what the sharing machine said from it. + BusDir = "/var/lib/mesh-mounts/from-bus" + ExportsFile = BusDir + "/exports.json" + // RunDir is the process's: what it found on its last look, which the tools and the bundle read. + RunDir = "/run/mesh-mounts" + StatusFile = RunDir + "/status.json" + // UnitDir is where the process writes each unit pair, beside the operator's own units. + UnitDir = "/etc/systemd/system" +) + +// None is the value of an empty setting: a definition's default may not be empty (ADR 0262). +const None = "none" + +// Config is the rendered file, read. +type Config struct { + Shares []Wanted `json:"shares"` + Sources []Source `json:"sources"` + // Server is the machine sharing the provision `nfs-share` (the binding's `from`), and At where it is + // reached on the private network (the binding's `at`). + Server string `json:"server,omitempty"` + At string `json:"at,omitempty"` + // Secret is the path of the module's own secret `smb-credentials`; State is the module's state + // directory, where the per-source credentials files are written. + Secret string `json:"secret,omitempty"` + State string `json:"state,omitempty"` + Account string `json:"account,omitempty"` + // Node is this machine's name: a share this machine serves itself is never mounted onto it. + Node string `json:"node,omitempty"` + // Problems are what in the file could not be read; each is said, and its share is left out. + Problems []string `json:"problems,omitempty"` + // Unreadable are the entries of shares or sources that could not be read. While there is one, no + // pair is removed: an entry the operator mistyped names a mount point nobody can tell, so the + // module cannot tell which of its pairs it still names (review of 2026-10-08, finding 8). + Unreadable []string `json:"unreadable,omitempty"` +} + +// Wanted is one NFS share this machine mounts: the share's name on the sharing machine, and where. +type Wanted struct { + Name string `json:"name"` + Mountpoint string `json:"mountpoint"` + ReadOnly bool `json:"read_only"` +} + +// Source is one occasional source: a device that is not a node and is not always there. +type Source struct { + Name string `json:"name"` + Kind string `json:"kind"` // smb or disk + Host string `json:"host,omitempty"` + Share string `json:"share,omitempty"` + UUID string `json:"uuid,omitempty"` + Mountpoint string `json:"mountpoint"` + ReadOnly bool `json:"read_only"` +} + +var ( + nameRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`) + hostRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9.-]{0,252}$`) + shareRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9 ._$-]{0,79}$`) + uuidRe = regexp.MustCompile(`^[A-Za-z0-9-]{4,64}$`) +) + +// CheckMountpoint refuses a mount point the module may not use: not absolute, not clean, the root, or +// under a directory the system owns. +func CheckMountpoint(p string) error { + switch { + case !strings.HasPrefix(p, "/"): + return fmt.Errorf("the mount point %q is not an absolute path", p) + case filepath.Clean(p) != p: + return fmt.Errorf("the mount point %q is not written plainly (it would be %q)", p, filepath.Clean(p)) + case strings.ContainsAny(p, " \t\n\\\"'"): + return fmt.Errorf("the mount point %q holds a space, a quote or a backslash", p) + } + for _, sys := range []string{"/", "/etc", "/usr", "/bin", "/sbin", "/lib", "/lib64", "/boot", "/proc", "/sys", "/dev", "/run", "/var", "/home", "/root", "/tmp"} { + if p == sys { + return fmt.Errorf("the mount point %q is the system's own directory", p) + } + } + for _, sys := range []string{"/etc/", "/usr/", "/bin/", "/sbin/", "/lib/", "/boot/", "/proc/", "/sys/", "/dev/", "/run/"} { + if strings.HasPrefix(p, sys) { + return fmt.Errorf("the mount point %q is under %s, which is the system's", p, strings.TrimSuffix(sys, "/")) + } + } + return nil +} + +func entries(v string) []string { + v = strings.TrimSpace(v) + if v == "" || v == None { + return nil + } + return strings.FieldsFunc(v, func(r rune) bool { return r == ' ' || r == ',' || r == '\n' || r == '\t' }) +} + +func readOnly(s string) (string, bool) { + if strings.HasSuffix(s, ":ro") { + return strings.TrimSuffix(s, ":ro"), true + } + return strings.TrimSuffix(s, ":rw"), false +} + +// ParseShares reads the setting `shares`: space-separated `name=mountpoint`, each optionally `:ro`. +// What cannot be read is said, and only that entry is left out. +func ParseShares(v string) ([]Wanted, []string) { + var out []Wanted + var problems []string + seen := map[string]bool{} + for _, e := range entries(v) { + name, rest, ok := strings.Cut(e, "=") + if !ok { + problems = append(problems, fmt.Sprintf("the share %q is not name=mountpoint", e)) + continue + } + point, ro := readOnly(rest) + if !nameRe.MatchString(name) { + problems = append(problems, fmt.Sprintf("the share name %q is not lower-case letters, digits and dashes", name)) + continue + } + if err := CheckMountpoint(point); err != nil { + problems = append(problems, fmt.Sprintf("the share %s: %v", name, err)) + continue + } + if seen[name] || seen["@"+point] { + problems = append(problems, fmt.Sprintf("the share %s or its mount point %s is named twice", name, point)) + continue + } + seen[name], seen["@"+point] = true, true + out = append(out, Wanted{Name: name, Mountpoint: point, ReadOnly: ro}) + } + return out, problems +} + +// ParseSources reads the setting `sources`: space-separated `name=smb:///@` or +// `name=disk:@`, each optionally `:ro`. +func ParseSources(v string) ([]Source, []string) { + var out []Source + var problems []string + seen := map[string]bool{} + for _, e := range entries(v) { + name, rest, ok := strings.Cut(e, "=") + if !ok { + problems = append(problems, fmt.Sprintf("the source %q is not name=@", e)) + continue + } + if !nameRe.MatchString(name) { + problems = append(problems, fmt.Sprintf("the source name %q is not lower-case letters, digits and dashes", name)) + continue + } + at := strings.LastIndex(rest, "@") + if at < 0 { + problems = append(problems, fmt.Sprintf("the source %s names no mount point (…@)", name)) + continue + } + where, pointRO := rest[:at], rest[at+1:] + point, ro := readOnly(pointRO) + if err := CheckMountpoint(point); err != nil { + problems = append(problems, fmt.Sprintf("the source %s: %v", name, err)) + continue + } + s := Source{Name: name, Mountpoint: point, ReadOnly: ro} + switch { + case strings.HasPrefix(where, "smb://"): + host, share, ok := strings.Cut(strings.TrimPrefix(where, "smb://"), "/") + if !ok || !hostRe.MatchString(host) || !shareRe.MatchString(share) { + problems = append(problems, fmt.Sprintf("the source %s: %q is not smb:///", name, where)) + continue + } + s.Kind, s.Host, s.Share = "smb", host, share + case strings.HasPrefix(where, "disk:"): + uuid := strings.TrimPrefix(where, "disk:") + if !uuidRe.MatchString(uuid) { + problems = append(problems, fmt.Sprintf("the source %s: %q is not disk:", name, where)) + continue + } + s.Kind, s.UUID = "disk", uuid + default: + problems = append(problems, fmt.Sprintf("the source %s: %q is neither smb:/// nor disk:", name, where)) + continue + } + if seen[name] || seen["@"+point] { + problems = append(problems, fmt.Sprintf("the source %s or its mount point %s is named twice", name, point)) + continue + } + seen[name], seen["@"+point] = true, true + out = append(out, s) + } + return out, problems +} + +// ParseConfig reads the rendered file: `key=value` lines. +func ParseConfig(text string) Config { + kv := map[string]string{} + for _, line := range strings.Split(text, "\n") { + line = strings.TrimSpace(line) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + k, v, ok := strings.Cut(line, "=") + if ok { + kv[strings.TrimSpace(k)] = strings.TrimSpace(v) + } + } + var c Config + var p []string + c.Shares, p = ParseShares(kv["shares"]) + c.Problems = append(c.Problems, p...) + c.Unreadable = append(c.Unreadable, p...) + c.Sources, p = ParseSources(kv["sources"]) + c.Problems = append(c.Problems, p...) + c.Unreadable = append(c.Unreadable, p...) + c.Server, c.At, c.Secret, c.State, c.Account = kv["server"], kv["at"], kv["secret"], kv["state"], kv["account"] + c.Node = kv["node"] + if c.At != "" && !hostRe.MatchString(c.At) { + c.Problems = append(c.Problems, fmt.Sprintf("the sharing machine's address %q is not a name or an address", c.At)) + c.At = "" + } + points := map[string]string{} + for _, w := range c.Shares { + points[w.Mountpoint] = w.Name + } + kept := c.Sources[:0] + for _, s := range c.Sources { + if other, dup := points[s.Mountpoint]; dup { + c.Problems = append(c.Problems, fmt.Sprintf("the source %s mounts at %s, where the share %s already does", s.Name, s.Mountpoint, other)) + continue + } + kept = append(kept, s) + } + c.Sources = kept + return c +} + +// ServerNode is the sharing machine, the key of its exports on the bus: the binding's `from`, else the first label of `at`. +func (c Config) ServerNode() string { + if c.Server != "" { + return c.Server + } + if c.At == "" { + return "" + } + first, _, _ := strings.Cut(c.At, ".") + return first +} + +// SMBCredential is one source's login, from the module's own secret. +type SMBCredential struct { + Username string + Password string + Domain string +} + +// ParseSMBCredentials reads the secret `smb-credentials`: one section per source, +// +// [name] +// username=… +// password=… +// domain=… (optional) +// +// Its values are never answered by any tool; a problem names the section and the key, never the value. +func ParseSMBCredentials(text string) (map[string]SMBCredential, []string) { + out := map[string]SMBCredential{} + var problems []string + section := "" + for i, raw := range strings.Split(text, "\n") { + line := strings.TrimSpace(raw) + if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") { + continue + } + if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") { + section = strings.TrimSpace(line[1 : len(line)-1]) + if _, dup := out[section]; dup { + problems = append(problems, fmt.Sprintf("the section [%s] is given twice; the first is kept", section)) + section = "" + continue + } + out[section] = SMBCredential{} + continue + } + k, v, ok := strings.Cut(line, "=") + if !ok || section == "" { + problems = append(problems, fmt.Sprintf("line %d is not key=value inside a [section]", i+1)) + continue + } + c := out[section] + switch strings.ToLower(strings.TrimSpace(k)) { + case "username", "user": + c.Username = strings.TrimSpace(v) + case "password", "pass": + c.Password = strings.TrimRight(v, "\r") + case "domain": + c.Domain = strings.TrimSpace(v) + default: + problems = append(problems, fmt.Sprintf("[%s] has a key %q this module does not read", section, strings.TrimSpace(k))) + } + out[section] = c + } + for name, c := range out { + if c.Username == "" { + problems = append(problems, fmt.Sprintf("[%s] gives no username", name)) + } + if strings.ContainsAny(c.Username+c.Password+c.Domain, "\n") { + problems = append(problems, fmt.Sprintf("[%s] holds a line break", name)) + delete(out, name) + } + } + return out, problems +} + +// CredentialsFile is a source's credentials in mount.cifs's format. +func (c SMBCredential) CredentialsFile() string { + s := "username=" + c.Username + "\npassword=" + c.Password + "\n" + if c.Domain != "" { + s += "domain=" + c.Domain + "\n" + } + return s +} diff --git a/modules/mounts/cmd/mounts/data.go b/modules/mounts/cmd/mounts/data.go new file mode 100644 index 00000000..4550ed81 --- /dev/null +++ b/modules/mounts/cmd/mounts/data.go @@ -0,0 +1,214 @@ +package main + +// What the module may not mount over (the review of 2026-10-08, findings 4a–4d): the mesh's own data on +// this machine, a mount point that something else uses, a folder that holds files, and a share this +// machine serves itself. And what the bundle leaves for the process so that it can tell: the paths the +// mesh keeps data in on this machine, from the controller's `data` verb over the bus. + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "strings" +) + +// DataFile is what the bundle left from mesh-controller.data for this machine. +const DataFile = BusDir + "/data.json" + +// DataPaths is where the mesh keeps data on this machine, as its controller says. +type DataPaths struct { + Machine string `json:"machine"` + Asked string `json:"asked"` + Paths []string `json:"paths"` + // Error is why the last ask got no answer; the paths are then the last answer's, or none. + Error string `json:"error,omitempty"` + Answered string `json:"answered,omitempty"` +} + +// Want is one share this machine asks of one server, put on the bus so that the server exports the share +// to this machine and to nobody who does not ask (finding 5). The server decides the access it gives; this +// is only what is asked. +type Want struct { + Server string `json:"server"` + Share string `json:"share"` + Access string `json:"access"` +} + +// Wants are what a config asks of its server, by share name. +func (c Config) Wants() []Want { + out := []Want{} + server := c.ServerNode() + if server == "" { + return out + } + for _, w := range c.Shares { + access := "rw" + if w.ReadOnly { + access = "ro" + } + out = append(out, Want{Server: server, Share: w.Name, Access: access}) + } + sortWants(out) + return out +} + +func sortWants(w []Want) { + for i := 1; i < len(w); i++ { + for j := i; j > 0 && w[j].Share < w[j-1].Share; j-- { + w[j], w[j-1] = w[j-1], w[j] + } + } +} + +// DataFrom reads the paths out of mesh-controller.data's answer: `answer.data`, `data`, or the same as +// text in `output`. An answer with no data list is an error, never an empty list: an empty list would +// let every mount point through. +func DataFrom(raw json.RawMessage) ([]string, error) { + type item struct { + Path string `json:"path"` + } + var whole struct { + Answer *struct { + Data *[]item `json:"data"` + } `json:"answer"` + Data *[]item `json:"data"` + Output string `json:"output"` + } + if err := json.Unmarshal(raw, &whole); err != nil { + return nil, fmt.Errorf("the controller's data answer is not JSON: %v", err) + } + var items *[]item + switch { + case whole.Answer != nil && whole.Answer.Data != nil: + items = whole.Answer.Data + case whole.Data != nil: + items = whole.Data + case whole.Output != "": + var inner struct { + Data *[]item `json:"data"` + } + if err := json.Unmarshal([]byte(whole.Output), &inner); err == nil && inner.Data != nil { + items = inner.Data + } + } + if items == nil { + return nil, errors.New("the controller's data answer names no data list") + } + paths := []string{} + for _, it := range *items { + if strings.HasPrefix(it.Path, "/") { + paths = append(paths, filepath.Clean(it.Path)) + } + } + return paths, nil +} + +// nearData refuses a mount point at, under or above a path the mesh keeps data in here: at or under, the +// mount would hide the data; above, it would hide it too, and everything beside it. +func nearData(point string, paths []string) error { + for _, d := range paths { + switch { + case point == d: + return fmt.Errorf("the mount point %s is at %s, where the mesh keeps data on this machine; it is never mounted over", point, d) + case strings.HasPrefix(point, d+"/"): + return fmt.Errorf("the mount point %s is under %s, where the mesh keeps data on this machine; it is never mounted over", point, d) + case strings.HasPrefix(d, point+"/"): + return fmt.Errorf("the mount point %s is above %s, where the mesh keeps data on this machine; mounting there would hide it", point, d) + } + } + return nil +} + +// sourceIs says whether what is mounted at a point is a pair's own source. For a network share the source +// is compared as the kernel writes it; for a disk, the device its by-uuid link resolves to. +func (m *Machine) sourceIs(kind, what string, mt Mounted) bool { + switch kind { + case "nfs": + return strings.HasPrefix(mt.Type, "nfs") && strings.TrimSuffix(mt.Source, "/") == strings.TrimSuffix(what, "/") + case "smb": + return (mt.Type == "cifs" || mt.Type == "smb3") && strings.EqualFold(strings.TrimSuffix(mt.Source, "/"), strings.TrimSuffix(what, "/")) + case "disk": + dev, err := filepath.EvalSymlinks(filepath.Join(m.Root, what)) + if err != nil { + return false + } + src, err := filepath.EvalSymlinks(filepath.Join(m.Root, mt.Source)) + return err == nil && src == dev + } + return false +} + +// kindOf is a unit's kind from what it mounts. +func kindOf(what, fstype string) string { + switch { + case strings.HasPrefix(fstype, "nfs"): + return "nfs" + case fstype == "cifs" || strings.HasPrefix(what, "//"): + return "smb" + case strings.HasPrefix(what, "/dev/"): + return "disk" + } + return "" +} + +// notOwnSource says why a mounted filesystem at a point is not the source a unit of the module mounts +// there, or nothing when it is, or nothing is mounted there but an automount. +func (m *Machine) notOwnSource(kind, what, point string, mounted map[string]Mounted) error { + mt, there := mounted[point] + if !there || mt.Type == "autofs" { + return nil + } + if kind != "" && m.sourceIs(kind, what, mt) { + return nil + } + return fmt.Errorf("%s is mounted at %s (%s), which is not what the module mounts there (%s)", mt.Source, point, mt.Type, what) +} + +// mountpointFree refuses a mount point another mount uses, or a folder that holds files. A point in +// mountinfo is never opened, so a network server that is away cannot hold the look. +func (m *Machine) mountpointFree(p Pair, own map[string]bool, mounted map[string]Mounted) error { + ours := own[p.MountUnit] && own[p.AutoUnit] + if mt, there := mounted[p.Mountpoint]; there { + if mt.Type == "autofs" { + if ours { + return nil + } + return fmt.Errorf("%s is already an automount that is not the module's (%s)", p.Mountpoint, mt.Source) + } + if ours && m.sourceIs(p.Kind, p.What, mt) { + return nil + } + return fmt.Errorf("%s is already mounted: %s (%s) is there, so this share is not mounted over it", p.Mountpoint, mt.Source, mt.Type) + } + dir, err := os.Open(filepath.Join(m.Root, p.Mountpoint)) + if err != nil { + return nil // missing: it is made + } + defer dir.Close() + info, err := dir.Stat() + if err != nil { + return fmt.Errorf("%s cannot be looked at: %v", p.Mountpoint, err) + } + if !info.IsDir() { + return fmt.Errorf("%s is a file, not a folder", p.Mountpoint) + } + if names, err := dir.Readdirnames(1); err == nil && len(names) > 0 { + return fmt.Errorf("%s is not empty (it holds %s); a mount would hide what is in it, so it is not mounted there", p.Mountpoint, names[0]) + } else if err != nil && err != io.EOF { + return fmt.Errorf("%s cannot be read: %v", p.Mountpoint, err) + } + return nil +} + +// unitField reads one Key= of a unit file. +func unitField(text, key string) string { + for _, line := range strings.Split(text, "\n") { + if v, ok := strings.CutPrefix(strings.TrimSpace(line), key+"="); ok { + return strings.TrimSpace(v) + } + } + return "" +} diff --git a/modules/mounts/cmd/mounts/exports.go b/modules/mounts/cmd/mounts/exports.go new file mode 100644 index 00000000..921bdef3 --- /dev/null +++ b/modules/mounts/cmd/mounts/exports.go @@ -0,0 +1,281 @@ +package main + +import ( + "context" + "fmt" + "sort" + "strings" +) + +// Export is one path offered to one client over NFS. +type Export struct { + Path string `json:"path"` + Client string `json:"client"` + Options string `json:"options,omitempty"` + File string `json:"file"` + Line int `json:"line,omitempty"` +} + +// ParseExports reads an exports(5) file: a path, then clients each with its options in brackets; a +// `-options` word sets the default for the clients after it; a backslash continues a line. +func ParseExports(text, file string) []Export { + out := []Export{} + lines := strings.Split(text, "\n") + for i := 0; i < len(lines); i++ { + start := i + 1 + line := lines[i] + for strings.HasSuffix(strings.TrimRight(line, " \t"), `\`) && i+1 < len(lines) { + line = strings.TrimSuffix(strings.TrimRight(line, " \t"), `\`) + " " + lines[i+1] + i++ + } + if c := strings.Index(line, "#"); c >= 0 { + line = line[:c] + } + f := splitExportLine(line) + if len(f) == 0 { + continue + } + path := unescape(strings.Trim(f[0], `"`)) + defaults := "" + clients := 0 + for _, w := range f[1:] { + if strings.HasPrefix(w, "-") { + defaults = strings.TrimPrefix(w, "-") + continue + } + client, opts := w, defaults + if o := strings.Index(w, "("); o >= 0 { + client = w[:o] + opts = strings.TrimSuffix(w[o+1:], ")") + } + if client == "" { + client = "*" + } + out = append(out, Export{Path: path, Client: client, Options: opts, File: file, Line: start}) + clients++ + } + if clients == 0 { + out = append(out, Export{Path: path, Client: "*", Options: defaults, File: file, Line: start}) + } + } + return out +} + +// splitExportLine splits on blanks outside double quotes. +func splitExportLine(line string) []string { + var out []string + var cur strings.Builder + quoted := false + for _, r := range line { + switch { + case r == '"': + quoted = !quoted + cur.WriteRune(r) + case (r == ' ' || r == '\t') && !quoted: + if cur.Len() > 0 { + out = append(out, cur.String()) + cur.Reset() + } + default: + cur.WriteRune(r) + } + } + if cur.Len() > 0 { + out = append(out, cur.String()) + } + return out +} + +// ZFSShare is one dataset's sharenfs or sharesmb property. +type ZFSShare struct { + Dataset string `json:"dataset"` + Property string `json:"property"` + Value string `json:"value"` + Source string `json:"source"` +} + +// ParseZFSGet reads `zfs get -H -o name,property,value,source`. +func ParseZFSGet(text string) []ZFSShare { + out := []ZFSShare{} + for _, line := range strings.Split(text, "\n") { + f := strings.Split(line, "\t") + if len(f) < 4 { + continue + } + out = append(out, ZFSShare{Dataset: f[0], Property: f[1], Value: f[2], Source: f[3]}) + } + return out +} + +// SMBShare is one Samba share: what it serves and to whom, never a password. +type SMBShare struct { + Name string `json:"name"` + Path string `json:"path,omitempty"` + Settings map[string]string `json:"settings"` +} + +// smbKept are the share settings answered. Everything else in a section — and the [global] section +// whole — is left out, so no password, password backend or secret a file may hold is ever answered. +var smbKept = map[string]bool{ + "comment": true, "valid users": true, "invalid users": true, "read only": true, "writable": true, + "writeable": true, "write ok": true, "write list": true, "read list": true, "force user": true, "force group": true, + "guest ok": true, "public": true, "browseable": true, "browsable": true, "hosts allow": true, "hosts deny": true, + "available": true, "create mask": true, "directory mask": true, +} + +// ParseSMBConf reads smb.conf's share sections. +func ParseSMBConf(text string) []SMBShare { + out := []SMBShare{} + var cur *SMBShare + for _, raw := range strings.Split(text, "\n") { + line := strings.TrimSpace(raw) + if line == "" || line[0] == '#' || line[0] == ';' { + continue + } + if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") { + name := strings.TrimSpace(line[1 : len(line)-1]) + if strings.EqualFold(name, "global") { + cur = nil + continue + } + out = append(out, SMBShare{Name: name, Settings: map[string]string{}}) + cur = &out[len(out)-1] + continue + } + if cur == nil { + continue + } + k, v, ok := strings.Cut(line, "=") + if !ok { + continue + } + key := strings.Join(strings.Fields(strings.ToLower(k)), " ") + v = strings.TrimSpace(v) + if key == "path" { + cur.Path = v + continue + } + if smbKept[key] { + cur.Settings[key] = v + } + } + return out +} + +// ExportsReport is what mounts_exports answers. +type ExportsReport struct { + Said []string `json:"said"` + Services map[string]string `json:"services"` + Exports []Export `json:"exports"` + Live []Export `json:"live_exports"` + ZFS []ZFSShare `json:"zfs_shares,omitempty"` + SMB []SMBShare `json:"smb_shares"` + Notes []string `json:"notes,omitempty"` +} + +// Exports reads what this machine shares over the network. +func (m *Machine) Exports() ExportsReport { + r := ExportsReport{Said: []string{}, Services: map[string]string{}, Exports: []Export{}, Live: []Export{}, SMB: []SMBShare{}} + ctx := context.Background() + + units := []string{"nfs-server.service", "smb.service"} + active, err := m.Run(ctx, "systemctl", append([]string{"is-active"}, units...)...) + states := strings.Fields(active) + for i, u := range units { + if i < len(states) { + r.Services[u] = states[i] + } else if err != nil { + r.Services[u] = "not read" + } + } + + files := []string{"/etc/exports"} + files = append(files, m.glob("/etc/exports.d/*.exports")...) + for _, f := range files { + text, err := m.read(f) + if err != nil { + r.Notes = append(r.Notes, unreadable(f, err)) + continue + } + r.Exports = append(r.Exports, ParseExports(text, f)...) + } + if etab, err := m.read("/var/lib/nfs/etab"); err != nil { + r.Notes = append(r.Notes, unreadable("/var/lib/nfs/etab", err)+" — the kernel's live exports are not known") + } else { + for _, line := range strings.Split(etab, "\n") { + path, rest, ok := strings.Cut(line, "\t") + if !ok { + continue + } + for _, e := range ParseExports("x "+rest, "/var/lib/nfs/etab") { + r.Live = append(r.Live, Export{Path: unescape(path), Client: e.Client, Options: e.Options, File: e.File}) + } + } + } + + if m.LookPath != nil && m.LookPath("zfs") { + out, err := m.Run(ctx, "zfs", "get", "-H", "-o", "name,property,value,source", "-t", "filesystem", "sharenfs,sharesmb") + if err != nil { + r.Notes = append(r.Notes, fmt.Sprintf("the ZFS share properties were not read: %v", err)) + } + for _, z := range ParseZFSGet(out) { + if z.Value != "off" && z.Value != "-" { + r.ZFS = append(r.ZFS, z) + } + } + } + + if conf, err := m.read("/etc/samba/smb.conf"); err != nil { + r.Notes = append(r.Notes, unreadable("/etc/samba/smb.conf", err)) + } else { + r.SMB = ParseSMBConf(conf) + } + + r.Said = r.say() + return r +} + +func (r ExportsReport) say() []string { + said := []string{} + for _, u := range []string{"nfs-server.service", "smb.service"} { + said = append(said, fmt.Sprintf("%s: %s", u, r.Services[u])) + } + live := map[string]bool{} + for _, e := range r.Live { + live[e.Path+" "+e.Client] = true + } + for _, e := range r.Exports { + line := fmt.Sprintf("NFS %s to %s (%s), in %s", e.Path, e.Client, e.Options, e.File) + if len(r.Live) > 0 && !live[e.Path+" "+e.Client] { + line += " — NOT in the kernel's live exports" + } + said = append(said, line) + } + declared := map[string]bool{} + for _, e := range r.Exports { + declared[e.Path+" "+e.Client] = true + } + for _, e := range r.Live { + if !declared[e.Path+" "+e.Client] { + said = append(said, fmt.Sprintf("NFS %s to %s is live but in no exports file (set by hand with exportfs, or by ZFS before its file was written)", e.Path, e.Client)) + } + } + for _, z := range r.ZFS { + said = append(said, fmt.Sprintf("ZFS %s %s=%s (%s)", z.Dataset, z.Property, z.Value, z.Source)) + } + sort.SliceStable(r.SMB, func(i, j int) bool { return r.SMB[i].Name < r.SMB[j].Name }) + for _, s := range r.SMB { + line := fmt.Sprintf("SMB [%s] %s", s.Name, s.Path) + if v := s.Settings["valid users"]; v != "" { + line += ", for " + v + } + if v := s.Settings["force user"]; v != "" { + line += ", writes as " + v + } + said = append(said, line) + } + if len(r.Exports) == 0 && len(r.Live) == 0 && len(r.ZFS) == 0 && len(r.SMB) == 0 { + said = append(said, "this machine shares nothing over NFS or SMB") + } + return said +} diff --git a/modules/mounts/cmd/mounts/exports_test.go b/modules/mounts/cmd/mounts/exports_test.go new file mode 100644 index 00000000..afcee3b7 --- /dev/null +++ b/modules/mounts/cmd/mounts/exports_test.go @@ -0,0 +1,108 @@ +package main + +import ( + "encoding/json" + "strings" + "testing" +) + +func TestParseExports(t *testing.T) { + text := `# comment +/srv/media 192.0.2.0/24(rw,sync,root_squash) 198.51.100.7(ro) +"/srv/with space" -ro,sync host.example \ + other.example(rw) +/srv/bare +` + got := ParseExports(text, "/etc/exports") + want := []Export{ + {"/srv/media", "192.0.2.0/24", "rw,sync,root_squash", "/etc/exports", 2}, + {"/srv/media", "198.51.100.7", "ro", "/etc/exports", 2}, + {"/srv/with space", "host.example", "ro,sync", "/etc/exports", 3}, + {"/srv/with space", "other.example", "rw", "/etc/exports", 3}, + {"/srv/bare", "*", "", "/etc/exports", 5}, + } + if len(got) != len(want) { + t.Fatalf("got %d, want %d: %+v", len(got), len(want), got) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("export %d = %+v, want %+v", i, got[i], want[i]) + } + } +} + +func TestParseSMBConfLeavesSecretsOut(t *testing.T) { + text := `[global] +workgroup = WORKGROUP +passdb backend = tdbsam +; comment +[media] +comment = Media +path = /srv/media/ +valid users = media +Read Only = no +force user = media +smb encrypt = required +username map script = /bin/secret +[homes] +browseable = no +` + got := ParseSMBConf(text) + if len(got) != 2 || got[0].Name != "media" || got[0].Path != "/srv/media/" { + t.Fatalf("got %+v", got) + } + if got[0].Settings["read only"] != "no" || got[0].Settings["force user"] != "media" { + t.Errorf("settings = %+v", got[0].Settings) + } + b, _ := json.Marshal(got) + for _, leak := range []string{"tdbsam", "WORKGROUP", "/bin/secret"} { + if strings.Contains(string(b), leak) { + t.Errorf("%q was answered: %s", leak, b) + } + } +} + +func TestExports(t *testing.T) { + f := newFake(t) + f.zfs = true + f.file("/etc/exports", "# only comments\n") + f.file("/etc/exports.d/zfs.exports", "/storage/media 192.0.2.0/24(sec=sys,rw,no_subtree_check,mountpoint)\n") + f.file("/var/lib/nfs/etab", "/storage/media\t192.0.2.0/24(rw,sync,root_squash,sec=sys)\n/old\t198.51.100.0/24(ro)\n") + f.file("/etc/samba/smb.conf", "[global]\nworkgroup = W\n[media]\npath = /storage/media/\nvalid users = media\nforce user = media\n") + f.answers["systemctl is-active nfs-server.service smb.service"] = "active\ninactive\n" + f.answers["zfs get -H -o name,property,value,source -t filesystem sharenfs,sharesmb"] = + "storage\tsharenfs\toff\tdefault\nstorage/media\tsharenfs\trw=@192.0.2.0/24\tlocal\nstorage/media\tsharesmb\ton\tlocal\n" + r := f.machine().Exports() + said := strings.Join(r.Said, "\n") + for _, want := range []string{ + "nfs-server.service: active", + "smb.service: inactive", + "NFS /storage/media to 192.0.2.0/24 (sec=sys,rw,no_subtree_check,mountpoint), in /etc/exports.d/zfs.exports", + "NFS /old to 198.51.100.0/24 is live but in no exports file", + "ZFS storage/media sharenfs=rw=@192.0.2.0/24 (local)", + "SMB [media] /storage/media/, for media, writes as media", + } { + if !strings.Contains(said, want) { + t.Errorf("said lacks %q:\n%s", want, said) + } + } + if len(r.ZFS) != 2 { + t.Errorf("a share property that is off must be left out: %+v", r.ZFS) + } +} + +func TestExportsUnreadableIsSaid(t *testing.T) { + r := newFake(t).machine().Exports() + notes := strings.Join(r.Notes, "\n") + for _, want := range []string{"/etc/exports: not present", "/var/lib/nfs/etab: not present", "/etc/samba/smb.conf: not present"} { + if !strings.Contains(notes, want) { + t.Errorf("notes lack %q: %s", want, notes) + } + } + if r.Services["nfs-server.service"] != "not read" { + t.Errorf("services = %+v", r.Services) + } + if !strings.Contains(strings.Join(r.Said, "\n"), "shares nothing") { + t.Errorf("said = %v", r.Said) + } +} diff --git a/modules/mounts/cmd/mounts/health.go b/modules/mounts/cmd/mounts/health.go new file mode 100644 index 00000000..cdd5ce0b --- /dev/null +++ b/modules/mounts/cmd/mounts/health.go @@ -0,0 +1,97 @@ +package main + +// The module's health (novox/hq ADR 0240, ADR 0263 rule 8), judged from what the process last found. +// +// Unhealthy when a share that should always be there has been unreachable or failed for five minutes, +// when an automount is not armed, when something the settings name cannot be set up, or when the +// process has not looked for two minutes. An occasional source that is absent is healthy: it is said as +// not present since when, and raises nothing. + +import ( + "fmt" + "strings" + "time" +) + +// Health is the answer the node-engine reads from a tool check. +type Health struct { + Healthy bool `json:"healthy"` + Why string `json:"why"` + Found []string `json:"found,omitempty"` + Said []string `json:"said,omitempty"` +} + +// Grace is how long a share may be away before it is a fault; Stale how old the process's look may be. +const ( + Grace = 5 * time.Minute + Stale = 2 * time.Minute +) + +func clock(ts string) string { + t, err := time.Parse(time.RFC3339, ts) + if err != nil { + return ts + } + return t.Local().Format("15:04") +} + +// Judge is the health of one status at a moment. +func Judge(st Status, readErr error, now time.Time) Health { + if readErr != nil { + return Health{Healthy: false, Why: "The module's process has left no account of this machine's mounts (" + readErr.Error() + ").", + Found: []string{"no status"}} + } + var found, said []string + if looked, err := time.Parse(time.RFC3339, st.Looked); err != nil || now.Sub(looked) > Stale { + found = append(found, fmt.Sprintf("The module's process last looked at %s, more than two minutes ago.", clock(st.Looked))) + } + for _, p := range st.Problems { + found = append(found, "The module's settings or secret: "+p+".") + } + from := "" + if st.Server != "" { + from = " of " + st.Server + } + for _, s := range st.Shares { + since, _ := time.Parse(time.RFC3339, s.Since) + long := now.Sub(since) >= Grace + switch s.State { + case StateMounted, StateArmed: + said = append(said, fmt.Sprintf("%s at %s: %s since %s.", s.Name, s.Mountpoint, s.State, clock(s.Since))) + case StateAbsent: + said = append(said, fmt.Sprintf("%s at %s: not present since %s.", s.Name, s.Mountpoint, clock(s.Since))) + case StateUnreachable: + line := fmt.Sprintf("The share %s%s is not reachable from this machine since %s (%s).", s.Name, from, clock(s.Since), s.Why) + if s.Kind != "nfs" { + line = fmt.Sprintf("The source %s is not reachable from this machine since %s (%s).", s.Name, clock(s.Since), s.Why) + } + if long { + found = append(found, line) + } else { + said = append(said, line+" Raised after five minutes.") + } + default: + line := fmt.Sprintf("The %s %s at %s cannot be mounted since %s: %s.", map[bool]string{true: "source", false: "share"}[s.Occasional], + s.Name, s.Mountpoint, clock(s.Since), strings.TrimSuffix(s.Why, ".")) + if long || strings.Contains(s.Why, "not armed") || strings.Contains(s.Why, "not the module's") || strings.Contains(s.Why, "not given") { + found = append(found, line) + } else { + said = append(said, line+" Raised after five minutes.") + } + } + } + if len(found) > 0 { + return Health{Healthy: false, Why: strings.Join(found, " "), Found: found, Said: said} + } + why := "Every share this machine is asked to mount is armed or mounted, and every occasional source is armed, mounted or not present." + if len(st.Shares) == 0 { + why = "This machine is asked to mount nothing." + } + return Health{Healthy: true, Why: why, Said: said} +} + +// Health reads the process's last look and judges it. +func (m *Machine) Health(now time.Time) Health { + st, err := readJSON[Status](m, StatusFile) + return Judge(st, err, now) +} diff --git a/modules/mounts/cmd/mounts/machine.go b/modules/mounts/cmd/mounts/machine.go new file mode 100644 index 00000000..b0c8722b --- /dev/null +++ b/modules/mounts/cmd/mounts/machine.go @@ -0,0 +1,118 @@ +package main + +import ( + "bytes" + "context" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "time" +) + +// CommandTimeout bounds every command this module runs. No command it runs touches a mounted +// filesystem, so none should block on a server that is gone; the bound is there in case one does. +const CommandTimeout = 5 * time.Second + +// Runner runs one command and answers its standard output. It is injected so that the tools are tested +// against recorded answers rather than this machine's service manager. +type Runner func(ctx context.Context, name string, args ...string) (string, error) + +// ExecRunner runs a command on the machine, bounded by CommandTimeout. +func ExecRunner(ctx context.Context, name string, args ...string) (string, error) { + ctx, cancel := context.WithTimeout(ctx, CommandTimeout) + defer cancel() + cmd := exec.CommandContext(ctx, name, args...) + var stdout, stderr bytes.Buffer + cmd.Stdout, cmd.Stderr = &stdout, &stderr + if err := cmd.Run(); err != nil { + said := strings.TrimSpace(stderr.String()) + if said != "" { + return stdout.String(), fmt.Errorf("%s: %w: %s", name, err, said) + } + return stdout.String(), fmt.Errorf("%s: %w", name, err) + } + return stdout.String(), nil +} + +// Machine is what the module reads: a filesystem root (the real one, or a test's tree standing in for +// /etc, /proc and /var/lib), a way to run a command, and a way to look a command up. +type Machine struct { + Root string + Run Runner + LookPath func(name string) bool + // Escalate runs a command as root: as given where this process is root, else through `sudo -n`, as the + // bundle runs as the operator account (novox/hq ADR 0175 §4) and the sudo module lets it. + Escalate Runner +} + +// Here is the machine this process runs on. +func Here() *Machine { + return &Machine{Root: "/", Run: ExecRunner, Escalate: EscalatedRunner, + LookPath: func(n string) bool { _, err := exec.LookPath(n); return err == nil }} +} + +// read answers one file's text under the root. Only regular files the module names are read; a +// mountpoint is never opened, so a network filesystem whose server is gone cannot block a call. +func (m *Machine) read(path string) (string, error) { + b, err := os.ReadFile(filepath.Join(m.Root, path)) + return string(b), err +} + +// glob answers the files under the root matching a pattern, as paths seen from the machine's root. +func (m *Machine) glob(pattern string) []string { + found, _ := filepath.Glob(filepath.Join(m.Root, pattern)) + out := make([]string, 0, len(found)) + for _, f := range found { + rel, err := filepath.Rel(m.Root, f) + if err != nil { + continue + } + out = append(out, "/"+filepath.ToSlash(rel)) + } + return out +} + +// unreadable says why a file could not be read, in the module's words. +func unreadable(path string, err error) string { + switch { + case os.IsNotExist(err): + return path + ": not present" + case os.IsPermission(err): + return path + ": not readable by the tool runner's account" + default: + return fmt.Sprintf("%s: not read (%v)", path, err) + } +} + +// EscalatedRunner runs a command as root, through sudo without a prompt where this process is not root. +// A refusal by sudo is said as one, never read as an empty answer. +func EscalatedRunner(ctx context.Context, name string, args ...string) (string, error) { + ctx, cancel := context.WithTimeout(ctx, 25*time.Second) + defer cancel() + program, argv := name, args + if os.Getuid() != 0 { + program, argv = "sudo", append([]string{"-n", name}, args...) + } + cmd := exec.CommandContext(ctx, program, argv...) + var stdout, stderr bytes.Buffer + cmd.Stdout, cmd.Stderr = &stdout, &stderr + if err := cmd.Run(); err != nil { + said := strings.TrimSpace(stderr.String()) + if program == "sudo" && strings.HasPrefix(said, "sudo:") { + return stdout.String(), fmt.Errorf("%s needs root, and the tool runner's account may not run it without a prompt: %s", name, said) + } + if said != "" { + return stdout.String(), fmt.Errorf("%s: %w: %s", name, err, said) + } + return stdout.String(), fmt.Errorf("%s: %w", name, err) + } + return stdout.String(), nil +} + +// present says whether a device or a file is there. Never used on a mount point. +func (m *Machine) present(p string) bool { + _, err := os.Stat(filepath.Join(m.Root, p)) + return err == nil +} diff --git a/modules/mounts/cmd/mounts/main.go b/modules/mounts/cmd/mounts/main.go new file mode 100644 index 00000000..ea5b32c0 --- /dev/null +++ b/modules/mounts/cmd/mounts/main.go @@ -0,0 +1,79 @@ +// The mounts module's Go bundle (novox/hq research 038, ADR 0263). One binary, three ways to run: +// +// - with no argument it is the bundle the node's tool runner launches and speaks MCP to over stdio, as the +// operator account: the tools that read what a machine mounts and shares, the seat node-mounts' verbs +// and the health check; and beside them the module's half on the bus (bus.go); +// - `apply` and `watch` are the module's two long-running processes, run by the node-engine as root: +// one writes the unit pair of every share and source the machine is asked to mount and arms the +// automounts, the other tries each source and says what it found (apply.go); +// - `adopt ` is the root half of the verb adopt, run by the bundle through sudo (adopt.go). +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "os/signal" + "syscall" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +func main() { + if len(os.Args) > 1 { + switch os.Args[1] { + case "apply", "watch": + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt) + defer stop() + a := NewApplier() + a.Act, a.Judge = os.Args[1] == "apply", os.Args[1] == "watch" + RunApply(ctx, a) + return + case "adopt": + os.Exit(RunAdopt(os.Args[2:])) + default: + fmt.Fprintf(os.Stderr, "mounts: %q is not a way to run; with no argument it serves its tools, with apply it "+ + "writes and arms the machine's mounts, with watch it says how they are, with adopt it takes over an fstab line\n", os.Args[1]) + os.Exit(2) + } + } + m := Here() + go shareOnTheBus(m) + // An empty name serves as the module the runtime names (MESH_SERVED_MODULE): mounts. + if err := stdio.Serve("", Tools(m)); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +// shareOnTheBus is the bundle's half on the bus, every LookEvery for as long as the bundle runs. +func shareOnTheBus(m *Machine) { + node := os.Getenv("MESH_NODE") + get := func(key string) (json.RawMessage, error) { + e, err := stdio.State(ExportsState).Get(key) + if err != nil || e == nil { + return nil, err + } + return e.Value, nil + } + put := func(key string, value any) error { + _, err := stdio.State(SharesState).Put(key, value) + return err + } + ask := func(key string, body any) (json.RawMessage, error) { return stdio.Ask(key, body) } + last, lastSaid := "", "" + for n := 0; ; n++ { + said := m.Share(node, get, put, &last, time.Now()) + // Where the mesh keeps data here changes only with an assignment: asked at start and every five minutes. + if n%10 == 0 { + said = append(said, m.LeaveData(node, ask, time.Now())...) + } + if s := fmt.Sprint(said); len(said) > 0 && s != lastSaid { + fmt.Fprintln(os.Stderr, "mounts:", s) + lastSaid = s + } + time.Sleep(LookEvery) + } +} diff --git a/modules/mounts/cmd/mounts/manifest_test.go b/modules/mounts/cmd/mounts/manifest_test.go new file mode 100644 index 00000000..9676f26f --- /dev/null +++ b/modules/mounts/cmd/mounts/manifest_test.go @@ -0,0 +1,148 @@ +package main + +import ( + "encoding/json" + "os" + "regexp" + "sort" + "strings" + "testing" +) + +type manifest struct { + Tools []string `json:"tools"` + Requires []string `json:"requires"` + Reads []string `json:"reads"` + Invokes []string `json:"invokes"` + Claims []map[string]any `json:"claims"` + State []map[string]any `json:"state"` + Settings map[string]map[string]any `json:"settings"` + OwnSecrets map[string]map[string]string `json:"own-secrets"` + Resources []map[string]any `json:"resources"` +} + +func readManifest(t *testing.T) (manifest, string) { + t.Helper() + raw, err := os.ReadFile("../../module.json") + if err != nil { + t.Fatal(err) + } + var m manifest + if err := json.Unmarshal(raw, &m); err != nil { + t.Fatal(err) + } + return m, string(raw) +} + +// The tools the bundle serves are the manifest's tools and the seat's verbs, each saying whether it reads +// or acts; the module requires the share, asks the server's seat and holds node-mounts. +func TestTheManifestNamesWhatTheBundleServesAndWhatItNeeds(t *testing.T) { + m, raw := readManifest(t) + var tools, verbs []string + for _, tl := range Tools(newSystem(t).machine()) { + if !strings.HasSuffix(tl.Description, "(r)") && !strings.HasSuffix(tl.Description, "(a)") { + t.Errorf("%s does not say whether it reads (r) or acts (a)", tl.Name) + } + if v, ok := strings.CutPrefix(tl.Name, Seat+"."); ok { + verbs = append(verbs, v) + } else { + tools = append(tools, tl.Name) + } + } + sort.Strings(tools) + sort.Strings(m.Tools) + if strings.Join(tools, ",") != strings.Join(m.Tools, ",") { + t.Errorf("served %v, listed %v", tools, m.Tools) + } + if len(m.Claims) != 1 || m.Claims[0]["name"] != Seat || m.Claims[0]["scope"] != "node" { + t.Fatalf("claims %v", m.Claims) + } + var serves []string + for _, v := range m.Claims[0]["serves"].([]any) { + serves = append(serves, v.(string)) + } + sort.Strings(serves) + sort.Strings(verbs) + if strings.Join(serves, ",") != strings.Join(verbs, ",") || strings.Join(verbs, ",") != "adopt,list,mount,test,unmount" { + t.Errorf("the seat serves %v, the bundle %v", serves, verbs) + } + if strings.Join(m.Requires, ",") != "nfs-share" { + t.Errorf("requires %v", m.Requires) + } + if strings.Join(m.Reads, ",") != ExportsState { + t.Errorf("the module reads nfs-server's exports from the bus: reads %v", m.Reads) + } + // It calls one verb: the controller's, for where the mesh keeps data on this machine (review finding 4d). + if strings.Join(m.Invokes, ",") != DataVerb { + t.Errorf("invokes %v, want %s", m.Invokes, DataVerb) + } + if !strings.Contains(raw, `node=${machine:name}`) { + t.Error("the config gives the process this machine's name, so it never mounts a share onto the machine that serves it") + } + if len(m.State) != 1 || m.State[0]["name"] != SharesState || m.State[0]["per-machine"] != true { + t.Errorf("state %v", m.State) + } + sec := m.OwnSecrets["smb-credentials"] + if sec["issued-by"] != "outside" || sec["taken"] != "at-start" || !strings.HasPrefix(sec["path"], "${dir:state}/") { + t.Errorf("the SMB logins are an outside party's, given with secret accept: %v", sec) + } + for _, key := range []string{"shares", "sources"} { + s := m.Settings[key] + if s["kind"] != "preference" || s["default"] != None || s["why"] == "" || !strings.Contains(raw, "${setting:"+key+"}") { + t.Errorf("setting %s: %v", key, s) + } + } + // A machine's or a person's name, as a word: "novox/hq" names the record, not the machine. + if hit := regexp.MustCompile(`\b(g14|novox|ace|shanks|jochen|recalbox)\b|/home/|/storage`).FindString(strings.ReplaceAll(raw, "novox/hq", "")); hit != "" { + t.Errorf("the manifest says %q", hit) + } + if strings.Contains(raw, "/etc/fstab") { + t.Error("the manifest places nothing in /etc/fstab: the operator's lines are taken over only by adopt") + } +} + +// Every long-running resource says how it is healthy (ADR 0240): apply by its unit, watch by mounts_health, +// both as root; the config file the process reads is the one the manifest places. +func TestTheProcessesAreRootAndJudgedByTheirUnitAndTheHealthTool(t *testing.T) { + m, raw := readManifest(t) + byID := map[string]map[string]any{} + for _, r := range m.Resources { + byID[r["id"].(string)] = r + if r["type"] == "process" || r["type"] == "service" || r["type"] == "container" { + if r["health"] == nil { + t.Errorf("%s runs and does not say how it is healthy", r["id"]) + } + } + } + apply, watch := byID["apply"], byID["watch"] + if h, _ := apply["health"].(map[string]any); apply["user"] != nil || h["kind"] != "unit" { + t.Errorf("apply runs as root and is judged by its unit: %v", apply) + } + if h, _ := watch["health"].(map[string]any); watch["user"] != nil || h["kind"] != "tool" || h["tool"] != "mounts_health" { + t.Errorf("watch runs as root and is judged by mounts_health: %v", watch) + } + for id, mode := range map[string]string{"apply": "apply", "watch": "watch"} { + if run, _ := byID[id]["run"].([]any); len(run) != 2 || run[1] != mode { + t.Errorf("%s runs the bundle's %s: %v", id, mode, byID[id]["run"]) + } + } + if byID["config"]["path"] != ConfigFile { + t.Errorf("the config file is %v, the code reads %s", byID["config"]["path"], ConfigFile) + } + content, _ := byID["config"]["content"].(string) + cfg := ParseConfig(content) + if len(cfg.Problems) != 0 && !strings.Contains(strings.Join(cfg.Problems, " "), "${") { + t.Errorf("the config's template reads: %v", cfg.Problems) + } + for _, want := range []string{"server=${bound:nfs-share:from}", "at=${bound:nfs-share:at}", "secret=${dir:state}/smb-credentials.secret"} { + if !strings.Contains(content, want) { + t.Errorf("the config lacks %s", want) + } + } + if byID["bus-dir"]["path"] != BusDir || byID["bus-dir"]["owner"] != "${machine:account}" { + t.Errorf("the bundle, as the operator account, writes the bus's answer: %v", byID["bus-dir"]) + } + if strings.Contains(raw, "password") { + t.Error("the manifest holds no password") + } +} diff --git a/modules/mounts/cmd/mounts/mounts.go b/modules/mounts/cmd/mounts/mounts.go new file mode 100644 index 00000000..f96992aa --- /dev/null +++ b/modules/mounts/cmd/mounts/mounts.go @@ -0,0 +1,430 @@ +package main + +import ( + "context" + "fmt" + "sort" + "strconv" + "strings" +) + +// networkTypes are the filesystem types whose files live on another machine: a mount of one waits on a +// server that may be away, which is what makes it worth telling apart. +var networkTypes = map[string]bool{ + "nfs": true, "nfs4": true, "cifs": true, "smb3": true, "smbfs": true, + "sshfs": true, "fuse.sshfs": true, "9p": true, "ceph": true, "glusterfs": true, "fuse.glusterfs": true, + "davfs": true, "fuse.rclone": true, +} + +// secretKeys are the mount options whose value is a credential. Their value is never answered. +var secretKeys = map[string]bool{"password": true, "pass": true, "passwd": true, "password2": true} + +// Redacted stands in for a credential's value. +const Redacted = "(redacted)" + +// RedactOptions answers a comma-separated option list with every credential's value replaced. The path +// of a credentials= file is kept: it names where the secret is, and the file is never read. +func RedactOptions(opts string) string { + if opts == "" { + return opts + } + parts := strings.Split(opts, ",") + var out []string + for i := 0; i < len(parts); i++ { + p := parts[i] + k, v, has := strings.Cut(p, "=") + key := strings.ToLower(strings.TrimSpace(k)) + switch { + case has && secretKeys[key]: + // mount.cifs writes a comma inside a password as two, so the value runs on over every ",,": an + // empty part, and the part after an empty one, are the password's. Read on the side of hiding: + // a run of three commas could be either, and is taken as the password's. + for i+1 < len(parts) && (parts[i+1] == "" || parts[i] == "") { + i++ + } + out = append(out, k+"="+Redacted) + case has && (key == "user" || key == "username") && strings.Contains(v, "%"): + // user=name%password: mount.cifs takes the password after the percent sign. + name, _, _ := strings.Cut(v, "%") + out = append(out, k+"="+name+"%"+Redacted) + default: + out = append(out, p) + } + } + return strings.Join(out, ",") +} + +// IsNetwork says whether a mount's files live on another machine, by its type or, for a type that +// does not say (fuse), by a source naming a host. +func IsNetwork(fstype, source string) bool { + if networkTypes[fstype] { + return true + } + return strings.HasPrefix(fstype, "fuse.") && strings.Contains(source, ":") && !strings.HasPrefix(source, "/") +} + +// unescape decodes the octal escapes fstab and mountinfo use for a space, a tab, a newline and a +// backslash in a path (\040, \011, \012, \134). +func unescape(s string) string { + if !strings.Contains(s, `\`) { + return s + } + var b strings.Builder + for i := 0; i < len(s); i++ { + if s[i] == '\\' && i+4 <= len(s) { + if n, err := strconv.ParseUint(s[i+1:i+4], 8, 8); err == nil { + b.WriteByte(byte(n)) + i += 3 + continue + } + } + b.WriteByte(s[i]) + } + return b.String() +} + +// FstabEntry is one line of /etc/fstab. +type FstabEntry struct { + Line int `json:"line"` + Source string `json:"source"` + Mountpoint string `json:"mountpoint"` + Type string `json:"type"` + Options string `json:"options"` + Dump string `json:"dump,omitempty"` + Pass string `json:"pass,omitempty"` + Network bool `json:"network"` +} + +// ParseFstab reads fstab text: comments and blank lines are skipped, and every credential in the +// options is redacted before it leaves this function. +func ParseFstab(text string) []FstabEntry { + out := []FstabEntry{} + for i, line := range strings.Split(text, "\n") { + line = strings.TrimSpace(line) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + f := strings.Fields(line) + if len(f) < 2 { + continue + } + e := FstabEntry{Line: i + 1, Source: unescape(f[0]), Mountpoint: unescape(f[1])} + if len(f) > 2 { + e.Type = f[2] + } + if len(f) > 3 { + e.Options = RedactOptions(f[3]) + } + if len(f) > 4 { + e.Dump = f[4] + } + if len(f) > 5 { + e.Pass = f[5] + } + e.Network = IsNetwork(e.Type, e.Source) + out = append(out, e) + } + return out +} + +// Mounted is one filesystem mounted now, as /proc/self/mountinfo says. +type Mounted struct { + Mountpoint string `json:"mountpoint"` + Source string `json:"source"` + Type string `json:"type"` + Options string `json:"options,omitempty"` +} + +// ParseMountinfo reads /proc/self/mountinfo: per line the mount point (field 5), and after the " - " +// separator the type, the source and the superblock options. The last mount at a point wins, as it +// is the one a path resolves to. +func ParseMountinfo(text string) map[string]Mounted { + out := map[string]Mounted{} + for _, line := range strings.Split(text, "\n") { + before, after, ok := strings.Cut(line, " - ") + if !ok { + continue + } + f := strings.Fields(before) + a := strings.Fields(after) + if len(f) < 5 || len(a) < 2 { + continue + } + mt := Mounted{Mountpoint: unescape(f[4]), Type: a[0], Source: unescape(a[1])} + if len(a) > 2 { + mt.Options = RedactOptions(a[2]) + } + out[mt.Mountpoint] = mt + } + return out +} + +// Unit is one mount or automount unit, as the service manager knows it. +type Unit struct { + Unit string `json:"unit"` + Kind string `json:"kind"` + Where string `json:"where"` + What string `json:"what,omitempty"` + Type string `json:"type,omitempty"` + Options string `json:"options,omitempty"` + Active string `json:"active"` + Sub string `json:"sub"` + Result string `json:"result,omitempty"` + From string `json:"from"` + FragmentPath string `json:"fragment_path,omitempty"` + SourcePath string `json:"source_path,omitempty"` + Timeout string `json:"timeout,omitempty"` + UnitFile string `json:"unit_file_state,omitempty"` +} + +// ShowProperties are what is asked of each unit. +var ShowProperties = "Id,What,Where,Type,Options,Result,ActiveState,SubState,FragmentPath,SourcePath,TimeoutUSec,UnitFileState" + +// ParseListUnits reads `systemctl list-units --plain --no-legend`: the unit name is the first field. +func ParseListUnits(text string) []string { + out := []string{} + for _, line := range strings.Split(text, "\n") { + f := strings.Fields(line) + if len(f) == 0 { + continue + } + name := strings.TrimLeft(f[0], "●* ") + if strings.HasSuffix(name, ".mount") || strings.HasSuffix(name, ".automount") { + out = append(out, name) + } + } + return out +} + +// ParseShow reads `systemctl show -p ... a b c`: one block of Key=Value lines per unit, blocks +// separated by a blank line. +func ParseShow(text string) []Unit { + out := []Unit{} + for _, block := range strings.Split(strings.ReplaceAll(text, "\r", ""), "\n\n") { + kv := map[string]string{} + for _, line := range strings.Split(block, "\n") { + k, v, ok := strings.Cut(line, "=") + if ok { + kv[k] = v + } + } + if kv["Id"] == "" { + continue + } + u := Unit{ + Unit: kv["Id"], Where: kv["Where"], What: kv["What"], Type: kv["Type"], + Options: RedactOptions(kv["Options"]), Active: kv["ActiveState"], Sub: kv["SubState"], + Result: kv["Result"], FragmentPath: kv["FragmentPath"], SourcePath: kv["SourcePath"], + Timeout: kv["TimeoutUSec"], UnitFile: kv["UnitFileState"], + } + u.Kind = "mount" + if strings.HasSuffix(u.Unit, ".automount") { + u.Kind = "automount" + } + switch { + case strings.HasPrefix(u.FragmentPath, "/run/systemd/generator"): + u.From = "generated from " + orDefault(u.SourcePath, "/etc/fstab") + case u.FragmentPath != "": + u.From = "unit file " + u.FragmentPath + default: + u.From = "made by the service manager at runtime (no unit file)" + } + out = append(out, u) + } + return out +} + +func orDefault(s, d string) string { + if s == "" { + return d + } + return s +} + +// Mount is one mount point, everything the machine knows of it joined: its fstab line, its mount and +// automount units, and what is mounted there now. +type Mount struct { + Mountpoint string `json:"mountpoint"` + Network bool `json:"network"` + Fstab *FstabEntry `json:"fstab,omitempty"` + MountUnit *Unit `json:"mount_unit,omitempty"` + Automount *Unit `json:"automount_unit,omitempty"` + MountedNow *Mounted `json:"mounted_now,omitempty"` +} + +// MountsReport is what mounts_list answers. +type MountsReport struct { + Said []string `json:"said"` + Mounts []Mount `json:"mounts"` + Notes []string `json:"notes,omitempty"` +} + +// virtualTypes are the kernel's own filesystems; a unit for one is never a storage question. +var virtualTypes = map[string]bool{ + "proc": true, "sysfs": true, "devtmpfs": true, "devpts": true, "tmpfs": true, "cgroup": true, "cgroup2": true, + "securityfs": true, "pstore": true, "efivarfs": true, "bpf": true, "debugfs": true, "tracefs": true, + "configfs": true, "fusectl": true, "mqueue": true, "hugetlbfs": true, "binfmt_misc": true, "autofs": true, + "nsfs": true, "overlay": true, "ramfs": true, "rpc_pipefs": true, "nfsd": true, +} + +// List joins fstab, the service manager's units and mountinfo, narrowed as asked. +func (m *Machine) List(onlyNetwork bool, match string) MountsReport { + r := MountsReport{Said: []string{}, Mounts: []Mount{}} + byPoint := map[string]*Mount{} + get := func(p string) *Mount { + if mt, ok := byPoint[p]; ok { + return mt + } + mt := &Mount{Mountpoint: p} + byPoint[p] = mt + return mt + } + + fstab, err := m.read("/etc/fstab") + if err != nil { + r.Notes = append(r.Notes, unreadable("/etc/fstab", err)) + } + for _, e := range ParseFstab(fstab) { + if e.Type == "swap" || e.Mountpoint == "none" { + continue + } + e := e + get(e.Mountpoint).Fstab = &e + } + + info, err := m.read("/proc/self/mountinfo") + if err != nil { + r.Notes = append(r.Notes, unreadable("/proc/self/mountinfo", err)) + } + now := ParseMountinfo(info) + + ctx := context.Background() + listed, err := m.Run(ctx, "systemctl", "list-units", "--type=mount,automount", "--all", "--no-legend", "--plain", "--no-pager") + if err != nil { + r.Notes = append(r.Notes, fmt.Sprintf("the service manager's units were not read: %v", err)) + } + if names := ParseListUnits(listed); len(names) > 0 { + args := append([]string{"show", "-p", ShowProperties, "--no-pager"}, names...) + shown, err := m.Run(ctx, "systemctl", args...) + if err != nil && shown == "" { + r.Notes = append(r.Notes, fmt.Sprintf("the units' properties were not read: %v", err)) + } + for _, u := range ParseShow(shown) { + if u.Where == "" || virtualTypes[u.Type] { + continue + } + u := u + mt := get(u.Where) + if u.Kind == "automount" { + mt.Automount = &u + } else { + mt.MountUnit = &u + } + } + } + + for p, mt := range byPoint { + if cur, ok := now[p]; ok { + cur := cur + mt.MountedNow = &cur + } + mt.Network = mt.Fstab != nil && mt.Fstab.Network || + mt.MountUnit != nil && IsNetwork(mt.MountUnit.Type, mt.MountUnit.What) || + mt.MountedNow != nil && IsNetwork(mt.MountedNow.Type, mt.MountedNow.Source) + } + + points := make([]string, 0, len(byPoint)) + for p := range byPoint { + points = append(points, p) + } + sort.Strings(points) + for _, p := range points { + mt := byPoint[p] + if onlyNetwork && !mt.Network { + continue + } + if match != "" && !strings.Contains(p, match) && !strings.Contains(mt.source(), match) { + continue + } + r.Mounts = append(r.Mounts, *mt) + r.Said = append(r.Said, mt.say()) + } + return r +} + +func (mt *Mount) source() string { + switch { + case mt.Fstab != nil: + return mt.Fstab.Source + case mt.MountUnit != nil: + return mt.MountUnit.What + case mt.MountedNow != nil: + return mt.MountedNow.Source + } + return "" +} + +func (mt *Mount) fstype() string { + switch { + case mt.Fstab != nil && mt.Fstab.Type != "": + return mt.Fstab.Type + case mt.MountUnit != nil && mt.MountUnit.Type != "": + return mt.MountUnit.Type + case mt.MountedNow != nil: + return mt.MountedNow.Type + } + return "?" +} + +func (mt *Mount) options() string { + if mt.Fstab != nil { + return mt.Fstab.Options + } + if mt.MountUnit != nil { + return mt.MountUnit.Options + } + return "" +} + +// say is the mount in one line: what it is, its state, where it is declared, and for a network +// mount whether it can hold up a boot. +func (mt *Mount) say() string { + head := fmt.Sprintf("%s (%s from %s)", mt.Mountpoint, mt.fstype(), mt.source()) + var state string + switch { + case mt.MountUnit != nil && mt.MountUnit.Active == "failed": + state = "failed" + if mt.MountUnit.Result != "" && mt.MountUnit.Result != "success" { + state += ", " + mt.MountUnit.Result + } + case mt.MountedNow != nil: + state = "mounted" + case mt.Automount != nil && mt.Automount.Active == "active": + state = "not mounted, waits for its first use (automount " + mt.Automount.Sub + ")" + case mt.MountUnit != nil: + state = "not mounted (" + mt.MountUnit.Active + ", " + mt.MountUnit.Sub + ")" + default: + state = "not mounted" + } + var from []string + switch { + case mt.Fstab != nil: + from = append(from, fmt.Sprintf("from /etc/fstab line %d", mt.Fstab.Line)) + case mt.MountUnit != nil: + from = append(from, mt.MountUnit.From) + } + if mt.Network { + opts := "," + mt.options() + "," + if mt.Automount == nil && !strings.Contains(opts, ",x-systemd.automount,") { + from = append(from, "no automount") + } + if !strings.Contains(opts, ",nofail,") { + from = append(from, "no nofail") + } + } + line := head + ": " + state + if len(from) > 0 { + line += "; " + strings.Join(from, ", ") + } + return line +} diff --git a/modules/mounts/cmd/mounts/mounts_test.go b/modules/mounts/cmd/mounts/mounts_test.go new file mode 100644 index 00000000..d342029d --- /dev/null +++ b/modules/mounts/cmd/mounts/mounts_test.go @@ -0,0 +1,211 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" +) + +// fake is a machine for a test: a tree standing in for /etc, /proc and /var/lib, and a runner +// answering from a table. +type fake struct { + t *testing.T + root string + answers map[string]string + zfs bool +} + +func newFake(t *testing.T) *fake { + t.Helper() + return &fake{t: t, root: t.TempDir(), answers: map[string]string{}} +} + +func (f *fake) file(path, text string) { + f.t.Helper() + p := filepath.Join(f.root, path) + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { + f.t.Fatal(err) + } + if err := os.WriteFile(p, []byte(text), 0o644); err != nil { + f.t.Fatal(err) + } +} + +func (f *fake) machine() *Machine { + return &Machine{ + Root: f.root, + Run: func(_ context.Context, name string, args ...string) (string, error) { + line := strings.TrimSpace(name + " " + strings.Join(args, " ")) + if out, ok := f.answers[line]; ok { + return out, nil + } + return "", fmt.Errorf("%s: not found", line) + }, + LookPath: func(n string) bool { return n == "zfs" && f.zfs }, + } +} + +func TestRedactOptions(t *testing.T) { + cases := []struct{ in, want string }{ + {"", ""}, + {"defaults,_netdev", "defaults,_netdev"}, + {"username=root,password=hunter2,iocharset=utf8", "username=root,password=(redacted),iocharset=utf8"}, + {"user=a,pass=x,PASSWD=y", "user=a,pass=(redacted),PASSWD=(redacted)"}, + {"credentials=/etc/samba/creds,sec=ntlmssp", "credentials=/etc/samba/creds,sec=ntlmssp"}, + {"password=", "password=(redacted)"}, + {"password2=z,passwordless", "password2=(redacted),passwordless"}, + } + for _, c := range cases { + if got := RedactOptions(c.in); got != c.want { + t.Errorf("RedactOptions(%q) = %q, want %q", c.in, got, c.want) + } + } +} + +const sampleFstab = `# /etc/fstab +UUID=aaaa / ext4 rw,relatime 0 1 +UUID=bbbb none swap defaults 0 0 +192.0.2.10:/storage/media /storage/media nfs defaults,_netdev 0 0 +//192.0.2.20/Share /mnt/my\040share cifs username=root,password=s3cret,iocharset=utf8 0 0 +192.0.2.10:/backup /mnt/backup nfs4 nofail,x-systemd.automount 0 0 +` + +func TestParseFstab(t *testing.T) { + got := ParseFstab(sampleFstab) + if len(got) != 5 { + t.Fatalf("got %d entries, want 5: %+v", len(got), got) + } + cases := []struct { + i int + point string + network bool + line int + }{ + {0, "/", false, 2}, {2, "/storage/media", true, 4}, {3, "/mnt/my share", true, 5}, {4, "/mnt/backup", true, 6}, + } + for _, c := range cases { + e := got[c.i] + if e.Mountpoint != c.point || e.Network != c.network || e.Line != c.line { + t.Errorf("entry %d = %+v, want point %q network %v line %d", c.i, e, c.point, c.network, c.line) + } + } + if strings.Contains(got[3].Options, "s3cret") { + t.Errorf("a password was answered: %q", got[3].Options) + } +} + +func TestIsNetwork(t *testing.T) { + cases := []struct { + typ, src string + want bool + }{ + {"nfs", "h:/x", true}, {"cifs", "//h/s", true}, {"fuse.sshfs", "u@h:/x", true}, {"ext4", "/dev/sda1", false}, + {"fuseblk", "/dev/sda2", false}, {"fuse.rclone", "remote:", true}, {"fuse.portal", "portal", false}, + } + for _, c := range cases { + if got := IsNetwork(c.typ, c.src); got != c.want { + t.Errorf("IsNetwork(%q, %q) = %v, want %v", c.typ, c.src, got, c.want) + } + } +} + +const sampleMountinfo = `22 1 259:3 / / rw,relatime shared:1 - ext4 /dev/nvme0n1p3 rw +40 22 0:50 / /mnt/backup rw,relatime shared:30 - autofs systemd-1 rw,fd=50 +61 40 0:60 / /mnt/backup rw,relatime shared:31 - nfs4 192.0.2.10:/backup rw,vers=4.2 +70 22 0:70 / /mnt/my\040share rw - cifs //192.0.2.20/Share rw,username=root,password=leaked +` + +func TestParseMountinfo(t *testing.T) { + got := ParseMountinfo(sampleMountinfo) + if got["/mnt/backup"].Type != "nfs4" { + t.Errorf("the last mount at a point must win: %+v", got["/mnt/backup"]) + } + share, ok := got["/mnt/my share"] + if !ok { + t.Fatalf("an escaped mount point was not decoded: %+v", got) + } + if strings.Contains(share.Options, "leaked") { + t.Errorf("a password in mountinfo was answered: %q", share.Options) + } +} + +const sampleListUnits = `-.mount loaded active mounted Root Mount +mnt-backup.automount loaded active waiting mnt-backup.automount +mnt-backup.mount loaded active mounted /mnt/backup +mnt-my\x20share.mount loaded failed failed /mnt/my share +storage-media.mount loaded failed failed /storage/media +dev-mqueue.mount loaded active mounted POSIX Message Queue File System +` + +var sampleShow = strings.Join([]string{ + "Id=-.mount\nWhere=/\nWhat=/dev/nvme0n1p3\nType=ext4\nOptions=rw\nResult=success\nActiveState=active\nSubState=mounted\nFragmentPath=/run/systemd/generator/-.mount\nSourcePath=/etc/fstab\nTimeoutUSec=1min 30s\nUnitFileState=generated", + "Id=mnt-backup.automount\nWhere=/mnt/backup\nResult=success\nActiveState=active\nSubState=waiting\nFragmentPath=/run/systemd/generator/mnt-backup.automount\nSourcePath=/etc/fstab\nUnitFileState=generated", + "Id=mnt-backup.mount\nWhere=/mnt/backup\nWhat=192.0.2.10:/backup\nType=nfs4\nOptions=rw,vers=4.2\nResult=success\nActiveState=active\nSubState=mounted\nFragmentPath=/run/systemd/generator/mnt-backup.mount\nSourcePath=/etc/fstab\nTimeoutUSec=1min 30s\nUnitFileState=generated", + "Id=mnt-my\\x20share.mount\nWhere=/mnt/my share\nWhat=//192.0.2.20/Share\nType=cifs\nOptions=username=root,password=s3cret\nResult=exit-code\nActiveState=failed\nSubState=failed\nFragmentPath=/run/systemd/generator/mnt-my\\x20share.mount\nSourcePath=/etc/fstab\nUnitFileState=generated", + "Id=storage-media.mount\nWhere=/storage/media\nWhat=192.0.2.10:/storage/media\nType=nfs\nOptions=defaults,_netdev\nResult=timeout\nActiveState=failed\nSubState=failed\nFragmentPath=/run/systemd/generator/storage-media.mount\nSourcePath=/etc/fstab\nTimeoutUSec=1min 30s\nUnitFileState=generated", + "Id=dev-mqueue.mount\nWhere=/dev/mqueue\nWhat=mqueue\nType=mqueue\nActiveState=active\nSubState=mounted\nFragmentPath=/usr/lib/systemd/system/dev-mqueue.mount\nUnitFileState=static", +}, "\n\n") + "\n" + +func listFake(t *testing.T) *fake { + f := newFake(t) + f.file("/etc/fstab", sampleFstab) + f.file("/proc/self/mountinfo", sampleMountinfo) + f.answers["systemctl list-units --type=mount,automount --all --no-legend --plain --no-pager"] = sampleListUnits + names := strings.Join(ParseListUnits(sampleListUnits), " ") + f.answers["systemctl show -p "+ShowProperties+" --no-pager "+names] = sampleShow + return f +} + +func TestList(t *testing.T) { + r := listFake(t).machine().List(false, "") + said := strings.Join(r.Said, "\n") + for _, want := range []string{ + "/storage/media (nfs from 192.0.2.10:/storage/media): failed, timeout; from /etc/fstab line 4, no automount, no nofail", + "/mnt/my share (cifs from //192.0.2.20/Share): failed, exit-code; from /etc/fstab line 5, no automount, no nofail", + "/mnt/backup (nfs4 from 192.0.2.10:/backup): mounted; from /etc/fstab line 6", + "/ (ext4 from UUID=aaaa): mounted; from /etc/fstab line 2", + } { + if !strings.Contains(said, want) { + t.Errorf("said lacks %q:\n%s", want, said) + } + } + if strings.Contains(said, "mqueue") { + t.Errorf("a kernel filesystem was said:\n%s", said) + } + for _, mt := range r.Mounts { + if mt.Mountpoint == "/mnt/backup" && (mt.Automount == nil || mt.MountUnit == nil) { + t.Errorf("/mnt/backup must carry both units: %+v", mt) + } + if mt.Mountpoint == "/storage/media" && mt.MountUnit.From != "generated from /etc/fstab" { + t.Errorf("from = %q", mt.MountUnit.From) + } + } + all, _ := json.Marshal(r) + if strings.Contains(string(all), "s3cret") || strings.Contains(string(all), "leaked") { + t.Errorf("a credential was answered: %s", all) + } +} + +func TestListNarrowed(t *testing.T) { + m := listFake(t).machine() + if r := m.List(true, ""); len(r.Mounts) != 3 { + t.Errorf("network only: got %d mounts, want 3: %v", len(r.Mounts), r.Said) + } + if r := m.List(false, "192.0.2.20"); len(r.Mounts) != 1 || r.Mounts[0].Mountpoint != "/mnt/my share" { + t.Errorf("match on source: %v", r.Said) + } +} + +func TestListWithoutFstabSaysSo(t *testing.T) { + f := newFake(t) + f.file("/proc/self/mountinfo", sampleMountinfo) + r := f.machine().List(false, "") + notes := strings.Join(r.Notes, "\n") + if !strings.Contains(notes, "/etc/fstab: not present") || !strings.Contains(notes, "units were not read") { + t.Errorf("a file or command that could not be read must be said: %q", notes) + } +} diff --git a/modules/mounts/cmd/mounts/review_test.go b/modules/mounts/cmd/mounts/review_test.go new file mode 100644 index 00000000..9e1c4c68 --- /dev/null +++ b/modules/mounts/cmd/mounts/review_test.go @@ -0,0 +1,329 @@ +package main + +// The review of 2026-10-08 (mesh-catalog #136): each finding reproduced by a test before it was fixed. + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// dataAnswer is mesh-controller.data's answer for this machine, as the bundle leaves it for the process. +func (f *fakeSystem) dataPaths(paths ...string) { + f.t.Helper() + if paths == nil { + paths = []string{} + } + raw, _ := json.Marshal(DataPaths{Machine: "workstation", Asked: time.Now().Format(time.RFC3339), Paths: paths}) + f.write(DataFile, string(raw)) +} + +func shareOf(st Status, name string) ShareStatus { + for _, s := range st.Shares { + if s.Name == name { + return s + } + } + return ShareStatus{} +} + +const libraryOnly = "shares=library=/srv/library\nserver=homeserver\nat=192.0.2.10\nnode=workstation\n" + +// Finding 8: a setting entry that cannot be read removed the module's pair for a mount point it had named +// before, because only what parsed counted as named. +func TestAnUnreadableEntryRemovesNoMount(t *testing.T) { + f := newSystem(t) + f.dataPaths() + f.write(ExportsFile, exportsAnswer(t, library)) + f.write(filepath.Join(UnitDir, "srv-library.mount"), Marker+"\n[Mount]\nWhat=192.0.2.10:/tank/media\nWhere=/srv/library\n") + f.write(filepath.Join(UnitDir, "srv-library.automount"), Marker+"\n[Automount]\nWhere=/srv/library\n") + // The operator meant library=/srv/library and wrote it without its slash. + f.write(ConfigFile, "shares=library=srv/library\nserver=homeserver\nat=192.0.2.10\nnode=workstation\n") + st := f.applier(time.Now()).Look(context.Background()) + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-library.mount")); err != nil { + t.Fatal("an entry that cannot be read removes no pair") + } + if f.called("systemctl stop srv-library.mount") || f.called("systemctl disable --now srv-library.automount") { + t.Errorf("nothing is stopped: %q", f.calls) + } + said := strings.Join(st.Problems, " ") + if !strings.Contains(said, "srv/library") || !strings.Contains(said, "nothing is removed") { + t.Errorf("the bad entry and what is kept are said: %q", st.Problems) + } +} + +// Finding 4a: a mount point already mounted by something else, or a folder with files in it, was taken. +func TestAMountPointInUseOrNotEmptyIsRefused(t *testing.T) { + f := newSystem(t) + f.dataPaths() + f.write(ConfigFile, "shares=library=/srv/library downloads=/srv/downloads\nserver=homeserver\nat=192.0.2.10\nnode=workstation\n") + f.write(ExportsFile, exportsAnswer(t, library, downloads)) + // Something else is mounted at the library's mount point; the downloads' folder holds a file. + f.write("/proc/self/mountinfo", "36 1 8:2 / /srv/library rw,relatime shared:1 - ext4 /dev/sdb1 rw\n") + f.write("/srv/downloads/notes.txt", "the operator's") + st := f.applier(time.Now()).Look(context.Background()) + for _, name := range []string{"library", "downloads"} { + s := shareOf(st, name) + if s.State != StateFailed { + t.Errorf("%s: %+v", name, s) + } + if f.called("systemctl enable --now srv-" + name + ".automount") { + t.Errorf("%s is not armed", name) + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-"+name+".mount")); !os.IsNotExist(err) { + t.Errorf("%s: no unit is written", name) + } + } + if s := shareOf(st, "library"); !strings.Contains(s.Why, "/dev/sdb1") { + t.Errorf("the library says what is mounted there: %q", s.Why) + } + if s := shareOf(st, "downloads"); !strings.Contains(s.Why, "not empty") { + t.Errorf("the downloads say the folder is not empty: %q", s.Why) + } + if f.read("/srv/downloads/notes.txt") != "the operator's" { + t.Error("the operator's file is untouched") + } +} + +// Its own share, mounted by its own unit, is not "in use by something else". +func TestTheModulesOwnMountIsNotRefused(t *testing.T) { + f := newSystem(t) + f.dataPaths() + f.write(ConfigFile, libraryOnly) + f.write(ExportsFile, exportsAnswer(t, library)) + f.write("/proc/self/mountinfo", "40 1 0:50 / /srv/library rw - autofs systemd-1 rw\n"+ + "41 40 0:51 / /srv/library rw - nfs4 192.0.2.10:/tank/media rw\n") + f.write(filepath.Join(UnitDir, "srv-library.mount"), Marker+"\n") + f.write(filepath.Join(UnitDir, "srv-library.automount"), Marker+"\n") + f.reachable["192.0.2.10:2049"] = true + st := f.applier(time.Now()).Look(context.Background()) + if s := shareOf(st, "library"); s.State != StateMounted { + t.Errorf("library %+v", s) + } +} + +// Finding 4b: a share the machine serves itself was mounted onto itself. +func TestAShareThisMachineServesIsRefused(t *testing.T) { + f := newSystem(t) + f.dataPaths() + f.write(ConfigFile, "shares=library=/srv/library\nserver=homeserver\nat=192.0.2.10\nnode=homeserver\n") + f.write(ExportsFile, exportsAnswer(t, library)) + st := f.applier(time.Now()).Look(context.Background()) + s := shareOf(st, "library") + if s.State != StateFailed || !strings.Contains(s.Why, "this machine shares it") { + t.Errorf("library %+v", s) + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-library.mount")); !os.IsNotExist(err) { + t.Error("no unit is written") + } +} + +// Finding 4c: removing a pair stopped its .mount even when what was mounted there was not its source. +func TestAMountThatIsNotItsOwnSourceIsNeverStopped(t *testing.T) { + f := newSystem(t) + f.dataPaths() + f.write(ConfigFile, "shares=none\nnode=workstation\n") + f.write(filepath.Join(UnitDir, "srv-old.mount"), Marker+"\n[Mount]\nWhat=192.0.2.10:/tank/old\nWhere=/srv/old\nType=nfs4\n") + f.write(filepath.Join(UnitDir, "srv-old.automount"), Marker+"\n[Automount]\nWhere=/srv/old\n") + f.write("/proc/self/mountinfo", "36 1 8:2 / /srv/old rw - ext4 /dev/sdb1 rw\n") + st := f.applier(time.Now()).Look(context.Background()) + if f.called("systemctl stop srv-old.mount") || f.called("systemctl disable --now srv-old.automount") { + t.Errorf("a mount that is not its own source is not stopped: %q", f.calls) + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-old.mount")); err != nil { + t.Error("and its pair is kept") + } + if !strings.Contains(strings.Join(st.Problems, " "), "/dev/sdb1") { + t.Errorf("said: %q", st.Problems) + } + // The verb unmount refuses the same. + f.write(StatusFile, `{"shares":[{"name":"old","kind":"nfs","mountpoint":"/srv/old","what":"192.0.2.10:/tank/old","state":"mounted","mount_unit":"srv-old.mount","automount_unit":"srv-old.automount"}]}`) + f.calls = nil + _, err := callVerb(t, f.machine(), "unmount", map[string]any{"name": "old"}) + if err == nil || !strings.Contains(err.Error(), "/dev/sdb1") || f.called("systemctl stop") { + t.Errorf("unmount refuses: %v %q", err, f.calls) + } +} + +// Finding 4d: a mount point at, under or above data the mesh keeps on this machine was taken. +func TestAMountPointNearTheMeshsDataIsRefused(t *testing.T) { + cases := map[string]string{ + "/mnt/plex": "above", + "/mnt/plex/data": "at", + "/mnt/plex/data/x": "under", + } + for point, how := range cases { + f := newSystem(t) + f.dataPaths("/mnt/plex/data", "/mnt/plex/config") + f.write(ConfigFile, "shares=library="+point+"\nserver=homeserver\nat=192.0.2.10\nnode=workstation\n") + f.write(ExportsFile, exportsAnswer(t, library)) + st := f.applier(time.Now()).Look(context.Background()) + s := shareOf(st, "library") + if s.State != StateFailed || !strings.Contains(s.Why, "/mnt/plex/data") || !strings.Contains(s.Why, how) { + t.Errorf("%s: %+v", point, s) + } + if f.called("systemctl enable") { + t.Errorf("%s is not armed", point) + } + } +} + +// When what the mesh keeps here cannot be read, nothing new is set up; what exists stays. +func TestWithoutTheMeshsDataNothingNewIsSetUp(t *testing.T) { + f := newSystem(t) + os.Remove(filepath.Join(f.root, DataFile)) + f.write(ConfigFile, libraryOnly) + f.write(ExportsFile, exportsAnswer(t, library)) + st := f.applier(time.Now()).Look(context.Background()) + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-library.mount")); !os.IsNotExist(err) { + t.Error("nothing new is written without the data list") + } + if s := shareOf(st, "library"); s.State != StateFailed || !strings.Contains(s.Why, "data") { + t.Errorf("library %+v", s) + } + // A pair that is already the module's stays and stays armed. + f.write(filepath.Join(UnitDir, "srv-library.mount"), Marker+"\n") + f.write(filepath.Join(UnitDir, "srv-library.automount"), Marker+"\n") + f.applier(time.Now()).Look(context.Background()) + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-library.mount")); err != nil { + t.Error("an existing pair stays") + } +} + +// The bundle asks the controller for this machine's data and leaves the paths for the process. +func TestTheBundleLeavesTheMeshsDataForTheProcess(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, libraryOnly) + answer := json.RawMessage(`{"answer":{"data":[{"machine":"workstation","module":"plex","item":"data","path":"/mnt/plex/data"}]},"ok":true}`) + paths, err := DataFrom(answer) + if err != nil || len(paths) != 1 || paths[0] != "/mnt/plex/data" { + t.Fatalf("paths %q %v", paths, err) + } + // The controller's verbs also answer as text in output. + paths, err = DataFrom(json.RawMessage(`{"ok":true,"output":"{\"data\":[{\"path\":\"/srv/x\"}]}"}`)) + if err != nil || len(paths) != 1 || paths[0] != "/srv/x" { + t.Fatalf("paths from output %q %v", paths, err) + } + if _, err := DataFrom(json.RawMessage(`{"ok":false}`)); err == nil { + t.Error("an answer with no data list is an error, not an empty list") + } +} + +// Finding 5: the server exports a share only to the nodes that want it, so the bundle says what this node +// wants of which server, also before the process has looked. +func TestTheBundleSaysWhatThisNodeWants(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, "shares=library=/srv/library downloads=/srv/downloads:ro\nserver=homeserver\nat=192.0.2.10\nnode=workstation\n") + var put []any + last := "" + ask := func(string) (json.RawMessage, error) { return json.RawMessage(`{"shares":[]}`), nil } + f.machine().Share("workstation", ask, func(_ string, v any) error { put = append(put, v); return nil }, &last, time.Now()) + if len(put) != 1 { + t.Fatalf("the wants are put before the process has looked: %d", len(put)) + } + raw, _ := json.Marshal(put[0]) + var got struct { + Wants []Want `json:"wants"` + } + json.Unmarshal(raw, &got) + want := []Want{{Server: "homeserver", Share: "downloads", Access: "ro"}, {Server: "homeserver", Share: "library", Access: "rw"}} + if len(got.Wants) != 2 || got.Wants[0] != want[0] || got.Wants[1] != want[1] { + t.Errorf("wants %s", raw) + } +} + +// Finding 9: the seat passes confirm as the string "true". +func TestConfirmIsTrueAsABooleanOrAString(t *testing.T) { + for _, v := range []any{true, "true"} { + if !flag(map[string]any{"confirm": v}, "confirm") { + t.Errorf("%#v is a confirmation", v) + } + } + for _, v := range []any{false, "false", "yes", "1", nil} { + if flag(map[string]any{"confirm": v}, "confirm") { + t.Errorf("%#v is not a confirmation", v) + } + } +} + +// Low: a password hidden in user=name%pass, and one holding a comma written as two (",,"). +func TestEveryWayAPasswordIsWrittenIsRedacted(t *testing.T) { + for _, opts := range []string{ + "user=bob%" + testPassword + ",vers=3.0", + "username=bob%" + testPassword, + "password=" + testPassword + ",,tail,vers=3.0", + "password=,,," + testPassword, + "pass=" + testPassword, + } { + if got := RedactOptions(opts); strings.Contains(got, testPassword) || strings.Contains(got, "tail") { + t.Errorf("RedactOptions(%q) = %q", opts, got) + } + line := "//192.0.2.76/Share /mnt/games cifs " + opts + " 0 0" + if got := RedactLine(line); strings.Contains(got, testPassword) || strings.Contains(got, "tail") { + t.Errorf("RedactLine(%q) = %q", line, got) + } + } + if got := RedactOptions("user=bob,vers=3.0"); got != "user=bob,vers=3.0" { + t.Errorf("a user name alone is kept: %q", got) + } +} + +func callVerb(t *testing.T, m *Machine, name string, args map[string]any) (any, error) { + t.Helper() + for _, tool := range Verbs(m) { + if tool.Name == Seat+"."+name { + return tool.Run(args) + } + } + t.Fatalf("no verb %s", name) + return nil, nil +} + +// The bundle asks the controller, leaves the paths, and keeps the last ones when an ask fails. +func TestTheBundleKeepsTheLastDataWhenAnAskFails(t *testing.T) { + f := newSystem(t) + m := f.machine() + var asked []string + ok := func(key string, body any) (json.RawMessage, error) { + raw, _ := json.Marshal(body) + asked = append(asked, key+" "+string(raw)) + return json.RawMessage(`{"answer":{"data":[{"path":"/mnt/plex/data"}]}}`), nil + } + if said := m.LeaveData("workstation", ok, time.Now()); len(said) != 0 { + t.Fatalf("said %q", said) + } + if len(asked) != 1 || asked[0] != `seat:mesh-controller.data {"machine":"workstation"}` { + t.Errorf("asked %q", asked) + } + failing := func(string, any) (json.RawMessage, error) { return nil, os.ErrDeadlineExceeded } + said := m.LeaveData("workstation", failing, time.Now()) + d, err := readJSON[DataPaths](m, DataFile) + if err != nil || len(d.Paths) != 1 || d.Paths[0] != "/mnt/plex/data" || d.Error == "" || len(said) == 0 { + t.Errorf("a failed ask keeps the last paths and says why: %+v %v %q", d, err, said) + } +} + +// The server decides each node's access (finding 5): a share it exports read-write may reach this node +// read-only, and a share it exports to other nodes only does not reach this one at all. The mount takes +// the access the server gave this node, never the share's own. +func TestTheServersAccessForThisNodeIsTheMountsAccess(t *testing.T) { + library := ServerShare{Name: "library", Path: "/srv/library", Access: "rw", Live: true, + To: []ShareClient{{Node: "ws", Address: "10.0.0.3", Access: "rw"}, {Node: "laptop", Address: "10.0.0.4", Access: "ro"}}} + got, ok := library.ForNode("laptop") + if !ok || got.Access != "ro" { + t.Errorf("the laptop, granted read-only, gets %q (exported to it: %v)", got.Access, ok) + } + if p := NFSPair(Wanted{Name: "library", Mountpoint: "/srv/library"}, "192.0.2.10", got); !strings.Contains(p.Mount, "Options=ro,") { + t.Errorf("a read-only grant mounted read-write:\n%s", p.Mount) + } + if got, ok := library.ForNode("ws"); !ok || got.Access != "rw" { + t.Errorf("the workstation, granted read-write, gets %q (%v)", got.Access, ok) + } + if _, ok := library.ForNode("control"); ok { + t.Errorf("a node the server does not export to is treated as exported to") + } +} diff --git a/modules/mounts/cmd/mounts/tools.go b/modules/mounts/cmd/mounts/tools.go new file mode 100644 index 00000000..3e29881f --- /dev/null +++ b/modules/mounts/cmd/mounts/tools.go @@ -0,0 +1,56 @@ +package main + +import ( + "strings" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +// Tools is the module's tools over one machine: the two that read what the machine has, the seat's verbs +// and the health check (verbs.go). +func Tools(m *Machine) []stdio.Tool { + return append([]stdio.Tool{ + { + Name: "mounts_list", + Description: "This machine's mounts as fstab and the service manager know them, joined by mount point: each /etc/fstab line " + + "(source, type, options, dump and pass), its mount and automount units (state, result, timeout, and whether generated " + + "from fstab or a unit file), and whether it is mounted now (from /proc/self/mountinfo, never by touching the mount point, " + + "so a server that is away cannot hang the call). Network filesystems (NFS, SMB, sshfs) are marked, and a network mount " + + "without automount or nofail is said. Credentials in options are never answered. Each mount is said in a line under " + + "`said`. Replaces cat /etc/fstab, findmnt and systemctl list-units --type=mount. (r)", + Input: map[string]any{ + "network": map[string]any{"type": "boolean", "description": "only network filesystems (default false)"}, + "match": map[string]any{"type": "string", "description": "only mounts whose mount point or source contains this"}, + }, + Run: func(args map[string]any) (any, error) { + return m.List(flag(args, "network"), text(args, "match")), nil + }, + }, + { + Name: "mounts_exports", + Description: "What this machine shares over the network: every NFS export in /etc/exports and /etc/exports.d (path, client, " + + "options, file), the kernel's live export table, the ZFS sharenfs and sharesmb properties where ZFS is present, the " + + "Samba shares in smb.conf (path, valid users, read only or writable, force user — never a password), and whether " + + "nfs-server and smb are active. An export that is declared and not live, or live and declared nowhere, is said. " + + "Replaces cat /etc/exports, exportfs -v, zfs get sharenfs and testparm -s. (r)", + Run: func(map[string]any) (any, error) { + return m.Exports(), nil + }, + }, + }, Verbs(m)...) +} + +func text(args map[string]any, key string) string { + s, _ := args[key].(string) + return strings.TrimSpace(s) +} + +func flag(args map[string]any, key string) bool { + switch v := args[key].(type) { + case bool: + return v + case string: + return v == "true" + } + return false +} diff --git a/modules/mounts/cmd/mounts/units.go b/modules/mounts/cmd/mounts/units.go new file mode 100644 index 00000000..bac0357e --- /dev/null +++ b/modules/mounts/cmd/mounts/units.go @@ -0,0 +1,179 @@ +package main + +// The unit pair for one share (novox/hq ADR 0263 rule 4, to-be 52 §2): a `.mount` that is never enabled +// and an `.automount` that is. Nothing is mounted at boot and nothing can fail a boot: the mount is made +// at first use and released after ten idle minutes, and a mount point is always an autofs mount, so +// nothing writes into the empty directory under it. + +import ( + "fmt" + "strconv" + "strings" +) + +// Marker opens every unit file this module writes. A unit file without it is not the module's, and the +// module never writes over, stops or removes one. +const Marker = "# Written by the mesh (module mounts, novox/hq ADR 0263)." + +// Timing, in seconds. +const ( + IdleSeconds = 600 // released after ten idle minutes + MountSeconds = 30 // a share of a node + OccasionalSeconds = 10 // a source that comes and goes +) + +// EscapePath is `systemd-escape --path`: the name systemd gives the units for a mount point. +func EscapePath(p string) string { + p = strings.Trim(p, "/") + if p == "" { + return "-" + } + var b strings.Builder + for i := 0; i < len(p); i++ { + c := p[i] + switch { + case c == '/': + b.WriteByte('-') + case c == '.' && i == 0: + fmt.Fprintf(&b, `\x%02x`, c) + case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == ':', c == '_', c == '.': + b.WriteByte(c) + default: + fmt.Fprintf(&b, `\x%02x`, c) + } + } + return b.String() +} + +// Pair is one share's two units, as written. +type Pair struct { + Name string `json:"name"` + Kind string `json:"kind"` // nfs, smb or disk + Occasional bool `json:"occasional"` + Mountpoint string `json:"mountpoint"` + What string `json:"what"` + // Probe is what is tried to tell whether the source is there: host:port, or a device path. + Probe string `json:"probe"` + MountUnit string `json:"mount_unit"` + AutoUnit string `json:"automount_unit"` + Mount string `json:"-"` + Automount string `json:"-"` + // Credentials is the path of the source's credentials file (SMB), written beside the units. + Credentials string `json:"credentials,omitempty"` +} + +// ServerShare is one share as the sharing machine says it exports it (its `node-nfs-server.exports`). +type ServerShare struct { + Name string `json:"name"` + Path string `json:"path"` + Access string `json:"access"` + UID int `json:"uid"` + GID int `json:"gid"` + Clients string `json:"clients,omitempty"` + Live bool `json:"live"` + // To is each node the server exports the share to, at the access it decided for that node. + To []ShareClient `json:"to,omitempty"` +} + +// ShareClient is one node a share is exported to, as the server decided it. +type ShareClient struct { + Node string `json:"node"` + Address string `json:"address"` + Access string `json:"access"` +} + +// ForNode is the share as the server exports it to one node: its access is the lower of the share's own +// and the node's, and false when the server does not export it to that node at all. +func (e ServerShare) ForNode(node string) (ServerShare, bool) { + for _, c := range e.To { + if c.Node == node { + if c.Access == "ro" || e.Access == "ro" { + e.Access = "ro" + } + return e, true + } + } + return e, false +} + +func header(name, kind string) string { + return Marker + "\n# The " + kind + " " + name + ". Replaced whenever the module's settings or the sharing machine's\n" + + "# answer change, and removed when nothing asks for it; edit nothing here: change the module's settings.\n" +} + +// NFSPair is the pair for a share of a node: NFS version 4.2 from the sharing machine's address on the +// private network. Read-write is mounted hard, so a write is never dropped while the server is away; +// read-only, soft, so a reader gets an error rather than waiting. +func NFSPair(w Wanted, at string, e ServerShare) Pair { + ro := w.ReadOnly || e.Access == "ro" + opts := "vers=4.2,hard,timeo=150,retrans=3,_netdev" + if ro { + opts = "ro,vers=4.2,soft,timeo=150,retrans=3,_netdev" + } + p := Pair{Name: w.Name, Kind: "nfs", Mountpoint: w.Mountpoint, What: at + ":" + e.Path, Probe: hostPort(at, 2049)} + p.render("share", "nfs4", opts, MountSeconds, "remote-fs.target", "") + return p +} + +// SMBPair is the pair for an occasional SMB source. Its login is in a file only root reads. +func SMBPair(s Source, credentials string, uid, gid int) Pair { + opts := "credentials=" + credentials + ",iocharset=utf8,_netdev,nofail" + if uid >= 0 { + opts += ",uid=" + strconv.Itoa(uid) + ",gid=" + strconv.Itoa(gid) + } + if s.ReadOnly { + opts = "ro," + opts + } + p := Pair{Name: s.Name, Kind: "smb", Occasional: true, Mountpoint: s.Mountpoint, + What: "//" + s.Host + "/" + s.Share, Probe: hostPort(s.Host, 445), Credentials: credentials} + p.render("source", "cifs", opts, OccasionalSeconds, "remote-fs.target", "") + return p +} + +// DiskPair is the pair for a disk that is plugged in now and then: bound to its device, so the mount +// goes when the disk does. +func DiskPair(s Source) Pair { + dev := "/dev/disk/by-uuid/" + s.UUID + opts := "nofail" + if s.ReadOnly { + opts = "ro,nofail" + } + p := Pair{Name: s.Name, Kind: "disk", Occasional: true, Mountpoint: s.Mountpoint, What: dev, Probe: dev} + p.render("source", "", opts, OccasionalSeconds, "local-fs.target", EscapePath(dev)+".device") + return p +} + +func hostPort(host string, port int) string { + if strings.Contains(host, ":") && !strings.HasPrefix(host, "[") { + return "[" + host + "]:" + strconv.Itoa(port) + } + return host + ":" + strconv.Itoa(port) +} + +func (p *Pair) render(kind, fstype, opts string, timeout int, target, device string) { + base := EscapePath(p.Mountpoint) + p.MountUnit, p.AutoUnit = base+".mount", base+".automount" + var m strings.Builder + m.WriteString(header(p.Name, kind)) + m.WriteString("[Unit]\nDescription=" + p.Name + " (" + p.What + "), mounted by the mesh\n") + if device != "" { + m.WriteString("BindsTo=" + device + "\nAfter=" + device + "\n") + } else { + m.WriteString("Wants=network-online.target\nAfter=network-online.target\n") + } + m.WriteString("\n[Mount]\nWhat=" + p.What + "\nWhere=" + p.Mountpoint + "\n") + if fstype != "" { + m.WriteString("Type=" + fstype + "\n") + } + m.WriteString("Options=" + opts + "\nTimeoutSec=" + strconv.Itoa(timeout) + "\n") + p.Mount = m.String() + + var a strings.Builder + a.WriteString(header(p.Name, kind)) + // The automount is armed whether or not the source is there; only the mount waits on it, so an + // absent disk leaves an armed automount, never a failed unit. + a.WriteString("[Unit]\nDescription=" + p.Name + ", mounted by the mesh at first use\n") + a.WriteString("\n[Automount]\nWhere=" + p.Mountpoint + "\nTimeoutIdleSec=" + strconv.Itoa(IdleSeconds) + "\n") + a.WriteString("\n[Install]\nWantedBy=" + target + "\n") + p.Automount = a.String() +} diff --git a/modules/mounts/cmd/mounts/verbs.go b/modules/mounts/cmd/mounts/verbs.go new file mode 100644 index 00000000..aadf29bd --- /dev/null +++ b/modules/mounts/cmd/mounts/verbs.go @@ -0,0 +1,153 @@ +package main + +// The seat node-mounts' verbs (novox/hq ADR 0263), served as `node-mounts.` so the runtime answers +// them on the seat's subject, and the module's health check. + +import ( + "context" + "fmt" + "path/filepath" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +// Seat is the node seat this module holds. +const Seat = "node-mounts" + +func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool { + return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run} +} + +var nameArg = map[string]any{"type": "string", "description": "the share's or source's name, as the module's settings give it"} + +// share is one share or source the process last found, by name. +func (m *Machine) share(name string) (ShareStatus, error) { + st, err := readJSON[Status](m, StatusFile) + if err != nil { + return ShareStatus{}, fmt.Errorf("the module's process has left no account yet: %v", err) + } + for _, s := range st.Shares { + if s.Name == name { + return s, nil + } + } + return ShareStatus{}, fmt.Errorf("this machine is asked to mount nothing named %q", name) +} + +// ownPair refuses a share whose units are not both the module's. +func (m *Machine) ownPair(s ShareStatus) error { + if s.MountUnit == "" || !IsOwnUnit(m, filepath.Join(UnitDir, s.MountUnit)) || !IsOwnUnit(m, filepath.Join(UnitDir, s.AutoUnit)) { + return fmt.Errorf("%s has no unit pair of the module's (%s): %s", s.Name, s.State, s.Why) + } + return nil +} + +// Verbs are the seat's verbs and the health check, over one machine. +func Verbs(m *Machine) []stdio.Tool { + ctx := context.Background() + return []stdio.Tool{ + verb("list", "Every mount point on this machine — its fstab line, its mount and automount units, whether it is "+ + "mounted now — and every share and occasional source the module is asked to mount, with its state (armed, "+ + "mounted, absent, unreachable, failed), since when and why. Credentials are never answered. Replaces cat "+ + "/etc/fstab, findmnt and systemctl list-units --type=mount. (r)", + map[string]any{"network": map[string]any{"type": "boolean", "description": "only network filesystems (default false)"}}, + func(a map[string]any) (any, error) { + st, err := readJSON[Status](m, StatusFile) + out := map[string]any{"machine": m.List(flag(a, "network"), "")} + if err != nil { + out["shares"] = "the module's process has left no account yet: " + err.Error() + } else { + out["shares"], out["looked"], out["problems"] = st.Shares, st.Looked, st.Problems + } + return out, nil + }), + verb("test", "Whether one share's or source's server answers now (TCP 2049 for NFS, 445 for SMB; for a disk, "+ + "whether it is plugged in), tried without touching its mount point. Replaces showmount -e and ping. (r)", + map[string]any{"name": nameArg}, + func(a map[string]any) (any, error) { + s, err := m.share(text(a, "name")) + if err != nil { + return nil, err + } + if s.Probe == "" { + return map[string]any{"name": s.Name, "answers": false, "said": "nothing to try: " + s.Why}, nil + } + ap := NewApplier() + ap.M = m + p := Pair{Kind: s.Kind, Probe: s.Probe} + start := time.Now() + var tried error + if p.Kind == "disk" { + if !m.present(p.Probe) { + tried = fmt.Errorf("the disk %s is not plugged in", p.Probe) + } + } else { + tried = ap.Dial(p.Probe, ProbeTimeout) + } + out := map[string]any{"name": s.Name, "tried": s.Probe, "answers": tried == nil, "took_ms": time.Since(start).Milliseconds()} + if tried != nil { + out["said"] = tried.Error() + } + return out, nil + }), + verb("mount", "Mount one share or source now, instead of waiting for its first use: its mount unit is "+ + "started. Replaces mount. (a)", + map[string]any{"name": nameArg}, + func(a map[string]any) (any, error) { + s, err := m.share(text(a, "name")) + if err == nil { + err = m.ownPair(s) + } + if err != nil { + return nil, err + } + if _, err := m.Escalate(ctx, "systemctl", "start", s.MountUnit); err != nil { + return nil, err + } + return map[string]any{"name": s.Name, "said": fmt.Sprintf("%s is mounted at %s.", s.Name, s.Mountpoint)}, nil + }), + verb("unmount", "Release one share or source now: its mount unit is stopped, and its automount stays armed, "+ + "so the next use mounts it again. Replaces umount. (a)", + map[string]any{"name": nameArg}, + func(a map[string]any) (any, error) { + s, err := m.share(text(a, "name")) + if err == nil { + err = m.ownPair(s) + } + if err != nil { + return nil, err + } + // Never stop a mount whose filesystem is not its own source (review of 2026-10-08, finding 4c). + info, _ := m.read("/proc/self/mountinfo") + if err := m.notOwnSource(s.Kind, s.What, s.Mountpoint, ParseMountinfo(info)); err != nil { + return nil, fmt.Errorf("%s is not released: %v", s.Name, err) + } + if _, err := m.Escalate(ctx, "systemctl", "stop", s.MountUnit); err != nil { + return nil, err + } + return map[string]any{"name": s.Name, "said": fmt.Sprintf("%s is released; its automount waits for the next use.", s.Name)}, nil + }), + verb("adopt", "Take over one hand-written /etc/fstab line, on a person's word: refused unless this module's own "+ + "unit pair for the same mount point is written and its automount armed. Then the line is commented out with "+ + "a dated marker (any password in it redacted), the file as it was is kept beside it readable by root only, "+ + "and the service manager reloads. Without confirm, a dry run that changes nothing. Replaces sed -i /etc/fstab. (a)", + map[string]any{ + "mountpoint": map[string]any{"type": "string", "description": "the mount point of the line, as fstab names it"}, + "confirm": map[string]any{"type": "boolean", "description": "true to change the file; otherwise a dry run"}, + "why": map[string]any{"type": "string", "description": "why the line is taken over (required with confirm)"}, + }, + func(a map[string]any) (any, error) { + return m.Adopt(ctx, text(a, "mountpoint"), flag(a, "confirm"), text(a, "why"), time.Now()) + }), + { + Name: "mounts_health", + Description: "Whether this machine's shares are healthy, as the module's health check (novox/hq ADR 0263): unhealthy " + + "when a share that should always be there has been unreachable or failed for five minutes, an automount is not " + + "armed, something the settings name cannot be set up (a missing login, a unit file that is not the module's), or " + + "the module's process has not looked for two minutes; with why, one sentence per finding. An occasional source " + + "that is absent is healthy, and said as not present since when. (r)", + Run: func(map[string]any) (any, error) { return m.Health(time.Now()), nil }, + }, + } +} diff --git a/modules/mounts/cmd/mounts/write_test.go b/modules/mounts/cmd/mounts/write_test.go new file mode 100644 index 00000000..bf06d3d4 --- /dev/null +++ b/modules/mounts/cmd/mounts/write_test.go @@ -0,0 +1,562 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// The secret every test that has one uses; it must never appear in anything the module says. +const testPassword = "s3cr3t-Pa55" + +// fakeSystem is a machine under a temporary root with a recorded service manager. +type fakeSystem struct { + t *testing.T + root string + calls []string + // units are the service manager's answers: ActiveState and UnitFileState by unit. + units map[string]map[string]string + // reachable are the host:port pairs that answer. + reachable map[string]bool +} + +func newSystem(t *testing.T) *fakeSystem { + f := &fakeSystem{t: t, root: t.TempDir(), units: map[string]map[string]string{}, reachable: map[string]bool{}} + for _, d := range []string{UnitDir, RunDir, BusDir, "/etc/mesh-mounts", "/proc/self", "/var/lib/mesh/mounts"} { + if err := os.MkdirAll(filepath.Join(f.root, d), 0o755); err != nil { + t.Fatal(err) + } + } + f.write("/proc/self/mountinfo", "") + // The mesh keeps no data on this machine, as the controller said through the bundle. + f.write(DataFile, `{"machine":"workstation","paths":[]}`) + return f +} + +func (f *fakeSystem) write(p, content string) { + f.t.Helper() + full := filepath.Join(f.root, p) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + f.t.Fatal(err) + } + if err := os.WriteFile(full, []byte(content), 0o644); err != nil { + f.t.Fatal(err) + } +} + +func (f *fakeSystem) read(p string) string { + b, _ := os.ReadFile(filepath.Join(f.root, p)) + return string(b) +} + +func (f *fakeSystem) run(_ context.Context, name string, args ...string) (string, error) { + f.calls = append(f.calls, name+" "+strings.Join(args, " ")) + if name != "systemctl" || len(args) == 0 { + return "", nil + } + switch args[0] { + case "show": + if len(args) >= 5 && args[3] == "--value" { + return f.units[args[4]]["ActiveState"] + "\n", nil + } + var blocks []string + for _, u := range args { + if !strings.HasSuffix(u, ".mount") && !strings.HasSuffix(u, ".automount") { + continue + } + s := f.units[u] + blocks = append(blocks, "Id="+u+"\nActiveState="+orDefault(s["ActiveState"], "inactive")+"\nSubState=dead\nResult="+ + orDefault(s["Result"], "success")+"\nUnitFileState="+orDefault(s["UnitFileState"], "disabled")) + } + return strings.Join(blocks, "\n\n") + "\n", nil + case "enable": + u := args[len(args)-1] + f.units[u] = map[string]string{"ActiveState": "active", "UnitFileState": "enabled"} + } + return "", nil +} + +func (f *fakeSystem) machine() *Machine { + return &Machine{Root: f.root, Run: f.run, Escalate: f.run, LookPath: func(string) bool { return true }} +} + +func (f *fakeSystem) applier(now time.Time) *Applier { + return &Applier{M: f.machine(), Act: true, Judge: true, + Dial: func(addr string, _ time.Duration) error { + if f.reachable[addr] { + return nil + } + return errors.New("dial tcp " + addr + ": connect: no route to host") + }, + Now: func() time.Time { return now }, + Lookup: func(string) (int, int, error) { return 1000, 1000, nil }, + } +} + +func (f *fakeSystem) called(prefix string) bool { + for _, c := range f.calls { + if strings.HasPrefix(c, prefix) { + return true + } + } + return false +} + +// everything is every file under the root, for a search that a secret appears nowhere it should not. +func (f *fakeSystem) everything(except ...string) string { + var b strings.Builder + filepath.Walk(f.root, func(p string, info os.FileInfo, err error) error { + if err != nil || info.IsDir() { + return nil + } + for _, e := range except { + if strings.HasSuffix(p, e) { + return nil + } + } + raw, _ := os.ReadFile(p) + b.Write(raw) + return nil + }) + return b.String() +} + +const config = "shares=library=/srv/library downloads=/srv/downloads:ro\n" + + "sources=games=smb://192.0.2.76/Share@/mnt/games usb=disk:1234-ABCD@/mnt/usb\n" + + "server=homeserver\nat=192.0.2.10\nnode=workstation\naccount=operator\n" + + "secret=/var/lib/mesh/mounts/smb-credentials.secret\nstate=/var/lib/mesh/mounts\n" + +func exportsAnswer(t *testing.T, shares ...ServerShare) string { + raw, err := json.Marshal(ExportsAnswer{Server: "homeserver", At: "192.0.2.10", Shares: shares}) + if err != nil { + t.Fatal(err) + } + return string(raw) +} + +var ( + library = ServerShare{Name: "library", Path: "/tank/media", Access: "rw", UID: 1001, GID: 2000, Live: true, To: []ShareClient{{Node: "workstation", Address: "10.0.0.3", Access: "rw"}}} + downloads = ServerShare{Name: "downloads", Path: "/tank/downloads", Access: "rw", UID: 1001, GID: 2000, Live: true, To: []ShareClient{{Node: "workstation", Address: "10.0.0.3", Access: "rw"}}} +) + +func TestTheSettingsAreReadAndWhatCannotBeIsSaidPlainly(t *testing.T) { + shares, problems := ParseShares("library=/srv/library downloads=/srv/downloads:ro,bad Upper=/x other=relative library=/srv/again root=/") + if len(shares) != 2 || shares[0] != (Wanted{"library", "/srv/library", false}) || shares[1] != (Wanted{"downloads", "/srv/downloads", true}) { + t.Fatalf("shares %+v", shares) + } + want := []string{`"bad" is not name=mountpoint`, `"Upper" is not lower-case`, `"relative" is not an absolute path`, "named twice", `"/" is the system's own directory`} + for i, w := range want { + if i >= len(problems) || !strings.Contains(problems[i], w) { + t.Errorf("problem %d: want %q in %q", i, w, problems) + } + } + if s, p := ParseShares("none"); len(s) != 0 || len(p) != 0 { + t.Errorf("none is nothing: %v %v", s, p) + } + sources, problems := ParseSources("games=smb://192.0.2.76/Share@/mnt/games:ro usb=disk:1234-ABCD@/mnt/usb ftp=ftp://x/y@/mnt/x nomount=smb://h/s") + if len(sources) != 2 || sources[0].Kind != "smb" || sources[0].Host != "192.0.2.76" || sources[0].Share != "Share" || !sources[0].ReadOnly || + sources[1].Kind != "disk" || sources[1].UUID != "1234-ABCD" { + t.Fatalf("sources %+v", sources) + } + if len(problems) != 2 || !strings.Contains(problems[0], "neither smb://") || !strings.Contains(problems[1], "names no mount point") { + t.Errorf("problems %q", problems) + } + if err := CheckMountpoint("/etc/x"); err == nil { + t.Error("a mount point under /etc is the system's") + } +} + +func TestAMountPointIsNamedAsSystemdEscapeNamesIt(t *testing.T) { + for in, want := range map[string]string{ + "/srv/library": "srv-library", + "/storage/media": "storage-media", + "/mnt/my-disk": `mnt-my\x2ddisk`, + "/mnt/.hidden": `mnt-.hidden`, + "/.x": `\x2ex`, + "/srv/a b": `srv-a\x20b`, + "/dev/disk/by-uuid/1234-ABCD": `dev-disk-by\x2duuid-1234\x2dABCD`, + } { + if got := EscapePath(in); got != want { + t.Errorf("EscapePath(%q) = %q, want %q", in, got, want) + } + } +} + +func TestANodesShareIsMountedHardWhenWritableAndSoftWhenNot(t *testing.T) { + rw := NFSPair(Wanted{Name: "library", Mountpoint: "/srv/library"}, "192.0.2.10", library) + for _, want := range []string{"What=192.0.2.10:/tank/media", "Where=/srv/library", "Type=nfs4", + "Options=vers=4.2,hard,timeo=150,retrans=3,_netdev", "TimeoutSec=30", "After=network-online.target"} { + if !strings.Contains(rw.Mount, want) { + t.Errorf("the read-write mount lacks %q:\n%s", want, rw.Mount) + } + } + if !strings.HasPrefix(rw.Mount, Marker) || !strings.HasPrefix(rw.Automount, Marker) { + t.Error("both units open with the module's marker") + } + if strings.Contains(rw.Mount, "[Install]") { + t.Error("the mount is never enabled: only the automount is") + } + for _, want := range []string{"[Automount]\nWhere=/srv/library\nTimeoutIdleSec=600", "WantedBy=remote-fs.target"} { + if !strings.Contains(rw.Automount, want) { + t.Errorf("the automount lacks %q:\n%s", want, rw.Automount) + } + } + if rw.MountUnit != "srv-library.mount" || rw.AutoUnit != "srv-library.automount" || rw.Probe != "192.0.2.10:2049" { + t.Errorf("names %+v", rw) + } + ro := NFSPair(Wanted{Name: "downloads", Mountpoint: "/srv/downloads", ReadOnly: true}, "192.0.2.10", downloads) + if !strings.Contains(ro.Mount, "Options=ro,vers=4.2,soft,") { + t.Errorf("a read-only share is mounted ro and soft:\n%s", ro.Mount) + } + served := library + served.Access = "ro" + if p := NFSPair(Wanted{Name: "library", Mountpoint: "/srv/library"}, "192.0.2.10", served); !strings.Contains(p.Mount, "Options=ro,") { + t.Error("a share the server exports read-only is mounted read-only") + } +} + +func TestAnOccasionalSourceWaitsTenSecondsAndNeverFailsABoot(t *testing.T) { + smb := SMBPair(Source{Name: "games", Kind: "smb", Host: "192.0.2.76", Share: "Share", Mountpoint: "/mnt/games"}, "/var/lib/mesh/mounts/smb-games.credentials", 1000, 1000) + for _, want := range []string{"What=//192.0.2.76/Share", "Type=cifs", "credentials=/var/lib/mesh/mounts/smb-games.credentials", "nofail", "uid=1000,gid=1000", "TimeoutSec=10"} { + if !strings.Contains(smb.Mount, want) { + t.Errorf("the SMB mount lacks %q:\n%s", want, smb.Mount) + } + } + if !smb.Occasional || smb.Probe != "192.0.2.76:445" || strings.Contains(smb.Mount, "password") { + t.Errorf("smb %+v", smb) + } + disk := DiskPair(Source{Name: "usb", Kind: "disk", UUID: "1234-ABCD", Mountpoint: "/mnt/usb"}) + dev := `dev-disk-by\x2duuid-1234\x2dABCD.device` + if !strings.Contains(disk.Mount, "BindsTo="+dev) || !strings.Contains(disk.Mount, "What=/dev/disk/by-uuid/1234-ABCD") { + t.Errorf("the disk's mount goes with the disk:\n%s", disk.Mount) + } + if strings.Contains(disk.Automount, "BindsTo") || !strings.Contains(disk.Automount, "WantedBy=local-fs.target") { + t.Errorf("the disk's automount stays armed when the disk is away:\n%s", disk.Automount) + } +} + +func TestALookWritesArmsAndRemovesOnlyTheModulesOwnUnits(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, config) + f.write(ExportsFile, exportsAnswer(t, library, downloads)) + f.write("/var/lib/mesh/mounts/smb-credentials.secret", "[games]\nusername=root\npassword="+testPassword+"\n") + // The operator's own unit for the downloads' mount point: never written over. + f.write(filepath.Join(UnitDir, "srv-downloads.mount"), "[Mount]\nWhat=elsewhere\n") + // A pair of the module's whose mount point nothing names any more. + f.write(filepath.Join(UnitDir, "srv-old.mount"), Marker+"\n[Mount]\n") + f.write(filepath.Join(UnitDir, "srv-old.automount"), Marker+"\n[Automount]\n") + f.reachable["192.0.2.10:2049"] = true + now := time.Date(2026, 10, 8, 19, 21, 0, 0, time.UTC) + + st := f.applier(now).Look(context.Background()) + + if got := f.read(filepath.Join(UnitDir, "srv-library.mount")); !strings.Contains(got, "What=192.0.2.10:/tank/media") { + t.Errorf("the library's mount: %q", got) + } + if got := f.read(filepath.Join(UnitDir, "srv-downloads.mount")); got != "[Mount]\nWhat=elsewhere\n" { + t.Errorf("a unit file that is not the module's is left byte for byte: %q", got) + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-old.mount")); !os.IsNotExist(err) { + t.Error("the module's pair nothing names any more is removed") + } + for _, c := range []string{"systemctl disable --now srv-old.automount", "systemctl stop srv-old.mount", "systemctl daemon-reload", + "systemctl enable --now srv-library.automount", "systemctl enable --now mnt-games.automount", "systemctl enable --now mnt-usb.automount"} { + if !f.called(c) { + t.Errorf("not called: %s (calls %q)", c, f.calls) + } + } + if f.called("systemctl enable --now srv-downloads.automount") { + t.Error("the refused share is not armed") + } + if info, err := os.Stat(filepath.Join(f.root, "/srv/library")); err != nil || !info.IsDir() { + t.Error("a missing mount point is made") + } + creds := filepath.Join(f.root, "/var/lib/mesh/mounts/smb-games.credentials") + if info, err := os.Stat(creds); err != nil || info.Mode().Perm() != 0o600 { + t.Fatalf("the source's login is a file only root reads: %v", err) + } + if b, _ := os.ReadFile(creds); string(b) != "username=root\npassword="+testPassword+"\n" { + t.Errorf("credentials file %q", b) + } + + states := map[string]ShareStatus{} + for _, s := range st.Shares { + states[s.Name] = s + } + if states["library"].State != StateArmed || states["games"].State != StateAbsent || states["usb"].State != StateAbsent { + t.Errorf("states %+v", states) + } + if d := states["downloads"]; d.State != StateFailed || !strings.Contains(d.Why, "not the module's") { + t.Errorf("downloads %+v", d) + } + if !strings.Contains(states["games"].Why, "not present") || states["games"].Since != now.Format(time.RFC3339) { + t.Errorf("games %+v", states["games"]) + } + // The password is in the secret and in the credentials file, and nowhere else. + if strings.Contains(f.everything("smb-credentials.secret", "smb-games.credentials"), testPassword) { + t.Error("the password leaked into a file the module wrote") + } + raw, _ := json.Marshal(f.machine().Health(now)) + if strings.Contains(string(raw), testPassword) { + t.Error("the password leaked into the health") + } + + // A second look a minute later changes nothing and keeps since. + f.calls = nil + later := now.Add(time.Minute) + st = f.applier(later).Look(context.Background()) + if f.called("systemctl daemon-reload") || f.called("systemctl enable") { + t.Errorf("a look with nothing to change changes nothing: %q", f.calls) + } + for _, s := range st.Shares { + if s.Since != now.Format(time.RFC3339) { + t.Errorf("%s since %s, want the first look's", s.Name, s.Since) + } + } +} + +func TestASilentServerRemovesNoMount(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, "shares=library=/srv/library\nserver=homeserver\nat=192.0.2.10\n") + f.write(filepath.Join(UnitDir, "srv-library.mount"), Marker+"\nkept\n") + f.write(filepath.Join(UnitDir, "srv-library.automount"), Marker+"\nkept\n") + st := f.applier(time.Now()).Look(context.Background()) + if f.read(filepath.Join(UnitDir, "srv-library.mount")) != Marker+"\nkept\n" { + t.Error("with no answer from the server the pair stays as it is") + } + if len(st.Shares) != 1 || st.Shares[0].State != StateUnreachable || !strings.Contains(st.Shares[0].Why, "has not been read from the bus yet") { + t.Errorf("shares %+v", st.Shares) + } + // And a missing config file writes and removes nothing. + g := newSystem(t) + g.write(filepath.Join(UnitDir, "srv-x.mount"), Marker+"\n") + st = g.applier(time.Now()).Look(context.Background()) + if _, err := os.Stat(filepath.Join(g.root, UnitDir, "srv-x.mount")); err != nil || len(st.Problems) == 0 { + t.Errorf("without its config the process removes nothing and says why: %v %v", err, st.Problems) + } +} + +func TestAMissingLoginIsSaidWithItsRemedyAndTheSourceIsNotMounted(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, "sources=games=smb://192.0.2.76/Share@/mnt/games\nsecret=/var/lib/mesh/mounts/smb-credentials.secret\nstate=/var/lib/mesh/mounts\n") + f.write("/var/lib/mesh/mounts/smb-credentials.secret", "[other]\nusername=x\npassword="+testPassword+"\n") + now := time.Now() + st := f.applier(now).Look(context.Background()) + if len(st.Shares) != 1 || st.Shares[0].State != StateFailed || + !strings.Contains(st.Shares[0].Why, "secret accept mounts smb-credentials --from ") { + t.Fatalf("shares %+v", st.Shares) + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "mnt-games.mount")); !os.IsNotExist(err) { + t.Error("a source without its login gets no units") + } + h := f.machine().Health(now) + if h.Healthy || !strings.Contains(h.Why, "login is not given") || strings.Contains(h.Why, testPassword) { + t.Errorf("health %+v", h) + } + creds, problems := ParseSMBCredentials("[a]\nusername=u\npassword=" + testPassword + "\nbogus=1\n[b]\npassword=p\n") + if creds["a"].Password != testPassword || len(problems) != 2 || strings.Contains(strings.Join(problems, " "), testPassword) { + t.Errorf("creds %v problems %q", len(creds), problems) + } +} + +func TestTheHealthGivesAShareFiveMinutesAndAnAbsentSourceNone(t *testing.T) { + now := time.Date(2026, 10, 8, 19, 30, 0, 0, time.Local) + at := func(d time.Duration) string { return now.Add(-d).Format(time.RFC3339) } + st := Status{Looked: at(10 * time.Second), Server: "homeserver", Shares: []ShareStatus{ + {Name: "library", Kind: "nfs", Mountpoint: "/srv/library", State: StateUnreachable, Since: at(4 * time.Minute), Why: "192.0.2.10:2049 does not answer"}, + {Name: "games", Kind: "smb", Occasional: true, Mountpoint: "/mnt/games", State: StateAbsent, Since: at(3 * time.Hour)}, + }} + if h := Judge(st, nil, now); !h.Healthy || !strings.Contains(strings.Join(h.Said, " "), "games at /mnt/games: not present since 16:30") { + t.Errorf("under five minutes, and an absent source, are healthy: %+v", h) + } + st.Shares[0].Since = at(9 * time.Minute) + h := Judge(st, nil, now) + if h.Healthy || !strings.Contains(h.Why, "The share library of homeserver is not reachable from this machine since 19:21") { + t.Errorf("five minutes away raises: %+v", h) + } + st.Shares = []ShareStatus{{Name: "library", Kind: "nfs", Mountpoint: "/srv/library", State: StateFailed, Since: at(time.Minute), + Why: "its automount srv-library.automount is not armed (inactive)"}} + if h := Judge(st, nil, now); h.Healthy { + t.Error("an automount not armed is raised at once") + } + st.Looked = at(3 * time.Minute) + st.Shares = nil + if h := Judge(st, nil, now); h.Healthy || !strings.Contains(h.Why, "last looked") { + t.Errorf("a stale look is raised: %+v", h) + } + if h := Judge(Status{}, errors.New("not present"), now); h.Healthy { + t.Error("no status is unhealthy") + } + if h := Judge(Status{Looked: at(time.Second)}, nil, now); !h.Healthy || h.Why != "This machine is asked to mount nothing." { + t.Errorf("nothing asked: %+v", h) + } +} + +// The workstation's two lines as they were found (research 038), in documentation addresses. +const operatorsFstab = "UUID=ad8b / ext4 rw,relatime 0 1\n" + + "192.0.2.10:/tank/media /srv/library nfs defaults,_netdev 0 0\n" + + "//192.0.2.76/Share /mnt/games cifs username=root,password=" + testPassword + ",iocharset=utf8,sec=ntlmssp 0 0\n" + +func TestAnFstabLineIsTakenOverOnlyOnceItsReplacementIsArmed(t *testing.T) { + f := newSystem(t) + f.write("/etc/fstab", operatorsFstab) + m := f.machine() + now := time.Date(2026, 10, 8, 20, 0, 0, 0, time.UTC) + ctx := context.Background() + + a, err := m.Adopt(ctx, "/mnt/games", false, "", now) + if err != nil || a.Done || !strings.HasPrefix(a.Said, "Would be refused now: the module has no unit pair") { + t.Fatalf("no pair yet: %+v %v", a, err) + } + if strings.Contains(a.Was, testPassword) || !strings.Contains(a.Was, "password=(redacted)") { + t.Errorf("the dry run says the line without its password: %q", a.Was) + } + if _, err := m.AdoptAsRoot(ctx, "/mnt/games", now); err == nil { + t.Fatal("the root half refuses too") + } + if f.read("/etc/fstab") != operatorsFstab { + t.Fatal("a refusal changes nothing") + } + + f.write(filepath.Join(UnitDir, "mnt-games.mount"), Marker+"\n") + f.write(filepath.Join(UnitDir, "mnt-games.automount"), Marker+"\n") + f.units["mnt-games.automount"] = map[string]string{"ActiveState": "inactive"} + if _, err := m.AdoptAsRoot(ctx, "/mnt/games", now); err == nil || !strings.Contains(err.Error(), "not armed") { + t.Fatalf("an automount not armed refuses: %v", err) + } + f.units["mnt-games.automount"] = map[string]string{"ActiveState": "active"} + if a, err := m.Adopt(ctx, "/mnt/games", false, "", now); err != nil || !strings.HasPrefix(a.Said, "A dry run") || a.Line != 3 { + t.Fatalf("dry run: %+v %v", a, err) + } + if _, err := m.Adopt(ctx, "/mnt/games", true, "", now); err == nil { + t.Error("confirm without why is refused") + } + + a, err = m.AdoptAsRoot(ctx, "/mnt/games", now) + if err != nil || !a.Done || a.Line != 3 || a.Copy != "/etc/fstab.mesh-adopted-20261008-200000" { + t.Fatalf("adopt: %+v %v", a, err) + } + got := f.read("/etc/fstab") + lines := strings.Split(got, "\n") + if lines[0] != "UUID=ad8b / ext4 rw,relatime 0 1" || lines[1] != "192.0.2.10:/tank/media /srv/library nfs defaults,_netdev 0 0" { + t.Errorf("every other line is kept byte for byte:\n%s", got) + } + if !strings.HasPrefix(lines[2], AdoptedPrefix+"2026-10-08: //192.0.2.76/Share") || strings.Contains(got, testPassword) { + t.Errorf("the line is commented out with its password redacted:\n%s", got) + } + info, err := os.Stat(filepath.Join(f.root, a.Copy)) + if err != nil || info.Mode().Perm() != 0o600 || f.read(a.Copy) != operatorsFstab { + t.Errorf("the file as it was is kept, root only: %v", err) + } + if !f.called("systemctl daemon-reload") { + t.Error("the service manager reloads") + } + raw, _ := json.Marshal(a) + if strings.Contains(string(raw), testPassword) { + t.Error("the answer holds the password") + } + if _, err := m.AdoptAsRoot(ctx, "/mnt/games", now); err == nil || !strings.Contains(err.Error(), "no line for /mnt/games") { + t.Errorf("a line taken over is not taken twice: %v", err) + } + if _, _, err := PlanAdoption("a /x nfs d 0 0\nb /x nfs d 0 0\n", "/x", now); err == nil || !strings.Contains(err.Error(), "more than one line") { + t.Errorf("two lines for one mount point are left to a person: %v", err) + } +} + +func TestAFailedAskKeepsTheLastAnswer(t *testing.T) { + cfg := ParseConfig("shares=library=/srv/library\nserver=homeserver\nat=192.0.2.10\n") + now := time.Now() + var asked string + ok := func(key string) (json.RawMessage, error) { + asked = key + return json.RawMessage(`{"shares":[{"name":"library","path":"/tank/media","access":"rw","uid":1001,"gid":2000,"live":true}]}`), nil + } + first := AskExports(cfg, ExportsAnswer{}, ok, now) + if asked != "homeserver" || len(first.Shares) != 1 || first.Error != "" { + t.Fatalf("first %+v asked %q", first, asked) + } + fail := func(string) (json.RawMessage, error) { return nil, errors.New("no responders") } + second := AskExports(cfg, first, fail, now.Add(time.Minute)) + empty := AskExports(cfg, first, func(string) (json.RawMessage, error) { return nil, nil }, now) + if !strings.Contains(empty.Error, "keeps nothing") || len(empty.Shares) != 1 { + t.Errorf("a machine with nothing kept yet: %+v", empty) + } + if second.Error != "no responders" || len(second.Shares) != 1 || second.Answered != first.Answered { + t.Errorf("second %+v", second) + } + if got := ParseConfig("at=homeserver.mesh.example\n").ServerNode(); got != "homeserver" { + t.Errorf("without from, the first label of at: %q", got) + } +} + +func TestTheBundlePutsWhatTheProcessFoundOnTheBus(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, "shares=library=/srv/library\nserver=homeserver\nat=192.0.2.10\n") + f.write(StatusFile, `{"looked":"2026-10-08T19:21:00Z","shares":[{"name":"library","state":"armed"}]}`) + var put []string + last := "" + ask := func(string) (json.RawMessage, error) { return json.RawMessage(`{"shares":[]}`), nil } + said := f.machine().Share("workstation", ask, func(key string, _ any) error { put = append(put, key); return nil }, &last, time.Now()) + if len(said) != 0 || len(put) != 1 || put[0] != "workstation" || !strings.Contains(f.read(ExportsFile), `"server": "homeserver"`) { + t.Errorf("said %q put %q exports %q", said, put, f.read(ExportsFile)) + } + f.machine().Share("workstation", ask, func(key string, _ any) error { put = append(put, key); return nil }, &last, time.Now()) + if len(put) != 1 { + t.Error("an unchanged status is not put again") + } +} + +// In production the acting process and the judging one are two: apply writes and arms and says nothing +// of how the shares are; watch writes and arms nothing and says how they are. +func TestApplyActsAndWatchJudgesAndNeitherDoesTheOthersWork(t *testing.T) { + f := newSystem(t) + f.write(ConfigFile, config) + f.write(ExportsFile, exportsAnswer(t, library, downloads)) + f.write("/var/lib/mesh/mounts/smb-credentials.secret", "[other]\nusername=x\npassword="+testPassword+"\n") + f.write(filepath.Join(UnitDir, "srv-downloads.mount"), "[Mount]\nWhat=elsewhere\n") + f.reachable["192.0.2.10:2049"] = true + now := time.Now() + + apply := f.applier(now) + apply.Judge = false + apply.Look(context.Background()) + if _, err := os.Stat(filepath.Join(f.root, StatusFile)); !os.IsNotExist(err) { + t.Error("apply leaves no status: that is watch's") + } + if !f.called("systemctl enable --now srv-library.automount") || f.read(filepath.Join(UnitDir, "srv-library.mount")) == "" { + t.Fatal("apply writes and arms") + } + applied, err := readJSON[Applied](f.machine(), AppliedFile) + if err != nil || len(applied.NotSet) != 2 { + t.Fatalf("apply says what it could not set up: %+v %v", applied, err) + } + + f.calls = nil + watch := f.applier(now) + watch.Act = false + f.write(filepath.Join(UnitDir, "srv-gone.mount"), Marker+"\n") + st := watch.Look(context.Background()) + for _, c := range f.calls { + if !strings.HasPrefix(c, "systemctl show") { + t.Errorf("watch only reads the service manager: %s", c) + } + } + if _, err := os.Stat(filepath.Join(f.root, UnitDir, "srv-gone.mount")); err != nil { + t.Error("watch removes nothing") + } + states := map[string]string{} + for _, s := range st.Shares { + states[s.Name] = s.State + } + if states["library"] != StateArmed || states["downloads"] != StateFailed || states["games"] != StateFailed || states["usb"] != StateAbsent { + t.Errorf("watch says how each is: %v", states) + } + if strings.Contains(f.everything("smb-credentials.secret"), testPassword) { + t.Error("the password leaked") + } +} diff --git a/modules/mounts/go.mod b/modules/mounts/go.mod new file mode 100644 index 00000000..9549ae83 --- /dev/null +++ b/modules/mounts/go.mod @@ -0,0 +1,5 @@ +module mounts + +go 1.22 + +require git.novox.be/novox/mesh-sdk/go v0.1.7 diff --git a/modules/mounts/go.sum b/modules/mounts/go.sum new file mode 100644 index 00000000..017ed9ad --- /dev/null +++ b/modules/mounts/go.sum @@ -0,0 +1,4 @@ +git.novox.be/novox/mesh-sdk/go v0.1.6 h1:9qzdYONYbJdWcu6sxQcq9v1LI0JxcfkiKYkMUzJSkVQ= +git.novox.be/novox/mesh-sdk/go v0.1.6/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY= +git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w= +git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY= diff --git a/modules/mounts/module.json b/modules/mounts/module.json new file mode 100644 index 00000000..c11f0283 --- /dev/null +++ b/modules/mounts/module.json @@ -0,0 +1,150 @@ +{ + "module": "mounts", + "version": "1", + "requires": [ + "nfs-share" + ], + "reads": [ + "nfs-server.exports" + ], + "invokes": [ + "seat:mesh-controller.data" + ], + "capabilities": [ + "package-manager", + "service-manager" + ], + "claims": [ + { + "name": "node-mounts", + "scope": "node", + "serves": [ + "list", + "test", + "mount", + "unmount", + "adopt" + ] + } + ], + "tools": [ + "mounts_list", + "mounts_exports", + "mounts_health" + ], + "state": [ + { + "name": "shares", + "per-machine": true + } + ], + "settings": { + "shares": { + "kind": "preference", + "default": "none", + "why": "a machine mounts no share of another until its assignment names one: space-separated name=mountpoint, each optionally :ro (hq ADR 0263 rule 5)" + }, + "sources": { + "kind": "preference", + "default": "none", + "why": "a machine has no occasional source until its assignment names one: space-separated name=smb:///@ or name=disk:@, each optionally :ro (hq ADR 0263 rule 6)" + } + }, + "own-secrets": { + "smb-credentials": { + "path": "${dir:state}/smb-credentials.secret", + "taken": "at-start", + "issued-by": "outside" + } + }, + "resources": [ + { + "id": "nfs-utils", + "type": "package", + "package": "nfs-utils" + }, + { + "id": "cifs-utils", + "type": "package", + "package": "cifs-utils" + }, + { + "id": "state", + "type": "directory", + "mode": "0700", + "place": "." + }, + { + "id": "config-dir", + "type": "directory", + "path": "/etc/mesh-mounts", + "mode": "0755" + }, + { + "id": "config", + "type": "file", + "path": "/etc/mesh-mounts/mounts.conf", + "mode": "0644", + "content": "# Written by the mesh (module mounts, novox/hq ADR 0263) from this machine's assignment: the settings\n# shares and sources, and the binding of nfs-share. Replaced on every push; change the settings, not this.\n# The module's process reads it as root every 30 seconds and writes one .mount and .automount per entry.\nshares=${setting:shares}\nsources=${setting:sources}\nserver=${bound:nfs-share:from}\nat=${bound:nfs-share:at}\naccount=${machine:account}\nnode=${machine:name}\nsecret=${dir:state}/smb-credentials.secret\nstate=${dir:state}\n" + }, + { + "id": "bus-dir-parent", + "type": "directory", + "path": "/var/lib/mesh-mounts", + "mode": "0755" + }, + { + "id": "bus-dir", + "type": "directory", + "path": "/var/lib/mesh-mounts/from-bus", + "mode": "0755", + "owner": "${machine:account}" + }, + { + "id": "apply", + "type": "process", + "name": "mesh-mounts-apply", + "artifact": "tools-go", + "run": [ + "./mounts", + "apply" + ], + "health": { + "kind": "unit" + } + }, + { + "id": "watch", + "type": "process", + "name": "mesh-mounts-watch", + "artifact": "tools-go", + "run": [ + "./mounts", + "watch" + ], + "health": { + "kind": "tool", + "tool": "mounts_health", + "interval": "60s", + "timeout": "10s", + "looks": 2, + "grace": "90s" + } + } + ], + "build": { + "artifacts": [ + { + "name": "tools-go", + "kind": "bundle", + "language": "go", + "system": "arch", + "from": "cmd/mounts", + "binary": "mounts", + "loads": [ + "mounts" + ] + } + ] + } +} diff --git a/modules/nfs-server/README.md b/modules/nfs-server/README.md new file mode 100644 index 00000000..4630ce55 --- /dev/null +++ b/modules/nfs-server/README.md @@ -0,0 +1,111 @@ +# nfs-server + +A node shares a folder with the mesh's machines, as a module (novox/hq ADR 0263, to-be 52). It holds +the node seat `node-nfs-server` and provides the provision `nfs-share`, which the `mounts` module on +another node requires. + +## The rules it keeps + +- **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or + `name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is + the operator's value (ADR 0112, ADR 0262). +- **To the nodes that ask, as far as this machine allows.** A share is exported to a node only when + this machine's `grants` setting names that node for it (`share=node:rw|ro[,node:rw|ro]`, separated by + spaces; `none`, the default, grants nothing) **and** that node's `mounts` module wants it from this + machine (its state `shares` on the bus). The access is the lower of the grant and the want, and of the + share's own `:ro`. The address is the node's private address from the mesh's roster + (`/etc/nfs-server/machines`, rendered by the mesh), never one a consumer gives, and it must lie inside + the private network's range. The control node, or any node, gets a share only when both name it. A + want that is not granted, and a grant nobody wants, are said by `exports`. Port 2049 is opened to the + mesh only; the home network is never named. +- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one + id names different accounts on different machines. Every line is `all_squash` with `anonuid` and + `anongid` read from the folder's owner on this machine, so a file written from any node lands owned by + the folder's owner, and root is squashed with everyone else. +- **Judged where it really is.** A share's path is resolved through every symlink, and every check runs + on the real folder, which is what the export line names. +- **Never root's.** A folder owned by root, or by root's group, is refused: squashing every client to + root would make every client root there. +- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, is inside a + backup (restic) repository (any parent up to `/`), or holds one within four levels below it is refused, + said by the tools, and makes the module unhealthy. The look below is bounded (200 000 folders); when it + is cut short, the share is refused rather than exported unchecked. +- **Only while its filesystem is mounted.** A share that is a filesystem of its own is exported with `mp`, + a folder inside one with `mp=`, so an unmounted dataset never exports the empty folder + under it. A folder of the root filesystem carries neither. +- **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It + never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its + own and nothing else, and a test holds it to that. +- **NFS version 4 only**, so 2049 is the only port a client needs: rpcbind and mountd stay closed. + +## What it owns + +- the `nfs-utils` package and `nfs-server.service`, running and enabled; +- `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on; +- `/etc/nfs-server/shares.conf`: the settings `shares` and `grants` and the range, rendered by the mesh; +- `/etc/nfs-server/machines`: every machine of the mesh and its private address, rendered by the mesh; +- `/var/lib/nfs-server/from-bus/wants.json`: what the nodes want of this machine, written by the tools + from the bus (the directory belongs to the operator account the tools run as); +- `/etc/exports.d/mesh.exports`: written by its process, with a marker header; +- `/run/nfs-server/status.json`: the process's last look, for the tools; +- the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module + and for a view (ADR 0201). + +## How it runs + +- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, the grants, + the roster and the wants, judges each folder, writes the export file when it changed, and runs + `exportfs -ra` when the file changed or a share is missing from the kernel's table. Then it writes its + status. With no wants taken from the bus yet, it exports nothing. +- **Its tools** also take every node's `mounts` state from the bus every 30 seconds and write what is + asked of this machine to the wants file; when the bus does not answer, the last wants stay. +- **Its tools** run as the operator account. They read the status and the kernel's table, and run what + only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools. +- **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS + server is not active, the process has not looked for five minutes, a share is refused or missing from + the kernel's table, or the kernel could not read the files again. + +## Tools + +| tool | | answers | +|---|---|---| +| `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) | +| `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 | +| `node-nfs-server.test` | r | whether one share is exported now and the server is active; with an address, whether it is inside the range and the share is exported to it | +| `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them | +| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live to at least one node; a dry run unless `confirm` is `true` (or "true"); a person's act, with why | +| `nfs_health` | r | the module's health, one sentence per finding | + +`exports` answers, for the `mounts` module: + +``` +{"shares": [{"name", "path", "access", "uid", "gid", "clients", "to": [{"node", "address", "access"}], + "mp", "line", "live"}], "unmatched": [...], + "refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}], + "service": "active", "range", "looked", "said", "notes"} +``` + +## When it is unassigned + +The process writes `/etc/exports.d/mesh.exports`, not the node-engine, because each line carries the +folder's owner and the nodes' addresses as this machine reads them. The node-engine removes a module's +resources in reverse order of declaration (mesh-host `internal/apply`), and the process is declared before +its config file, so on unassignment the config file is gone when the process is stopped. The process then +removes its export file — the one removal its code makes — and runs `exportfs -ra`, so the kernel lets the +shares go. A plain restart, a push or a shutdown leaves the config file in place, and the exports stay. + +## What it leaves +- **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not + a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could + reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so + a folder that owner cannot read stays closed. A share that must be strictly fenced is better given its + own dataset. +- **Exports that are not the mesh's** (`/etc/exports`, a ZFS dataset's `sharenfs`) are left as found and + shown, until `adopt` takes one over on a person's word. + +## Root acts and sudo + +The tools run as the operator account and act as root through `sudo -n`. The `sudo` module grants that +account every command, so **this module's code is the only line** between a tool call and root: each act +checks again what it may touch before it runs a command, and a test holds each check. Narrowing what sudo +grants the account to the commands these modules need is a separate decision, not made here. diff --git a/modules/nfs-server/cmd/nfs-server/machine.go b/modules/nfs-server/cmd/nfs-server/machine.go new file mode 100644 index 00000000..857a50ed --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/machine.go @@ -0,0 +1,305 @@ +package main + +import ( + "bytes" + "context" + "errors" + "fmt" + "io/fs" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "syscall" + "time" +) + +// The machine as the module sees it (novox/hq ADR 0263). +// +// **Two processes, two accounts.** The node's tool runtime runs the tools as the operator account, not +// root (novox/hq ADR 0175 §4): they read, and what only root may do — `exportfs -ra`, `zfs set` — goes +// through `sudo -n`, as the packet filter's and the service manager's tools do. The module's long-running +// process runs as root: it renders the export file and writes the status the tools read. It cannot reach +// the bus, so the tools' bundle puts that status in the module's state. +// +// **It writes two files and nothing else.** The export file and the status file are the module's own. +// Nothing here can create, chown, chmod or remove anything under a shared path: the machine offers no +// such call, and Write refuses every path but the module's two (novox/hq ADR 0051). + +// The module's files on the machine. +const ( + // ConfigFile is what the mesh renders from the assignment: the shares and the private network's range. + ConfigFile = "/etc/nfs-server/shares.conf" + // ExportFile is the module's own export file, which the distribution's exportfs reads with the rest of + // /etc/exports.d. + ExportFile = "/etc/exports.d/mesh.exports" + // StatusFile is the process's last look, for the tools. + StatusFile = "/run/nfs-server/status.json" + // Etab is the kernel's export table as nfs-utils keeps it. + Etab = "/var/lib/nfs/etab" + // Unit is the NFS server's unit. + Unit = "nfs-server.service" + // RosterFile is the mesh's machines, name and private address, rendered by the mesh (a fact). + RosterFile = "/etc/nfs-server/machines" + // WantsDir is where the tools, as the operator account, write what they read on the bus. + WantsDir = "/var/lib/nfs-server/from-bus" + // WantsFile is every node's wants of this machine, from the `mounts` modules' state on the bus. + WantsFile = WantsDir + "/wants.json" + // MountInfo is the kernel's mount table, read for `mp`. + MountInfo = "/proc/self/mountinfo" +) + +// CommandTimeout bounds every command. None touches a shared path's contents. +const CommandTimeout = 20 * time.Second + +// Ran is what one command did. +type Ran struct { + Out string + Err string + Code int +} + +// Runner runs one command; injected, so every tool is tested over recorded answers. +type Runner func(ctx context.Context, name string, args ...string) Ran + +// ExecRunner runs a command on the machine, bounded by CommandTimeout. +func ExecRunner(ctx context.Context, name string, args ...string) Ran { + ctx, cancel := context.WithTimeout(ctx, CommandTimeout) + defer cancel() + cmd := exec.CommandContext(ctx, name, args...) + var stdout, stderr bytes.Buffer + cmd.Stdout, cmd.Stderr = &stdout, &stderr + err := cmd.Run() + r := Ran{Out: stdout.String(), Err: strings.TrimSpace(stderr.String())} + if err != nil { + var exit *exec.ExitError + if errors.As(err, &exit) { + r.Code = exit.ExitCode() + } else { + r.Code = -1 + if r.Err == "" { + r.Err = err.Error() + } + } + } + return r +} + +// Owner is who owns a path, and whether it is a directory. +type Owner struct { + UID, GID int + Dir bool +} + +// Machine is a filesystem root (the real one, or a test's tree), a runner, who this process is, and a +// clock. +type Machine struct { + Root string + Run Runner + UID int + Now func() time.Time + // WalkLimit bounds the look for a backup repository below a share. + WalkLimit int + owners map[string]Owner + // ownerOf reads a file's owner from its inode; nil is the inode as the kernel says it. Tests set it + // so the folders they make stand for a non-root owner, whoever runs them. + ownerOf func(info os.FileInfo) (uid, gid int) + // clean is when a look below each real path last found no backup repository (LookKept). + clean map[string]time.Time + cleanMu sync.Mutex +} + +// Here is the machine this process runs on. +func Here() *Machine { + return &Machine{Root: "/", Run: ExecRunner, UID: os.Getuid(), Now: time.Now} +} + +func (m *Machine) at(path string) string { return filepath.Join(m.Root, path) } + +// Read is one file's text. +func (m *Machine) Read(path string) (string, error) { + b, err := os.ReadFile(m.at(path)) + return string(b), err +} + +// Exists says whether a path is there. +func (m *Machine) Exists(path string) bool { + _, err := os.Lstat(m.at(path)) + return err == nil +} + +// Stat is a path's owner. It reads the inode and nothing else. +func (m *Machine) Stat(path string) (Owner, error) { + if o, ok := m.owners[path]; ok { + return o, nil + } + info, err := os.Stat(m.at(path)) + if err != nil { + return Owner{}, err + } + o := Owner{Dir: info.IsDir()} + if m.ownerOf != nil { + o.UID, o.GID = m.ownerOf(info) + return o, nil + } + o.UID, o.GID = inodeOwner(info) + return o, nil +} + +// Real is a path with every symlink resolved, as the machine names it. A path that resolves outside the +// machine's root is refused. +func (m *Machine) Real(p string) (string, error) { + root, err := filepath.EvalSymlinks(m.Root) + if err != nil { + return "", err + } + full, err := filepath.EvalSymlinks(m.at(p)) + if err != nil { + return "", err + } + rel, err := filepath.Rel(root, full) + if err != nil || rel == ".." || strings.HasPrefix(rel, "../") { + return "", fmt.Errorf("%s resolves outside the machine (%s)", p, full) + } + return filepath.Clean("/" + rel), nil +} + +// MountPoints are the kernel's mount points, from its mount table; "/" alone when it cannot be read. +func (m *Machine) MountPoints() []string { + text, err := m.Read(MountInfo) + if err != nil { + return []string{"/"} + } + var out []string + for _, line := range strings.Split(text, "\n") { + f := strings.Fields(line) + if len(f) >= 5 { + out = append(out, unescapeMount(f[4])) + } + } + return out +} + +// unescapeMount undoes the kernel's octal escapes in a mount point (\040 for a space). +func unescapeMount(s string) string { + if !strings.Contains(s, "\\") { + return s + } + var b strings.Builder + for i := 0; i < len(s); i++ { + if s[i] == '\\' && i+3 < len(s) { + var v int + if _, err := fmt.Sscanf(s[i+1:i+4], "%o", &v); err == nil { + b.WriteByte(byte(v)) + i += 3 + continue + } + } + b.WriteByte(s[i]) + } + return b.String() +} + +// Children are the names of a directory's subdirectories. Used only to look for a backup repository one +// level below a share; it lists names and opens nothing. +func (m *Machine) Children(path string) []string { + entries, err := os.ReadDir(m.at(path)) + if err != nil { + return nil + } + var out []string + for _, e := range entries { + if e.IsDir() { + out = append(out, e.Name()) + } + } + return out +} + +// Glob is the paths under the root matching a pattern, as the machine names them. +func (m *Machine) Glob(pattern string) []string { + found, _ := filepath.Glob(m.at(pattern)) + out := make([]string, 0, len(found)) + for _, f := range found { + rel, err := filepath.Rel(m.Root, f) + if err != nil { + continue + } + out = append(out, "/"+rel) + } + return out +} + +// written are the only paths Write may touch. +var written = map[string]bool{ExportFile: true, StatusFile: true, WantsFile: true} + +// Write replaces one of the module's own files, atomically. Every other path is refused: the module owns +// these three files (the export file and the status, written as root; the wants, written by the tools) +// and nothing else on the machine. +func (m *Machine) Write(path, content string) error { + if !written[path] { + return fmt.Errorf("the module writes only %s, %s and %s, not %s", ExportFile, StatusFile, WantsFile, path) + } + full := m.at(path) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil && !errors.Is(err, fs.ErrExist) { + return err + } + tmp := full + ".tmp" + if err := os.WriteFile(tmp, []byte(content), 0o644); err != nil { + return err + } + return os.Rename(tmp, full) +} + +// unexport removes the module's own export file, and nothing else: the one removal the module makes, +// when it was unassigned (Stopped). +func (m *Machine) unexport() error { + if err := os.Remove(m.at(ExportFile)); err != nil && !errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("the export file %s could not be removed: %v", ExportFile, err) + } + return nil +} + +// Escalated is a command as it is run: as given when this process is root, else through sudo without a +// prompt. +func (m *Machine) Escalated(name string, args ...string) (string, []string) { + if m.UID == 0 { + return name, args + } + return "sudo", append([]string{"-n", name}, args...) +} + +// AsRoot runs what only root may do, and names a refusal by how it failed. +func (m *Machine) AsRoot(name string, args ...string) (Ran, error) { + program, argv := m.Escalated(name, args...) + r := m.Run(context.Background(), program, argv...) + if r.Code != 0 { + why := r.Err + if why == "" { + why = strings.TrimSpace(r.Out) + } + if program == "sudo" && (strings.Contains(why, "a password is required") || strings.Contains(why, "not allowed")) { + return r, fmt.Errorf("%s needs root, and sudo refused the operator account without a prompt: %s", name, why) + } + return r, fmt.Errorf("%s %s: exit %d: %s", name, strings.Join(args, " "), r.Code, why) + } + return r, nil +} + +// ServiceState is what the service manager says of the NFS server: active, inactive, failed, … +func (m *Machine) ServiceState() string { + r := m.Run(context.Background(), "systemctl", "is-active", Unit) + if s := strings.TrimSpace(r.Out); s != "" { + return s + } + return "unknown" +} + +// inodeOwner is a file's owner as its inode says. +func inodeOwner(info os.FileInfo) (uid, gid int) { + if st, ok := info.Sys().(*syscall.Stat_t); ok { + return int(st.Uid), int(st.Gid) + } + return -1, -1 +} diff --git a/modules/nfs-server/cmd/nfs-server/main.go b/modules/nfs-server/cmd/nfs-server/main.go new file mode 100644 index 00000000..12315c53 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/main.go @@ -0,0 +1,161 @@ +// The nfs-server module's Go bundle (novox/hq ADR 0263): a node shares a folder with the mesh's machines +// only, one share per folder, every client mapped to the folder's owner. +// +// Started with no arguments it is served by the node's runtime as the operator account, over MCP on stdio +// (ADR 0188, ADR 0193): the node-nfs-server seat's verbs and the module's health, and beside them it puts +// the process's last look in the module's state `exports`, keyed by this machine (ADR 0201). Started as +// `nfs-server exports` it is the module's long-running process, as root: it renders the export file from +// the assignment's shares and has the kernel read it (process.go). stdout is the MCP channel; everything +// else is said on stderr. +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "os/signal" + "sort" + "syscall" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +// ExportsState is the state the exports are kept in on the bus; MountsState the `mounts` modules' state, +// which says what each node wants of which machine. +const ( + ExportsState = "exports" + MountsState = "mounts.shares" +) + +func main() { + if len(os.Args) > 1 { + if os.Args[1] != "exports" || len(os.Args) != 2 { + fmt.Fprintln(os.Stderr, "usage: nfs-server [exports]") + os.Exit(2) + } + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt) + defer stop() + if err := Here().RunExports(ctx); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } + return + } + m := Here() + go ShareExports(m) + if err := stdio.Serve("", Tools(m)); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +// ShareExports puts the exports in the module's state every Every, for as long as the bundle runs, so a +// consumer's module and a view read them from the bus rather than from this machine. +func ShareExports(m *Machine) { + machine := os.Getenv("MESH_NODE") + if machine == "" { + fmt.Fprintln(os.Stderr, "nfs-server: the runtime gave no machine name (MESH_NODE); the exports are not shared") + return + } + last := "" + put := func() { + m.TakeWants(machine) + _, err := stdio.State(ExportsState).Put(machine, m.Exports()) + said := "put" + if err != nil { + said = err.Error() + } + if said != last { + fmt.Fprintln(os.Stderr, "nfs-server: exports state:", said) + } + last = said + } + // The SDK reaches the bus once Serve runs. + time.Sleep(2 * time.Second) + put() + for range time.Tick(Every) { + put() + } +} + +// TakeWants reads every node's `mounts` state from the bus and writes what is asked of this machine to +// the wants file, for the root process. When the bus does not answer, the last wants stay. +func (m *Machine) TakeWants(machine string) { + state := stdio.State(MountsState) + keys, err := state.Keys() + if err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the mounts modules' state could not be listed:", err) + return + } + raw := map[string][]byte{} + for _, k := range keys { + e, err := state.Get(k) + if err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the mounts state of", k, "could not be read:", err) + return + } + if e != nil { + raw[k] = e.Value + } + } + wants, notes := WantsFromStates(machine, raw) + for _, n := range notes { + fmt.Fprintln(os.Stderr, "nfs-server:", n) + } + text, _ := json.MarshalIndent(WantsFromBus{Looked: m.Now().UTC().Format(time.RFC3339), Wants: wants}, "", " ") + have, _ := m.Read(WantsFile) + if wantsOf(have) == wantsOf(string(text)) { + return + } + if err := m.Write(WantsFile, string(text)+"\n"); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the wants could not be written:", err) + } +} + +// wantsOf is a wants file without its time, to compare two. +func wantsOf(text string) string { + var w WantsFromBus + if json.Unmarshal([]byte(text), &w) != nil { + return text + } + raw, _ := json.Marshal(w.Wants) + return string(raw) +} + +// WantsFromStates is what each node's `mounts` state asks of this machine: its wants whose server is this +// machine, keyed by the node. A state that cannot be read is said and skipped. +func WantsFromStates(machine string, states map[string][]byte) (map[string][]Want, []string) { + out := map[string][]Want{} + var notes []string + keys := make([]string, 0, len(states)) + for k := range states { + keys = append(keys, k) + } + sort.Strings(keys) + for _, node := range keys { + var v struct { + Wants []struct { + Server string `json:"server"` + Share string `json:"share"` + Access string `json:"access"` + } `json:"wants"` + } + if err := json.Unmarshal(states[node], &v); err != nil { + notes = append(notes, fmt.Sprintf("the mounts state of %s is not one this module reads: %v", node, err)) + continue + } + for _, w := range v.Wants { + if w.Server != machine || !validName(w.Share) { + continue + } + access := "rw" + if w.Access == "ro" { + access = "ro" + } + out[node] = append(out[node], Want{Share: w.Share, Access: access}) + } + } + return out, notes +} diff --git a/modules/nfs-server/cmd/nfs-server/manifest_test.go b/modules/nfs-server/cmd/nfs-server/manifest_test.go new file mode 100644 index 00000000..548371d3 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/manifest_test.go @@ -0,0 +1,164 @@ +package main + +import ( + "encoding/json" + "os" + "regexp" + "sort" + "strings" + "testing" +) + +type manifest struct { + Tools []string `json:"tools"` + Claims []struct { + Name string `json:"name"` + Scope string `json:"scope"` + Serves []string `json:"serves"` + } `json:"claims"` + Provides []map[string]any `json:"provides"` + Listens []struct { + Port int `json:"port"` + Protocol string `json:"protocol"` + From string `json:"from"` + } `json:"listens"` + Settings map[string]any `json:"settings"` + Resources []map[string]any `json:"resources"` +} + +func readManifest(t *testing.T) (manifest, string) { + t.Helper() + raw, err := os.ReadFile("../../module.json") + if err != nil { + t.Fatal(err) + } + var m manifest + if err := json.Unmarshal(raw, &m); err != nil { + t.Fatal(err) + } + return m, string(raw) +} + +// The seat's verbs are served as `node-nfs-server.`, the health tool is listed, and every tool says +// whether it reads or acts. +func TestTheSeatsVerbsAreAllServedAndTheHealthToolListed(t *testing.T) { + m, _ := readManifest(t) + if len(m.Claims) != 1 || m.Claims[0].Name != Seat || m.Claims[0].Scope != "node" { + t.Fatalf("claims: %+v", m.Claims) + } + var verbs, tools []string + for _, tl := range Tools(newFake(t).machine()) { + if name, ok := strings.CutPrefix(tl.Name, Seat+"."); ok { + verbs = append(verbs, name) + } else { + tools = append(tools, tl.Name) + } + if !strings.HasSuffix(tl.Description, "(r)") && !strings.HasSuffix(tl.Description, "(a)") { + t.Errorf("%s does not say whether it reads or acts", tl.Name) + } + } + sort.Strings(verbs) + serves := append([]string(nil), m.Claims[0].Serves...) + sort.Strings(serves) + if strings.Join(verbs, ",") != strings.Join(serves, ",") { + t.Errorf("served %v, the claim lists %v", verbs, serves) + } + if strings.Join(tools, ",") != strings.Join(m.Tools, ",") { + t.Errorf("tools %v, listed %v", tools, m.Tools) + } +} + +// Port 2049 to the mesh's machines and nothing else; the provision is the mesh's, keeping no name of its +// consumers. +func TestItListensOnTwoThousandFortyNineFromTheMeshOnlyAndProvidesTheShare(t *testing.T) { + m, raw := readManifest(t) + if len(m.Listens) != 1 || m.Listens[0].Port != 2049 || m.Listens[0].Protocol != "tcp" || m.Listens[0].From != "mesh" { + t.Errorf("listens: %+v", m.Listens) + } + if len(m.Provides) != 1 || m.Provides[0]["name"] != "nfs-share" || m.Provides[0]["scope"] != "mesh" || m.Provides[0]["identity"] != false { + t.Errorf("provides: %+v", m.Provides) + } + for _, port := range []string{"111", "20048", "\"port\": 445"} { + if strings.Contains(raw, `"port": `+port) { + t.Errorf("the module opens %s", port) + } + } + if !strings.Contains(raw, "vers3=n") { + t.Errorf("NFS version 3 is not switched off, so 2049 alone would not do") + } +} + +// The definition names no folder, no machine and no person (novox/hq ADR 0112): the shares are the +// assignment's setting, and the range the mesh's. +func TestTheDefinitionNamesNoFolderNoMachineAndGivesTheSharesNoDefault(t *testing.T) { + m, raw := readManifest(t) + if hit := regexp.MustCompile(`\b(g14|novox|ace|shanks|jochen)\b|/home/|/storage|/srv/`).FindString(strings.ReplaceAll(raw, "novox/hq", "")); hit != "" { + t.Errorf("the manifest says %q", hit) + } + if _, declared := m.Settings["shares"]; declared { + t.Errorf("the shares are the operator's value and have no default (ADR 0262)") + } + var config string + for _, r := range m.Resources { + if r["id"] == "config" { + config, _ = r["content"].(string) + } + } + if !strings.Contains(config, "shares=${setting:shares}") || !strings.Contains(config, "range=${machine:mesh-range}") { + t.Errorf("the config file: %q", config) + } + // Every long-running resource says how it is ready (ADR 0240 rule 8). + for _, r := range m.Resources { + if r["type"] == "service" || r["type"] == "process" { + if r["health"] == nil { + t.Errorf("%v has no health", r["id"]) + } + } + } +} + +// Review points 5 and 7: the module reads the nodes' wants, renders the mesh's roster, has grants that +// default to none, and its process is declared before its config, so the node-engine — removing a +// module's resources in reverse order — takes the config away before it stops the process, which then +// lets the shares go. +func TestTheManifestCarriesGrantsTheRosterTheWantsAndTheRemovalOrder(t *testing.T) { + m, raw := readManifest(t) + var full struct { + Reads []string `json:"reads"` + Facts map[string]map[string]string `json:"facts"` + Settings map[string]map[string]any `json:"settings"` + } + if err := json.Unmarshal([]byte(raw), &full); err != nil { + t.Fatal(err) + } + if len(full.Reads) != 1 || full.Reads[0] != MountsState { + t.Errorf("reads: %v", full.Reads) + } + roster := full.Facts["machines"] + if roster["path"] != RosterFile || !strings.Contains(roster["template"], "{{range .Machines}}{{.Name}} {{.Address}}") { + t.Errorf("the roster fact: %v", roster) + } + if g := full.Settings["grants"]; g["kind"] != "preference" || g["default"] != "none" { + t.Errorf("grants: %v", g) + } + order := map[string]int{} + var config string + for i, r := range m.Resources { + order[r["id"].(string)] = i + if r["id"] == "config" { + config, _ = r["content"].(string) + } + if r["path"] == WantsDir && r["owner"] != "${machine:account}" { + t.Errorf("the tools, as the operator account, cannot write the wants: %v", r) + } + } + if !strings.Contains(config, "grants=${setting:grants}") { + t.Errorf("the config lacks the grants: %q", config) + } + if _, ok := order["wants-dir"]; !ok { + t.Errorf("no directory for the wants") + } + if !(order["exports"] < order["config"] && order["exports"] < order["config-dir"]) { + t.Errorf("the process must be declared before its config, so unassignment removes the config first: %v", order) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/nfs_test.go b/modules/nfs-server/cmd/nfs-server/nfs_test.go new file mode 100644 index 00000000..dbc45970 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/nfs_test.go @@ -0,0 +1,449 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// fake is a machine under a temporary root, with recorded command answers. +type fake struct { + t *testing.T + root string + answers map[string]Ran + ran []string + now time.Time + uid int + owners map[string]Owner + // onExportfs, when set, writes the kernel's table as exportfs -ra would. + onExportfs func() +} + +func newFake(t *testing.T) *fake { + f := &fake{t: t, root: t.TempDir(), answers: map[string]Ran{}, now: time.Date(2026, 10, 8, 19, 21, 0, 0, time.UTC), uid: 0} + f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "active\n"} + f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Code: -1, Err: `exec: "zfs": executable file not found in $PATH`} + return f +} + +func (f *fake) machine() *Machine { + return &Machine{Root: f.root, UID: f.uid, owners: f.owners, ownerOf: asTheLibrarysOwner, Now: func() time.Time { return f.now }, Run: func(_ context.Context, name string, args ...string) Ran { + line := strings.Join(append([]string{name}, args...), " ") + f.ran = append(f.ran, line) + if (line == "exportfs -ra" || line == "sudo -n exportfs -ra") && f.onExportfs != nil { + f.onExportfs() + } + if r, ok := f.answers[line]; ok { + return r + } + return Ran{} + }} +} + +func (f *fake) dir(p string) { + if err := os.MkdirAll(filepath.Join(f.root, p), 0o755); err != nil { + f.t.Fatal(err) + } +} + +func (f *fake) file(p, content string) { + f.dir(filepath.Dir(p)) + if err := os.WriteFile(filepath.Join(f.root, p), []byte(content), 0o644); err != nil { + f.t.Fatal(err) + } +} + +func (f *fake) read(p string) string { + b, _ := os.ReadFile(filepath.Join(f.root, p)) + return string(b) +} + +func (f *fake) restic(p string) { + f.file(p+"/config", "x") + for _, d := range []string{"keys", "snapshots", "data"} { + f.dir(p + "/" + d) + } +} + +// kernelReads makes exportfs -ra copy the mesh's export file into the kernel's table, as it would. +func (f *fake) kernelReads() { + f.onExportfs = func() { + var b strings.Builder + for _, e := range ParseExports(f.read(ExportFile), ExportFile) { + fmt.Fprintf(&b, "%s\t%s(%s)\n", e.Path, e.Client, e.Options) + } + f.file(Etab, b.String()) + } +} + +const meshRange = "10.0.0.0/24" + +// ws is the node every older test serves: granted every share and wanting it. +const wsAddress = "10.0.0.3" + +// config is a config granting every share to ws; serve writes the roster and ws's wants for those shares. +func config(shares string) string { + parsed, _ := ParseShares(shares) + var grants []string + for _, s := range parsed { + grants = append(grants, s.Name+"=ws:rw") + } + g := strings.Join(grants, " ") + if g == "" { + g = "none" + } + return "# test\nshares=" + shares + "\ngrants=" + g + "\nrange=" + meshRange + "\n" +} + +// serve has ws want these shares of this machine, at the mesh's address in the roster. +func (f *fake) serve(names ...string) { + f.roster("ws " + wsAddress) + var w []Want + for _, n := range names { + w = append(w, Want{Share: n, Access: "rw"}) + } + f.wants(map[string][]Want{"ws": w}) +} + +func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *testing.T) { + f := newFake(t) + f.dir("/srv/library") + f.dir("/srv/downloads") + f.file(ConfigFile, config("library=/srv/library downloads=/srv/downloads:ro")) + f.serve("library", "downloads") + f.kernelReads() + s := f.machine().Pass(context.Background(), nil) + if len(s.Shares) != 2 || len(s.Refused) != 0 { + t.Fatalf("status: %+v", s) + } + uid, gid := 1001, 2000 // the owner the tests' folders stand for (asTheLibrarysOwner) + text := f.read(ExportFile) + for _, want := range []string{ + fmt.Sprintf("/srv/library %s(rw,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", wsAddress, uid, gid), + fmt.Sprintf("/srv/downloads %s(ro,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", wsAddress, uid, gid), + } { + if !strings.Contains(text, want+"\n") { + t.Errorf("the export file lacks %q:\n%s", want, text) + } + } + for _, e := range ParseExports(text, ExportFile) { + if e.Client != wsAddress { + t.Errorf("a client other than the granted and wanting node: %+v", e) + } + if !strings.Contains(e.Options, "all_squash") || strings.Contains(e.Options, "no_root_squash") { + t.Errorf("every client is squashed: %+v", e) + } + } + if !s.Shares[0].Live || !s.Shares[1].Live { + t.Errorf("both shares are live after exportfs -ra: %+v", s.Shares) + } + if s.Reloaded == nil || s.ReloadWhy != "the export file changed" { + t.Errorf("the kernel read the file again because it changed: %+v", s) + } + // Rendering is stable: a second pass changes nothing and asks the kernel nothing. + f.ran = nil + f.machine().Pass(context.Background(), &s) + for _, r := range f.ran { + if strings.Contains(r, "exportfs") { + t.Errorf("a pass with nothing changed reloaded: %v", f.ran) + } + } +} + +func TestTheRootABackupRepositoryAndAMissingFolderAreRefusedAndNothingIsCreated(t *testing.T) { + f := newFake(t) + f.restic("/srv/pool/restic") + f.dir("/srv/pool/media") + f.restic("/srv/backups") + f.file("/srv/a-file", "x") + f.file(ConfigFile, config("root=/ pool=/srv/pool backups=/srv/backups missing=/srv/nowhere file=/srv/a-file rel=srv/x media=/srv/pool/media")) + f.serve("root", "pool", "backups", "missing", "file", "rel", "media") + f.kernelReads() + s := f.machine().Pass(context.Background(), nil) + refused := map[string]string{} + for _, r := range s.Refused { + refused[r.Name] = r.Why + } + for name, word := range map[string]string{ + "root": "root", "pool": "holds a backup repository", "backups": "is a backup repository", + "missing": "not there", "file": "not a folder", "rel": "not an absolute path", + } { + if !strings.Contains(refused[name], word) { + t.Errorf("%s refused with %q, want %q", name, refused[name], word) + } + } + if len(s.Shares) != 1 || s.Shares[0].Name != "media" { + t.Errorf("only the folder beside the backups is exported: %+v", s.Shares) + } + if strings.Contains(f.read(ExportFile), "/srv/pool ") || strings.Contains(f.read(ExportFile), "/srv/backups") { + t.Errorf("a refused folder is in the export file:\n%s", f.read(ExportFile)) + } + if _, err := os.Stat(filepath.Join(f.root, "/srv/nowhere")); err == nil { + t.Errorf("the module created a missing folder") + } + h := f.machine().Health() + if h.Healthy || !strings.Contains(h.Why, "backups are never exported") { + t.Errorf("a refused share makes the module unhealthy, said: %+v", h) + } +} + +func TestTheModuleWritesOnlyItsTwoFilesAndHasNoWayToChangeAnOwnerOrMode(t *testing.T) { + m := newFake(t).machine() + for _, p := range []string{"/srv/library/x", "/etc/exports", "/etc/exports.d/zfs.exports", ConfigFile} { + if err := m.Write(p, "x"); err == nil { + t.Errorf("Write accepted %s", p) + } + } + // No code path changes an owner or a mode, or removes anything: the words are not in the bundle. + files, _ := filepath.Glob("*.go") + for _, file := range files { + if strings.HasSuffix(file, "_test.go") { + continue + } + raw, _ := os.ReadFile(file) + text := string(raw) + // The one removal: unexport takes the module's own export file away on unassignment. + if file == "machine.go" { + start := strings.Index(text, "func (m *Machine) unexport() error {") + end := strings.Index(text[start:], "\n}\n") + body := text[start : start+end] + if strings.Count(body, "os.Remove(") != 1 || !strings.Contains(body, "os.Remove(m.at(ExportFile))") { + t.Errorf("unexport removes something other than the export file:\n%s", body) + } + text = text[:start] + text[start+end:] + } + for _, word := range []string{"Chown", "Lchown", "Chmod", "os.Remove", "RemoveAll", `"chown"`, `"chmod"`, `"rm"`, "Mkdir(", "MkdirAll(filepath.Join(m.Root"} { + if strings.Contains(text, word) { + t.Errorf("%s says %s", file, word) + } + } + } +} + +func TestParseSharesAndConfig(t *testing.T) { + shares, problems := ParseShares("library=/srv/library, downloads=/srv/downloads:ro Bad=/x noequals library=/again") + if len(shares) != 2 || shares[0].Name != "downloads" || !shares[0].ReadOnly || shares[1].Access != "rw" { + t.Errorf("shares: %+v", shares) + } + if len(problems) != 3 { + t.Errorf("problems: %v", problems) + } + c, problems := ParseConfig("shares=\nrange=not-a-range\n") + if c.Range != "" || len(problems) != 1 { + t.Errorf("a range that is not one is said: %+v %v", c, problems) + } + _, problems = ParseConfig("shares=a=/x\n") + if len(problems) != 1 || !strings.Contains(problems[0], "no private network range") { + t.Errorf("no range is said: %v", problems) + } +} + +func TestParseExportsReadsTheKernelsTableAndZFSFile(t *testing.T) { + etab := "/srv/library\t192.0.2.0/24(rw,sync,wdelay,root_squash,sec=sys)\n/srv/x\n# comment\n/srv/y a(ro) b(rw)\n" + got := ParseExports(etab, Etab) + if len(got) != 4 || got[0].Client != "192.0.2.0/24" || got[1].Client != "*" || got[3].Client != "b" || got[3].Options != "rw" { + t.Errorf("%+v", got) + } +} + +func zfsWith(f *fake, sharenfs string) { + f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Out: "pool\t/srv/pool\toff\npool/library\t/srv/library\t" + sharenfs + "\n"} +} + +func TestAdoptIsADryRunWithoutConfirmAndRefusesWhileTheMeshExportIsNotLive(t *testing.T) { + f := newFake(t) + f.uid = 1000 + f.dir("/srv/library") + zfsWith(f, "rw=@192.0.2.0/24") + f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") + // The kernel has not read the mesh's file: no exportfs effect. + f.machine().Pass(context.Background(), nil) + if _, err := f.machine().Adopt("/srv/library", true, "take it over"); err == nil || !strings.Contains(err.Error(), "not live") { + t.Fatalf("adopt while not live: %v", err) + } + f.kernelReads() + f.machine().Pass(context.Background(), nil) + f.ran = nil + dry, err := f.machine().Adopt("/srv/library", false, "") + if err != nil || dry.Done || !dry.DryRun || !strings.Contains(dry.Said, "would set sharenfs=off on the dataset pool/library") { + t.Fatalf("dry run: %+v %v", dry, err) + } + for _, r := range f.ran { + if strings.Contains(r, "zfs set") { + t.Fatalf("a dry run changed something: %v", f.ran) + } + } + if _, err := f.machine().Adopt("/srv/library", true, ""); err == nil { + t.Errorf("adopt with confirm and no why was done") + } + done, err := f.machine().Adopt("/srv/library", true, "the mesh exports it now") + if err != nil || !done.Done { + t.Fatalf("adopt: %+v %v", done, err) + } + var set []string + for _, r := range f.ran { + if strings.Contains(r, "zfs set") { + set = append(set, r) + } + } + if len(set) != 1 || set[0] != "sudo -n zfs set sharenfs=off pool/library" { + t.Errorf("exactly one property of exactly one dataset, through sudo: %v", set) + } + zfsWith(f, "off") + if _, err := f.machine().Adopt("/srv/library", true, "again"); err == nil || !strings.Contains(err.Error(), "already has sharenfs off") { + t.Errorf("adopt of a dataset already off: %v", err) + } + if _, err := f.machine().Adopt("/srv/elsewhere", false, ""); err == nil { + t.Errorf("adopt of a folder no dataset is mounted at") + } +} + +func TestAdoptRefusesAFolderTheMeshDoesNotExport(t *testing.T) { + f := newFake(t) + f.dir("/srv/library") + zfsWith(f, "on") + f.file(ConfigFile, config("")) + f.kernelReads() + f.machine().Pass(context.Background(), nil) + if _, err := f.machine().Adopt("/srv/library", false, ""); err == nil || !strings.Contains(err.Error(), "does not export") { + t.Errorf("%v", err) + } +} + +func TestHealthSaysAStoppedServerAStaleProcessAndAShareNotInTheTable(t *testing.T) { + f := newFake(t) + f.dir("/srv/library") + f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") + f.kernelReads() + f.machine().Pass(context.Background(), nil) + if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "exports library to ws (10.0.0.3, rw)") { + t.Errorf("healthy: %+v", h) + } + f.file(Etab, "") + f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "failed\n", Code: 3} + f.now = f.now.Add(10 * time.Minute) + h := f.machine().Health() + for _, want := range []string{"is failed", "last looked 10m0s ago", "The share library (/srv/library) is not in the kernel's export table."} { + if h.Healthy || !strings.Contains(h.Why, want) { + t.Errorf("health lacks %q: %+v", want, h) + } + } + if h := newFake(t).machine().Health(); h.Healthy || !strings.Contains(h.Why, "written no status") { + t.Errorf("no status is unhealthy: %+v", h) + } +} + +func TestExportsAnswersTheSharesAndWhatIsNotTheMeshs(t *testing.T) { + f := newFake(t) + f.dir("/srv/library") + f.file(ConfigFile, config("library=/srv/library")) + f.file("/etc/exports.d/zfs.exports", "# !!! DO NOT EDIT\n/srv/library 192.0.2.0/24(sec=sys,rw,no_subtree_check)\n") + f.file("/etc/exports", "# only comments\n") + f.serve("library") + zfsWith(f, "rw=@192.0.2.0/24") + f.kernelReads() + f.machine().Pass(context.Background(), nil) + a := f.machine().Exports() + raw, _ := json.Marshal(a) + var shape struct { + Shares []struct { + Name, Path, Access, Clients string + UID, GID int + Live bool + } + Foreign []map[string]any + Service string + } + if err := json.Unmarshal(raw, &shape); err != nil { + t.Fatal(err) + } + if len(shape.Shares) != 1 || shape.Shares[0].Name != "library" || !shape.Shares[0].Live || shape.Shares[0].Clients != wsAddress || shape.Service != "active" { + t.Errorf("%s", raw) + } + if len(shape.Foreign) != 2 { + t.Errorf("the ZFS file's line and the dataset's property are not the mesh's: %s", raw) + } + for _, key := range []string{`"shares"`, `"foreign"`, `"service"`, `"uid"`, `"gid"`, `"live"`, `"clients"`, `"access"`} { + if !strings.Contains(string(raw), key) { + t.Errorf("the answer lacks %s, which the mounts module reads", key) + } + } +} + +func TestTestSaysWhetherAShareIsServedAndWhetherAnAddressIsInRange(t *testing.T) { + f := newFake(t) + f.dir("/srv/library") + f.file(ConfigFile, config("library=/srv/library")) + f.serve("library") + f.kernelReads() + f.machine().Pass(context.Background(), nil) + ok, _ := f.machine().Test("library", "10.0.0.3") + if !ok.OK || ok.InRange == nil || !*ok.InRange { + t.Errorf("%+v", ok) + } + out, _ := f.machine().Test("library", "192.0.2.7") + if out.OK || !strings.Contains(out.Why, "not inside the private network's range") { + t.Errorf("%+v", out) + } + other, _ := f.machine().Test("library", "10.0.0.9") + if other.OK || !strings.Contains(other.Why, "is not exported to 10.0.0.9") { + t.Errorf("an address in the range the share is not exported to: %+v", other) + } + none, _ := f.machine().Test("photos", "") + if none.OK || !strings.Contains(none.Why, "no share named photos") { + t.Errorf("%+v", none) + } +} + +func TestClientsReadTheKernelsListOrFallBackToConnections(t *testing.T) { + f := newFake(t) + f.file("/proc/fs/nfsd/clients/7/info", "clientid: 0xabc\naddress: \"10.0.0.3:785\"\nname: \"Linux NFSv4.2 workstation\"\nminor version: 2\n") + a := f.machine().Clients() + if len(a.Clients) != 1 || a.Clients[0].Address != "10.0.0.3:785" || a.Clients[0].Minor != "2" { + t.Errorf("%+v", a) + } + g := newFake(t) + g.answers["ss -tnH state established ( sport = :2049 )"] = Ran{Out: "0 0 10.0.0.2:2049 10.0.0.3:785\n"} + b := g.machine().Clients() + if len(b.Clients) != 1 || b.Clients[0].Address != "10.0.0.3:785" || b.From != "connections to port 2049" { + t.Errorf("%+v", b) + } +} + +func TestAnActNeedingRootGoesThroughSudoOnlyWhenNotRoot(t *testing.T) { + f := newFake(t) + f.uid = 1000 + f.machine().AsRoot("exportfs", "-ra") + f.uid = 0 + f.machine().AsRoot("exportfs", "-ra") + if len(f.ran) != 2 || f.ran[0] != "sudo -n exportfs -ra" || f.ran[1] != "exportfs -ra" { + t.Errorf("%v", f.ran) + } + f.answers["sudo -n zfs set sharenfs=off x"] = Ran{Code: 1, Err: "sudo: a password is required"} + f.uid = 1000 + if _, err := f.machine().AsRoot("zfs", "set", "sharenfs=off", "x"); err == nil || !strings.Contains(err.Error(), "sudo refused") { + t.Errorf("%v", err) + } +} + +// asTheLibrarysOwner is how the tests see the folders they make: owned by the library's owner (1001:2000), +// not by whoever runs them. The build seat runs the suite as root, where every temporary folder is root's +// and the refusal of a root-owned share would fire first; on a person's machine they are that person's. +// A folder the test runner does not own keeps its real owner, so a real root-owned folder is still root's. +func asTheLibrarysOwner(info os.FileInfo) (uid, gid int) { + uid, gid = inodeOwner(info) + if uid == os.Getuid() { + uid = 1001 + } + if gid == os.Getgid() { + gid = 2000 + } + return uid, gid +} diff --git a/modules/nfs-server/cmd/nfs-server/process.go b/modules/nfs-server/cmd/nfs-server/process.go new file mode 100644 index 00000000..b28b933d --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/process.go @@ -0,0 +1,191 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "strings" + "time" +) + +// The module's process, as root: every Every it reads the assignment's shares, renders the export file, +// has the kernel read it again when anything changed or a share is not live, and writes what it saw for +// the tools (novox/hq ADR 0263). + +// Every is how often the process looks. +const Every = 30 * time.Second + +// Status is the process's last look, as the tools read it. +type Status struct { + At time.Time `json:"at"` + Range string `json:"range"` + Shares []Exported `json:"shares"` + Refused []Refused `json:"refused,omitempty"` + Problems []string `json:"problems,omitempty"` + // Said is what does not match between the grants and the wants: a node wanting a share it is not + // granted, a grant nobody wants, a share exported to no node. + Said []string `json:"said,omitempty"` + // Reloaded is when the process last had the kernel read the export files again, and why. + Reloaded *time.Time `json:"reloaded,omitempty"` + ReloadWhy string `json:"reload_why,omitempty"` + ReloadError string `json:"reload_error,omitempty"` +} + +// Pass is one look: what it did is in the status it answers, which it has written. +func (m *Machine) Pass(ctx context.Context, last *Status) Status { + s := Status{At: m.Now().UTC()} + if last != nil { + s.Reloaded, s.ReloadWhy = last.Reloaded, last.ReloadWhy + } + text, err := m.Read(ConfigFile) + if err != nil { + s.Problems = append(s.Problems, fmt.Sprintf("the assignment's shares (%s) could not be read: %v; nothing is exported", ConfigFile, err)) + } + c, problems := ParseConfig(text) + s.Problems = append(s.Problems, problems...) + s.Range = c.Range + roster, note := m.Roster() + if note != "" { + s.Problems = append(s.Problems, note) + } + wants, note := m.Wants() + if note != "" { + s.Said = append(s.Said, note) + } + exports, refused, said := m.Judge(c, roster, wants) + s.Refused = refused + s.Said = append(s.Said, said...) + + want := ExportFileText(exports) + have, _ := m.Read(ExportFile) + reload := "" + if have != want { + if err := m.Write(ExportFile, want); err != nil { + s.Problems = append(s.Problems, fmt.Sprintf("the export file could not be written: %v", err)) + } else { + reload = "the export file changed" + } + } + live, note := m.Live() + if reload == "" { + for _, e := range exports { + if !SharedLive(live, e) { + reload = fmt.Sprintf("the share %s was not in the kernel's table", e.Name) + break + } + } + } + if reload != "" { + if _, err := m.AsRoot("exportfs", "-ra"); err != nil { + s.ReloadError = err.Error() + } else { + at := m.Now().UTC() + s.Reloaded, s.ReloadWhy = &at, reload + } + live, note = m.Live() + } + if note != "" && len(exports) > 0 { + s.Problems = append(s.Problems, note) + } + for i := range exports { + exports[i].Live = SharedLive(live, exports[i]) + } + s.Shares = exports + raw, _ := json.MarshalIndent(s, "", " ") + if err := m.Write(StatusFile, string(raw)+"\n"); err != nil { + s.Problems = append(s.Problems, fmt.Sprintf("the status file could not be written: %v", err)) + } + return s +} + +// RunExports is the process: a pass now and every Every until the context ends, then Stopped. +func (m *Machine) RunExports(ctx context.Context) error { + last := m.Pass(ctx, nil) + t := time.NewTicker(Every) + defer t.Stop() + for { + select { + case <-ctx.Done(): + m.Stopped() + return nil + case <-t.C: + last = m.Pass(ctx, &last) + } + } +} + +// LastStatus is the process's last look, or why there is none. +func (m *Machine) LastStatus() (*Status, error) { + text, err := m.Read(StatusFile) + if err != nil { + return nil, fmt.Errorf("the module's process has written no status (%s): %v", StatusFile, err) + } + var s Status + if err := json.Unmarshal([]byte(text), &s); err != nil { + return nil, fmt.Errorf("the module's status file is not one: %v", err) + } + return &s, nil +} + +// Roster is the mesh's machines by name, with their private addresses, as the mesh rendered them. +func (m *Machine) Roster() (map[string]string, string) { + text, err := m.Read(RosterFile) + if err != nil { + return map[string]string{}, fmt.Sprintf("the mesh's roster (%s) could not be read: %v; no node's address is known, so nothing is exported", RosterFile, err) + } + out := map[string]string{} + for _, line := range strings.Split(text, "\n") { + f := strings.Fields(line) + if len(f) != 2 || strings.HasPrefix(f[0], "#") { + continue + } + out[f[0]] = f[1] + } + return out, "" +} + +// Wants is every node's wants of this machine, as the tools last took them from the bus. With none taken +// yet, nothing is wanted, and nothing is exported. +func (m *Machine) Wants() (map[string][]Want, string) { + text, err := m.Read(WantsFile) + if err != nil { + return map[string][]Want{}, "no node's wants have been taken from the bus yet, so nothing is exported" + } + var w WantsFromBus + if err := json.Unmarshal([]byte(text), &w); err != nil { + return map[string][]Want{}, fmt.Sprintf("the wants taken from the bus (%s) could not be read: %v; nothing is exported", WantsFile, err) + } + if w.Wants == nil { + w.Wants = map[string][]Want{} + } + return w.Wants, "" +} + +// SharedLive says whether the kernel exports a share to every node it is meant for. +func SharedLive(live []Entry, e Exported) bool { + for _, g := range e.To { + if !IsLive(live, e.Path, g.Address) { + return false + } + } + return true +} + +// Stopped is the process ending. When the module was unassigned — the node-engine removes the module's +// files in reverse declaration order, so the config file, declared after this process, is already gone +// when the process is stopped (mesh-host internal/apply: orphans removed in reverse order) — the +// module's export file goes and the kernel lets the shares go. A plain restart, a push or the machine +// shutting down leaves the config file, and the exports stay. +func (m *Machine) Stopped() { + if _, err := m.Read(ConfigFile); err == nil || !os.IsNotExist(err) { + return + } + if err := m.unexport(); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server:", err) + return + } + if _, err := m.AsRoot("exportfs", "-ra"); err != nil { + fmt.Fprintln(os.Stderr, "nfs-server: the export file is gone, and exportfs -ra failed:", err) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/review_test.go b/modules/nfs-server/cmd/nfs-server/review_test.go new file mode 100644 index 00000000..c0893d11 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/review_test.go @@ -0,0 +1,381 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// The review of 2026-10-08 (mesh-catalog #136): each finding, reproduced. + +// roster writes the machines the mesh renders, name and address. +func (f *fake) roster(lines ...string) { + f.file(RosterFile, "# test\n"+strings.Join(lines, "\n")+"\n") +} + +// wants writes what the nodes' mounts modules want of this machine, as the tools take it from the bus. +func (f *fake) wants(w map[string][]Want) { + raw, _ := json.Marshal(WantsFromBus{Wants: w}) + f.file(WantsFile, string(raw)) +} + +// mountinfo writes the kernel's mount table with these mount points. +func (f *fake) mountinfo(points ...string) { + var b strings.Builder + for i, p := range points { + fmt.Fprintf(&b, "%d 1 0:%d / %s rw,relatime shared:1 - zfs pool%d rw\n", 20+i, 40+i, p, i) + } + f.file(MountInfo, b.String()) +} + +func (f *fake) symlink(target, at string) { + f.dir(filepath.Dir(at)) + if err := os.Symlink(target, filepath.Join(f.root, at)); err != nil { + f.t.Fatal(err) + } +} + +// grantsConfig is a config with grants. +func grantsConfig(shares, grants string) string { + return "# test\nshares=" + shares + "\ngrants=" + grants + "\nrange=" + meshRange + "\n" +} + +func refusedWhy(s Status, name string) string { + for _, r := range s.Refused { + if r.Name == name { + return r.Why + } + } + return "" +} + +// Finding 1: a share owned by root would squash every client to root. +func TestAShareWhoseOwnerIsRootIsRefused(t *testing.T) { + f := newFake(t) + f.dir("/srv/rootowned") + f.dir("/srv/rootgroup") + f.mountinfo("/") + f.owners = map[string]Owner{"/srv/rootowned": {UID: 0, GID: 2000, Dir: true}, "/srv/rootgroup": {UID: 1001, GID: 0, Dir: true}} + f.file(ConfigFile, grantsConfig("a=/srv/rootowned b=/srv/rootgroup", "a=ws:rw b=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "a", Access: "rw"}, {Share: "b", Access: "rw"}}}) + s := f.machine().Pass(context.Background(), nil) + for _, name := range []string{"a", "b"} { + if !strings.Contains(refusedWhy(s, name), "root") { + t.Errorf("%s is not refused for being root's: %+v", name, s) + } + } + if strings.Contains(f.read(ExportFile), "anonuid=0") || strings.Contains(f.read(ExportFile), "anongid=0") { + t.Errorf("an export squashes to root:\n%s", f.read(ExportFile)) + } +} + +// Finding 1, on a real inode: a folder root owns, read as the kernel says, is refused. Run as root (the build +// seat, or `unshare -r`), the test's own temporary folder is root's; run as anyone else, a system folder is. +func TestARealRootOwnedFolderIsRefused(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + share := "/srv/mine" + f.dir(share) + root := f.root + if os.Getuid() != 0 { + info, err := os.Stat("/usr") + if err != nil { + t.Skip("no /usr to stand for a root-owned folder:", err) + } + if uid, _ := inodeOwner(info); uid != 0 { + t.Skipf("/usr is not root's here (uid %d); run as root or under unshare -r", uid) + } + root, share = "/", "/usr" + } + f.file(ConfigFile, grantsConfig("system="+share, "system=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "system", Access: "rw"}}}) + m := f.machine() + m.ownerOf = nil // the inode as the kernel says it + if root == "/" { + // Judge only reads: the folder's real path, its inode and the mount table. Nothing is written. + sys := &Machine{Root: "/", Run: func(context.Context, string, ...string) Ran { return Ran{} }} + _, refused, _ := sys.Judge(Config{Shares: []Share{{Name: "system", Path: share, Access: "rw"}}, + Grants: map[string]map[string]string{"system": {"ws": "rw"}}, Range: "10.0.0.0/24"}, + map[string]string{"ws": "10.0.0.3"}, map[string][]Want{"ws": {{Share: "system", Access: "rw"}}}) + if len(refused) != 1 || !strings.Contains(refused[0].Why, "owned by root") { + t.Fatalf("the system folder %s, root's on its inode, is not refused for being root's: %+v", share, refused) + } + return + } + s := m.Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "system"), "root") { + t.Errorf("a folder root owns on its inode is not refused: %+v", s) + } +} + +// Finding 2: a symlink is judged where it points, and the export names the real folder. +func TestASymlinkIsJudgedAndExportedAtItsRealPath(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.restic("/srv/backups") + f.dir("/srv/real-library") + f.symlink("backups", "/srv/innocent") + f.symlink("real-library", "/srv/library") + f.file(ConfigFile, grantsConfig("innocent=/srv/innocent library=/srv/library", "innocent=ws:rw library=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "innocent", Access: "rw"}, {Share: "library", Access: "rw"}}}) + s := f.machine().Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "innocent"), "backup") { + t.Errorf("a symlink to the backups is not refused: %+v", s) + } + text := f.read(ExportFile) + if strings.Contains(text, "/srv/innocent") || strings.Contains(text, "/srv/backups") { + t.Errorf("the backups are exported through a symlink:\n%s", text) + } + if !strings.Contains(text, "\n/srv/real-library ") || strings.Contains(text, "\n/srv/library ") { + t.Errorf("the export does not name the real folder:\n%s", text) + } +} + +// Finding 3: a folder inside a backup repository, or one holding a repository deeper than one level, +// is refused; a walk cut short refuses rather than exports. +func TestABackupRepositoryAboveOrDeepBelowIsRefusedAndAnUnfinishedLookRefuses(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.restic("/srv/repo") + f.dir("/srv/repo/data/ab") + f.restic("/srv/media/films/old/backup") + f.dir("/srv/wide/a") + f.dir("/srv/wide/b") + f.dir("/srv/wide/c") + f.dir("/srv/clean/films/one") + f.file(ConfigFile, grantsConfig("inside=/srv/repo/data/ab deep=/srv/media wide=/srv/wide clean=/srv/clean", "inside=ws:rw deep=ws:rw wide=ws:rw clean=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "inside", Access: "rw"}, {Share: "deep", Access: "rw"}, {Share: "wide", Access: "rw"}, {Share: "clean", Access: "rw"}}}) + m := f.machine() + m.WalkLimit = 4 + s := m.Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "inside"), "inside the backup repository /srv/repo") { + t.Errorf("a folder inside a backup repository: %q", refusedWhy(s, "inside")) + } + if !strings.Contains(refusedWhy(s, "deep"), "/srv/media/films/old/backup") { + t.Errorf("a repository three levels down: %q", refusedWhy(s, "deep")) + } + m2 := f.machine() + m2.WalkLimit = 2 + s = m2.Pass(context.Background(), nil) + if !strings.Contains(refusedWhy(s, "wide"), "could not make sure") { + t.Errorf("a look cut short by its bound did not refuse: %q", refusedWhy(s, "wide")) + } + m3 := f.machine() + s = m3.Pass(context.Background(), nil) + if refusedWhy(s, "clean") != "" || refusedWhy(s, "wide") != "" { + t.Errorf("a clean folder is refused: %+v", s.Refused) + } +} + +// Finding 5: a share is exported to the nodes the server grants it to and whose mounts want it, at the +// lower of the two accesses, at the roster's address only; the control node and strangers get nothing. +func TestAShareGoesOnlyToTheNodesGrantedAndWantingItAtTheLowerAccess(t *testing.T) { + f := newFake(t) + f.mountinfo("/", "/srv/library", "/srv/pool") + f.dir("/srv/library") + f.dir("/srv/pool/downloads") + f.dir("/srv/pool/unwanted") + f.file(ConfigFile, grantsConfig("library=/srv/library downloads=/srv/pool/downloads unwanted=/srv/pool/unwanted", + "library=ws:rw,laptop:ro downloads=ws:rw unwanted=ws:rw")) + f.roster("ws 10.0.0.3", "laptop 10.0.0.4", "control 10.0.0.1", "outside 192.0.2.9") + f.wants(map[string][]Want{ + "ws": {{Share: "library", Access: "ro"}, {Share: "downloads", Access: "rw"}}, + "laptop": {{Share: "library", Access: "rw"}}, + "control": {{Share: "library", Access: "rw"}}, + "stranger": {{Share: "library", Access: "rw"}}, + "outside": {{Share: "downloads", Access: "rw"}}, + }) + f.kernelReads() + s := f.machine().Pass(context.Background(), nil) + text := f.read(ExportFile) + var lib, dl string + for _, line := range strings.Split(text, "\n") { + if strings.HasPrefix(line, "/srv/library ") { + lib = line + } + if strings.HasPrefix(line, "/srv/pool/downloads ") { + dl = line + } + } + if !strings.Contains(lib, " 10.0.0.3(ro,") || !strings.Contains(lib, " 10.0.0.4(ro,") { + t.Errorf("the library goes ro to the workstation (it wants ro) and ro to the laptop (granted ro): %q", lib) + } + if strings.Contains(text, "10.0.0.1") || strings.Contains(text, meshRange) || strings.Contains(text, "192.0.2.9") { + t.Errorf("the control node, the whole range or an address outside the range is exported to:\n%s", text) + } + if !strings.Contains(dl, " 10.0.0.3(rw,") || strings.Count(dl, "(") != 1 { + t.Errorf("downloads goes rw to the workstation only: %q", dl) + } + if strings.Contains(text, "/srv/pool/unwanted") { + t.Errorf("a share no node wants is exported:\n%s", text) + } + var said = strings.Join(s.Said, "\n") + for _, want := range []string{"control wants library", "stranger", "unwanted is granted to ws, which does not want it"} { + if !strings.Contains(said, want) { + t.Errorf("the status does not say %q:\n%s", want, said) + } + } + for _, e := range s.Shares { + if e.Name == "library" && (len(e.To) != 2 || !e.Live) { + t.Errorf("the library's clients: %+v", e) + } + } + if !strings.Contains(f.read(StatusFile), `"to"`) { + t.Errorf("the status lacks each share's clients") + } +} + +// With no wants file (the bus never answered), nothing is exported: closed, not open. +func TestWithNoWantsNothingIsExported(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.dir("/srv/library") + f.file(ConfigFile, grantsConfig("library=/srv/library", "library=ws:rw")) + f.roster("ws 10.0.0.3") + f.machine().Pass(context.Background(), nil) + if strings.Contains(f.read(ExportFile), "/srv/library") { + t.Errorf("exported with no node asking:\n%s", f.read(ExportFile)) + } +} + +// Finding 6: `mp` keeps a share unexported while its filesystem is not mounted. +func TestTheExportCarriesMpForItsFilesystem(t *testing.T) { + f := newFake(t) + f.mountinfo("/", "/srv/library", "/srv/pool") + f.dir("/srv/library") + f.dir("/srv/pool/downloads") + f.dir("/srv/plain") + f.file(ConfigFile, grantsConfig("library=/srv/library downloads=/srv/pool/downloads plain=/srv/plain", "library=ws:rw downloads=ws:rw plain=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "library", Access: "rw"}, {Share: "downloads", Access: "rw"}, {Share: "plain", Access: "rw"}}}) + f.machine().Pass(context.Background(), nil) + text := f.read(ExportFile) + if !strings.Contains(text, "/srv/library 10.0.0.3(rw,mp,") { + t.Errorf("a dataset share lacks mp:\n%s", text) + } + if !strings.Contains(text, "/srv/pool/downloads 10.0.0.3(rw,mp=/srv/pool,") { + t.Errorf("a folder in a dataset lacks mp=:\n%s", text) + } + if strings.Contains(text, "/srv/plain 10.0.0.3(rw,mp") { + t.Errorf("a folder of the root filesystem names mp:\n%s", text) + } +} + +// Finding 7: the module's export file goes when the module is unassigned, and stays on a restart. +func TestTheExportFileGoesOnUnassignmentAndStaysOnARestart(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.dir("/srv/library") + f.file(ConfigFile, grantsConfig("library=/srv/library", "library=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "library", Access: "rw"}}}) + f.kernelReads() + m := f.machine() + m.Pass(context.Background(), nil) + f.ran = nil + m.Stopped() + if !strings.Contains(f.read(ExportFile), "/srv/library") { + t.Errorf("a restart (config still there) removed the exports") + } + for _, r := range f.ran { + if strings.Contains(r, "exportfs") { + t.Errorf("a restart reloaded the exports: %v", f.ran) + } + } + if err := os.Remove(filepath.Join(f.root, ConfigFile)); err != nil { + t.Fatal(err) + } + m.Stopped() + if _, err := os.Stat(filepath.Join(f.root, ExportFile)); err == nil { + t.Errorf("the export file stays after the module's config was removed") + } + if strings.Contains(f.read(Etab), "/srv/library") { + t.Errorf("the kernel still exports the folder after unassignment: %q", f.read(Etab)) + } +} + +// Finding 9: the seat's schema passes `confirm` as a string. +func TestConfirmIsTrueAsABooleanOrTheStringTrue(t *testing.T) { + for _, c := range []struct { + v any + want bool + }{{true, true}, {"true", true}, {"TRUE", true}, {"false", false}, {"yes", false}, {nil, false}, {false, false}} { + if got := confirmed(map[string]any{"confirm": c.v}); got != c.want { + t.Errorf("confirm %#v read as %v", c.v, got) + } + } +} + +// Grants are read plainly, and "none" grants nothing. +func TestParseGrants(t *testing.T) { + g, problems := ParseGrants("library=ws:rw,laptop:ro downloads=ws:rw") + if len(problems) != 0 || g["library"]["laptop"] != "ro" || g["downloads"]["ws"] != "rw" { + t.Errorf("%v %v", g, problems) + } + g, problems = ParseGrants("none") + if len(g) != 0 || len(problems) != 0 { + t.Errorf("none: %v %v", g, problems) + } + _, problems = ParseGrants("library=ws library=x:rw bad=ws:rx") + if len(problems) != 3 { + t.Errorf("bad grants are said: %v", problems) + } +} + +// The wants are taken from each node's `mounts` state: only what is asked of this machine, keyed by the +// node that asked. +func TestWantsFromTheMountsStateKeepOnlyWhatIsAskedOfThisMachine(t *testing.T) { + states := map[string][]byte{ + "ws": []byte(`{"wants":[{"server":"home","share":"library","access":"rw"},{"server":"other","share":"x","access":"rw"}],"shares":[]}`), + "laptop": []byte(`{"wants":[{"server":"home","share":"library","access":"ro"}]}`), + "broken": []byte(`not json`), + "none": []byte(`{"shares":[]}`), + } + w, notes := WantsFromStates("home", states) + if len(w) != 2 || len(w["ws"]) != 1 || w["ws"][0].Share != "library" || w["laptop"][0].Access != "ro" { + t.Errorf("%+v", w) + } + if len(notes) != 1 || !strings.Contains(notes[0], "broken") { + t.Errorf("a state that cannot be read is said: %v", notes) + } + m := newFake(t).machine() + if err := m.Write(WantsFile, "{}"); err != nil { + t.Errorf("the tools may write the wants file: %v", err) + } +} + +// The look below a share for a backup repository walks up to WalkLimit folders; on a library of tens of +// thousands of folders that must not run on every 30 s pass. A clean result is kept for LookKept, and a +// repository put there later is found once that has passed. +func TestTheLookBelowIsKeptForAWhileAndThenTakenAgain(t *testing.T) { + f := newFake(t) + f.mountinfo("/") + f.dir("/srv/clean/films/one") + f.file(ConfigFile, grantsConfig("clean=/srv/clean", "clean=ws:rw")) + f.roster("ws 10.0.0.3") + f.wants(map[string][]Want{"ws": {{Share: "clean", Access: "rw"}}}) + m := f.machine() + now := time.Date(2026, 10, 8, 19, 0, 0, 0, time.UTC) + m.Now = func() time.Time { return now } + if s := m.Pass(context.Background(), nil); refusedWhy(s, "clean") != "" { + t.Fatalf("a clean folder is refused: %+v", s.Refused) + } + f.restic("/srv/clean/films/one/backup") + now = now.Add(time.Minute) + if s := m.Pass(context.Background(), nil); refusedWhy(s, "clean") != "" { + t.Errorf("the look was taken again within a minute: %+v", s.Refused) + } + now = now.Add(LookKept) + if s := m.Pass(context.Background(), nil); !strings.Contains(refusedWhy(s, "clean"), "/srv/clean/films/one/backup") { + t.Errorf("a repository put there later was not found once the kept look expired: %+v", s.Refused) + } +} diff --git a/modules/nfs-server/cmd/nfs-server/shares.go b/modules/nfs-server/cmd/nfs-server/shares.go new file mode 100644 index 00000000..eb656442 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/shares.go @@ -0,0 +1,519 @@ +package main + +import ( + "fmt" + "net/netip" + "path" + "sort" + "strings" + "time" +) + +// The shares, from the assignment, and the export lines made from them (novox/hq ADR 0263). +// +// **One share per folder.** A share is a name and a path the operator gives in the `shares` setting: +// `library=/srv/library downloads=/srv/downloads`, `:ro` after a path for read-only. The path is the +// operator's data (novox/hq ADR 0051): the module reads its owner and exports it, and never creates, +// chowns, chmods or removes it. +// +// **Every client is the folder's owner.** NFS with sec=sys believes the user id the client sends, and the +// same id names different accounts on different machines. So every line squashes every client +// (`all_squash`) to the owner of the shared folder as this machine reads it (`anonuid`, `anongid`): a file +// written from any node lands owned by the folder's owner, and no client acts as any other account here. +// Root is squashed with everyone else. +// +// **To the nodes that ask, as far as this machine allows** (review of mesh-catalog #136, point 5). A share +// is exported to a node's address only when this machine's `grants` setting names that node for it, and +// that node's `mounts` module wants it from this machine (its state `shares` on the bus, which the tools +// write to WantsFile). The access is the lower of the two: a grant of ro is never widened by a want. The +// address is the mesh's own, from the roster the mesh renders (RosterFile), never one a consumer gave, +// and it must lie in the private network's range. A share nobody is granted and wants is exported to +// nobody. The home network is never named. +// +// **Never the backups, never root's.** A share is judged at its real path (symlinks resolved). It is +// refused when it is the machine's root, is not there, is not a folder, is owned by root or root's group +// (squashing to root would make every client root there), lies inside a backup repository, or holds one +// within WalkDepth levels below — and when that look is cut short by its bound, it is refused rather +// than exported. +// +// **Only while its filesystem is mounted.** Every line carries `mp`: a share that is a filesystem of its +// own is exported only while it is mounted, and a folder inside one only while that filesystem is +// (`mp=`). Without it, an unmounted dataset would export the empty folder under it. + +// Share is one share the operator asked for. +type Share struct { + Name string `json:"name"` + Path string `json:"path"` + ReadOnly bool `json:"-"` + Access string `json:"access"` +} + +// Grantee is one node a share is exported to: its name, its address on the private network, and the +// access it is given. +type Grantee struct { + Node string `json:"node"` + Address string `json:"address"` + Access string `json:"access"` +} + +// Exported is a share as this machine exports it. +type Exported struct { + Name string `json:"name"` + Path string `json:"path"` + Access string `json:"access"` + UID int `json:"uid"` + GID int `json:"gid"` + // Clients is the addresses it is exported to, separated by commas; empty when no node is both granted + // it and wants it. To says who each is. + Clients string `json:"clients"` + To []Grantee `json:"to"` + // Mp is the export's mp option: "mp", "mp=", or empty for a folder of the root filesystem. + Mp string `json:"mp,omitempty"` + Line string `json:"line"` + Live bool `json:"live"` +} + +// Refused is a share that is not exported, and why. +type Refused struct { + Name string `json:"name"` + Path string `json:"path"` + Why string `json:"why"` +} + +// Config is what the mesh rendered from the assignment. +type Config struct { + Shares []Share + // Grants is, per share, the nodes this machine lets mount it and with which access. + Grants map[string]map[string]string + Range string +} + +// Want is one share a node's mounts module wants of this machine, as it put it on the bus. +type Want struct { + Share string `json:"share"` + Access string `json:"access"` +} + +// WantsFromBus is the file the tools write from the bus: every node's wants of this machine. +type WantsFromBus struct { + Looked string `json:"looked,omitempty"` + Wants map[string][]Want `json:"wants"` +} + +// ParseConfig reads the config file: `shares=`, `grants=` and `range=` lines, comments and blanks +// ignored. +func ParseConfig(text string) (Config, []string) { + var c Config + var problems []string + for _, line := range strings.Split(text, "\n") { + line = strings.TrimSpace(line) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + key, value, ok := strings.Cut(line, "=") + if !ok { + problems = append(problems, fmt.Sprintf("the line %q of %s says nothing this module reads", line, ConfigFile)) + continue + } + switch strings.TrimSpace(key) { + case "shares": + shares, bad := ParseShares(value) + c.Shares = shares + problems = append(problems, bad...) + case "grants": + grants, bad := ParseGrants(value) + c.Grants = grants + problems = append(problems, bad...) + case "range": + c.Range = strings.TrimSpace(value) + default: + problems = append(problems, fmt.Sprintf("%s names %q, which this module does not read", ConfigFile, key)) + } + } + if c.Range == "" { + problems = append(problems, fmt.Sprintf("%s gives no private network range, so nothing can be exported", ConfigFile)) + } else if _, err := netip.ParsePrefix(c.Range); err != nil { + problems = append(problems, fmt.Sprintf("the private network's range %q is not a range: %v", c.Range, err)) + c.Range = "" + } + return c, problems +} + +// ParseShares reads the `shares` setting: `name=path` or `name=path:ro`, separated by spaces or commas. +func ParseShares(value string) ([]Share, []string) { + var shares []Share + var problems []string + seen := map[string]bool{} + for _, item := range strings.FieldsFunc(value, func(r rune) bool { return r == ' ' || r == ',' || r == '\t' }) { + name, p, ok := strings.Cut(item, "=") + if !ok || name == "" || p == "" { + problems = append(problems, fmt.Sprintf("the share %q is not name=path", item)) + continue + } + if !validName(name) { + problems = append(problems, fmt.Sprintf("the share name %q is not lower-case letters, digits and dashes", name)) + continue + } + if seen[name] { + problems = append(problems, fmt.Sprintf("the share %q is named twice", name)) + continue + } + seen[name] = true + s := Share{Name: name, Path: p, Access: "rw"} + if strings.HasSuffix(p, ":ro") { + s.Path, s.ReadOnly, s.Access = strings.TrimSuffix(p, ":ro"), true, "ro" + } else if strings.HasSuffix(p, ":rw") { + s.Path = strings.TrimSuffix(p, ":rw") + } + shares = append(shares, s) + } + sort.Slice(shares, func(i, j int) bool { return shares[i].Name < shares[j].Name }) + return shares, problems +} + +// ParseGrants reads the `grants` setting: `share=node:rw|ro[,node:rw|ro]`, entries separated by spaces; +// `none` grants nothing. +func ParseGrants(value string) (map[string]map[string]string, []string) { + out := map[string]map[string]string{} + seen := map[string]bool{} + var problems []string + value = strings.TrimSpace(value) + if value == "" || value == "none" { + return out, nil + } + for _, item := range strings.Fields(value) { + share, list, ok := strings.Cut(item, "=") + if !ok || !validName(share) || list == "" { + problems = append(problems, fmt.Sprintf("the grant %q is not share=node:rw|ro[,node:rw|ro]", item)) + continue + } + if seen[share] { + problems = append(problems, fmt.Sprintf("the share %q is granted twice; one entry per share", share)) + continue + } + seen[share] = true + nodes := map[string]string{} + good := true + for _, g := range strings.Split(list, ",") { + node, access, ok := strings.Cut(g, ":") + if !ok || !validName(node) || (access != "rw" && access != "ro") { + problems = append(problems, fmt.Sprintf("the grant %q of share %s is not node:rw or node:ro", g, share)) + good = false + break + } + nodes[node] = access + } + if good { + out[share] = nodes + } + } + return out, problems +} + +func validName(n string) bool { + if n == "" || n[0] == '-' { + return false + } + for _, r := range n { + if !(r >= 'a' && r <= 'z' || r >= '0' && r <= '9' || r == '-') { + return false + } + } + return true +} + +// IsBackupRepository says whether a directory is a restic repository: a `config` file beside `keys`, +// `snapshots` and `data`. It looks at names only. +func (m *Machine) IsBackupRepository(dir string) bool { + if o, err := m.Stat(path.Join(dir, "config")); err != nil || o.Dir { + return false + } + for _, sub := range []string{"keys", "snapshots", "data"} { + if o, err := m.Stat(path.Join(dir, sub)); err != nil || !o.Dir { + return false + } + } + return true +} + +// WalkDepth is how many levels below a share are looked through for a backup repository; WalkLimit how +// many folders at most, after which the share is refused rather than exported unchecked. +const ( + WalkDepth = 4 + WalkLimit = 200000 +) + +// LookKept is how long a look below a share that found no backup repository is believed. The look walks up +// to WalkLimit folders, which on a large library is too much for every 30 s pass; a repository put under a +// share is found within this long. A look that found one, or was cut short, is never kept: the share stays +// refused and is looked at again on the next pass. +const LookKept = time.Hour + +// repositoryBelow finds a backup repository within WalkDepth levels below dir. complete is false when +// the look stopped at its bound before it saw everything. +func (m *Machine) repositoryBelow(dir string) (found string, complete bool) { + now := time.Now() + if m.Now != nil { + now = m.Now() + } + m.cleanMu.Lock() + defer m.cleanMu.Unlock() + if at, kept := m.clean[dir]; kept && now.Sub(at) < LookKept { + return "", true + } + found, complete = m.walkBelow(dir) + if found == "" && complete { + if m.clean == nil { + m.clean = map[string]time.Time{} + } + m.clean[dir] = now + } else { + delete(m.clean, dir) + } + return found, complete +} + +// walkBelow is the look itself, breadth first, bounded by WalkDepth and WalkLimit. +func (m *Machine) walkBelow(dir string) (found string, complete bool) { + limit := m.WalkLimit + if limit <= 0 { + limit = WalkLimit + } + seen := 0 + level := []string{dir} + for depth := 1; depth <= WalkDepth && len(level) > 0; depth++ { + var next []string + for _, d := range level { + for _, child := range m.Children(d) { + seen++ + if seen > limit { + return "", false + } + p := path.Join(d, child) + if m.IsBackupRepository(p) { + return p, true + } + next = append(next, p) + } + } + level = next + } + return "", true +} + +// mpFor is the export's mp option for a real path: `mp` when it is a mount point of its own, `mp=` when that is not the root filesystem, else nothing. +func mpFor(points []string, real string) string { + best := "" + for _, p := range points { + if p == real { + return "mp" + } + if p != "/" && strings.HasPrefix(real, p+"/") && len(p) > len(best) { + best = p + } + } + if best == "" { + return "" + } + return "mp=" + best +} + +// lower is the narrower of two accesses: ro wins. +func lower(a, b string) string { + if a == "ro" || b == "ro" { + return "ro" + } + return "rw" +} + +// Judge turns the asked-for shares into export lines, or refusals, for the nodes granted and wanting +// each; what does not match is said. +func (m *Machine) Judge(c Config, roster map[string]string, wants map[string][]Want) ([]Exported, []Refused, []string) { + var out []Exported + var refused []Refused + var said []string + var prefix netip.Prefix + if c.Range != "" { + prefix, _ = netip.ParsePrefix(c.Range) + } + points := m.MountPoints() + shares := map[string]bool{} + for _, s := range c.Shares { + shares[s.Name] = true + } + for _, s := range c.Shares { + no := func(why string) { refused = append(refused, Refused{Name: s.Name, Path: s.Path, Why: why}) } + clean := path.Clean(s.Path) + switch { + case !path.IsAbs(s.Path): + no(fmt.Sprintf("The share %s names %q, which is not an absolute path.", s.Name, s.Path)) + continue + case strings.ContainsAny(s.Path, " \t\"\\#"): + no(fmt.Sprintf("The share %s's path %q has a character an export line cannot carry.", s.Name, s.Path)) + continue + } + real, err := m.Real(clean) + if err != nil { + no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, clean, err)) + continue + } + via := "" + if real != clean { + via = fmt.Sprintf(" (%s is a symlink to %s)", clean, real) + } + if real == "/" { + no(fmt.Sprintf("The share %s is the machine's root%s, which is never exported.", s.Name, via)) + continue + } + if strings.ContainsAny(real, " \t\"\\#") { + no(fmt.Sprintf("The share %s's real path %q has a character an export line cannot carry.", s.Name, real)) + continue + } + o, err := m.Stat(real) + if err != nil { + no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, real, err)) + continue + } + if !o.Dir { + no(fmt.Sprintf("The share %s's path %s%s is not a folder.", s.Name, real, via)) + continue + } + if o.UID == 0 || o.GID == 0 { + no(fmt.Sprintf("The share %s's folder %s%s is owned by root (%d:%d), and every client would act as root there: give it an owner of its own.", s.Name, real, via, o.UID, o.GID)) + continue + } + if m.IsBackupRepository(real) { + no(fmt.Sprintf("The share %s's folder %s%s is a backup repository, and backups are never exported.", s.Name, real, via)) + continue + } + inside := "" + for p := path.Dir(real); ; p = path.Dir(p) { + if m.IsBackupRepository(p) { + inside = p + break + } + if p == "/" { + break + } + } + if inside != "" { + no(fmt.Sprintf("The share %s's folder %s%s is inside the backup repository %s, and backups are never exported.", s.Name, real, via, inside)) + continue + } + holds, complete := m.repositoryBelow(real) + if holds != "" { + no(fmt.Sprintf("The share %s's folder %s%s holds a backup repository (%s), and backups are never exported: share the folders beside it instead.", s.Name, real, via, holds)) + continue + } + if !complete { + no(fmt.Sprintf("The share %s's folder %s%s holds more folders than the module looks through (%d levels, at most %d folders), so it could not make sure no backup repository is under it: share smaller folders.", s.Name, real, via, WalkDepth, m.limit())) + continue + } + if c.Range == "" { + no(fmt.Sprintf("The share %s is not exported: the private network's range is not known.", s.Name)) + continue + } + e := Exported{Name: s.Name, Path: real, Access: s.Access, UID: o.UID, GID: o.GID, Mp: mpFor(points, real), To: []Grantee{}} + granted := c.Grants[s.Name] + nodes := make([]string, 0, len(wants)) + for n := range wants { + nodes = append(nodes, n) + } + sort.Strings(nodes) + wanting := map[string]bool{} + for _, node := range nodes { + for _, w := range wants[node] { + if w.Share != s.Name { + continue + } + wanting[node] = true + g, ok := granted[node] + if !ok { + said = append(said, fmt.Sprintf("%s wants %s, and this machine's grants do not name it, so it is not exported to it", node, s.Name)) + continue + } + addr, known := roster[node] + if !known { + said = append(said, fmt.Sprintf("%s wants %s, and the mesh's roster names no such machine, so it is not exported to it", node, s.Name)) + continue + } + a, err := netip.ParseAddr(addr) + if err != nil || !prefix.Contains(a) { + said = append(said, fmt.Sprintf("%s's address %s is not inside the private network's range %s, so %s is not exported to it", node, addr, c.Range, s.Name)) + continue + } + access := lower(lower(g, w.Access), s.Access) + e.To = append(e.To, Grantee{Node: node, Address: addr, Access: access}) + } + } + grantedNodes := make([]string, 0, len(granted)) + for n := range granted { + if !wanting[n] { + grantedNodes = append(grantedNodes, n) + } + } + sort.Strings(grantedNodes) + for _, n := range grantedNodes { + said = append(said, fmt.Sprintf("%s is granted to %s, which does not want it", s.Name, n)) + } + var addrs []string + for _, g := range e.To { + addrs = append(addrs, g.Address) + } + e.Clients = strings.Join(addrs, ",") + e.Line = ExportLine(e) + if len(e.To) == 0 { + said = append(said, fmt.Sprintf("%s is exported to no node: none is both granted it and wants it", s.Name)) + } + out = append(out, e) + } + for share := range c.Grants { + if !shares[share] { + said = append(said, fmt.Sprintf("the grants name %s, which is not one of this machine's shares", share)) + } + } + sort.Strings(said) + return out, refused, said +} + +func (m *Machine) limit() int { + if m.WalkLimit > 0 { + return m.WalkLimit + } + return WalkLimit +} + +// ExportLine is one share's line in the export file, or empty when it goes to no node. +func ExportLine(e Exported) string { + if len(e.To) == 0 { + return "" + } + var b strings.Builder + b.WriteString(e.Path) + for _, g := range e.To { + opts := g.Access + if e.Mp != "" { + opts += "," + e.Mp + } + fmt.Fprintf(&b, " %s(%s,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", g.Address, opts, e.UID, e.GID) + } + return b.String() +} + +// ExportFileText is the whole export file. +func ExportFileText(exports []Exported) string { + var b strings.Builder + b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered by the module's own process\n") + b.WriteString("# from its assignment's `shares` and `grants` and the nodes that want each share; edit the settings,\n") + b.WriteString("# never this file. One share per folder, to each granted and wanting node's private address only,\n") + b.WriteString("# every client squashed to the folder's owner.\n") + for _, e := range exports { + if e.Line == "" { + fmt.Fprintf(&b, "# share %s: exported to no node\n", e.Name) + continue + } + fmt.Fprintf(&b, "# share %s\n%s\n", e.Name, e.Line) + } + return b.String() +} diff --git a/modules/nfs-server/cmd/nfs-server/table.go b/modules/nfs-server/cmd/nfs-server/table.go new file mode 100644 index 00000000..b2014ebf --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/table.go @@ -0,0 +1,133 @@ +package main + +import ( + "context" + "path" + "strings" +) + +// What the machine exports, read: the kernel's table, the other export files, and ZFS's property. + +// Entry is one export: a path, one client, its options, and where it was found. +type Entry struct { + Path string `json:"path"` + Client string `json:"client"` + Options string `json:"options,omitempty"` + From string `json:"from"` +} + +// ParseExports reads an exports(5) file or the kernel's etab: a path, then clients each with options. +func ParseExports(text, from string) []Entry { + var out []Entry + for _, line := range strings.Split(text, "\n") { + if i := strings.Index(line, "#"); i >= 0 { + line = line[:i] + } + fields := strings.Fields(line) + if len(fields) < 1 { + continue + } + p := fields[0] + if len(fields) == 1 { + out = append(out, Entry{Path: p, Client: "*", From: from}) + continue + } + for _, c := range fields[1:] { + client, opts := c, "" + if i := strings.Index(c, "("); i >= 0 { + client, opts = c[:i], strings.TrimSuffix(c[i+1:], ")") + } + if client == "" { + client = "*" + } + out = append(out, Entry{Path: p, Client: client, Options: opts, From: from}) + } + } + return out +} + +// Live is the kernel's export table, or nil and why when it cannot be read. +func (m *Machine) Live() ([]Entry, string) { + text, err := m.Read(Etab) + if err != nil { + return nil, "the kernel's export table (" + Etab + ") could not be read: " + err.Error() + } + return ParseExports(text, Etab), "" +} + +// IsLive says whether the kernel exports a path to a client. +func IsLive(live []Entry, p, client string) bool { + for _, e := range live { + if path.Clean(e.Path) == path.Clean(p) && e.Client == client { + return true + } + } + return false +} + +// Dataset is one ZFS filesystem as `zfs list` answers it. +type Dataset struct { + Name string `json:"name"` + Mountpoint string `json:"mountpoint"` + ShareNFS string `json:"sharenfs"` +} + +// Datasets are the ZFS filesystems, when zfs is there; nil without it. +func (m *Machine) Datasets() ([]Dataset, string) { + r := m.Run(context.Background(), "zfs", "list", "-H", "-o", "name,mountpoint,sharenfs", "-t", "filesystem") + if r.Code != 0 { + if r.Code == -1 && strings.Contains(r.Err, "executable file not found") { + return nil, "" + } + return nil, "zfs list did not answer: " + firstLine(r.Err) + } + var out []Dataset + for _, line := range strings.Split(r.Out, "\n") { + f := strings.Split(line, "\t") + if len(f) != 3 { + continue + } + out = append(out, Dataset{Name: f[0], Mountpoint: f[1], ShareNFS: f[2]}) + } + return out, "" +} + +// Foreign is every export on the machine that is not the mesh's: /etc/exports, the other files in +// /etc/exports.d (ZFS writes one), and every dataset whose sharenfs is on. +func (m *Machine) Foreign() ([]Entry, []Dataset, []string) { + var entries []Entry + var notes []string + files := append([]string{"/etc/exports"}, m.Glob("/etc/exports.d/*.exports")...) + for _, f := range files { + if f == ExportFile { + continue + } + text, err := m.Read(f) + if err != nil { + if m.Exists(f) { + notes = append(notes, f+" could not be read: "+err.Error()) + } + continue + } + entries = append(entries, ParseExports(text, f)...) + } + sets, note := m.Datasets() + if note != "" { + notes = append(notes, note) + } + var shared []Dataset + for _, d := range sets { + if d.ShareNFS != "off" && d.ShareNFS != "-" && d.ShareNFS != "" { + shared = append(shared, d) + } + } + return entries, shared, notes +} + +func firstLine(s string) string { + s = strings.TrimSpace(s) + if i := strings.IndexByte(s, '\n'); i >= 0 { + return s[:i] + } + return s +} diff --git a/modules/nfs-server/cmd/nfs-server/tools.go b/modules/nfs-server/cmd/nfs-server/tools.go new file mode 100644 index 00000000..114fce40 --- /dev/null +++ b/modules/nfs-server/cmd/nfs-server/tools.go @@ -0,0 +1,474 @@ +package main + +import ( + "context" + "fmt" + "net/netip" + "path" + "sort" + "strings" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +// Seat is the role this module holds on its machine (novox/hq ADR 0263). +const Seat = "node-nfs-server" + +// Stale is how old the process's last look may be before the tools say it stopped looking. +const Stale = 5 * time.Minute + +// verb is one of the seat's verbs, listed as `.` so the runtime serves it on the seat's +// subject, as /node-nfs-server.. +func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool { + return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run} +} + +func str(description string) map[string]any { + return map[string]any{"type": "string", "description": description} +} + +func arg(a map[string]any, k string) string { + v, _ := a[k].(string) + return strings.TrimSpace(v) +} + +func flag(a map[string]any, k string) bool { + switch v := a[k].(type) { + case bool: + return v + case string: + return strings.EqualFold(strings.TrimSpace(v), "true") + } + return false +} + +// confirmed says whether an act was confirmed: the boolean true, or the string "true" the seat's schema +// passes. +func confirmed(a map[string]any) bool { return flag(a, "confirm") } + +// Tools are the seat's verbs and the module's health. +func Tools(m *Machine) []stdio.Tool { + return []stdio.Tool{ + verb("exports", "Every share this machine exports for the mesh: its name, folder, rw or ro, the owner every "+ + "client is mapped to (uid, gid), the nodes it is exported to (each granted by this machine and wanting it, "+ + "at its private address, with its access) and whether the kernel holds it now; what does not match "+ + "between the grants and the wants; the shares the assignment asked for that were refused, and why; and every export "+ + "on the machine that is not the mesh's (/etc/exports, the other files in /etc/exports.d, a ZFS dataset's "+ + "sharenfs). Replaces exportfs -v, cat /etc/exports and zfs get sharenfs. (r)", + map[string]any{}, func(map[string]any) (any, error) { return m.Exports(), nil }), + verb("clients", "Which machines have a share of this machine mounted now: each NFS client the kernel knows, "+ + "with its address and the name it gave, or — when the kernel's list cannot be read — the connections "+ + "to port 2049. Replaces cat /proc/fs/nfsd/clients/*/info and ss -tn sport = :2049. (r)", + map[string]any{}, func(map[string]any) (any, error) { return m.Clients(), nil }), + verb("test", "Whether this machine exports one share for the mesh now: the share is asked for and not "+ + "refused, the kernel holds it for every node it is exported to, and the NFS server is active; with an "+ + "address, also whether that address is inside the range and the share is exported to it. Replaces showmount -e. (r)", + map[string]any{"share": str("the share's name"), "address": str("a client's address on the private network (optional)")}, + func(a map[string]any) (any, error) { return m.Test(arg(a, "share"), arg(a, "address")) }), + verb("reload", "Have the kernel read the export files again now (exportfs -ra), and answer the mesh's shares "+ + "as it then holds them. The module's process does this itself whenever its file changes or a share is "+ + "missing; this is for after a change made outside it. Replaces exportfs -ra. (a)", + map[string]any{}, func(map[string]any) (any, error) { return m.Reload() }), + verb("adopt", "Take over the export a ZFS dataset's sharenfs property makes for a folder the mesh now "+ + "exports: sets sharenfs=off on that one dataset and nothing else, then exportfs -ra. Only when the "+ + "mesh's own export of the same folder is live in the kernel to at least one node, so the folder is never "+ + "unexported for everyone; unless confirm is true, a dry run that says what it would do. A person's act, with why. Replaces zfs set "+ + "sharenfs=off. (a)", + map[string]any{ + "path": str("the shared folder, as the share names it"), + "confirm": str("\"true\" to do it; anything else, a dry run"), + "why": str("why, in a sentence (required with confirm)"), + }, + func(a map[string]any) (any, error) { return m.Adopt(arg(a, "path"), confirmed(a), arg(a, "why")) }), + { + Name: "nfs_health", + Description: "Whether this machine's shares are served, as the module's health check (novox/hq ADR 0240, " + + "ADR 0263): unhealthy when the NFS server is not active, the module's process has not looked for five " + + "minutes, a share the assignment asks for is refused or missing from the kernel's table, or the " + + "export file could not be written or read again; with why, one sentence per finding. (r)", + Run: func(map[string]any) (any, error) { return m.Health(), nil }, + }, + } +} + +// ExportsAnswer is what `exports` answers. +type ExportsAnswer struct { + Shares []Exported `json:"shares"` + Refused []Refused `json:"refused,omitempty"` + Foreign []Entry `json:"foreign"` + Service string `json:"service"` + Range string `json:"range,omitempty"` + // Unmatched is what does not match between this machine's grants and the nodes' wants. + Unmatched []string `json:"unmatched,omitempty"` + Looked string `json:"looked,omitempty"` + Said []string `json:"said"` + Notes []string `json:"notes,omitempty"` +} + +// Exports is the machine's exports now. +func (m *Machine) Exports() ExportsAnswer { + a := ExportsAnswer{Shares: []Exported{}, Foreign: []Entry{}, Service: m.ServiceState()} + st, err := m.LastStatus() + if err != nil { + a.Notes = append(a.Notes, err.Error()) + } else { + a.Range, a.Refused = st.Range, st.Refused + a.Looked = st.At.Format(time.RFC3339) + live, note := m.Live() + if note != "" { + a.Notes = append(a.Notes, note) + } + for _, e := range st.Shares { + if note == "" { + e.Live = SharedLive(live, e) + } + a.Shares = append(a.Shares, e) + } + a.Notes = append(a.Notes, st.Problems...) + a.Unmatched = st.Said + } + entries, sets, notes := m.Foreign() + a.Notes = append(a.Notes, notes...) + a.Foreign = append(a.Foreign, entries...) + for _, d := range sets { + a.Foreign = append(a.Foreign, Entry{Path: d.Mountpoint, Client: d.ShareNFS, From: "zfs sharenfs on " + d.Name}) + } + for _, e := range a.Shares { + if len(e.To) == 0 { + a.Said = append(a.Said, fmt.Sprintf("%s: %s, exported to no node", e.Name, e.Path)) + continue + } + state := "live" + if !e.Live { + state = "NOT in the kernel's table" + } + a.Said = append(a.Said, fmt.Sprintf("%s: %s to %s, every client as %d:%d — %s", e.Name, e.Path, grantees(e.To), e.UID, e.GID, state)) + } + a.Said = append(a.Said, a.Unmatched...) + for _, r := range a.Refused { + a.Said = append(a.Said, "refused: "+r.Why) + } + for _, f := range a.Foreign { + a.Said = append(a.Said, fmt.Sprintf("not the mesh's: %s to %s (%s)", f.Path, f.Client, f.From)) + } + if len(a.Shares) == 0 && len(a.Refused) == 0 { + a.Said = append(a.Said, "the mesh exports nothing here") + } + a.Said = append(a.Said, "the NFS server is "+a.Service) + return a +} + +// Client is one machine that has a share mounted. +type Client struct { + Address string `json:"address"` + Name string `json:"name,omitempty"` + Minor string `json:"minor_version,omitempty"` +} + +// ClientsAnswer is what `clients` answers. +type ClientsAnswer struct { + Clients []Client `json:"clients"` + From string `json:"from"` + Notes []string `json:"notes,omitempty"` +} + +// ParseClientInfo reads one /proc/fs/nfsd/clients//info. +func ParseClientInfo(text string) Client { + var c Client + for _, line := range strings.Split(text, "\n") { + k, v, ok := strings.Cut(line, ":") + if !ok { + continue + } + v = strings.Trim(strings.TrimSpace(v), `"`) + switch strings.TrimSpace(k) { + case "address": + c.Address = v + case "name": + c.Name = v + case "minor version": + c.Minor = v + } + } + return c +} + +// Clients are the NFS clients the kernel knows, or the connections to 2049. +func (m *Machine) Clients() ClientsAnswer { + a := ClientsAnswer{Clients: []Client{}} + const dir = "/proc/fs/nfsd/clients" + infos := m.Glob(dir + "/*/info") + if len(infos) == 0 && m.UID != 0 { + if r, err := m.AsRoot("find", dir, "-maxdepth", "2", "-name", "info"); err == nil { + for _, l := range strings.Fields(r.Out) { + infos = append(infos, l) + } + } else { + a.Notes = append(a.Notes, err.Error()) + } + } + for _, f := range infos { + text, err := m.Read(f) + if err != nil && m.UID != 0 { + r, e := m.AsRoot("cat", f) + text, err = r.Out, e + } + if err != nil { + a.Notes = append(a.Notes, f+": "+err.Error()) + continue + } + a.Clients = append(a.Clients, ParseClientInfo(text)) + } + if len(infos) > 0 { + a.From = dir + return a + } + r := m.Run(context.Background(), "ss", "-tnH", "state", "established", "( sport = :2049 )") + a.From = "connections to port 2049" + for _, line := range strings.Split(r.Out, "\n") { + f := strings.Fields(line) + if len(f) >= 4 { + a.Clients = append(a.Clients, Client{Address: f[len(f)-1]}) + } + } + return a +} + +// TestAnswer is what `test` answers. +type TestAnswer struct { + Share string `json:"share"` + Path string `json:"path,omitempty"` + Access string `json:"access,omitempty"` + Live bool `json:"live"` + Service string `json:"service"` + InRange *bool `json:"in_range,omitempty"` + OK bool `json:"ok"` + Why string `json:"why"` +} + +// Test says whether one share is served now. +func (m *Machine) Test(share, address string) (TestAnswer, error) { + if share == "" { + return TestAnswer{}, fmt.Errorf("name the share") + } + a := TestAnswer{Share: share, Service: m.ServiceState()} + st, err := m.LastStatus() + if err != nil { + a.Why = err.Error() + return a, nil + } + for _, r := range st.Refused { + if r.Name == share { + a.Path, a.Why = r.Path, r.Why + return a, nil + } + } + var found *Exported + for i := range st.Shares { + if st.Shares[i].Name == share { + found = &st.Shares[i] + } + } + if found == nil { + a.Why = fmt.Sprintf("This machine's assignment asks for no share named %s.", share) + return a, nil + } + a.Path, a.Access = found.Path, found.Access + live, note := m.Live() + a.Live = note == "" && SharedLive(live, *found) + var why []string + if address != "" { + addr, err1 := netip.ParseAddr(address) + prefix, err2 := netip.ParsePrefix(st.Range) + in := err1 == nil && err2 == nil && prefix.Contains(addr) + a.InRange = &in + if !in { + why = append(why, fmt.Sprintf("%s is not inside the private network's range %s.", address, st.Range)) + } else { + to := false + for _, g := range found.To { + if g.Address == address { + to = true + a.Access = g.Access + } + } + if !to { + why = append(why, fmt.Sprintf("The share %s is not exported to %s: no node there is both granted it and wants it.", share, address)) + } else if note != "" || !IsLive(live, found.Path, address) { + why = append(why, fmt.Sprintf("The kernel does not export %s to %s now.", found.Path, address)) + } + } + } + if len(found.To) == 0 { + why = append(why, fmt.Sprintf("The share %s is exported to no node.", share)) + } else if !a.Live { + why = append(why, fmt.Sprintf("The kernel does not export %s to every node it is meant for (%s) now.", found.Path, grantees(found.To))) + } + if a.Service != "active" { + why = append(why, "The NFS server is "+a.Service+".") + } + a.OK = len(why) == 0 + if a.OK { + a.Why = fmt.Sprintf("The share %s (%s) is exported to %s and the NFS server is active.", share, found.Path, grantees(found.To)) + } else { + a.Why = strings.Join(why, " ") + } + return a, nil +} + +// Reload has the kernel read the export files again. +func (m *Machine) Reload() (any, error) { + if _, err := m.AsRoot("exportfs", "-ra"); err != nil { + return nil, err + } + return m.Exports(), nil +} + +// AdoptAnswer is what `adopt` answers. +type AdoptAnswer struct { + Path string `json:"path"` + Dataset string `json:"dataset,omitempty"` + Was string `json:"sharenfs_was,omitempty"` + Done bool `json:"done"` + DryRun bool `json:"dry_run"` + Said string `json:"said"` +} + +// Adopt clears one dataset's sharenfs once the mesh's own export of its folder is live. +func (m *Machine) Adopt(p string, confirm bool, why string) (AdoptAnswer, error) { + if p == "" { + return AdoptAnswer{}, fmt.Errorf("name the shared folder (path)") + } + p = path.Clean(p) + a := AdoptAnswer{Path: p, DryRun: !confirm} + sets, note := m.Datasets() + if note != "" { + return a, fmt.Errorf("%s", note) + } + var ds *Dataset + for i := range sets { + if path.Clean(sets[i].Mountpoint) == p { + ds = &sets[i] + } + } + if ds == nil { + return a, fmt.Errorf("no ZFS dataset is mounted at %s, so there is no sharenfs to take over", p) + } + a.Dataset, a.Was = ds.Name, ds.ShareNFS + if ds.ShareNFS == "off" || ds.ShareNFS == "-" || ds.ShareNFS == "" { + return a, fmt.Errorf("the dataset %s already has sharenfs off; nothing to take over", ds.Name) + } + st, err := m.LastStatus() + if err != nil { + return a, err + } + var mine *Exported + for i := range st.Shares { + if st.Shares[i].Path == p { + mine = &st.Shares[i] + } + } + if mine == nil { + return a, fmt.Errorf("the mesh does not export %s: add it to this machine's nfs-server `shares` setting first, so the folder is never unexported", p) + } + if len(mine.To) == 0 { + return a, fmt.Errorf("the mesh exports %s to no node yet (no node is both granted it and wants it), so taking over would unexport it for everyone", p) + } + live, lnote := m.Live() + if lnote != "" || !SharedLive(live, *mine) { + return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, grantees(mine.To)) + } + plan := fmt.Sprintf("set sharenfs=off on the dataset %s (now %q) and run exportfs -ra; %s stays exported as the share %s to %s, every client as %d:%d", + ds.Name, ds.ShareNFS, p, mine.Name, grantees(mine.To), mine.UID, mine.GID) + if !confirm { + a.Said = "Dry run: would " + plan + "." + return a, nil + } + if why == "" { + return a, fmt.Errorf("say why: adopt is a person's act, and the why is kept with it") + } + if _, err := m.AsRoot("zfs", "set", "sharenfs=off", ds.Name); err != nil { + return a, err + } + if _, err := m.AsRoot("exportfs", "-ra"); err != nil { + return a, fmt.Errorf("sharenfs is off on %s, and exportfs -ra failed: %v", ds.Name, err) + } + a.Done = true + a.Said = fmt.Sprintf("Did: %s. Why: %s", plan, why) + return a, nil +} + +// Health is the module's health, as the node-engine asks it. +type Health struct { + Healthy bool `json:"healthy"` + Why string `json:"why"` + Found []string `json:"found,omitempty"` +} + +// Health judges the shares. +func (m *Machine) Health() Health { + var found []string + service := m.ServiceState() + if service != "active" { + found = append(found, fmt.Sprintf("The NFS server (%s) is %s.", Unit, service)) + } + st, err := m.LastStatus() + if err != nil { + found = append(found, err.Error()+".") + return Health{Healthy: false, Why: strings.Join(found, " "), Found: found} + } + if age := m.Now().Sub(st.At); age > Stale { + found = append(found, fmt.Sprintf("The module's process last looked %s ago, at %s.", age.Round(time.Second), st.At.Local().Format("15:04"))) + } + for _, p := range st.Problems { + found = append(found, sentence(p)) + } + for _, r := range st.Refused { + found = append(found, r.Why) + } + if st.ReloadError != "" { + found = append(found, sentence("the kernel could not read the export files again: "+st.ReloadError)) + } + live, note := m.Live() + for _, e := range st.Shares { + if note == "" && !SharedLive(live, e) { + found = append(found, fmt.Sprintf("The share %s (%s) is not in the kernel's export table.", e.Name, e.Path)) + } + } + sort.Strings(found) + if len(found) > 0 { + return Health{Healthy: false, Why: strings.Join(found, " "), Found: found} + } + var parts []string + for _, e := range st.Shares { + if len(e.To) > 0 { + parts = append(parts, fmt.Sprintf("%s to %s", e.Name, grantees(e.To))) + } + } + if len(parts) == 0 { + return Health{Healthy: true, Why: "The NFS server is active, and the mesh exports nothing to any node here."} + } + return Health{Healthy: true, Why: fmt.Sprintf("The NFS server is active and exports %s.", strings.Join(parts, "; "))} +} + +func sentence(s string) string { + s = strings.TrimSpace(s) + if s == "" { + return s + } + s = strings.ToUpper(s[:1]) + s[1:] + if !strings.HasSuffix(s, ".") { + s += "." + } + return s +} + +// grantees says who a share goes to: `ws (10.0.0.3, rw), laptop (10.0.0.4, ro)`. +func grantees(to []Grantee) string { + var out []string + for _, g := range to { + out = append(out, fmt.Sprintf("%s (%s, %s)", g.Node, g.Address, g.Access)) + } + return strings.Join(out, ", ") +} diff --git a/modules/nfs-server/go.mod b/modules/nfs-server/go.mod new file mode 100644 index 00000000..6e122025 --- /dev/null +++ b/modules/nfs-server/go.mod @@ -0,0 +1,5 @@ +module nfsserver + +go 1.22 + +require git.novox.be/novox/mesh-sdk/go v0.1.7 diff --git a/modules/nfs-server/go.sum b/modules/nfs-server/go.sum new file mode 100644 index 00000000..b474419e --- /dev/null +++ b/modules/nfs-server/go.sum @@ -0,0 +1,2 @@ +git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w= +git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY= diff --git a/modules/nfs-server/module.json b/modules/nfs-server/module.json new file mode 100644 index 00000000..f36e133c --- /dev/null +++ b/modules/nfs-server/module.json @@ -0,0 +1,171 @@ +{ + "module": "nfs-server", + "version": "1", + "upgrade": { + "policy": "record", + "why": "the folders other machines mount: a build that breaks the exports leaves every client's mount hanging or refused, and the gate on this one machine does not see the clients (hq ADR 0236, ADR 0263)" + }, + "capabilities": [ + "package-manager", + "service-manager" + ], + "provides": [ + { + "name": "nfs-share", + "scope": "mesh", + "identity": false + } + ], + "data": { + "consumers": { + "nfs-share": { + "class": "none", + "why": "the shared folders are the operator's data (hq ADR 0051): what a client writes lands in them, and they are protected where the operator declares them, never by this module, which keeps nothing of a consumer's" + } + } + }, + "claims": [ + { + "name": "node-nfs-server", + "scope": "node", + "serves": [ + "exports", + "clients", + "test", + "reload", + "adopt" + ] + } + ], + "state": [ + { + "name": "exports", + "ttl-seconds": 120, + "per-machine": true + } + ], + "reads": [ + "mounts.shares" + ], + "settings": { + "grants": { + "kind": "preference", + "default": "none", + "why": "which nodes may mount which share, and how, is this machine's to say (hq ADR 0263, review of mesh-catalog #136): share=node:rw|ro[,node:rw|ro], entries separated by spaces; none exports to no node, whatever a node wants" + } + }, + "facts": { + "machines": { + "path": "/etc/nfs-server/machines", + "template": "# Written by the mesh (module nfs-server, novox/hq ADR 0263): every machine of the mesh and its private\n# address, from which the module takes a node's address. Replaced on every push.\n{{range .Machines}}{{.Name}} {{.Address}}\n{{end}}" + } + }, + "tools": [ + "nfs_health" + ], + "listens": [ + { + "name": "nfs", + "port": 2049, + "protocol": "tcp", + "from": "mesh", + "fixed": true, + "why": "the shares, to the mesh's machines only (hq ADR 0263): NFS version 4 alone, which needs no other port, and never the home network, where a device that is not a node could claim any user id" + } + ], + "resources": [ + { + "id": "package", + "type": "package", + "package": "nfs-utils" + }, + { + "id": "nfs-conf", + "type": "file", + "path": "/etc/nfs.conf.d/50-mesh.conf", + "mode": "0644", + "content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263). Replaced on every push; a drop-in of\n# the operator's that sorts after this one overrides it, and is theirs.\n#\n# NFS version 4 only: a client needs port 2049 and nothing else, so the module opens nothing more\n# than that, to the private network. Version 3 needs rpcbind and mountd, on ports the mesh does not open.\n[nfsd]\nvers2=n\nvers3=n\nvers4=y\nvers4.0=n\nvers4.1=y\nvers4.2=y\n" + }, + { + "id": "run-dir", + "type": "directory", + "path": "/run/nfs-server", + "mode": "0755" + }, + { + "id": "server", + "type": "service", + "unit": "nfs-server.service", + "state": "running", + "boot": "enabled", + "restart-on": [ + "nfs-conf" + ], + "health": { + "kind": "unit" + } + }, + { + "id": "wants-dir-parent", + "type": "directory", + "path": "/var/lib/nfs-server", + "mode": "0755" + }, + { + "id": "wants-dir", + "type": "directory", + "path": "/var/lib/nfs-server/from-bus", + "mode": "0755", + "owner": "${machine:account}" + }, + { + "id": "exports", + "type": "process", + "name": "nfs-server-exports", + "artifact": "tools", + "run": [ + "./nfs-server", + "exports" + ], + "restart-on": [ + "config" + ], + "health": { + "kind": "tool", + "tool": "nfs_health", + "interval": "60s", + "timeout": "10s", + "looks": 2, + "grace": "90s" + } + }, + { + "id": "config-dir", + "type": "directory", + "path": "/etc/nfs-server", + "mode": "0755" + }, + { + "id": "config", + "type": "file", + "path": "/etc/nfs-server/shares.conf", + "mode": "0644", + "content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The grants: share=node:rw|ro[,…],\n# which nodes may mount each and how. The module's process exports a share to a node's private address\n# only when it is granted here and that node's mounts module wants it, every client mapped to the\n# folder's owner, and only to addresses inside the range below.\nshares=${setting:shares}\ngrants=${setting:grants}\nrange=${machine:mesh-range}\n" + } + ], + "build": { + "artifacts": [ + { + "name": "tools", + "kind": "bundle", + "language": "go", + "system": "arch", + "from": "cmd/nfs-server", + "binary": "nfs-server", + "loads": [ + "nfs-server" + ] + } + ] + } +}