Add the nfs-server module: a node shares a folder with the mesh's machines only, every client mapped to the folder's owner (hq ADR 0263)
mesh/merge-gate fail: builds new: modules/mounts, modules/nfs-server, sent nowhere; no bus step; a manifest the change touches fails the module check: modu…
mesh/repo-check fail: its merge-check.sh failed: long-running resources without health: 70
mesh/delivery-group group feat/mounts-module rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate fail: builds new: modules/mounts, modules/nfs-server, sent nowhere; no bus step; a manifest the change touches fails the module check: modu…
mesh/repo-check fail: its merge-check.sh failed: long-running resources without health: 70
mesh/delivery-group group feat/mounts-module rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
# nfs-server
|
||||
|
||||
A node shares a folder with the mesh's machines, as a module (novox/hq ADR 0263, to-be 52). It holds
|
||||
the node seat `node-nfs-server` and provides the provision `nfs-share`, which the `mounts` module on
|
||||
another node requires.
|
||||
|
||||
## The rules it keeps
|
||||
|
||||
- **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or
|
||||
`name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is
|
||||
the operator's value (ADR 0112, ADR 0262).
|
||||
- **To the mesh's machines only.** Every export line names the private network's range
|
||||
(`${machine:mesh-range}`) and nothing else, and the module opens port 2049 to the mesh only. The home
|
||||
network is never named.
|
||||
- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one
|
||||
id names different accounts on different machines. Every line is `all_squash` with `anonuid` and
|
||||
`anongid` read from the folder's owner on this machine, so a file written from any node lands owned by
|
||||
the folder's owner, and root is squashed with everyone else.
|
||||
- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, or holds a
|
||||
backup (restic) repository at its path or one level below is refused, said by the tools, and makes the
|
||||
module unhealthy.
|
||||
- **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It
|
||||
never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its
|
||||
own and nothing else, and a test holds it to that.
|
||||
- **NFS version 4 only**, so 2049 is the only port a client needs: rpcbind and mountd stay closed.
|
||||
|
||||
## What it owns
|
||||
|
||||
- the `nfs-utils` package and `nfs-server.service`, running and enabled;
|
||||
- `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on;
|
||||
- `/etc/nfs-server/shares.conf`: the setting and the range, rendered by the mesh;
|
||||
- `/etc/exports.d/mesh.exports`: written by its process, with a marker header;
|
||||
- `/run/nfs-server/status.json`: the process's last look, for the tools;
|
||||
- the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module
|
||||
and for a view (ADR 0201).
|
||||
|
||||
## How it runs
|
||||
|
||||
- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, judges each
|
||||
folder, writes the export file when it changed, and runs `exportfs -ra` when the file changed or a share
|
||||
is missing from the kernel's table. Then it writes its status.
|
||||
- **Its tools** run as the operator account. They read the status and the kernel's table, and run what
|
||||
only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools.
|
||||
- **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS
|
||||
server is not active, the process has not looked for five minutes, a share is refused or missing from
|
||||
the kernel's table, or the kernel could not read the files again.
|
||||
|
||||
## Tools
|
||||
|
||||
| tool | | answers |
|
||||
|---|---|---|
|
||||
| `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) |
|
||||
| `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 |
|
||||
| `node-nfs-server.test` | r | whether one share is exported to the range now, the server is active, and an address is inside the range |
|
||||
| `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them |
|
||||
| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live; a dry run without `confirm`; a person's act, with why |
|
||||
| `nfs_health` | r | the module's health, one sentence per finding |
|
||||
|
||||
`exports` answers, for the `mounts` module:
|
||||
|
||||
```
|
||||
{"shares": [{"name", "path", "access", "uid", "gid", "clients", "line", "live"}],
|
||||
"refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}],
|
||||
"service": "active", "range", "looked", "said", "notes"}
|
||||
```
|
||||
|
||||
## What it leaves
|
||||
|
||||
- **The export file after unassignment.** The process writes `/etc/exports.d/mesh.exports`, not the
|
||||
node-engine, because each line carries the folder's owner as this machine reads it. When the module is
|
||||
unassigned, the process stops and the file stays until removed. `exports` on any later holder, and the
|
||||
`mounts` module's `mounts_exports`, show it.
|
||||
- **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not
|
||||
a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could
|
||||
reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so
|
||||
a folder that owner cannot read stays closed. A share that must be strictly fenced is better given its
|
||||
own dataset.
|
||||
- **Exports that are not the mesh's** (`/etc/exports`, a ZFS dataset's `sharenfs`) are left as found and
|
||||
shown, until `adopt` takes one over on a person's word.
|
||||
@@ -0,0 +1,210 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The machine as the module sees it (novox/hq ADR 0263).
|
||||
//
|
||||
// **Two processes, two accounts.** The node's tool runtime runs the tools as the operator account, not
|
||||
// root (novox/hq ADR 0175 §4): they read, and what only root may do — `exportfs -ra`, `zfs set` — goes
|
||||
// through `sudo -n`, as the packet filter's and the service manager's tools do. The module's long-running
|
||||
// process runs as root: it renders the export file and writes the status the tools read. It cannot reach
|
||||
// the bus, so the tools' bundle puts that status in the module's state.
|
||||
//
|
||||
// **It writes two files and nothing else.** The export file and the status file are the module's own.
|
||||
// Nothing here can create, chown, chmod or remove anything under a shared path: the machine offers no
|
||||
// such call, and Write refuses every path but the module's two (novox/hq ADR 0051).
|
||||
|
||||
// The module's files on the machine.
|
||||
const (
|
||||
// ConfigFile is what the mesh renders from the assignment: the shares and the private network's range.
|
||||
ConfigFile = "/etc/nfs-server/shares.conf"
|
||||
// ExportFile is the module's own export file, which the distribution's exportfs reads with the rest of
|
||||
// /etc/exports.d.
|
||||
ExportFile = "/etc/exports.d/mesh.exports"
|
||||
// StatusFile is the process's last look, for the tools.
|
||||
StatusFile = "/run/nfs-server/status.json"
|
||||
// Etab is the kernel's export table as nfs-utils keeps it.
|
||||
Etab = "/var/lib/nfs/etab"
|
||||
// Unit is the NFS server's unit.
|
||||
Unit = "nfs-server.service"
|
||||
)
|
||||
|
||||
// CommandTimeout bounds every command. None touches a shared path's contents.
|
||||
const CommandTimeout = 20 * time.Second
|
||||
|
||||
// Ran is what one command did.
|
||||
type Ran struct {
|
||||
Out string
|
||||
Err string
|
||||
Code int
|
||||
}
|
||||
|
||||
// Runner runs one command; injected, so every tool is tested over recorded answers.
|
||||
type Runner func(ctx context.Context, name string, args ...string) Ran
|
||||
|
||||
// ExecRunner runs a command on the machine, bounded by CommandTimeout.
|
||||
func ExecRunner(ctx context.Context, name string, args ...string) Ran {
|
||||
ctx, cancel := context.WithTimeout(ctx, CommandTimeout)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, name, args...)
|
||||
var stdout, stderr bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = &stdout, &stderr
|
||||
err := cmd.Run()
|
||||
r := Ran{Out: stdout.String(), Err: strings.TrimSpace(stderr.String())}
|
||||
if err != nil {
|
||||
var exit *exec.ExitError
|
||||
if errors.As(err, &exit) {
|
||||
r.Code = exit.ExitCode()
|
||||
} else {
|
||||
r.Code = -1
|
||||
if r.Err == "" {
|
||||
r.Err = err.Error()
|
||||
}
|
||||
}
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// Owner is who owns a path, and whether it is a directory.
|
||||
type Owner struct {
|
||||
UID, GID int
|
||||
Dir bool
|
||||
}
|
||||
|
||||
// Machine is a filesystem root (the real one, or a test's tree), a runner, who this process is, and a
|
||||
// clock.
|
||||
type Machine struct {
|
||||
Root string
|
||||
Run Runner
|
||||
UID int
|
||||
Now func() time.Time
|
||||
}
|
||||
|
||||
// Here is the machine this process runs on.
|
||||
func Here() *Machine {
|
||||
return &Machine{Root: "/", Run: ExecRunner, UID: os.Getuid(), Now: time.Now}
|
||||
}
|
||||
|
||||
func (m *Machine) at(path string) string { return filepath.Join(m.Root, path) }
|
||||
|
||||
// Read is one file's text.
|
||||
func (m *Machine) Read(path string) (string, error) {
|
||||
b, err := os.ReadFile(m.at(path))
|
||||
return string(b), err
|
||||
}
|
||||
|
||||
// Exists says whether a path is there.
|
||||
func (m *Machine) Exists(path string) bool {
|
||||
_, err := os.Lstat(m.at(path))
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// Stat is a path's owner. It reads the inode and nothing else.
|
||||
func (m *Machine) Stat(path string) (Owner, error) {
|
||||
info, err := os.Stat(m.at(path))
|
||||
if err != nil {
|
||||
return Owner{}, err
|
||||
}
|
||||
o := Owner{Dir: info.IsDir()}
|
||||
if st, ok := info.Sys().(*syscall.Stat_t); ok {
|
||||
o.UID, o.GID = int(st.Uid), int(st.Gid)
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Children are the names of a directory's subdirectories. Used only to look for a backup repository one
|
||||
// level below a share; it lists names and opens nothing.
|
||||
func (m *Machine) Children(path string) []string {
|
||||
entries, err := os.ReadDir(m.at(path))
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
for _, e := range entries {
|
||||
if e.IsDir() {
|
||||
out = append(out, e.Name())
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Glob is the paths under the root matching a pattern, as the machine names them.
|
||||
func (m *Machine) Glob(pattern string) []string {
|
||||
found, _ := filepath.Glob(m.at(pattern))
|
||||
out := make([]string, 0, len(found))
|
||||
for _, f := range found {
|
||||
rel, err := filepath.Rel(m.Root, f)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, "/"+rel)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// written are the only paths Write may touch.
|
||||
var written = map[string]bool{ExportFile: true, StatusFile: true}
|
||||
|
||||
// Write replaces one of the module's own files, atomically. Every other path is refused: the module owns
|
||||
// these two files and nothing else on the machine.
|
||||
func (m *Machine) Write(path, content string) error {
|
||||
if !written[path] {
|
||||
return fmt.Errorf("the module writes only %s and %s, not %s", ExportFile, StatusFile, path)
|
||||
}
|
||||
full := m.at(path)
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil && !errors.Is(err, fs.ErrExist) {
|
||||
return err
|
||||
}
|
||||
tmp := full + ".tmp"
|
||||
if err := os.WriteFile(tmp, []byte(content), 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, full)
|
||||
}
|
||||
|
||||
// Escalated is a command as it is run: as given when this process is root, else through sudo without a
|
||||
// prompt.
|
||||
func (m *Machine) Escalated(name string, args ...string) (string, []string) {
|
||||
if m.UID == 0 {
|
||||
return name, args
|
||||
}
|
||||
return "sudo", append([]string{"-n", name}, args...)
|
||||
}
|
||||
|
||||
// AsRoot runs what only root may do, and names a refusal by how it failed.
|
||||
func (m *Machine) AsRoot(name string, args ...string) (Ran, error) {
|
||||
program, argv := m.Escalated(name, args...)
|
||||
r := m.Run(context.Background(), program, argv...)
|
||||
if r.Code != 0 {
|
||||
why := r.Err
|
||||
if why == "" {
|
||||
why = strings.TrimSpace(r.Out)
|
||||
}
|
||||
if program == "sudo" && (strings.Contains(why, "a password is required") || strings.Contains(why, "not allowed")) {
|
||||
return r, fmt.Errorf("%s needs root, and sudo refused the operator account without a prompt: %s", name, why)
|
||||
}
|
||||
return r, fmt.Errorf("%s %s: exit %d: %s", name, strings.Join(args, " "), r.Code, why)
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
// ServiceState is what the service manager says of the NFS server: active, inactive, failed, …
|
||||
func (m *Machine) ServiceState() string {
|
||||
r := m.Run(context.Background(), "systemctl", "is-active", Unit)
|
||||
if s := strings.TrimSpace(r.Out); s != "" {
|
||||
return s
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
// The nfs-server module's Go bundle (novox/hq ADR 0263): a node shares a folder with the mesh's machines
|
||||
// only, one share per folder, every client mapped to the folder's owner.
|
||||
//
|
||||
// Started with no arguments it is served by the node's runtime as the operator account, over MCP on stdio
|
||||
// (ADR 0188, ADR 0193): the node-nfs-server seat's verbs and the module's health, and beside them it puts
|
||||
// the process's last look in the module's state `exports`, keyed by this machine (ADR 0201). Started as
|
||||
// `nfs-server exports` it is the module's long-running process, as root: it renders the export file from
|
||||
// the assignment's shares and has the kernel read it (process.go). stdout is the MCP channel; everything
|
||||
// else is said on stderr.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// ExportsState is the state the exports are kept in on the bus.
|
||||
const ExportsState = "exports"
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 {
|
||||
if os.Args[1] != "exports" || len(os.Args) != 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: nfs-server [exports]")
|
||||
os.Exit(2)
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt)
|
||||
defer stop()
|
||||
if err := Here().RunExports(ctx); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
m := Here()
|
||||
go ShareExports(m)
|
||||
if err := stdio.Serve("", Tools(m)); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// ShareExports puts the exports in the module's state every Every, for as long as the bundle runs, so a
|
||||
// consumer's module and a view read them from the bus rather than from this machine.
|
||||
func ShareExports(m *Machine) {
|
||||
machine := os.Getenv("MESH_NODE")
|
||||
if machine == "" {
|
||||
fmt.Fprintln(os.Stderr, "nfs-server: the runtime gave no machine name (MESH_NODE); the exports are not shared")
|
||||
return
|
||||
}
|
||||
last := ""
|
||||
put := func() {
|
||||
_, err := stdio.State(ExportsState).Put(machine, m.Exports())
|
||||
said := "put"
|
||||
if err != nil {
|
||||
said = err.Error()
|
||||
}
|
||||
if said != last {
|
||||
fmt.Fprintln(os.Stderr, "nfs-server: exports state:", said)
|
||||
}
|
||||
last = said
|
||||
}
|
||||
// The SDK reaches the bus once Serve runs.
|
||||
time.Sleep(2 * time.Second)
|
||||
put()
|
||||
for range time.Tick(Every) {
|
||||
put()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type manifest struct {
|
||||
Tools []string `json:"tools"`
|
||||
Claims []struct {
|
||||
Name string `json:"name"`
|
||||
Scope string `json:"scope"`
|
||||
Serves []string `json:"serves"`
|
||||
} `json:"claims"`
|
||||
Provides []map[string]any `json:"provides"`
|
||||
Listens []struct {
|
||||
Port int `json:"port"`
|
||||
Protocol string `json:"protocol"`
|
||||
From string `json:"from"`
|
||||
} `json:"listens"`
|
||||
Settings map[string]any `json:"settings"`
|
||||
Resources []map[string]any `json:"resources"`
|
||||
}
|
||||
|
||||
func readManifest(t *testing.T) (manifest, string) {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m manifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return m, string(raw)
|
||||
}
|
||||
|
||||
// The seat's verbs are served as `node-nfs-server.<verb>`, the health tool is listed, and every tool says
|
||||
// whether it reads or acts.
|
||||
func TestTheSeatsVerbsAreAllServedAndTheHealthToolListed(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || m.Claims[0].Scope != "node" {
|
||||
t.Fatalf("claims: %+v", m.Claims)
|
||||
}
|
||||
var verbs, tools []string
|
||||
for _, tl := range Tools(newFake(t).machine()) {
|
||||
if name, ok := strings.CutPrefix(tl.Name, Seat+"."); ok {
|
||||
verbs = append(verbs, name)
|
||||
} else {
|
||||
tools = append(tools, tl.Name)
|
||||
}
|
||||
if !strings.HasSuffix(tl.Description, "(r)") && !strings.HasSuffix(tl.Description, "(a)") {
|
||||
t.Errorf("%s does not say whether it reads or acts", tl.Name)
|
||||
}
|
||||
}
|
||||
sort.Strings(verbs)
|
||||
serves := append([]string(nil), m.Claims[0].Serves...)
|
||||
sort.Strings(serves)
|
||||
if strings.Join(verbs, ",") != strings.Join(serves, ",") {
|
||||
t.Errorf("served %v, the claim lists %v", verbs, serves)
|
||||
}
|
||||
if strings.Join(tools, ",") != strings.Join(m.Tools, ",") {
|
||||
t.Errorf("tools %v, listed %v", tools, m.Tools)
|
||||
}
|
||||
}
|
||||
|
||||
// Port 2049 to the mesh's machines and nothing else; the provision is the mesh's, keeping no name of its
|
||||
// consumers.
|
||||
func TestItListensOnTwoThousandFortyNineFromTheMeshOnlyAndProvidesTheShare(t *testing.T) {
|
||||
m, raw := readManifest(t)
|
||||
if len(m.Listens) != 1 || m.Listens[0].Port != 2049 || m.Listens[0].Protocol != "tcp" || m.Listens[0].From != "mesh" {
|
||||
t.Errorf("listens: %+v", m.Listens)
|
||||
}
|
||||
if len(m.Provides) != 1 || m.Provides[0]["name"] != "nfs-share" || m.Provides[0]["scope"] != "mesh" || m.Provides[0]["identity"] != false {
|
||||
t.Errorf("provides: %+v", m.Provides)
|
||||
}
|
||||
for _, port := range []string{"111", "20048", "\"port\": 445"} {
|
||||
if strings.Contains(raw, `"port": `+port) {
|
||||
t.Errorf("the module opens %s", port)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(raw, "vers3=n") {
|
||||
t.Errorf("NFS version 3 is not switched off, so 2049 alone would not do")
|
||||
}
|
||||
}
|
||||
|
||||
// The definition names no folder, no machine and no person (novox/hq ADR 0112): the shares are the
|
||||
// assignment's setting, and the range the mesh's.
|
||||
func TestTheDefinitionNamesNoFolderNoMachineAndGivesTheSharesNoDefault(t *testing.T) {
|
||||
m, raw := readManifest(t)
|
||||
if hit := regexp.MustCompile(`\b(g14|novox|ace|shanks|jochen)\b|/home/|/storage|/srv/`).FindString(strings.ReplaceAll(raw, "novox/hq", "")); hit != "" {
|
||||
t.Errorf("the manifest says %q", hit)
|
||||
}
|
||||
if _, declared := m.Settings["shares"]; declared {
|
||||
t.Errorf("the shares are the operator's value and have no default (ADR 0262)")
|
||||
}
|
||||
var config string
|
||||
for _, r := range m.Resources {
|
||||
if r["id"] == "config" {
|
||||
config, _ = r["content"].(string)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(config, "shares=${setting:shares}") || !strings.Contains(config, "range=${machine:mesh-range}") {
|
||||
t.Errorf("the config file: %q", config)
|
||||
}
|
||||
// Every long-running resource says how it is ready (ADR 0240 rule 8).
|
||||
for _, r := range m.Resources {
|
||||
if r["type"] == "service" || r["type"] == "process" {
|
||||
if r["health"] == nil {
|
||||
t.Errorf("%v has no health", r["id"])
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,389 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fake is a machine under a temporary root, with recorded command answers.
|
||||
type fake struct {
|
||||
t *testing.T
|
||||
root string
|
||||
answers map[string]Ran
|
||||
ran []string
|
||||
now time.Time
|
||||
uid int
|
||||
// onExportfs, when set, writes the kernel's table as exportfs -ra would.
|
||||
onExportfs func()
|
||||
}
|
||||
|
||||
func newFake(t *testing.T) *fake {
|
||||
f := &fake{t: t, root: t.TempDir(), answers: map[string]Ran{}, now: time.Date(2026, 10, 8, 19, 21, 0, 0, time.UTC), uid: 0}
|
||||
f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "active\n"}
|
||||
f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Code: -1, Err: `exec: "zfs": executable file not found in $PATH`}
|
||||
return f
|
||||
}
|
||||
|
||||
func (f *fake) machine() *Machine {
|
||||
return &Machine{Root: f.root, UID: f.uid, Now: func() time.Time { return f.now }, Run: func(_ context.Context, name string, args ...string) Ran {
|
||||
line := strings.Join(append([]string{name}, args...), " ")
|
||||
f.ran = append(f.ran, line)
|
||||
if (line == "exportfs -ra" || line == "sudo -n exportfs -ra") && f.onExportfs != nil {
|
||||
f.onExportfs()
|
||||
}
|
||||
if r, ok := f.answers[line]; ok {
|
||||
return r
|
||||
}
|
||||
return Ran{}
|
||||
}}
|
||||
}
|
||||
|
||||
func (f *fake) dir(p string) {
|
||||
if err := os.MkdirAll(filepath.Join(f.root, p), 0o755); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *fake) file(p, content string) {
|
||||
f.dir(filepath.Dir(p))
|
||||
if err := os.WriteFile(filepath.Join(f.root, p), []byte(content), 0o644); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *fake) read(p string) string {
|
||||
b, _ := os.ReadFile(filepath.Join(f.root, p))
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func (f *fake) restic(p string) {
|
||||
f.file(p+"/config", "x")
|
||||
for _, d := range []string{"keys", "snapshots", "data"} {
|
||||
f.dir(p + "/" + d)
|
||||
}
|
||||
}
|
||||
|
||||
// kernelReads makes exportfs -ra copy the mesh's export file into the kernel's table, as it would.
|
||||
func (f *fake) kernelReads() {
|
||||
f.onExportfs = func() {
|
||||
var b strings.Builder
|
||||
for _, e := range ParseExports(f.read(ExportFile), ExportFile) {
|
||||
fmt.Fprintf(&b, "%s\t%s(%s)\n", e.Path, e.Client, e.Options)
|
||||
}
|
||||
f.file(Etab, b.String())
|
||||
}
|
||||
}
|
||||
|
||||
const meshRange = "10.0.0.0/24"
|
||||
|
||||
func config(shares string) string {
|
||||
return "# test\nshares=" + shares + "\nrange=" + meshRange + "\n"
|
||||
}
|
||||
|
||||
func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/library")
|
||||
f.dir("/srv/downloads")
|
||||
f.file(ConfigFile, config("library=/srv/library downloads=/srv/downloads:ro"))
|
||||
f.kernelReads()
|
||||
s := f.machine().Pass(context.Background(), nil)
|
||||
if len(s.Shares) != 2 || len(s.Refused) != 0 {
|
||||
t.Fatalf("status: %+v", s)
|
||||
}
|
||||
uid, gid := os.Getuid(), os.Getgid()
|
||||
text := f.read(ExportFile)
|
||||
for _, want := range []string{
|
||||
fmt.Sprintf("/srv/library %s(rw,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid),
|
||||
fmt.Sprintf("/srv/downloads %s(ro,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid),
|
||||
} {
|
||||
if !strings.Contains(text, want+"\n") {
|
||||
t.Errorf("the export file lacks %q:\n%s", want, text)
|
||||
}
|
||||
}
|
||||
for _, e := range ParseExports(text, ExportFile) {
|
||||
if e.Client != meshRange {
|
||||
t.Errorf("a client other than the private network's range: %+v", e)
|
||||
}
|
||||
if !strings.Contains(e.Options, "all_squash") || strings.Contains(e.Options, "no_root_squash") {
|
||||
t.Errorf("every client is squashed: %+v", e)
|
||||
}
|
||||
}
|
||||
if !s.Shares[0].Live || !s.Shares[1].Live {
|
||||
t.Errorf("both shares are live after exportfs -ra: %+v", s.Shares)
|
||||
}
|
||||
if s.Reloaded == nil || s.ReloadWhy != "the export file changed" {
|
||||
t.Errorf("the kernel read the file again because it changed: %+v", s)
|
||||
}
|
||||
// Rendering is stable: a second pass changes nothing and asks the kernel nothing.
|
||||
f.ran = nil
|
||||
f.machine().Pass(context.Background(), &s)
|
||||
for _, r := range f.ran {
|
||||
if strings.Contains(r, "exportfs") {
|
||||
t.Errorf("a pass with nothing changed reloaded: %v", f.ran)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheRootABackupRepositoryAndAMissingFolderAreRefusedAndNothingIsCreated(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.restic("/srv/pool/restic")
|
||||
f.dir("/srv/pool/media")
|
||||
f.restic("/srv/backups")
|
||||
f.file("/srv/a-file", "x")
|
||||
f.file(ConfigFile, config("root=/ pool=/srv/pool backups=/srv/backups missing=/srv/nowhere file=/srv/a-file rel=srv/x media=/srv/pool/media"))
|
||||
f.kernelReads()
|
||||
s := f.machine().Pass(context.Background(), nil)
|
||||
refused := map[string]string{}
|
||||
for _, r := range s.Refused {
|
||||
refused[r.Name] = r.Why
|
||||
}
|
||||
for name, word := range map[string]string{
|
||||
"root": "root", "pool": "holds a backup repository", "backups": "is a backup repository",
|
||||
"missing": "not there", "file": "not a folder", "rel": "not an absolute path",
|
||||
} {
|
||||
if !strings.Contains(refused[name], word) {
|
||||
t.Errorf("%s refused with %q, want %q", name, refused[name], word)
|
||||
}
|
||||
}
|
||||
if len(s.Shares) != 1 || s.Shares[0].Name != "media" {
|
||||
t.Errorf("only the folder beside the backups is exported: %+v", s.Shares)
|
||||
}
|
||||
if strings.Contains(f.read(ExportFile), "/srv/pool ") || strings.Contains(f.read(ExportFile), "/srv/backups") {
|
||||
t.Errorf("a refused folder is in the export file:\n%s", f.read(ExportFile))
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(f.root, "/srv/nowhere")); err == nil {
|
||||
t.Errorf("the module created a missing folder")
|
||||
}
|
||||
h := f.machine().Health()
|
||||
if h.Healthy || !strings.Contains(h.Why, "backups are never exported") {
|
||||
t.Errorf("a refused share makes the module unhealthy, said: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheModuleWritesOnlyItsTwoFilesAndHasNoWayToChangeAnOwnerOrMode(t *testing.T) {
|
||||
m := newFake(t).machine()
|
||||
for _, p := range []string{"/srv/library/x", "/etc/exports", "/etc/exports.d/zfs.exports", ConfigFile} {
|
||||
if err := m.Write(p, "x"); err == nil {
|
||||
t.Errorf("Write accepted %s", p)
|
||||
}
|
||||
}
|
||||
// No code path changes an owner or a mode, or removes anything: the words are not in the bundle.
|
||||
files, _ := filepath.Glob("*.go")
|
||||
for _, file := range files {
|
||||
if strings.HasSuffix(file, "_test.go") {
|
||||
continue
|
||||
}
|
||||
raw, _ := os.ReadFile(file)
|
||||
for _, word := range []string{"Chown", "Lchown", "Chmod", "os.Remove", "RemoveAll", `"chown"`, `"chmod"`, `"rm"`, "Mkdir(", "MkdirAll(filepath.Join(m.Root"} {
|
||||
if strings.Contains(string(raw), word) {
|
||||
t.Errorf("%s says %s", file, word)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseSharesAndConfig(t *testing.T) {
|
||||
shares, problems := ParseShares("library=/srv/library, downloads=/srv/downloads:ro Bad=/x noequals library=/again")
|
||||
if len(shares) != 2 || shares[0].Name != "downloads" || !shares[0].ReadOnly || shares[1].Access != "rw" {
|
||||
t.Errorf("shares: %+v", shares)
|
||||
}
|
||||
if len(problems) != 3 {
|
||||
t.Errorf("problems: %v", problems)
|
||||
}
|
||||
c, problems := ParseConfig("shares=\nrange=not-a-range\n")
|
||||
if c.Range != "" || len(problems) != 1 {
|
||||
t.Errorf("a range that is not one is said: %+v %v", c, problems)
|
||||
}
|
||||
_, problems = ParseConfig("shares=a=/x\n")
|
||||
if len(problems) != 1 || !strings.Contains(problems[0], "no private network range") {
|
||||
t.Errorf("no range is said: %v", problems)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseExportsReadsTheKernelsTableAndZFSFile(t *testing.T) {
|
||||
etab := "/srv/library\t192.0.2.0/24(rw,sync,wdelay,root_squash,sec=sys)\n/srv/x\n# comment\n/srv/y a(ro) b(rw)\n"
|
||||
got := ParseExports(etab, Etab)
|
||||
if len(got) != 4 || got[0].Client != "192.0.2.0/24" || got[1].Client != "*" || got[3].Client != "b" || got[3].Options != "rw" {
|
||||
t.Errorf("%+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func zfsWith(f *fake, sharenfs string) {
|
||||
f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Out: "pool\t/srv/pool\toff\npool/library\t/srv/library\t" + sharenfs + "\n"}
|
||||
}
|
||||
|
||||
func TestAdoptIsADryRunWithoutConfirmAndRefusesWhileTheMeshExportIsNotLive(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.uid = 1000
|
||||
f.dir("/srv/library")
|
||||
zfsWith(f, "rw=@192.0.2.0/24")
|
||||
f.file(ConfigFile, config("library=/srv/library"))
|
||||
// The kernel has not read the mesh's file: no exportfs effect.
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
if _, err := f.machine().Adopt("/srv/library", true, "take it over"); err == nil || !strings.Contains(err.Error(), "not live") {
|
||||
t.Fatalf("adopt while not live: %v", err)
|
||||
}
|
||||
f.kernelReads()
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
f.ran = nil
|
||||
dry, err := f.machine().Adopt("/srv/library", false, "")
|
||||
if err != nil || dry.Done || !dry.DryRun || !strings.Contains(dry.Said, "would set sharenfs=off on the dataset pool/library") {
|
||||
t.Fatalf("dry run: %+v %v", dry, err)
|
||||
}
|
||||
for _, r := range f.ran {
|
||||
if strings.Contains(r, "zfs set") {
|
||||
t.Fatalf("a dry run changed something: %v", f.ran)
|
||||
}
|
||||
}
|
||||
if _, err := f.machine().Adopt("/srv/library", true, ""); err == nil {
|
||||
t.Errorf("adopt with confirm and no why was done")
|
||||
}
|
||||
done, err := f.machine().Adopt("/srv/library", true, "the mesh exports it now")
|
||||
if err != nil || !done.Done {
|
||||
t.Fatalf("adopt: %+v %v", done, err)
|
||||
}
|
||||
var set []string
|
||||
for _, r := range f.ran {
|
||||
if strings.Contains(r, "zfs set") {
|
||||
set = append(set, r)
|
||||
}
|
||||
}
|
||||
if len(set) != 1 || set[0] != "sudo -n zfs set sharenfs=off pool/library" {
|
||||
t.Errorf("exactly one property of exactly one dataset, through sudo: %v", set)
|
||||
}
|
||||
zfsWith(f, "off")
|
||||
if _, err := f.machine().Adopt("/srv/library", true, "again"); err == nil || !strings.Contains(err.Error(), "already has sharenfs off") {
|
||||
t.Errorf("adopt of a dataset already off: %v", err)
|
||||
}
|
||||
if _, err := f.machine().Adopt("/srv/elsewhere", false, ""); err == nil {
|
||||
t.Errorf("adopt of a folder no dataset is mounted at")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdoptRefusesAFolderTheMeshDoesNotExport(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/library")
|
||||
zfsWith(f, "on")
|
||||
f.file(ConfigFile, config(""))
|
||||
f.kernelReads()
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
if _, err := f.machine().Adopt("/srv/library", false, ""); err == nil || !strings.Contains(err.Error(), "does not export") {
|
||||
t.Errorf("%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthSaysAStoppedServerAStaleProcessAndAShareNotInTheTable(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/library")
|
||||
f.file(ConfigFile, config("library=/srv/library"))
|
||||
f.kernelReads()
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "exports library to 10.0.0.0/24") {
|
||||
t.Errorf("healthy: %+v", h)
|
||||
}
|
||||
f.file(Etab, "")
|
||||
f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "failed\n", Code: 3}
|
||||
f.now = f.now.Add(10 * time.Minute)
|
||||
h := f.machine().Health()
|
||||
for _, want := range []string{"is failed", "last looked 10m0s ago", "The share library (/srv/library) is not in the kernel's export table."} {
|
||||
if h.Healthy || !strings.Contains(h.Why, want) {
|
||||
t.Errorf("health lacks %q: %+v", want, h)
|
||||
}
|
||||
}
|
||||
if h := newFake(t).machine().Health(); h.Healthy || !strings.Contains(h.Why, "written no status") {
|
||||
t.Errorf("no status is unhealthy: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExportsAnswersTheSharesAndWhatIsNotTheMeshs(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/library")
|
||||
f.file(ConfigFile, config("library=/srv/library"))
|
||||
f.file("/etc/exports.d/zfs.exports", "# !!! DO NOT EDIT\n/srv/library 192.0.2.0/24(sec=sys,rw,no_subtree_check)\n")
|
||||
f.file("/etc/exports", "# only comments\n")
|
||||
zfsWith(f, "rw=@192.0.2.0/24")
|
||||
f.kernelReads()
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
a := f.machine().Exports()
|
||||
raw, _ := json.Marshal(a)
|
||||
var shape struct {
|
||||
Shares []struct {
|
||||
Name, Path, Access, Clients string
|
||||
UID, GID int
|
||||
Live bool
|
||||
}
|
||||
Foreign []map[string]any
|
||||
Service string
|
||||
}
|
||||
if err := json.Unmarshal(raw, &shape); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(shape.Shares) != 1 || shape.Shares[0].Name != "library" || !shape.Shares[0].Live || shape.Shares[0].Clients != meshRange || shape.Service != "active" {
|
||||
t.Errorf("%s", raw)
|
||||
}
|
||||
if len(shape.Foreign) != 2 {
|
||||
t.Errorf("the ZFS file's line and the dataset's property are not the mesh's: %s", raw)
|
||||
}
|
||||
for _, key := range []string{`"shares"`, `"foreign"`, `"service"`, `"uid"`, `"gid"`, `"live"`, `"clients"`, `"access"`} {
|
||||
if !strings.Contains(string(raw), key) {
|
||||
t.Errorf("the answer lacks %s, which the mounts module reads", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTestSaysWhetherAShareIsServedAndWhetherAnAddressIsInRange(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/library")
|
||||
f.file(ConfigFile, config("library=/srv/library"))
|
||||
f.kernelReads()
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
ok, _ := f.machine().Test("library", "10.0.0.3")
|
||||
if !ok.OK || ok.InRange == nil || !*ok.InRange {
|
||||
t.Errorf("%+v", ok)
|
||||
}
|
||||
out, _ := f.machine().Test("library", "192.0.2.7")
|
||||
if out.OK || !strings.Contains(out.Why, "not inside the private network's range") {
|
||||
t.Errorf("%+v", out)
|
||||
}
|
||||
none, _ := f.machine().Test("photos", "")
|
||||
if none.OK || !strings.Contains(none.Why, "no share named photos") {
|
||||
t.Errorf("%+v", none)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientsReadTheKernelsListOrFallBackToConnections(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.file("/proc/fs/nfsd/clients/7/info", "clientid: 0xabc\naddress: \"10.0.0.3:785\"\nname: \"Linux NFSv4.2 workstation\"\nminor version: 2\n")
|
||||
a := f.machine().Clients()
|
||||
if len(a.Clients) != 1 || a.Clients[0].Address != "10.0.0.3:785" || a.Clients[0].Minor != "2" {
|
||||
t.Errorf("%+v", a)
|
||||
}
|
||||
g := newFake(t)
|
||||
g.answers["ss -tnH state established ( sport = :2049 )"] = Ran{Out: "0 0 10.0.0.2:2049 10.0.0.3:785\n"}
|
||||
b := g.machine().Clients()
|
||||
if len(b.Clients) != 1 || b.Clients[0].Address != "10.0.0.3:785" || b.From != "connections to port 2049" {
|
||||
t.Errorf("%+v", b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnActNeedingRootGoesThroughSudoOnlyWhenNotRoot(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.uid = 1000
|
||||
f.machine().AsRoot("exportfs", "-ra")
|
||||
f.uid = 0
|
||||
f.machine().AsRoot("exportfs", "-ra")
|
||||
if len(f.ran) != 2 || f.ran[0] != "sudo -n exportfs -ra" || f.ran[1] != "exportfs -ra" {
|
||||
t.Errorf("%v", f.ran)
|
||||
}
|
||||
f.answers["sudo -n zfs set sharenfs=off x"] = Ran{Code: 1, Err: "sudo: a password is required"}
|
||||
f.uid = 1000
|
||||
if _, err := f.machine().AsRoot("zfs", "set", "sharenfs=off", "x"); err == nil || !strings.Contains(err.Error(), "sudo refused") {
|
||||
t.Errorf("%v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The module's process, as root: every Every it reads the assignment's shares, renders the export file,
|
||||
// has the kernel read it again when anything changed or a share is not live, and writes what it saw for
|
||||
// the tools (novox/hq ADR 0263).
|
||||
|
||||
// Every is how often the process looks.
|
||||
const Every = 30 * time.Second
|
||||
|
||||
// Status is the process's last look, as the tools read it.
|
||||
type Status struct {
|
||||
At time.Time `json:"at"`
|
||||
Range string `json:"range"`
|
||||
Shares []Exported `json:"shares"`
|
||||
Refused []Refused `json:"refused,omitempty"`
|
||||
Problems []string `json:"problems,omitempty"`
|
||||
// Reloaded is when the process last had the kernel read the export files again, and why.
|
||||
Reloaded *time.Time `json:"reloaded,omitempty"`
|
||||
ReloadWhy string `json:"reload_why,omitempty"`
|
||||
ReloadError string `json:"reload_error,omitempty"`
|
||||
}
|
||||
|
||||
// Pass is one look: what it did is in the status it answers, which it has written.
|
||||
func (m *Machine) Pass(ctx context.Context, last *Status) Status {
|
||||
s := Status{At: m.Now().UTC()}
|
||||
if last != nil {
|
||||
s.Reloaded, s.ReloadWhy = last.Reloaded, last.ReloadWhy
|
||||
}
|
||||
text, err := m.Read(ConfigFile)
|
||||
if err != nil {
|
||||
s.Problems = append(s.Problems, fmt.Sprintf("the assignment's shares (%s) could not be read: %v; nothing is exported", ConfigFile, err))
|
||||
}
|
||||
c, problems := ParseConfig(text)
|
||||
s.Problems = append(s.Problems, problems...)
|
||||
s.Range = c.Range
|
||||
exports, refused := m.Judge(c)
|
||||
s.Refused = refused
|
||||
|
||||
want := ExportFileText(exports)
|
||||
have, _ := m.Read(ExportFile)
|
||||
reload := ""
|
||||
if have != want {
|
||||
if err := m.Write(ExportFile, want); err != nil {
|
||||
s.Problems = append(s.Problems, fmt.Sprintf("the export file could not be written: %v", err))
|
||||
} else {
|
||||
reload = "the export file changed"
|
||||
}
|
||||
}
|
||||
live, note := m.Live()
|
||||
if reload == "" {
|
||||
for _, e := range exports {
|
||||
if !IsLive(live, e.Path, e.Clients) {
|
||||
reload = fmt.Sprintf("the share %s was not in the kernel's table", e.Name)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if reload != "" {
|
||||
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
|
||||
s.ReloadError = err.Error()
|
||||
} else {
|
||||
at := m.Now().UTC()
|
||||
s.Reloaded, s.ReloadWhy = &at, reload
|
||||
}
|
||||
live, note = m.Live()
|
||||
}
|
||||
if note != "" && len(exports) > 0 {
|
||||
s.Problems = append(s.Problems, note)
|
||||
}
|
||||
for i := range exports {
|
||||
exports[i].Live = IsLive(live, exports[i].Path, exports[i].Clients)
|
||||
}
|
||||
s.Shares = exports
|
||||
raw, _ := json.MarshalIndent(s, "", " ")
|
||||
if err := m.Write(StatusFile, string(raw)+"\n"); err != nil {
|
||||
s.Problems = append(s.Problems, fmt.Sprintf("the status file could not be written: %v", err))
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// RunExports is the process: a pass now and every Every until the context ends.
|
||||
func (m *Machine) RunExports(ctx context.Context) error {
|
||||
last := m.Pass(ctx, nil)
|
||||
t := time.NewTicker(Every)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case <-t.C:
|
||||
last = m.Pass(ctx, &last)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// LastStatus is the process's last look, or why there is none.
|
||||
func (m *Machine) LastStatus() (*Status, error) {
|
||||
text, err := m.Read(StatusFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the module's process has written no status (%s): %v", StatusFile, err)
|
||||
}
|
||||
var s Status
|
||||
if err := json.Unmarshal([]byte(text), &s); err != nil {
|
||||
return nil, fmt.Errorf("the module's status file is not one: %v", err)
|
||||
}
|
||||
return &s, nil
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The shares, from the assignment, and the export lines made from them (novox/hq ADR 0263).
|
||||
//
|
||||
// **One share per folder.** A share is a name and a path the operator gives in the `shares` setting:
|
||||
// `library=/srv/library downloads=/srv/downloads`, `:ro` after a path for read-only. The path is the
|
||||
// operator's data (novox/hq ADR 0051): the module reads its owner and exports it, and never creates,
|
||||
// chowns, chmods or removes it.
|
||||
//
|
||||
// **Every client is the folder's owner.** NFS with sec=sys believes the user id the client sends, and the
|
||||
// same id names different accounts on different machines. So every line squashes every client
|
||||
// (`all_squash`) to the owner of the shared folder as this machine reads it (`anonuid`, `anongid`): a file
|
||||
// written from any node lands owned by the folder's owner, and no client acts as any other account here.
|
||||
// Root is squashed with everyone else.
|
||||
//
|
||||
// **To the mesh's machines only.** The clients are the private network's range and nothing else: every
|
||||
// address there is a node. The home network is never named.
|
||||
//
|
||||
// **Never the backups.** A share that is the machine's root, that does not exist, that is not a
|
||||
// directory, or that holds a backup repository at its path or one level below is refused, and said.
|
||||
|
||||
// Share is one share the operator asked for.
|
||||
type Share struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
ReadOnly bool `json:"-"`
|
||||
Access string `json:"access"`
|
||||
}
|
||||
|
||||
// Exported is a share as this machine exports it.
|
||||
type Exported struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Access string `json:"access"`
|
||||
UID int `json:"uid"`
|
||||
GID int `json:"gid"`
|
||||
Clients string `json:"clients"`
|
||||
Line string `json:"line"`
|
||||
Live bool `json:"live"`
|
||||
}
|
||||
|
||||
// Refused is a share that is not exported, and why.
|
||||
type Refused struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
|
||||
// Config is what the mesh rendered from the assignment.
|
||||
type Config struct {
|
||||
Shares []Share
|
||||
Range string
|
||||
}
|
||||
|
||||
// ParseConfig reads the config file: `shares=` and `range=` lines, comments and blanks ignored.
|
||||
func ParseConfig(text string) (Config, []string) {
|
||||
var c Config
|
||||
var problems []string
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
key, value, ok := strings.Cut(line, "=")
|
||||
if !ok {
|
||||
problems = append(problems, fmt.Sprintf("the line %q of %s says nothing this module reads", line, ConfigFile))
|
||||
continue
|
||||
}
|
||||
switch strings.TrimSpace(key) {
|
||||
case "shares":
|
||||
shares, bad := ParseShares(value)
|
||||
c.Shares = shares
|
||||
problems = append(problems, bad...)
|
||||
case "range":
|
||||
c.Range = strings.TrimSpace(value)
|
||||
default:
|
||||
problems = append(problems, fmt.Sprintf("%s names %q, which this module does not read", ConfigFile, key))
|
||||
}
|
||||
}
|
||||
if c.Range == "" {
|
||||
problems = append(problems, fmt.Sprintf("%s gives no private network range, so nothing can be exported", ConfigFile))
|
||||
} else if _, err := netip.ParsePrefix(c.Range); err != nil {
|
||||
problems = append(problems, fmt.Sprintf("the private network's range %q is not a range: %v", c.Range, err))
|
||||
c.Range = ""
|
||||
}
|
||||
return c, problems
|
||||
}
|
||||
|
||||
// ParseShares reads the `shares` setting: `name=path` or `name=path:ro`, separated by spaces or commas.
|
||||
func ParseShares(value string) ([]Share, []string) {
|
||||
var shares []Share
|
||||
var problems []string
|
||||
seen := map[string]bool{}
|
||||
for _, item := range strings.FieldsFunc(value, func(r rune) bool { return r == ' ' || r == ',' || r == '\t' }) {
|
||||
name, p, ok := strings.Cut(item, "=")
|
||||
if !ok || name == "" || p == "" {
|
||||
problems = append(problems, fmt.Sprintf("the share %q is not name=path", item))
|
||||
continue
|
||||
}
|
||||
if !validName(name) {
|
||||
problems = append(problems, fmt.Sprintf("the share name %q is not lower-case letters, digits and dashes", name))
|
||||
continue
|
||||
}
|
||||
if seen[name] {
|
||||
problems = append(problems, fmt.Sprintf("the share %q is named twice", name))
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
s := Share{Name: name, Path: p, Access: "rw"}
|
||||
if strings.HasSuffix(p, ":ro") {
|
||||
s.Path, s.ReadOnly, s.Access = strings.TrimSuffix(p, ":ro"), true, "ro"
|
||||
} else if strings.HasSuffix(p, ":rw") {
|
||||
s.Path = strings.TrimSuffix(p, ":rw")
|
||||
}
|
||||
shares = append(shares, s)
|
||||
}
|
||||
sort.Slice(shares, func(i, j int) bool { return shares[i].Name < shares[j].Name })
|
||||
return shares, problems
|
||||
}
|
||||
|
||||
func validName(n string) bool {
|
||||
if n == "" || n[0] == '-' {
|
||||
return false
|
||||
}
|
||||
for _, r := range n {
|
||||
if !(r >= 'a' && r <= 'z' || r >= '0' && r <= '9' || r == '-') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// IsBackupRepository says whether a directory is a restic repository: a `config` file beside `keys`,
|
||||
// `snapshots` and `data`. It looks at names only.
|
||||
func (m *Machine) IsBackupRepository(dir string) bool {
|
||||
if !m.Exists(path.Join(dir, "config")) {
|
||||
return false
|
||||
}
|
||||
for _, sub := range []string{"keys", "snapshots", "data"} {
|
||||
if o, err := m.Stat(path.Join(dir, sub)); err != nil || !o.Dir {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Judge turns the asked-for shares into export lines, or refusals.
|
||||
func (m *Machine) Judge(c Config) ([]Exported, []Refused) {
|
||||
var out []Exported
|
||||
var refused []Refused
|
||||
for _, s := range c.Shares {
|
||||
no := func(why string) { refused = append(refused, Refused{Name: s.Name, Path: s.Path, Why: why}) }
|
||||
clean := path.Clean(s.Path)
|
||||
switch {
|
||||
case !path.IsAbs(s.Path):
|
||||
no(fmt.Sprintf("The share %s names %q, which is not an absolute path.", s.Name, s.Path))
|
||||
continue
|
||||
case clean == "/":
|
||||
no(fmt.Sprintf("The share %s is the machine's root, which is never exported.", s.Name))
|
||||
continue
|
||||
case strings.ContainsAny(s.Path, " \t\"\\#"):
|
||||
no(fmt.Sprintf("The share %s's path %q has a character an export line cannot carry.", s.Name, s.Path))
|
||||
continue
|
||||
}
|
||||
o, err := m.Stat(clean)
|
||||
if err != nil {
|
||||
no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, clean, err))
|
||||
continue
|
||||
}
|
||||
if !o.Dir {
|
||||
no(fmt.Sprintf("The share %s's path %s is not a folder.", s.Name, clean))
|
||||
continue
|
||||
}
|
||||
if m.IsBackupRepository(clean) {
|
||||
no(fmt.Sprintf("The share %s's folder %s is a backup repository, and backups are never exported.", s.Name, clean))
|
||||
continue
|
||||
}
|
||||
holds := ""
|
||||
for _, child := range m.Children(clean) {
|
||||
if m.IsBackupRepository(path.Join(clean, child)) {
|
||||
holds = path.Join(clean, child)
|
||||
break
|
||||
}
|
||||
}
|
||||
if holds != "" {
|
||||
no(fmt.Sprintf("The share %s's folder %s holds a backup repository (%s), and backups are never exported: share the folders beside it instead.", s.Name, clean, holds))
|
||||
continue
|
||||
}
|
||||
if c.Range == "" {
|
||||
no(fmt.Sprintf("The share %s is not exported: the private network's range is not known.", s.Name))
|
||||
continue
|
||||
}
|
||||
e := Exported{Name: s.Name, Path: clean, Access: s.Access, UID: o.UID, GID: o.GID, Clients: c.Range}
|
||||
e.Line = ExportLine(e)
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, refused
|
||||
}
|
||||
|
||||
// ExportLine is one share's line in the export file.
|
||||
func ExportLine(e Exported) string {
|
||||
return fmt.Sprintf("%s %s(%s,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)",
|
||||
e.Path, e.Clients, e.Access, e.UID, e.GID)
|
||||
}
|
||||
|
||||
// ExportFileText is the whole export file.
|
||||
func ExportFileText(exports []Exported) string {
|
||||
var b strings.Builder
|
||||
b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered from its assignment's\n")
|
||||
b.WriteString("# `shares` setting by the module's own process; edit the setting, never this file.\n")
|
||||
b.WriteString("# One share per folder, to the private network's range only, every client squashed to the\n")
|
||||
b.WriteString("# folder's owner.\n")
|
||||
for _, e := range exports {
|
||||
fmt.Fprintf(&b, "# share %s\n%s\n", e.Name, e.Line)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// What the machine exports, read: the kernel's table, the other export files, and ZFS's property.
|
||||
|
||||
// Entry is one export: a path, one client, its options, and where it was found.
|
||||
type Entry struct {
|
||||
Path string `json:"path"`
|
||||
Client string `json:"client"`
|
||||
Options string `json:"options,omitempty"`
|
||||
From string `json:"from"`
|
||||
}
|
||||
|
||||
// ParseExports reads an exports(5) file or the kernel's etab: a path, then clients each with options.
|
||||
func ParseExports(text, from string) []Entry {
|
||||
var out []Entry
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
if i := strings.Index(line, "#"); i >= 0 {
|
||||
line = line[:i]
|
||||
}
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 1 {
|
||||
continue
|
||||
}
|
||||
p := fields[0]
|
||||
if len(fields) == 1 {
|
||||
out = append(out, Entry{Path: p, Client: "*", From: from})
|
||||
continue
|
||||
}
|
||||
for _, c := range fields[1:] {
|
||||
client, opts := c, ""
|
||||
if i := strings.Index(c, "("); i >= 0 {
|
||||
client, opts = c[:i], strings.TrimSuffix(c[i+1:], ")")
|
||||
}
|
||||
if client == "" {
|
||||
client = "*"
|
||||
}
|
||||
out = append(out, Entry{Path: p, Client: client, Options: opts, From: from})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Live is the kernel's export table, or nil and why when it cannot be read.
|
||||
func (m *Machine) Live() ([]Entry, string) {
|
||||
text, err := m.Read(Etab)
|
||||
if err != nil {
|
||||
return nil, "the kernel's export table (" + Etab + ") could not be read: " + err.Error()
|
||||
}
|
||||
return ParseExports(text, Etab), ""
|
||||
}
|
||||
|
||||
// IsLive says whether the kernel exports a path to a client.
|
||||
func IsLive(live []Entry, p, client string) bool {
|
||||
for _, e := range live {
|
||||
if path.Clean(e.Path) == path.Clean(p) && e.Client == client {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Dataset is one ZFS filesystem as `zfs list` answers it.
|
||||
type Dataset struct {
|
||||
Name string `json:"name"`
|
||||
Mountpoint string `json:"mountpoint"`
|
||||
ShareNFS string `json:"sharenfs"`
|
||||
}
|
||||
|
||||
// Datasets are the ZFS filesystems, when zfs is there; nil without it.
|
||||
func (m *Machine) Datasets() ([]Dataset, string) {
|
||||
r := m.Run(context.Background(), "zfs", "list", "-H", "-o", "name,mountpoint,sharenfs", "-t", "filesystem")
|
||||
if r.Code != 0 {
|
||||
if r.Code == -1 && strings.Contains(r.Err, "executable file not found") {
|
||||
return nil, ""
|
||||
}
|
||||
return nil, "zfs list did not answer: " + firstLine(r.Err)
|
||||
}
|
||||
var out []Dataset
|
||||
for _, line := range strings.Split(r.Out, "\n") {
|
||||
f := strings.Split(line, "\t")
|
||||
if len(f) != 3 {
|
||||
continue
|
||||
}
|
||||
out = append(out, Dataset{Name: f[0], Mountpoint: f[1], ShareNFS: f[2]})
|
||||
}
|
||||
return out, ""
|
||||
}
|
||||
|
||||
// Foreign is every export on the machine that is not the mesh's: /etc/exports, the other files in
|
||||
// /etc/exports.d (ZFS writes one), and every dataset whose sharenfs is on.
|
||||
func (m *Machine) Foreign() ([]Entry, []Dataset, []string) {
|
||||
var entries []Entry
|
||||
var notes []string
|
||||
files := append([]string{"/etc/exports"}, m.Glob("/etc/exports.d/*.exports")...)
|
||||
for _, f := range files {
|
||||
if f == ExportFile {
|
||||
continue
|
||||
}
|
||||
text, err := m.Read(f)
|
||||
if err != nil {
|
||||
if m.Exists(f) {
|
||||
notes = append(notes, f+" could not be read: "+err.Error())
|
||||
}
|
||||
continue
|
||||
}
|
||||
entries = append(entries, ParseExports(text, f)...)
|
||||
}
|
||||
sets, note := m.Datasets()
|
||||
if note != "" {
|
||||
notes = append(notes, note)
|
||||
}
|
||||
var shared []Dataset
|
||||
for _, d := range sets {
|
||||
if d.ShareNFS != "off" && d.ShareNFS != "-" && d.ShareNFS != "" {
|
||||
shared = append(shared, d)
|
||||
}
|
||||
}
|
||||
return entries, shared, notes
|
||||
}
|
||||
|
||||
func firstLine(s string) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if i := strings.IndexByte(s, '\n'); i >= 0 {
|
||||
return s[:i]
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,427 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// Seat is the role this module holds on its machine (novox/hq ADR 0263).
|
||||
const Seat = "node-nfs-server"
|
||||
|
||||
// Stale is how old the process's last look may be before the tools say it stopped looking.
|
||||
const Stale = 5 * time.Minute
|
||||
|
||||
// verb is one of the seat's verbs, listed as `<seat>.<verb>` so the runtime serves it on the seat's
|
||||
// subject, as <node>/node-nfs-server.<verb>.
|
||||
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
|
||||
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run}
|
||||
}
|
||||
|
||||
func str(description string) map[string]any {
|
||||
return map[string]any{"type": "string", "description": description}
|
||||
}
|
||||
|
||||
func arg(a map[string]any, k string) string {
|
||||
v, _ := a[k].(string)
|
||||
return strings.TrimSpace(v)
|
||||
}
|
||||
|
||||
func flag(a map[string]any, k string) bool {
|
||||
v, _ := a[k].(bool)
|
||||
return v
|
||||
}
|
||||
|
||||
// Tools are the seat's verbs and the module's health.
|
||||
func Tools(m *Machine) []stdio.Tool {
|
||||
return []stdio.Tool{
|
||||
verb("exports", "Every share this machine exports for the mesh: its name, folder, rw or ro, the owner every "+
|
||||
"client is mapped to (uid, gid), the clients (the private network's range, nothing else) and whether the "+
|
||||
"kernel holds it now; the shares the assignment asked for that were refused, and why; and every export "+
|
||||
"on the machine that is not the mesh's (/etc/exports, the other files in /etc/exports.d, a ZFS dataset's "+
|
||||
"sharenfs). Replaces exportfs -v, cat /etc/exports and zfs get sharenfs. (r)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Exports(), nil }),
|
||||
verb("clients", "Which machines have a share of this machine mounted now: each NFS client the kernel knows, "+
|
||||
"with its address and the name it gave, or — when the kernel's list cannot be read — the connections "+
|
||||
"to port 2049. Replaces cat /proc/fs/nfsd/clients/*/info and ss -tn sport = :2049. (r)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Clients(), nil }),
|
||||
verb("test", "Whether this machine exports one share for the mesh now: the share is asked for and not "+
|
||||
"refused, the kernel holds it for the private network's range, and the NFS server is active; with an "+
|
||||
"address, also whether that address is inside the range. Replaces showmount -e. (r)",
|
||||
map[string]any{"share": str("the share's name"), "address": str("a client's address on the private network (optional)")},
|
||||
func(a map[string]any) (any, error) { return m.Test(arg(a, "share"), arg(a, "address")) }),
|
||||
verb("reload", "Have the kernel read the export files again now (exportfs -ra), and answer the mesh's shares "+
|
||||
"as it then holds them. The module's process does this itself whenever its file changes or a share is "+
|
||||
"missing; this is for after a change made outside it. Replaces exportfs -ra. (a)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Reload() }),
|
||||
verb("adopt", "Take over the export a ZFS dataset's sharenfs property makes for a folder the mesh now "+
|
||||
"exports: sets sharenfs=off on that one dataset and nothing else, then exportfs -ra. Only when the "+
|
||||
"mesh's own export of the same folder is live in the kernel, so the folder is never unexported; "+
|
||||
"without confirm, a dry run that says what it would do. A person's act, with why. Replaces zfs set "+
|
||||
"sharenfs=off. (a)",
|
||||
map[string]any{
|
||||
"path": str("the shared folder, as the share names it"),
|
||||
"confirm": map[string]any{"type": "boolean", "description": "do it; without, a dry run"},
|
||||
"why": str("why, in a sentence (required with confirm)"),
|
||||
},
|
||||
func(a map[string]any) (any, error) { return m.Adopt(arg(a, "path"), flag(a, "confirm"), arg(a, "why")) }),
|
||||
{
|
||||
Name: "nfs_health",
|
||||
Description: "Whether this machine's shares are served, as the module's health check (novox/hq ADR 0240, " +
|
||||
"ADR 0263): unhealthy when the NFS server is not active, the module's process has not looked for five " +
|
||||
"minutes, a share the assignment asks for is refused or missing from the kernel's table, or the " +
|
||||
"export file could not be written or read again; with why, one sentence per finding. (r)",
|
||||
Run: func(map[string]any) (any, error) { return m.Health(), nil },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// ExportsAnswer is what `exports` answers.
|
||||
type ExportsAnswer struct {
|
||||
Shares []Exported `json:"shares"`
|
||||
Refused []Refused `json:"refused,omitempty"`
|
||||
Foreign []Entry `json:"foreign"`
|
||||
Service string `json:"service"`
|
||||
Range string `json:"range,omitempty"`
|
||||
Looked string `json:"looked,omitempty"`
|
||||
Said []string `json:"said"`
|
||||
Notes []string `json:"notes,omitempty"`
|
||||
}
|
||||
|
||||
// Exports is the machine's exports now.
|
||||
func (m *Machine) Exports() ExportsAnswer {
|
||||
a := ExportsAnswer{Shares: []Exported{}, Foreign: []Entry{}, Service: m.ServiceState()}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
a.Notes = append(a.Notes, err.Error())
|
||||
} else {
|
||||
a.Range, a.Refused = st.Range, st.Refused
|
||||
a.Looked = st.At.Format(time.RFC3339)
|
||||
live, note := m.Live()
|
||||
if note != "" {
|
||||
a.Notes = append(a.Notes, note)
|
||||
}
|
||||
for _, e := range st.Shares {
|
||||
if note == "" {
|
||||
e.Live = IsLive(live, e.Path, e.Clients)
|
||||
}
|
||||
a.Shares = append(a.Shares, e)
|
||||
}
|
||||
a.Notes = append(a.Notes, st.Problems...)
|
||||
}
|
||||
entries, sets, notes := m.Foreign()
|
||||
a.Notes = append(a.Notes, notes...)
|
||||
a.Foreign = append(a.Foreign, entries...)
|
||||
for _, d := range sets {
|
||||
a.Foreign = append(a.Foreign, Entry{Path: d.Mountpoint, Client: d.ShareNFS, From: "zfs sharenfs on " + d.Name})
|
||||
}
|
||||
for _, e := range a.Shares {
|
||||
state := "live"
|
||||
if !e.Live {
|
||||
state = "NOT in the kernel's table"
|
||||
}
|
||||
a.Said = append(a.Said, fmt.Sprintf("%s: %s %s to %s, every client as %d:%d — %s", e.Name, e.Path, e.Access, e.Clients, e.UID, e.GID, state))
|
||||
}
|
||||
for _, r := range a.Refused {
|
||||
a.Said = append(a.Said, "refused: "+r.Why)
|
||||
}
|
||||
for _, f := range a.Foreign {
|
||||
a.Said = append(a.Said, fmt.Sprintf("not the mesh's: %s to %s (%s)", f.Path, f.Client, f.From))
|
||||
}
|
||||
if len(a.Shares) == 0 && len(a.Refused) == 0 {
|
||||
a.Said = append(a.Said, "the mesh exports nothing here")
|
||||
}
|
||||
a.Said = append(a.Said, "the NFS server is "+a.Service)
|
||||
return a
|
||||
}
|
||||
|
||||
// Client is one machine that has a share mounted.
|
||||
type Client struct {
|
||||
Address string `json:"address"`
|
||||
Name string `json:"name,omitempty"`
|
||||
Minor string `json:"minor_version,omitempty"`
|
||||
}
|
||||
|
||||
// ClientsAnswer is what `clients` answers.
|
||||
type ClientsAnswer struct {
|
||||
Clients []Client `json:"clients"`
|
||||
From string `json:"from"`
|
||||
Notes []string `json:"notes,omitempty"`
|
||||
}
|
||||
|
||||
// ParseClientInfo reads one /proc/fs/nfsd/clients/<id>/info.
|
||||
func ParseClientInfo(text string) Client {
|
||||
var c Client
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
k, v, ok := strings.Cut(line, ":")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
v = strings.Trim(strings.TrimSpace(v), `"`)
|
||||
switch strings.TrimSpace(k) {
|
||||
case "address":
|
||||
c.Address = v
|
||||
case "name":
|
||||
c.Name = v
|
||||
case "minor version":
|
||||
c.Minor = v
|
||||
}
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// Clients are the NFS clients the kernel knows, or the connections to 2049.
|
||||
func (m *Machine) Clients() ClientsAnswer {
|
||||
a := ClientsAnswer{Clients: []Client{}}
|
||||
const dir = "/proc/fs/nfsd/clients"
|
||||
infos := m.Glob(dir + "/*/info")
|
||||
if len(infos) == 0 && m.UID != 0 {
|
||||
if r, err := m.AsRoot("find", dir, "-maxdepth", "2", "-name", "info"); err == nil {
|
||||
for _, l := range strings.Fields(r.Out) {
|
||||
infos = append(infos, l)
|
||||
}
|
||||
} else {
|
||||
a.Notes = append(a.Notes, err.Error())
|
||||
}
|
||||
}
|
||||
for _, f := range infos {
|
||||
text, err := m.Read(f)
|
||||
if err != nil && m.UID != 0 {
|
||||
r, e := m.AsRoot("cat", f)
|
||||
text, err = r.Out, e
|
||||
}
|
||||
if err != nil {
|
||||
a.Notes = append(a.Notes, f+": "+err.Error())
|
||||
continue
|
||||
}
|
||||
a.Clients = append(a.Clients, ParseClientInfo(text))
|
||||
}
|
||||
if len(infos) > 0 {
|
||||
a.From = dir
|
||||
return a
|
||||
}
|
||||
r := m.Run(context.Background(), "ss", "-tnH", "state", "established", "( sport = :2049 )")
|
||||
a.From = "connections to port 2049"
|
||||
for _, line := range strings.Split(r.Out, "\n") {
|
||||
f := strings.Fields(line)
|
||||
if len(f) >= 4 {
|
||||
a.Clients = append(a.Clients, Client{Address: f[len(f)-1]})
|
||||
}
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
// TestAnswer is what `test` answers.
|
||||
type TestAnswer struct {
|
||||
Share string `json:"share"`
|
||||
Path string `json:"path,omitempty"`
|
||||
Access string `json:"access,omitempty"`
|
||||
Live bool `json:"live"`
|
||||
Service string `json:"service"`
|
||||
InRange *bool `json:"in_range,omitempty"`
|
||||
OK bool `json:"ok"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
|
||||
// Test says whether one share is served now.
|
||||
func (m *Machine) Test(share, address string) (TestAnswer, error) {
|
||||
if share == "" {
|
||||
return TestAnswer{}, fmt.Errorf("name the share")
|
||||
}
|
||||
a := TestAnswer{Share: share, Service: m.ServiceState()}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
a.Why = err.Error()
|
||||
return a, nil
|
||||
}
|
||||
for _, r := range st.Refused {
|
||||
if r.Name == share {
|
||||
a.Path, a.Why = r.Path, r.Why
|
||||
return a, nil
|
||||
}
|
||||
}
|
||||
var found *Exported
|
||||
for i := range st.Shares {
|
||||
if st.Shares[i].Name == share {
|
||||
found = &st.Shares[i]
|
||||
}
|
||||
}
|
||||
if found == nil {
|
||||
a.Why = fmt.Sprintf("This machine's assignment asks for no share named %s.", share)
|
||||
return a, nil
|
||||
}
|
||||
a.Path, a.Access = found.Path, found.Access
|
||||
live, note := m.Live()
|
||||
a.Live = note == "" && IsLive(live, found.Path, found.Clients)
|
||||
var why []string
|
||||
if address != "" {
|
||||
addr, err1 := netip.ParseAddr(address)
|
||||
prefix, err2 := netip.ParsePrefix(found.Clients)
|
||||
in := err1 == nil && err2 == nil && prefix.Contains(addr)
|
||||
a.InRange = &in
|
||||
if !in {
|
||||
why = append(why, fmt.Sprintf("%s is not inside the private network's range %s.", address, found.Clients))
|
||||
}
|
||||
}
|
||||
if !a.Live {
|
||||
why = append(why, fmt.Sprintf("The kernel does not export %s to %s now.", found.Path, found.Clients))
|
||||
}
|
||||
if a.Service != "active" {
|
||||
why = append(why, "The NFS server is "+a.Service+".")
|
||||
}
|
||||
a.OK = len(why) == 0
|
||||
if a.OK {
|
||||
a.Why = fmt.Sprintf("The share %s (%s, %s) is exported to %s and the NFS server is active.", share, found.Path, found.Access, found.Clients)
|
||||
} else {
|
||||
a.Why = strings.Join(why, " ")
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// Reload has the kernel read the export files again.
|
||||
func (m *Machine) Reload() (any, error) {
|
||||
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Exports(), nil
|
||||
}
|
||||
|
||||
// AdoptAnswer is what `adopt` answers.
|
||||
type AdoptAnswer struct {
|
||||
Path string `json:"path"`
|
||||
Dataset string `json:"dataset,omitempty"`
|
||||
Was string `json:"sharenfs_was,omitempty"`
|
||||
Done bool `json:"done"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
Said string `json:"said"`
|
||||
}
|
||||
|
||||
// Adopt clears one dataset's sharenfs once the mesh's own export of its folder is live.
|
||||
func (m *Machine) Adopt(p string, confirm bool, why string) (AdoptAnswer, error) {
|
||||
if p == "" {
|
||||
return AdoptAnswer{}, fmt.Errorf("name the shared folder (path)")
|
||||
}
|
||||
p = path.Clean(p)
|
||||
a := AdoptAnswer{Path: p, DryRun: !confirm}
|
||||
sets, note := m.Datasets()
|
||||
if note != "" {
|
||||
return a, fmt.Errorf("%s", note)
|
||||
}
|
||||
var ds *Dataset
|
||||
for i := range sets {
|
||||
if path.Clean(sets[i].Mountpoint) == p {
|
||||
ds = &sets[i]
|
||||
}
|
||||
}
|
||||
if ds == nil {
|
||||
return a, fmt.Errorf("no ZFS dataset is mounted at %s, so there is no sharenfs to take over", p)
|
||||
}
|
||||
a.Dataset, a.Was = ds.Name, ds.ShareNFS
|
||||
if ds.ShareNFS == "off" || ds.ShareNFS == "-" || ds.ShareNFS == "" {
|
||||
return a, fmt.Errorf("the dataset %s already has sharenfs off; nothing to take over", ds.Name)
|
||||
}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
var mine *Exported
|
||||
for i := range st.Shares {
|
||||
if st.Shares[i].Path == p {
|
||||
mine = &st.Shares[i]
|
||||
}
|
||||
}
|
||||
if mine == nil {
|
||||
return a, fmt.Errorf("the mesh does not export %s: add it to this machine's nfs-server `shares` setting first, so the folder is never unexported", p)
|
||||
}
|
||||
live, lnote := m.Live()
|
||||
if lnote != "" || !IsLive(live, p, mine.Clients) {
|
||||
return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, mine.Clients)
|
||||
}
|
||||
plan := fmt.Sprintf("set sharenfs=off on the dataset %s (now %q) and run exportfs -ra; %s stays exported as the share %s to %s, every client as %d:%d",
|
||||
ds.Name, ds.ShareNFS, p, mine.Name, mine.Clients, mine.UID, mine.GID)
|
||||
if !confirm {
|
||||
a.Said = "Dry run: would " + plan + "."
|
||||
return a, nil
|
||||
}
|
||||
if why == "" {
|
||||
return a, fmt.Errorf("say why: adopt is a person's act, and the why is kept with it")
|
||||
}
|
||||
if _, err := m.AsRoot("zfs", "set", "sharenfs=off", ds.Name); err != nil {
|
||||
return a, err
|
||||
}
|
||||
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
|
||||
return a, fmt.Errorf("sharenfs is off on %s, and exportfs -ra failed: %v", ds.Name, err)
|
||||
}
|
||||
a.Done = true
|
||||
a.Said = fmt.Sprintf("Did: %s. Why: %s", plan, why)
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// Health is the module's health, as the node-engine asks it.
|
||||
type Health struct {
|
||||
Healthy bool `json:"healthy"`
|
||||
Why string `json:"why"`
|
||||
Found []string `json:"found,omitempty"`
|
||||
}
|
||||
|
||||
// Health judges the shares.
|
||||
func (m *Machine) Health() Health {
|
||||
var found []string
|
||||
service := m.ServiceState()
|
||||
if service != "active" {
|
||||
found = append(found, fmt.Sprintf("The NFS server (%s) is %s.", Unit, service))
|
||||
}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
found = append(found, err.Error()+".")
|
||||
return Health{Healthy: false, Why: strings.Join(found, " "), Found: found}
|
||||
}
|
||||
if age := m.Now().Sub(st.At); age > Stale {
|
||||
found = append(found, fmt.Sprintf("The module's process last looked %s ago, at %s.", age.Round(time.Second), st.At.Local().Format("15:04")))
|
||||
}
|
||||
for _, p := range st.Problems {
|
||||
found = append(found, sentence(p))
|
||||
}
|
||||
for _, r := range st.Refused {
|
||||
found = append(found, r.Why)
|
||||
}
|
||||
if st.ReloadError != "" {
|
||||
found = append(found, sentence("the kernel could not read the export files again: "+st.ReloadError))
|
||||
}
|
||||
live, note := m.Live()
|
||||
for _, e := range st.Shares {
|
||||
if note == "" && !IsLive(live, e.Path, e.Clients) {
|
||||
found = append(found, fmt.Sprintf("The share %s (%s) is not in the kernel's export table.", e.Name, e.Path))
|
||||
}
|
||||
}
|
||||
sort.Strings(found)
|
||||
if len(found) > 0 {
|
||||
return Health{Healthy: false, Why: strings.Join(found, " "), Found: found}
|
||||
}
|
||||
names := make([]string, 0, len(st.Shares))
|
||||
for _, e := range st.Shares {
|
||||
names = append(names, e.Name)
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return Health{Healthy: true, Why: "The NFS server is active, and the mesh exports nothing here."}
|
||||
}
|
||||
return Health{Healthy: true, Why: fmt.Sprintf("The NFS server is active and exports %s to %s.", strings.Join(names, ", "), st.Range)}
|
||||
}
|
||||
|
||||
func sentence(s string) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if s == "" {
|
||||
return s
|
||||
}
|
||||
s = strings.ToUpper(s[:1]) + s[1:]
|
||||
if !strings.HasSuffix(s, ".") {
|
||||
s += "."
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
module nfsserver
|
||||
|
||||
go 1.22
|
||||
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.7
|
||||
@@ -0,0 +1,2 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"module": "nfs-server",
|
||||
"version": "1",
|
||||
"upgrade": {
|
||||
"policy": "record",
|
||||
"why": "the folders other machines mount: a build that breaks the exports leaves every client's mount hanging or refused, and the gate on this one machine does not see the clients (hq ADR 0236, ADR 0263)"
|
||||
},
|
||||
"capabilities": [
|
||||
"package-manager",
|
||||
"service-manager"
|
||||
],
|
||||
"provides": [
|
||||
{
|
||||
"name": "nfs-share",
|
||||
"scope": "mesh",
|
||||
"identity": false
|
||||
}
|
||||
],
|
||||
"data": {
|
||||
"consumers": {
|
||||
"nfs-share": {
|
||||
"class": "none",
|
||||
"why": "the shared folders are the operator's data (hq ADR 0051): what a client writes lands in them, and they are protected where the operator declares them, never by this module, which keeps nothing of a consumer's"
|
||||
}
|
||||
}
|
||||
},
|
||||
"claims": [
|
||||
{
|
||||
"name": "node-nfs-server",
|
||||
"scope": "node",
|
||||
"serves": [
|
||||
"exports",
|
||||
"clients",
|
||||
"test",
|
||||
"reload",
|
||||
"adopt"
|
||||
]
|
||||
}
|
||||
],
|
||||
"state": [
|
||||
{
|
||||
"name": "exports",
|
||||
"ttl-seconds": 120,
|
||||
"per-machine": true
|
||||
}
|
||||
],
|
||||
"tools": [
|
||||
"nfs_health"
|
||||
],
|
||||
"listens": [
|
||||
{
|
||||
"name": "nfs",
|
||||
"port": 2049,
|
||||
"protocol": "tcp",
|
||||
"from": "mesh",
|
||||
"fixed": true,
|
||||
"why": "the shares, to the mesh's machines only (hq ADR 0263): NFS version 4 alone, which needs no other port, and never the home network, where a device that is not a node could claim any user id"
|
||||
}
|
||||
],
|
||||
"resources": [
|
||||
{
|
||||
"id": "package",
|
||||
"type": "package",
|
||||
"package": "nfs-utils"
|
||||
},
|
||||
{
|
||||
"id": "nfs-conf",
|
||||
"type": "file",
|
||||
"path": "/etc/nfs.conf.d/50-mesh.conf",
|
||||
"mode": "0644",
|
||||
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263). Replaced on every push; a drop-in of\n# the operator's that sorts after this one overrides it, and is theirs.\n#\n# NFS version 4 only: a client needs port 2049 and nothing else, so the module opens nothing more\n# than that, to the private network. Version 3 needs rpcbind and mountd, on ports the mesh does not open.\n[nfsd]\nvers2=n\nvers3=n\nvers4=y\nvers4.0=n\nvers4.1=y\nvers4.2=y\n"
|
||||
},
|
||||
{
|
||||
"id": "config-dir",
|
||||
"type": "directory",
|
||||
"path": "/etc/nfs-server",
|
||||
"mode": "0755"
|
||||
},
|
||||
{
|
||||
"id": "config",
|
||||
"type": "file",
|
||||
"path": "/etc/nfs-server/shares.conf",
|
||||
"mode": "0644",
|
||||
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The module's process exports each\n# to the private network's range below, every client mapped to the folder's owner.\nshares=${setting:shares}\nrange=${machine:mesh-range}\n"
|
||||
},
|
||||
{
|
||||
"id": "run-dir",
|
||||
"type": "directory",
|
||||
"path": "/run/nfs-server",
|
||||
"mode": "0755"
|
||||
},
|
||||
{
|
||||
"id": "server",
|
||||
"type": "service",
|
||||
"unit": "nfs-server.service",
|
||||
"state": "running",
|
||||
"boot": "enabled",
|
||||
"restart-on": [
|
||||
"nfs-conf"
|
||||
],
|
||||
"health": {
|
||||
"kind": "unit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "exports",
|
||||
"type": "process",
|
||||
"name": "nfs-server-exports",
|
||||
"artifact": "tools",
|
||||
"run": [
|
||||
"./nfs-server",
|
||||
"exports"
|
||||
],
|
||||
"restart-on": [
|
||||
"config"
|
||||
],
|
||||
"health": {
|
||||
"kind": "tool",
|
||||
"tool": "nfs_health",
|
||||
"interval": "60s",
|
||||
"timeout": "10s",
|
||||
"looks": 2,
|
||||
"grace": "90s"
|
||||
}
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "tools",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/nfs-server",
|
||||
"binary": "nfs-server",
|
||||
"loads": [
|
||||
"nfs-server"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user