Add the nfs-server module: a node shares a folder with the mesh's machines only, every client mapped to the folder's owner (hq ADR 0263)
mesh/merge-gate fail: builds new: modules/mounts, modules/nfs-server, sent nowhere; no bus step; a manifest the change touches fails the module check: modu…
mesh/repo-check fail: its merge-check.sh failed: long-running resources without health: 70
mesh/delivery-group group feat/mounts-module rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request

This commit is contained in:
jochen
2026-10-08 18:36:01 +02:00
parent 2b1b9b0235
commit 48fba44471
12 changed files with 1918 additions and 0 deletions
+79
View File
@@ -0,0 +1,79 @@
# nfs-server
A node shares a folder with the mesh's machines, as a module (novox/hq ADR 0263, to-be 52). It holds
the node seat `node-nfs-server` and provides the provision `nfs-share`, which the `mounts` module on
another node requires.
## The rules it keeps
- **One share per folder.** The shares are the assignment's `shares` setting: `name=folder`, or
`name=folder:ro`, separated by spaces. The setting has no default: which folders a machine shares is
the operator's value (ADR 0112, ADR 0262).
- **To the mesh's machines only.** Every export line names the private network's range
(`${machine:mesh-range}`) and nothing else, and the module opens port 2049 to the mesh only. The home
network is never named.
- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one
id names different accounts on different machines. Every line is `all_squash` with `anonuid` and
`anongid` read from the folder's owner on this machine, so a file written from any node lands owned by
the folder's owner, and root is squashed with everyone else.
- **Never the backups.** A folder that is the machine's root, is not there, is not a folder, or holds a
backup (restic) repository at its path or one level below is refused, said by the tools, and makes the
module unhealthy.
- **The folders are the operator's** (ADR 0051). The module reads a folder's owner and exports it. It
never creates, chowns, chmods or removes anything under a shared path; its code writes two files of its
own and nothing else, and a test holds it to that.
- **NFS version 4 only**, so 2049 is the only port a client needs: rpcbind and mountd stay closed.
## What it owns
- the `nfs-utils` package and `nfs-server.service`, running and enabled;
- `/etc/nfs.conf.d/50-mesh.conf`: versions 2, 3 and 4.0 off, 4.1 and 4.2 on;
- `/etc/nfs-server/shares.conf`: the setting and the range, rendered by the mesh;
- `/etc/exports.d/mesh.exports`: written by its process, with a marker header;
- `/run/nfs-server/status.json`: the process's last look, for the tools;
- the state `exports` on the bus, one key per machine: what `exports` answers, for a consumer's module
and for a view (ADR 0201).
## How it runs
- **Its process** (`nfs-server exports`, as root) looks every 30 seconds. It reads the shares, judges each
folder, writes the export file when it changed, and runs `exportfs -ra` when the file changed or a share
is missing from the kernel's table. Then it writes its status.
- **Its tools** run as the operator account. They read the status and the kernel's table, and run what
only root may do (`exportfs -ra`, `zfs set`) through `sudo -n`, like the packet filter's tools.
- **Health** is the tool `nfs_health`, asked by the node-engine every minute: unhealthy when the NFS
server is not active, the process has not looked for five minutes, a share is refused or missing from
the kernel's table, or the kernel could not read the files again.
## Tools
| tool | | answers |
|---|---|---|
| `node-nfs-server.exports` | r | every share: name, folder, rw or ro, the owner every client is mapped to, the clients, whether it is live; refused shares and why; every export that is not the mesh's (`/etc/exports`, other files in `/etc/exports.d`, ZFS `sharenfs`) |
| `node-nfs-server.clients` | r | the NFS clients the kernel knows (address, name), or the connections to 2049 |
| `node-nfs-server.test` | r | whether one share is exported to the range now, the server is active, and an address is inside the range |
| `node-nfs-server.reload` | a | `exportfs -ra`, then the shares as the kernel holds them |
| `node-nfs-server.adopt` | a | take over a ZFS dataset's `sharenfs` export of a folder the mesh exports: `sharenfs=off` on that one dataset, then `exportfs -ra`; only when the mesh's export of the folder is live; a dry run without `confirm`; a person's act, with why |
| `nfs_health` | r | the module's health, one sentence per finding |
`exports` answers, for the `mounts` module:
```
{"shares": [{"name", "path", "access", "uid", "gid", "clients", "line", "live"}],
"refused": [{"name", "path", "why"}], "foreign": [{"path", "client", "options", "from"}],
"service": "active", "range", "looked", "said", "notes"}
```
## What it leaves
- **The export file after unassignment.** The process writes `/etc/exports.d/mesh.exports`, not the
node-engine, because each line carries the folder's owner as this machine reads it. When the module is
unassigned, the process stops and the file stays until removed. `exports` on any later holder, and the
`mounts` module's `mounts_exports`, show it.
- **A subfolder export shares its filesystem.** A share that is a folder inside a larger filesystem (not
a filesystem of its own) is exported with `no_subtree_check`, so a client that forges file handles could
reach other folders of that filesystem. Every client is a node and is squashed to the share's owner, so
a folder that owner cannot read stays closed. A share that must be strictly fenced is better given its
own dataset.
- **Exports that are not the mesh's** (`/etc/exports`, a ZFS dataset's `sharenfs`) are left as found and
shown, until `adopt` takes one over on a person's word.
@@ -0,0 +1,210 @@
package main
import (
"bytes"
"context"
"errors"
"fmt"
"io/fs"
"os"
"os/exec"
"path/filepath"
"strings"
"syscall"
"time"
)
// The machine as the module sees it (novox/hq ADR 0263).
//
// **Two processes, two accounts.** The node's tool runtime runs the tools as the operator account, not
// root (novox/hq ADR 0175 §4): they read, and what only root may do — `exportfs -ra`, `zfs set` — goes
// through `sudo -n`, as the packet filter's and the service manager's tools do. The module's long-running
// process runs as root: it renders the export file and writes the status the tools read. It cannot reach
// the bus, so the tools' bundle puts that status in the module's state.
//
// **It writes two files and nothing else.** The export file and the status file are the module's own.
// Nothing here can create, chown, chmod or remove anything under a shared path: the machine offers no
// such call, and Write refuses every path but the module's two (novox/hq ADR 0051).
// The module's files on the machine.
const (
// ConfigFile is what the mesh renders from the assignment: the shares and the private network's range.
ConfigFile = "/etc/nfs-server/shares.conf"
// ExportFile is the module's own export file, which the distribution's exportfs reads with the rest of
// /etc/exports.d.
ExportFile = "/etc/exports.d/mesh.exports"
// StatusFile is the process's last look, for the tools.
StatusFile = "/run/nfs-server/status.json"
// Etab is the kernel's export table as nfs-utils keeps it.
Etab = "/var/lib/nfs/etab"
// Unit is the NFS server's unit.
Unit = "nfs-server.service"
)
// CommandTimeout bounds every command. None touches a shared path's contents.
const CommandTimeout = 20 * time.Second
// Ran is what one command did.
type Ran struct {
Out string
Err string
Code int
}
// Runner runs one command; injected, so every tool is tested over recorded answers.
type Runner func(ctx context.Context, name string, args ...string) Ran
// ExecRunner runs a command on the machine, bounded by CommandTimeout.
func ExecRunner(ctx context.Context, name string, args ...string) Ran {
ctx, cancel := context.WithTimeout(ctx, CommandTimeout)
defer cancel()
cmd := exec.CommandContext(ctx, name, args...)
var stdout, stderr bytes.Buffer
cmd.Stdout, cmd.Stderr = &stdout, &stderr
err := cmd.Run()
r := Ran{Out: stdout.String(), Err: strings.TrimSpace(stderr.String())}
if err != nil {
var exit *exec.ExitError
if errors.As(err, &exit) {
r.Code = exit.ExitCode()
} else {
r.Code = -1
if r.Err == "" {
r.Err = err.Error()
}
}
}
return r
}
// Owner is who owns a path, and whether it is a directory.
type Owner struct {
UID, GID int
Dir bool
}
// Machine is a filesystem root (the real one, or a test's tree), a runner, who this process is, and a
// clock.
type Machine struct {
Root string
Run Runner
UID int
Now func() time.Time
}
// Here is the machine this process runs on.
func Here() *Machine {
return &Machine{Root: "/", Run: ExecRunner, UID: os.Getuid(), Now: time.Now}
}
func (m *Machine) at(path string) string { return filepath.Join(m.Root, path) }
// Read is one file's text.
func (m *Machine) Read(path string) (string, error) {
b, err := os.ReadFile(m.at(path))
return string(b), err
}
// Exists says whether a path is there.
func (m *Machine) Exists(path string) bool {
_, err := os.Lstat(m.at(path))
return err == nil
}
// Stat is a path's owner. It reads the inode and nothing else.
func (m *Machine) Stat(path string) (Owner, error) {
info, err := os.Stat(m.at(path))
if err != nil {
return Owner{}, err
}
o := Owner{Dir: info.IsDir()}
if st, ok := info.Sys().(*syscall.Stat_t); ok {
o.UID, o.GID = int(st.Uid), int(st.Gid)
}
return o, nil
}
// Children are the names of a directory's subdirectories. Used only to look for a backup repository one
// level below a share; it lists names and opens nothing.
func (m *Machine) Children(path string) []string {
entries, err := os.ReadDir(m.at(path))
if err != nil {
return nil
}
var out []string
for _, e := range entries {
if e.IsDir() {
out = append(out, e.Name())
}
}
return out
}
// Glob is the paths under the root matching a pattern, as the machine names them.
func (m *Machine) Glob(pattern string) []string {
found, _ := filepath.Glob(m.at(pattern))
out := make([]string, 0, len(found))
for _, f := range found {
rel, err := filepath.Rel(m.Root, f)
if err != nil {
continue
}
out = append(out, "/"+rel)
}
return out
}
// written are the only paths Write may touch.
var written = map[string]bool{ExportFile: true, StatusFile: true}
// Write replaces one of the module's own files, atomically. Every other path is refused: the module owns
// these two files and nothing else on the machine.
func (m *Machine) Write(path, content string) error {
if !written[path] {
return fmt.Errorf("the module writes only %s and %s, not %s", ExportFile, StatusFile, path)
}
full := m.at(path)
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil && !errors.Is(err, fs.ErrExist) {
return err
}
tmp := full + ".tmp"
if err := os.WriteFile(tmp, []byte(content), 0o644); err != nil {
return err
}
return os.Rename(tmp, full)
}
// Escalated is a command as it is run: as given when this process is root, else through sudo without a
// prompt.
func (m *Machine) Escalated(name string, args ...string) (string, []string) {
if m.UID == 0 {
return name, args
}
return "sudo", append([]string{"-n", name}, args...)
}
// AsRoot runs what only root may do, and names a refusal by how it failed.
func (m *Machine) AsRoot(name string, args ...string) (Ran, error) {
program, argv := m.Escalated(name, args...)
r := m.Run(context.Background(), program, argv...)
if r.Code != 0 {
why := r.Err
if why == "" {
why = strings.TrimSpace(r.Out)
}
if program == "sudo" && (strings.Contains(why, "a password is required") || strings.Contains(why, "not allowed")) {
return r, fmt.Errorf("%s needs root, and sudo refused the operator account without a prompt: %s", name, why)
}
return r, fmt.Errorf("%s %s: exit %d: %s", name, strings.Join(args, " "), r.Code, why)
}
return r, nil
}
// ServiceState is what the service manager says of the NFS server: active, inactive, failed, …
func (m *Machine) ServiceState() string {
r := m.Run(context.Background(), "systemctl", "is-active", Unit)
if s := strings.TrimSpace(r.Out); s != "" {
return s
}
return "unknown"
}
+74
View File
@@ -0,0 +1,74 @@
// The nfs-server module's Go bundle (novox/hq ADR 0263): a node shares a folder with the mesh's machines
// only, one share per folder, every client mapped to the folder's owner.
//
// Started with no arguments it is served by the node's runtime as the operator account, over MCP on stdio
// (ADR 0188, ADR 0193): the node-nfs-server seat's verbs and the module's health, and beside them it puts
// the process's last look in the module's state `exports`, keyed by this machine (ADR 0201). Started as
// `nfs-server exports` it is the module's long-running process, as root: it renders the export file from
// the assignment's shares and has the kernel read it (process.go). stdout is the MCP channel; everything
// else is said on stderr.
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// ExportsState is the state the exports are kept in on the bus.
const ExportsState = "exports"
func main() {
if len(os.Args) > 1 {
if os.Args[1] != "exports" || len(os.Args) != 2 {
fmt.Fprintln(os.Stderr, "usage: nfs-server [exports]")
os.Exit(2)
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt)
defer stop()
if err := Here().RunExports(ctx); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
m := Here()
go ShareExports(m)
if err := stdio.Serve("", Tools(m)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// ShareExports puts the exports in the module's state every Every, for as long as the bundle runs, so a
// consumer's module and a view read them from the bus rather than from this machine.
func ShareExports(m *Machine) {
machine := os.Getenv("MESH_NODE")
if machine == "" {
fmt.Fprintln(os.Stderr, "nfs-server: the runtime gave no machine name (MESH_NODE); the exports are not shared")
return
}
last := ""
put := func() {
_, err := stdio.State(ExportsState).Put(machine, m.Exports())
said := "put"
if err != nil {
said = err.Error()
}
if said != last {
fmt.Fprintln(os.Stderr, "nfs-server: exports state:", said)
}
last = said
}
// The SDK reaches the bus once Serve runs.
time.Sleep(2 * time.Second)
put()
for range time.Tick(Every) {
put()
}
}
@@ -0,0 +1,118 @@
package main
import (
"encoding/json"
"os"
"regexp"
"sort"
"strings"
"testing"
)
type manifest struct {
Tools []string `json:"tools"`
Claims []struct {
Name string `json:"name"`
Scope string `json:"scope"`
Serves []string `json:"serves"`
} `json:"claims"`
Provides []map[string]any `json:"provides"`
Listens []struct {
Port int `json:"port"`
Protocol string `json:"protocol"`
From string `json:"from"`
} `json:"listens"`
Settings map[string]any `json:"settings"`
Resources []map[string]any `json:"resources"`
}
func readManifest(t *testing.T) (manifest, string) {
t.Helper()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
return m, string(raw)
}
// The seat's verbs are served as `node-nfs-server.<verb>`, the health tool is listed, and every tool says
// whether it reads or acts.
func TestTheSeatsVerbsAreAllServedAndTheHealthToolListed(t *testing.T) {
m, _ := readManifest(t)
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || m.Claims[0].Scope != "node" {
t.Fatalf("claims: %+v", m.Claims)
}
var verbs, tools []string
for _, tl := range Tools(newFake(t).machine()) {
if name, ok := strings.CutPrefix(tl.Name, Seat+"."); ok {
verbs = append(verbs, name)
} else {
tools = append(tools, tl.Name)
}
if !strings.HasSuffix(tl.Description, "(r)") && !strings.HasSuffix(tl.Description, "(a)") {
t.Errorf("%s does not say whether it reads or acts", tl.Name)
}
}
sort.Strings(verbs)
serves := append([]string(nil), m.Claims[0].Serves...)
sort.Strings(serves)
if strings.Join(verbs, ",") != strings.Join(serves, ",") {
t.Errorf("served %v, the claim lists %v", verbs, serves)
}
if strings.Join(tools, ",") != strings.Join(m.Tools, ",") {
t.Errorf("tools %v, listed %v", tools, m.Tools)
}
}
// Port 2049 to the mesh's machines and nothing else; the provision is the mesh's, keeping no name of its
// consumers.
func TestItListensOnTwoThousandFortyNineFromTheMeshOnlyAndProvidesTheShare(t *testing.T) {
m, raw := readManifest(t)
if len(m.Listens) != 1 || m.Listens[0].Port != 2049 || m.Listens[0].Protocol != "tcp" || m.Listens[0].From != "mesh" {
t.Errorf("listens: %+v", m.Listens)
}
if len(m.Provides) != 1 || m.Provides[0]["name"] != "nfs-share" || m.Provides[0]["scope"] != "mesh" || m.Provides[0]["identity"] != false {
t.Errorf("provides: %+v", m.Provides)
}
for _, port := range []string{"111", "20048", "\"port\": 445"} {
if strings.Contains(raw, `"port": `+port) {
t.Errorf("the module opens %s", port)
}
}
if !strings.Contains(raw, "vers3=n") {
t.Errorf("NFS version 3 is not switched off, so 2049 alone would not do")
}
}
// The definition names no folder, no machine and no person (novox/hq ADR 0112): the shares are the
// assignment's setting, and the range the mesh's.
func TestTheDefinitionNamesNoFolderNoMachineAndGivesTheSharesNoDefault(t *testing.T) {
m, raw := readManifest(t)
if hit := regexp.MustCompile(`\b(g14|novox|ace|shanks|jochen)\b|/home/|/storage|/srv/`).FindString(strings.ReplaceAll(raw, "novox/hq", "")); hit != "" {
t.Errorf("the manifest says %q", hit)
}
if _, declared := m.Settings["shares"]; declared {
t.Errorf("the shares are the operator's value and have no default (ADR 0262)")
}
var config string
for _, r := range m.Resources {
if r["id"] == "config" {
config, _ = r["content"].(string)
}
}
if !strings.Contains(config, "shares=${setting:shares}") || !strings.Contains(config, "range=${machine:mesh-range}") {
t.Errorf("the config file: %q", config)
}
// Every long-running resource says how it is ready (ADR 0240 rule 8).
for _, r := range m.Resources {
if r["type"] == "service" || r["type"] == "process" {
if r["health"] == nil {
t.Errorf("%v has no health", r["id"])
}
}
}
}
@@ -0,0 +1,389 @@
package main
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// fake is a machine under a temporary root, with recorded command answers.
type fake struct {
t *testing.T
root string
answers map[string]Ran
ran []string
now time.Time
uid int
// onExportfs, when set, writes the kernel's table as exportfs -ra would.
onExportfs func()
}
func newFake(t *testing.T) *fake {
f := &fake{t: t, root: t.TempDir(), answers: map[string]Ran{}, now: time.Date(2026, 10, 8, 19, 21, 0, 0, time.UTC), uid: 0}
f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "active\n"}
f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Code: -1, Err: `exec: "zfs": executable file not found in $PATH`}
return f
}
func (f *fake) machine() *Machine {
return &Machine{Root: f.root, UID: f.uid, Now: func() time.Time { return f.now }, Run: func(_ context.Context, name string, args ...string) Ran {
line := strings.Join(append([]string{name}, args...), " ")
f.ran = append(f.ran, line)
if (line == "exportfs -ra" || line == "sudo -n exportfs -ra") && f.onExportfs != nil {
f.onExportfs()
}
if r, ok := f.answers[line]; ok {
return r
}
return Ran{}
}}
}
func (f *fake) dir(p string) {
if err := os.MkdirAll(filepath.Join(f.root, p), 0o755); err != nil {
f.t.Fatal(err)
}
}
func (f *fake) file(p, content string) {
f.dir(filepath.Dir(p))
if err := os.WriteFile(filepath.Join(f.root, p), []byte(content), 0o644); err != nil {
f.t.Fatal(err)
}
}
func (f *fake) read(p string) string {
b, _ := os.ReadFile(filepath.Join(f.root, p))
return string(b)
}
func (f *fake) restic(p string) {
f.file(p+"/config", "x")
for _, d := range []string{"keys", "snapshots", "data"} {
f.dir(p + "/" + d)
}
}
// kernelReads makes exportfs -ra copy the mesh's export file into the kernel's table, as it would.
func (f *fake) kernelReads() {
f.onExportfs = func() {
var b strings.Builder
for _, e := range ParseExports(f.read(ExportFile), ExportFile) {
fmt.Fprintf(&b, "%s\t%s(%s)\n", e.Path, e.Client, e.Options)
}
f.file(Etab, b.String())
}
}
const meshRange = "10.0.0.0/24"
func config(shares string) string {
return "# test\nshares=" + shares + "\nrange=" + meshRange + "\n"
}
func TestAnExportLineSquashesEveryClientToTheFoldersOwnerToTheMeshRangeOnly(t *testing.T) {
f := newFake(t)
f.dir("/srv/library")
f.dir("/srv/downloads")
f.file(ConfigFile, config("library=/srv/library downloads=/srv/downloads:ro"))
f.kernelReads()
s := f.machine().Pass(context.Background(), nil)
if len(s.Shares) != 2 || len(s.Refused) != 0 {
t.Fatalf("status: %+v", s)
}
uid, gid := os.Getuid(), os.Getgid()
text := f.read(ExportFile)
for _, want := range []string{
fmt.Sprintf("/srv/library %s(rw,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid),
fmt.Sprintf("/srv/downloads %s(ro,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)", meshRange, uid, gid),
} {
if !strings.Contains(text, want+"\n") {
t.Errorf("the export file lacks %q:\n%s", want, text)
}
}
for _, e := range ParseExports(text, ExportFile) {
if e.Client != meshRange {
t.Errorf("a client other than the private network's range: %+v", e)
}
if !strings.Contains(e.Options, "all_squash") || strings.Contains(e.Options, "no_root_squash") {
t.Errorf("every client is squashed: %+v", e)
}
}
if !s.Shares[0].Live || !s.Shares[1].Live {
t.Errorf("both shares are live after exportfs -ra: %+v", s.Shares)
}
if s.Reloaded == nil || s.ReloadWhy != "the export file changed" {
t.Errorf("the kernel read the file again because it changed: %+v", s)
}
// Rendering is stable: a second pass changes nothing and asks the kernel nothing.
f.ran = nil
f.machine().Pass(context.Background(), &s)
for _, r := range f.ran {
if strings.Contains(r, "exportfs") {
t.Errorf("a pass with nothing changed reloaded: %v", f.ran)
}
}
}
func TestTheRootABackupRepositoryAndAMissingFolderAreRefusedAndNothingIsCreated(t *testing.T) {
f := newFake(t)
f.restic("/srv/pool/restic")
f.dir("/srv/pool/media")
f.restic("/srv/backups")
f.file("/srv/a-file", "x")
f.file(ConfigFile, config("root=/ pool=/srv/pool backups=/srv/backups missing=/srv/nowhere file=/srv/a-file rel=srv/x media=/srv/pool/media"))
f.kernelReads()
s := f.machine().Pass(context.Background(), nil)
refused := map[string]string{}
for _, r := range s.Refused {
refused[r.Name] = r.Why
}
for name, word := range map[string]string{
"root": "root", "pool": "holds a backup repository", "backups": "is a backup repository",
"missing": "not there", "file": "not a folder", "rel": "not an absolute path",
} {
if !strings.Contains(refused[name], word) {
t.Errorf("%s refused with %q, want %q", name, refused[name], word)
}
}
if len(s.Shares) != 1 || s.Shares[0].Name != "media" {
t.Errorf("only the folder beside the backups is exported: %+v", s.Shares)
}
if strings.Contains(f.read(ExportFile), "/srv/pool ") || strings.Contains(f.read(ExportFile), "/srv/backups") {
t.Errorf("a refused folder is in the export file:\n%s", f.read(ExportFile))
}
if _, err := os.Stat(filepath.Join(f.root, "/srv/nowhere")); err == nil {
t.Errorf("the module created a missing folder")
}
h := f.machine().Health()
if h.Healthy || !strings.Contains(h.Why, "backups are never exported") {
t.Errorf("a refused share makes the module unhealthy, said: %+v", h)
}
}
func TestTheModuleWritesOnlyItsTwoFilesAndHasNoWayToChangeAnOwnerOrMode(t *testing.T) {
m := newFake(t).machine()
for _, p := range []string{"/srv/library/x", "/etc/exports", "/etc/exports.d/zfs.exports", ConfigFile} {
if err := m.Write(p, "x"); err == nil {
t.Errorf("Write accepted %s", p)
}
}
// No code path changes an owner or a mode, or removes anything: the words are not in the bundle.
files, _ := filepath.Glob("*.go")
for _, file := range files {
if strings.HasSuffix(file, "_test.go") {
continue
}
raw, _ := os.ReadFile(file)
for _, word := range []string{"Chown", "Lchown", "Chmod", "os.Remove", "RemoveAll", `"chown"`, `"chmod"`, `"rm"`, "Mkdir(", "MkdirAll(filepath.Join(m.Root"} {
if strings.Contains(string(raw), word) {
t.Errorf("%s says %s", file, word)
}
}
}
}
func TestParseSharesAndConfig(t *testing.T) {
shares, problems := ParseShares("library=/srv/library, downloads=/srv/downloads:ro Bad=/x noequals library=/again")
if len(shares) != 2 || shares[0].Name != "downloads" || !shares[0].ReadOnly || shares[1].Access != "rw" {
t.Errorf("shares: %+v", shares)
}
if len(problems) != 3 {
t.Errorf("problems: %v", problems)
}
c, problems := ParseConfig("shares=\nrange=not-a-range\n")
if c.Range != "" || len(problems) != 1 {
t.Errorf("a range that is not one is said: %+v %v", c, problems)
}
_, problems = ParseConfig("shares=a=/x\n")
if len(problems) != 1 || !strings.Contains(problems[0], "no private network range") {
t.Errorf("no range is said: %v", problems)
}
}
func TestParseExportsReadsTheKernelsTableAndZFSFile(t *testing.T) {
etab := "/srv/library\t192.0.2.0/24(rw,sync,wdelay,root_squash,sec=sys)\n/srv/x\n# comment\n/srv/y a(ro) b(rw)\n"
got := ParseExports(etab, Etab)
if len(got) != 4 || got[0].Client != "192.0.2.0/24" || got[1].Client != "*" || got[3].Client != "b" || got[3].Options != "rw" {
t.Errorf("%+v", got)
}
}
func zfsWith(f *fake, sharenfs string) {
f.answers["zfs list -H -o name,mountpoint,sharenfs -t filesystem"] = Ran{Out: "pool\t/srv/pool\toff\npool/library\t/srv/library\t" + sharenfs + "\n"}
}
func TestAdoptIsADryRunWithoutConfirmAndRefusesWhileTheMeshExportIsNotLive(t *testing.T) {
f := newFake(t)
f.uid = 1000
f.dir("/srv/library")
zfsWith(f, "rw=@192.0.2.0/24")
f.file(ConfigFile, config("library=/srv/library"))
// The kernel has not read the mesh's file: no exportfs effect.
f.machine().Pass(context.Background(), nil)
if _, err := f.machine().Adopt("/srv/library", true, "take it over"); err == nil || !strings.Contains(err.Error(), "not live") {
t.Fatalf("adopt while not live: %v", err)
}
f.kernelReads()
f.machine().Pass(context.Background(), nil)
f.ran = nil
dry, err := f.machine().Adopt("/srv/library", false, "")
if err != nil || dry.Done || !dry.DryRun || !strings.Contains(dry.Said, "would set sharenfs=off on the dataset pool/library") {
t.Fatalf("dry run: %+v %v", dry, err)
}
for _, r := range f.ran {
if strings.Contains(r, "zfs set") {
t.Fatalf("a dry run changed something: %v", f.ran)
}
}
if _, err := f.machine().Adopt("/srv/library", true, ""); err == nil {
t.Errorf("adopt with confirm and no why was done")
}
done, err := f.machine().Adopt("/srv/library", true, "the mesh exports it now")
if err != nil || !done.Done {
t.Fatalf("adopt: %+v %v", done, err)
}
var set []string
for _, r := range f.ran {
if strings.Contains(r, "zfs set") {
set = append(set, r)
}
}
if len(set) != 1 || set[0] != "sudo -n zfs set sharenfs=off pool/library" {
t.Errorf("exactly one property of exactly one dataset, through sudo: %v", set)
}
zfsWith(f, "off")
if _, err := f.machine().Adopt("/srv/library", true, "again"); err == nil || !strings.Contains(err.Error(), "already has sharenfs off") {
t.Errorf("adopt of a dataset already off: %v", err)
}
if _, err := f.machine().Adopt("/srv/elsewhere", false, ""); err == nil {
t.Errorf("adopt of a folder no dataset is mounted at")
}
}
func TestAdoptRefusesAFolderTheMeshDoesNotExport(t *testing.T) {
f := newFake(t)
f.dir("/srv/library")
zfsWith(f, "on")
f.file(ConfigFile, config(""))
f.kernelReads()
f.machine().Pass(context.Background(), nil)
if _, err := f.machine().Adopt("/srv/library", false, ""); err == nil || !strings.Contains(err.Error(), "does not export") {
t.Errorf("%v", err)
}
}
func TestHealthSaysAStoppedServerAStaleProcessAndAShareNotInTheTable(t *testing.T) {
f := newFake(t)
f.dir("/srv/library")
f.file(ConfigFile, config("library=/srv/library"))
f.kernelReads()
f.machine().Pass(context.Background(), nil)
if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "exports library to 10.0.0.0/24") {
t.Errorf("healthy: %+v", h)
}
f.file(Etab, "")
f.answers["systemctl is-active nfs-server.service"] = Ran{Out: "failed\n", Code: 3}
f.now = f.now.Add(10 * time.Minute)
h := f.machine().Health()
for _, want := range []string{"is failed", "last looked 10m0s ago", "The share library (/srv/library) is not in the kernel's export table."} {
if h.Healthy || !strings.Contains(h.Why, want) {
t.Errorf("health lacks %q: %+v", want, h)
}
}
if h := newFake(t).machine().Health(); h.Healthy || !strings.Contains(h.Why, "written no status") {
t.Errorf("no status is unhealthy: %+v", h)
}
}
func TestExportsAnswersTheSharesAndWhatIsNotTheMeshs(t *testing.T) {
f := newFake(t)
f.dir("/srv/library")
f.file(ConfigFile, config("library=/srv/library"))
f.file("/etc/exports.d/zfs.exports", "# !!! DO NOT EDIT\n/srv/library 192.0.2.0/24(sec=sys,rw,no_subtree_check)\n")
f.file("/etc/exports", "# only comments\n")
zfsWith(f, "rw=@192.0.2.0/24")
f.kernelReads()
f.machine().Pass(context.Background(), nil)
a := f.machine().Exports()
raw, _ := json.Marshal(a)
var shape struct {
Shares []struct {
Name, Path, Access, Clients string
UID, GID int
Live bool
}
Foreign []map[string]any
Service string
}
if err := json.Unmarshal(raw, &shape); err != nil {
t.Fatal(err)
}
if len(shape.Shares) != 1 || shape.Shares[0].Name != "library" || !shape.Shares[0].Live || shape.Shares[0].Clients != meshRange || shape.Service != "active" {
t.Errorf("%s", raw)
}
if len(shape.Foreign) != 2 {
t.Errorf("the ZFS file's line and the dataset's property are not the mesh's: %s", raw)
}
for _, key := range []string{`"shares"`, `"foreign"`, `"service"`, `"uid"`, `"gid"`, `"live"`, `"clients"`, `"access"`} {
if !strings.Contains(string(raw), key) {
t.Errorf("the answer lacks %s, which the mounts module reads", key)
}
}
}
func TestTestSaysWhetherAShareIsServedAndWhetherAnAddressIsInRange(t *testing.T) {
f := newFake(t)
f.dir("/srv/library")
f.file(ConfigFile, config("library=/srv/library"))
f.kernelReads()
f.machine().Pass(context.Background(), nil)
ok, _ := f.machine().Test("library", "10.0.0.3")
if !ok.OK || ok.InRange == nil || !*ok.InRange {
t.Errorf("%+v", ok)
}
out, _ := f.machine().Test("library", "192.0.2.7")
if out.OK || !strings.Contains(out.Why, "not inside the private network's range") {
t.Errorf("%+v", out)
}
none, _ := f.machine().Test("photos", "")
if none.OK || !strings.Contains(none.Why, "no share named photos") {
t.Errorf("%+v", none)
}
}
func TestClientsReadTheKernelsListOrFallBackToConnections(t *testing.T) {
f := newFake(t)
f.file("/proc/fs/nfsd/clients/7/info", "clientid: 0xabc\naddress: \"10.0.0.3:785\"\nname: \"Linux NFSv4.2 workstation\"\nminor version: 2\n")
a := f.machine().Clients()
if len(a.Clients) != 1 || a.Clients[0].Address != "10.0.0.3:785" || a.Clients[0].Minor != "2" {
t.Errorf("%+v", a)
}
g := newFake(t)
g.answers["ss -tnH state established ( sport = :2049 )"] = Ran{Out: "0 0 10.0.0.2:2049 10.0.0.3:785\n"}
b := g.machine().Clients()
if len(b.Clients) != 1 || b.Clients[0].Address != "10.0.0.3:785" || b.From != "connections to port 2049" {
t.Errorf("%+v", b)
}
}
func TestAnActNeedingRootGoesThroughSudoOnlyWhenNotRoot(t *testing.T) {
f := newFake(t)
f.uid = 1000
f.machine().AsRoot("exportfs", "-ra")
f.uid = 0
f.machine().AsRoot("exportfs", "-ra")
if len(f.ran) != 2 || f.ran[0] != "sudo -n exportfs -ra" || f.ran[1] != "exportfs -ra" {
t.Errorf("%v", f.ran)
}
f.answers["sudo -n zfs set sharenfs=off x"] = Ran{Code: 1, Err: "sudo: a password is required"}
f.uid = 1000
if _, err := f.machine().AsRoot("zfs", "set", "sharenfs=off", "x"); err == nil || !strings.Contains(err.Error(), "sudo refused") {
t.Errorf("%v", err)
}
}
@@ -0,0 +1,114 @@
package main
import (
"context"
"encoding/json"
"fmt"
"time"
)
// The module's process, as root: every Every it reads the assignment's shares, renders the export file,
// has the kernel read it again when anything changed or a share is not live, and writes what it saw for
// the tools (novox/hq ADR 0263).
// Every is how often the process looks.
const Every = 30 * time.Second
// Status is the process's last look, as the tools read it.
type Status struct {
At time.Time `json:"at"`
Range string `json:"range"`
Shares []Exported `json:"shares"`
Refused []Refused `json:"refused,omitempty"`
Problems []string `json:"problems,omitempty"`
// Reloaded is when the process last had the kernel read the export files again, and why.
Reloaded *time.Time `json:"reloaded,omitempty"`
ReloadWhy string `json:"reload_why,omitempty"`
ReloadError string `json:"reload_error,omitempty"`
}
// Pass is one look: what it did is in the status it answers, which it has written.
func (m *Machine) Pass(ctx context.Context, last *Status) Status {
s := Status{At: m.Now().UTC()}
if last != nil {
s.Reloaded, s.ReloadWhy = last.Reloaded, last.ReloadWhy
}
text, err := m.Read(ConfigFile)
if err != nil {
s.Problems = append(s.Problems, fmt.Sprintf("the assignment's shares (%s) could not be read: %v; nothing is exported", ConfigFile, err))
}
c, problems := ParseConfig(text)
s.Problems = append(s.Problems, problems...)
s.Range = c.Range
exports, refused := m.Judge(c)
s.Refused = refused
want := ExportFileText(exports)
have, _ := m.Read(ExportFile)
reload := ""
if have != want {
if err := m.Write(ExportFile, want); err != nil {
s.Problems = append(s.Problems, fmt.Sprintf("the export file could not be written: %v", err))
} else {
reload = "the export file changed"
}
}
live, note := m.Live()
if reload == "" {
for _, e := range exports {
if !IsLive(live, e.Path, e.Clients) {
reload = fmt.Sprintf("the share %s was not in the kernel's table", e.Name)
break
}
}
}
if reload != "" {
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
s.ReloadError = err.Error()
} else {
at := m.Now().UTC()
s.Reloaded, s.ReloadWhy = &at, reload
}
live, note = m.Live()
}
if note != "" && len(exports) > 0 {
s.Problems = append(s.Problems, note)
}
for i := range exports {
exports[i].Live = IsLive(live, exports[i].Path, exports[i].Clients)
}
s.Shares = exports
raw, _ := json.MarshalIndent(s, "", " ")
if err := m.Write(StatusFile, string(raw)+"\n"); err != nil {
s.Problems = append(s.Problems, fmt.Sprintf("the status file could not be written: %v", err))
}
return s
}
// RunExports is the process: a pass now and every Every until the context ends.
func (m *Machine) RunExports(ctx context.Context) error {
last := m.Pass(ctx, nil)
t := time.NewTicker(Every)
defer t.Stop()
for {
select {
case <-ctx.Done():
return nil
case <-t.C:
last = m.Pass(ctx, &last)
}
}
}
// LastStatus is the process's last look, or why there is none.
func (m *Machine) LastStatus() (*Status, error) {
text, err := m.Read(StatusFile)
if err != nil {
return nil, fmt.Errorf("the module's process has written no status (%s): %v", StatusFile, err)
}
var s Status
if err := json.Unmarshal([]byte(text), &s); err != nil {
return nil, fmt.Errorf("the module's status file is not one: %v", err)
}
return &s, nil
}
+225
View File
@@ -0,0 +1,225 @@
package main
import (
"fmt"
"net/netip"
"path"
"sort"
"strings"
)
// The shares, from the assignment, and the export lines made from them (novox/hq ADR 0263).
//
// **One share per folder.** A share is a name and a path the operator gives in the `shares` setting:
// `library=/srv/library downloads=/srv/downloads`, `:ro` after a path for read-only. The path is the
// operator's data (novox/hq ADR 0051): the module reads its owner and exports it, and never creates,
// chowns, chmods or removes it.
//
// **Every client is the folder's owner.** NFS with sec=sys believes the user id the client sends, and the
// same id names different accounts on different machines. So every line squashes every client
// (`all_squash`) to the owner of the shared folder as this machine reads it (`anonuid`, `anongid`): a file
// written from any node lands owned by the folder's owner, and no client acts as any other account here.
// Root is squashed with everyone else.
//
// **To the mesh's machines only.** The clients are the private network's range and nothing else: every
// address there is a node. The home network is never named.
//
// **Never the backups.** A share that is the machine's root, that does not exist, that is not a
// directory, or that holds a backup repository at its path or one level below is refused, and said.
// Share is one share the operator asked for.
type Share struct {
Name string `json:"name"`
Path string `json:"path"`
ReadOnly bool `json:"-"`
Access string `json:"access"`
}
// Exported is a share as this machine exports it.
type Exported struct {
Name string `json:"name"`
Path string `json:"path"`
Access string `json:"access"`
UID int `json:"uid"`
GID int `json:"gid"`
Clients string `json:"clients"`
Line string `json:"line"`
Live bool `json:"live"`
}
// Refused is a share that is not exported, and why.
type Refused struct {
Name string `json:"name"`
Path string `json:"path"`
Why string `json:"why"`
}
// Config is what the mesh rendered from the assignment.
type Config struct {
Shares []Share
Range string
}
// ParseConfig reads the config file: `shares=` and `range=` lines, comments and blanks ignored.
func ParseConfig(text string) (Config, []string) {
var c Config
var problems []string
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
key, value, ok := strings.Cut(line, "=")
if !ok {
problems = append(problems, fmt.Sprintf("the line %q of %s says nothing this module reads", line, ConfigFile))
continue
}
switch strings.TrimSpace(key) {
case "shares":
shares, bad := ParseShares(value)
c.Shares = shares
problems = append(problems, bad...)
case "range":
c.Range = strings.TrimSpace(value)
default:
problems = append(problems, fmt.Sprintf("%s names %q, which this module does not read", ConfigFile, key))
}
}
if c.Range == "" {
problems = append(problems, fmt.Sprintf("%s gives no private network range, so nothing can be exported", ConfigFile))
} else if _, err := netip.ParsePrefix(c.Range); err != nil {
problems = append(problems, fmt.Sprintf("the private network's range %q is not a range: %v", c.Range, err))
c.Range = ""
}
return c, problems
}
// ParseShares reads the `shares` setting: `name=path` or `name=path:ro`, separated by spaces or commas.
func ParseShares(value string) ([]Share, []string) {
var shares []Share
var problems []string
seen := map[string]bool{}
for _, item := range strings.FieldsFunc(value, func(r rune) bool { return r == ' ' || r == ',' || r == '\t' }) {
name, p, ok := strings.Cut(item, "=")
if !ok || name == "" || p == "" {
problems = append(problems, fmt.Sprintf("the share %q is not name=path", item))
continue
}
if !validName(name) {
problems = append(problems, fmt.Sprintf("the share name %q is not lower-case letters, digits and dashes", name))
continue
}
if seen[name] {
problems = append(problems, fmt.Sprintf("the share %q is named twice", name))
continue
}
seen[name] = true
s := Share{Name: name, Path: p, Access: "rw"}
if strings.HasSuffix(p, ":ro") {
s.Path, s.ReadOnly, s.Access = strings.TrimSuffix(p, ":ro"), true, "ro"
} else if strings.HasSuffix(p, ":rw") {
s.Path = strings.TrimSuffix(p, ":rw")
}
shares = append(shares, s)
}
sort.Slice(shares, func(i, j int) bool { return shares[i].Name < shares[j].Name })
return shares, problems
}
func validName(n string) bool {
if n == "" || n[0] == '-' {
return false
}
for _, r := range n {
if !(r >= 'a' && r <= 'z' || r >= '0' && r <= '9' || r == '-') {
return false
}
}
return true
}
// IsBackupRepository says whether a directory is a restic repository: a `config` file beside `keys`,
// `snapshots` and `data`. It looks at names only.
func (m *Machine) IsBackupRepository(dir string) bool {
if !m.Exists(path.Join(dir, "config")) {
return false
}
for _, sub := range []string{"keys", "snapshots", "data"} {
if o, err := m.Stat(path.Join(dir, sub)); err != nil || !o.Dir {
return false
}
}
return true
}
// Judge turns the asked-for shares into export lines, or refusals.
func (m *Machine) Judge(c Config) ([]Exported, []Refused) {
var out []Exported
var refused []Refused
for _, s := range c.Shares {
no := func(why string) { refused = append(refused, Refused{Name: s.Name, Path: s.Path, Why: why}) }
clean := path.Clean(s.Path)
switch {
case !path.IsAbs(s.Path):
no(fmt.Sprintf("The share %s names %q, which is not an absolute path.", s.Name, s.Path))
continue
case clean == "/":
no(fmt.Sprintf("The share %s is the machine's root, which is never exported.", s.Name))
continue
case strings.ContainsAny(s.Path, " \t\"\\#"):
no(fmt.Sprintf("The share %s's path %q has a character an export line cannot carry.", s.Name, s.Path))
continue
}
o, err := m.Stat(clean)
if err != nil {
no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, clean, err))
continue
}
if !o.Dir {
no(fmt.Sprintf("The share %s's path %s is not a folder.", s.Name, clean))
continue
}
if m.IsBackupRepository(clean) {
no(fmt.Sprintf("The share %s's folder %s is a backup repository, and backups are never exported.", s.Name, clean))
continue
}
holds := ""
for _, child := range m.Children(clean) {
if m.IsBackupRepository(path.Join(clean, child)) {
holds = path.Join(clean, child)
break
}
}
if holds != "" {
no(fmt.Sprintf("The share %s's folder %s holds a backup repository (%s), and backups are never exported: share the folders beside it instead.", s.Name, clean, holds))
continue
}
if c.Range == "" {
no(fmt.Sprintf("The share %s is not exported: the private network's range is not known.", s.Name))
continue
}
e := Exported{Name: s.Name, Path: clean, Access: s.Access, UID: o.UID, GID: o.GID, Clients: c.Range}
e.Line = ExportLine(e)
out = append(out, e)
}
return out, refused
}
// ExportLine is one share's line in the export file.
func ExportLine(e Exported) string {
return fmt.Sprintf("%s %s(%s,sync,secure,no_subtree_check,all_squash,anonuid=%d,anongid=%d)",
e.Path, e.Clients, e.Access, e.UID, e.GID)
}
// ExportFileText is the whole export file.
func ExportFileText(exports []Exported) string {
var b strings.Builder
b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered from its assignment's\n")
b.WriteString("# `shares` setting by the module's own process; edit the setting, never this file.\n")
b.WriteString("# One share per folder, to the private network's range only, every client squashed to the\n")
b.WriteString("# folder's owner.\n")
for _, e := range exports {
fmt.Fprintf(&b, "# share %s\n%s\n", e.Name, e.Line)
}
return b.String()
}
+133
View File
@@ -0,0 +1,133 @@
package main
import (
"context"
"path"
"strings"
)
// What the machine exports, read: the kernel's table, the other export files, and ZFS's property.
// Entry is one export: a path, one client, its options, and where it was found.
type Entry struct {
Path string `json:"path"`
Client string `json:"client"`
Options string `json:"options,omitempty"`
From string `json:"from"`
}
// ParseExports reads an exports(5) file or the kernel's etab: a path, then clients each with options.
func ParseExports(text, from string) []Entry {
var out []Entry
for _, line := range strings.Split(text, "\n") {
if i := strings.Index(line, "#"); i >= 0 {
line = line[:i]
}
fields := strings.Fields(line)
if len(fields) < 1 {
continue
}
p := fields[0]
if len(fields) == 1 {
out = append(out, Entry{Path: p, Client: "*", From: from})
continue
}
for _, c := range fields[1:] {
client, opts := c, ""
if i := strings.Index(c, "("); i >= 0 {
client, opts = c[:i], strings.TrimSuffix(c[i+1:], ")")
}
if client == "" {
client = "*"
}
out = append(out, Entry{Path: p, Client: client, Options: opts, From: from})
}
}
return out
}
// Live is the kernel's export table, or nil and why when it cannot be read.
func (m *Machine) Live() ([]Entry, string) {
text, err := m.Read(Etab)
if err != nil {
return nil, "the kernel's export table (" + Etab + ") could not be read: " + err.Error()
}
return ParseExports(text, Etab), ""
}
// IsLive says whether the kernel exports a path to a client.
func IsLive(live []Entry, p, client string) bool {
for _, e := range live {
if path.Clean(e.Path) == path.Clean(p) && e.Client == client {
return true
}
}
return false
}
// Dataset is one ZFS filesystem as `zfs list` answers it.
type Dataset struct {
Name string `json:"name"`
Mountpoint string `json:"mountpoint"`
ShareNFS string `json:"sharenfs"`
}
// Datasets are the ZFS filesystems, when zfs is there; nil without it.
func (m *Machine) Datasets() ([]Dataset, string) {
r := m.Run(context.Background(), "zfs", "list", "-H", "-o", "name,mountpoint,sharenfs", "-t", "filesystem")
if r.Code != 0 {
if r.Code == -1 && strings.Contains(r.Err, "executable file not found") {
return nil, ""
}
return nil, "zfs list did not answer: " + firstLine(r.Err)
}
var out []Dataset
for _, line := range strings.Split(r.Out, "\n") {
f := strings.Split(line, "\t")
if len(f) != 3 {
continue
}
out = append(out, Dataset{Name: f[0], Mountpoint: f[1], ShareNFS: f[2]})
}
return out, ""
}
// Foreign is every export on the machine that is not the mesh's: /etc/exports, the other files in
// /etc/exports.d (ZFS writes one), and every dataset whose sharenfs is on.
func (m *Machine) Foreign() ([]Entry, []Dataset, []string) {
var entries []Entry
var notes []string
files := append([]string{"/etc/exports"}, m.Glob("/etc/exports.d/*.exports")...)
for _, f := range files {
if f == ExportFile {
continue
}
text, err := m.Read(f)
if err != nil {
if m.Exists(f) {
notes = append(notes, f+" could not be read: "+err.Error())
}
continue
}
entries = append(entries, ParseExports(text, f)...)
}
sets, note := m.Datasets()
if note != "" {
notes = append(notes, note)
}
var shared []Dataset
for _, d := range sets {
if d.ShareNFS != "off" && d.ShareNFS != "-" && d.ShareNFS != "" {
shared = append(shared, d)
}
}
return entries, shared, notes
}
func firstLine(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexByte(s, '\n'); i >= 0 {
return s[:i]
}
return s
}
+427
View File
@@ -0,0 +1,427 @@
package main
import (
"context"
"fmt"
"net/netip"
"path"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Seat is the role this module holds on its machine (novox/hq ADR 0263).
const Seat = "node-nfs-server"
// Stale is how old the process's last look may be before the tools say it stopped looking.
const Stale = 5 * time.Minute
// verb is one of the seat's verbs, listed as `<seat>.<verb>` so the runtime serves it on the seat's
// subject, as <node>/node-nfs-server.<verb>.
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run}
}
func str(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
func arg(a map[string]any, k string) string {
v, _ := a[k].(string)
return strings.TrimSpace(v)
}
func flag(a map[string]any, k string) bool {
v, _ := a[k].(bool)
return v
}
// Tools are the seat's verbs and the module's health.
func Tools(m *Machine) []stdio.Tool {
return []stdio.Tool{
verb("exports", "Every share this machine exports for the mesh: its name, folder, rw or ro, the owner every "+
"client is mapped to (uid, gid), the clients (the private network's range, nothing else) and whether the "+
"kernel holds it now; the shares the assignment asked for that were refused, and why; and every export "+
"on the machine that is not the mesh's (/etc/exports, the other files in /etc/exports.d, a ZFS dataset's "+
"sharenfs). Replaces exportfs -v, cat /etc/exports and zfs get sharenfs. (r)",
map[string]any{}, func(map[string]any) (any, error) { return m.Exports(), nil }),
verb("clients", "Which machines have a share of this machine mounted now: each NFS client the kernel knows, "+
"with its address and the name it gave, or — when the kernel's list cannot be read — the connections "+
"to port 2049. Replaces cat /proc/fs/nfsd/clients/*/info and ss -tn sport = :2049. (r)",
map[string]any{}, func(map[string]any) (any, error) { return m.Clients(), nil }),
verb("test", "Whether this machine exports one share for the mesh now: the share is asked for and not "+
"refused, the kernel holds it for the private network's range, and the NFS server is active; with an "+
"address, also whether that address is inside the range. Replaces showmount -e. (r)",
map[string]any{"share": str("the share's name"), "address": str("a client's address on the private network (optional)")},
func(a map[string]any) (any, error) { return m.Test(arg(a, "share"), arg(a, "address")) }),
verb("reload", "Have the kernel read the export files again now (exportfs -ra), and answer the mesh's shares "+
"as it then holds them. The module's process does this itself whenever its file changes or a share is "+
"missing; this is for after a change made outside it. Replaces exportfs -ra. (a)",
map[string]any{}, func(map[string]any) (any, error) { return m.Reload() }),
verb("adopt", "Take over the export a ZFS dataset's sharenfs property makes for a folder the mesh now "+
"exports: sets sharenfs=off on that one dataset and nothing else, then exportfs -ra. Only when the "+
"mesh's own export of the same folder is live in the kernel, so the folder is never unexported; "+
"without confirm, a dry run that says what it would do. A person's act, with why. Replaces zfs set "+
"sharenfs=off. (a)",
map[string]any{
"path": str("the shared folder, as the share names it"),
"confirm": map[string]any{"type": "boolean", "description": "do it; without, a dry run"},
"why": str("why, in a sentence (required with confirm)"),
},
func(a map[string]any) (any, error) { return m.Adopt(arg(a, "path"), flag(a, "confirm"), arg(a, "why")) }),
{
Name: "nfs_health",
Description: "Whether this machine's shares are served, as the module's health check (novox/hq ADR 0240, " +
"ADR 0263): unhealthy when the NFS server is not active, the module's process has not looked for five " +
"minutes, a share the assignment asks for is refused or missing from the kernel's table, or the " +
"export file could not be written or read again; with why, one sentence per finding. (r)",
Run: func(map[string]any) (any, error) { return m.Health(), nil },
},
}
}
// ExportsAnswer is what `exports` answers.
type ExportsAnswer struct {
Shares []Exported `json:"shares"`
Refused []Refused `json:"refused,omitempty"`
Foreign []Entry `json:"foreign"`
Service string `json:"service"`
Range string `json:"range,omitempty"`
Looked string `json:"looked,omitempty"`
Said []string `json:"said"`
Notes []string `json:"notes,omitempty"`
}
// Exports is the machine's exports now.
func (m *Machine) Exports() ExportsAnswer {
a := ExportsAnswer{Shares: []Exported{}, Foreign: []Entry{}, Service: m.ServiceState()}
st, err := m.LastStatus()
if err != nil {
a.Notes = append(a.Notes, err.Error())
} else {
a.Range, a.Refused = st.Range, st.Refused
a.Looked = st.At.Format(time.RFC3339)
live, note := m.Live()
if note != "" {
a.Notes = append(a.Notes, note)
}
for _, e := range st.Shares {
if note == "" {
e.Live = IsLive(live, e.Path, e.Clients)
}
a.Shares = append(a.Shares, e)
}
a.Notes = append(a.Notes, st.Problems...)
}
entries, sets, notes := m.Foreign()
a.Notes = append(a.Notes, notes...)
a.Foreign = append(a.Foreign, entries...)
for _, d := range sets {
a.Foreign = append(a.Foreign, Entry{Path: d.Mountpoint, Client: d.ShareNFS, From: "zfs sharenfs on " + d.Name})
}
for _, e := range a.Shares {
state := "live"
if !e.Live {
state = "NOT in the kernel's table"
}
a.Said = append(a.Said, fmt.Sprintf("%s: %s %s to %s, every client as %d:%d — %s", e.Name, e.Path, e.Access, e.Clients, e.UID, e.GID, state))
}
for _, r := range a.Refused {
a.Said = append(a.Said, "refused: "+r.Why)
}
for _, f := range a.Foreign {
a.Said = append(a.Said, fmt.Sprintf("not the mesh's: %s to %s (%s)", f.Path, f.Client, f.From))
}
if len(a.Shares) == 0 && len(a.Refused) == 0 {
a.Said = append(a.Said, "the mesh exports nothing here")
}
a.Said = append(a.Said, "the NFS server is "+a.Service)
return a
}
// Client is one machine that has a share mounted.
type Client struct {
Address string `json:"address"`
Name string `json:"name,omitempty"`
Minor string `json:"minor_version,omitempty"`
}
// ClientsAnswer is what `clients` answers.
type ClientsAnswer struct {
Clients []Client `json:"clients"`
From string `json:"from"`
Notes []string `json:"notes,omitempty"`
}
// ParseClientInfo reads one /proc/fs/nfsd/clients/<id>/info.
func ParseClientInfo(text string) Client {
var c Client
for _, line := range strings.Split(text, "\n") {
k, v, ok := strings.Cut(line, ":")
if !ok {
continue
}
v = strings.Trim(strings.TrimSpace(v), `"`)
switch strings.TrimSpace(k) {
case "address":
c.Address = v
case "name":
c.Name = v
case "minor version":
c.Minor = v
}
}
return c
}
// Clients are the NFS clients the kernel knows, or the connections to 2049.
func (m *Machine) Clients() ClientsAnswer {
a := ClientsAnswer{Clients: []Client{}}
const dir = "/proc/fs/nfsd/clients"
infos := m.Glob(dir + "/*/info")
if len(infos) == 0 && m.UID != 0 {
if r, err := m.AsRoot("find", dir, "-maxdepth", "2", "-name", "info"); err == nil {
for _, l := range strings.Fields(r.Out) {
infos = append(infos, l)
}
} else {
a.Notes = append(a.Notes, err.Error())
}
}
for _, f := range infos {
text, err := m.Read(f)
if err != nil && m.UID != 0 {
r, e := m.AsRoot("cat", f)
text, err = r.Out, e
}
if err != nil {
a.Notes = append(a.Notes, f+": "+err.Error())
continue
}
a.Clients = append(a.Clients, ParseClientInfo(text))
}
if len(infos) > 0 {
a.From = dir
return a
}
r := m.Run(context.Background(), "ss", "-tnH", "state", "established", "( sport = :2049 )")
a.From = "connections to port 2049"
for _, line := range strings.Split(r.Out, "\n") {
f := strings.Fields(line)
if len(f) >= 4 {
a.Clients = append(a.Clients, Client{Address: f[len(f)-1]})
}
}
return a
}
// TestAnswer is what `test` answers.
type TestAnswer struct {
Share string `json:"share"`
Path string `json:"path,omitempty"`
Access string `json:"access,omitempty"`
Live bool `json:"live"`
Service string `json:"service"`
InRange *bool `json:"in_range,omitempty"`
OK bool `json:"ok"`
Why string `json:"why"`
}
// Test says whether one share is served now.
func (m *Machine) Test(share, address string) (TestAnswer, error) {
if share == "" {
return TestAnswer{}, fmt.Errorf("name the share")
}
a := TestAnswer{Share: share, Service: m.ServiceState()}
st, err := m.LastStatus()
if err != nil {
a.Why = err.Error()
return a, nil
}
for _, r := range st.Refused {
if r.Name == share {
a.Path, a.Why = r.Path, r.Why
return a, nil
}
}
var found *Exported
for i := range st.Shares {
if st.Shares[i].Name == share {
found = &st.Shares[i]
}
}
if found == nil {
a.Why = fmt.Sprintf("This machine's assignment asks for no share named %s.", share)
return a, nil
}
a.Path, a.Access = found.Path, found.Access
live, note := m.Live()
a.Live = note == "" && IsLive(live, found.Path, found.Clients)
var why []string
if address != "" {
addr, err1 := netip.ParseAddr(address)
prefix, err2 := netip.ParsePrefix(found.Clients)
in := err1 == nil && err2 == nil && prefix.Contains(addr)
a.InRange = &in
if !in {
why = append(why, fmt.Sprintf("%s is not inside the private network's range %s.", address, found.Clients))
}
}
if !a.Live {
why = append(why, fmt.Sprintf("The kernel does not export %s to %s now.", found.Path, found.Clients))
}
if a.Service != "active" {
why = append(why, "The NFS server is "+a.Service+".")
}
a.OK = len(why) == 0
if a.OK {
a.Why = fmt.Sprintf("The share %s (%s, %s) is exported to %s and the NFS server is active.", share, found.Path, found.Access, found.Clients)
} else {
a.Why = strings.Join(why, " ")
}
return a, nil
}
// Reload has the kernel read the export files again.
func (m *Machine) Reload() (any, error) {
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
return nil, err
}
return m.Exports(), nil
}
// AdoptAnswer is what `adopt` answers.
type AdoptAnswer struct {
Path string `json:"path"`
Dataset string `json:"dataset,omitempty"`
Was string `json:"sharenfs_was,omitempty"`
Done bool `json:"done"`
DryRun bool `json:"dry_run"`
Said string `json:"said"`
}
// Adopt clears one dataset's sharenfs once the mesh's own export of its folder is live.
func (m *Machine) Adopt(p string, confirm bool, why string) (AdoptAnswer, error) {
if p == "" {
return AdoptAnswer{}, fmt.Errorf("name the shared folder (path)")
}
p = path.Clean(p)
a := AdoptAnswer{Path: p, DryRun: !confirm}
sets, note := m.Datasets()
if note != "" {
return a, fmt.Errorf("%s", note)
}
var ds *Dataset
for i := range sets {
if path.Clean(sets[i].Mountpoint) == p {
ds = &sets[i]
}
}
if ds == nil {
return a, fmt.Errorf("no ZFS dataset is mounted at %s, so there is no sharenfs to take over", p)
}
a.Dataset, a.Was = ds.Name, ds.ShareNFS
if ds.ShareNFS == "off" || ds.ShareNFS == "-" || ds.ShareNFS == "" {
return a, fmt.Errorf("the dataset %s already has sharenfs off; nothing to take over", ds.Name)
}
st, err := m.LastStatus()
if err != nil {
return a, err
}
var mine *Exported
for i := range st.Shares {
if st.Shares[i].Path == p {
mine = &st.Shares[i]
}
}
if mine == nil {
return a, fmt.Errorf("the mesh does not export %s: add it to this machine's nfs-server `shares` setting first, so the folder is never unexported", p)
}
live, lnote := m.Live()
if lnote != "" || !IsLive(live, p, mine.Clients) {
return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, mine.Clients)
}
plan := fmt.Sprintf("set sharenfs=off on the dataset %s (now %q) and run exportfs -ra; %s stays exported as the share %s to %s, every client as %d:%d",
ds.Name, ds.ShareNFS, p, mine.Name, mine.Clients, mine.UID, mine.GID)
if !confirm {
a.Said = "Dry run: would " + plan + "."
return a, nil
}
if why == "" {
return a, fmt.Errorf("say why: adopt is a person's act, and the why is kept with it")
}
if _, err := m.AsRoot("zfs", "set", "sharenfs=off", ds.Name); err != nil {
return a, err
}
if _, err := m.AsRoot("exportfs", "-ra"); err != nil {
return a, fmt.Errorf("sharenfs is off on %s, and exportfs -ra failed: %v", ds.Name, err)
}
a.Done = true
a.Said = fmt.Sprintf("Did: %s. Why: %s", plan, why)
return a, nil
}
// Health is the module's health, as the node-engine asks it.
type Health struct {
Healthy bool `json:"healthy"`
Why string `json:"why"`
Found []string `json:"found,omitempty"`
}
// Health judges the shares.
func (m *Machine) Health() Health {
var found []string
service := m.ServiceState()
if service != "active" {
found = append(found, fmt.Sprintf("The NFS server (%s) is %s.", Unit, service))
}
st, err := m.LastStatus()
if err != nil {
found = append(found, err.Error()+".")
return Health{Healthy: false, Why: strings.Join(found, " "), Found: found}
}
if age := m.Now().Sub(st.At); age > Stale {
found = append(found, fmt.Sprintf("The module's process last looked %s ago, at %s.", age.Round(time.Second), st.At.Local().Format("15:04")))
}
for _, p := range st.Problems {
found = append(found, sentence(p))
}
for _, r := range st.Refused {
found = append(found, r.Why)
}
if st.ReloadError != "" {
found = append(found, sentence("the kernel could not read the export files again: "+st.ReloadError))
}
live, note := m.Live()
for _, e := range st.Shares {
if note == "" && !IsLive(live, e.Path, e.Clients) {
found = append(found, fmt.Sprintf("The share %s (%s) is not in the kernel's export table.", e.Name, e.Path))
}
}
sort.Strings(found)
if len(found) > 0 {
return Health{Healthy: false, Why: strings.Join(found, " "), Found: found}
}
names := make([]string, 0, len(st.Shares))
for _, e := range st.Shares {
names = append(names, e.Name)
}
if len(names) == 0 {
return Health{Healthy: true, Why: "The NFS server is active, and the mesh exports nothing here."}
}
return Health{Healthy: true, Why: fmt.Sprintf("The NFS server is active and exports %s to %s.", strings.Join(names, ", "), st.Range)}
}
func sentence(s string) string {
s = strings.TrimSpace(s)
if s == "" {
return s
}
s = strings.ToUpper(s[:1]) + s[1:]
if !strings.HasSuffix(s, ".") {
s += "."
}
return s
}
+5
View File
@@ -0,0 +1,5 @@
module nfsserver
go 1.22
require git.novox.be/novox/mesh-sdk/go v0.1.7
+2
View File
@@ -0,0 +1,2 @@
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
+142
View File
@@ -0,0 +1,142 @@
{
"module": "nfs-server",
"version": "1",
"upgrade": {
"policy": "record",
"why": "the folders other machines mount: a build that breaks the exports leaves every client's mount hanging or refused, and the gate on this one machine does not see the clients (hq ADR 0236, ADR 0263)"
},
"capabilities": [
"package-manager",
"service-manager"
],
"provides": [
{
"name": "nfs-share",
"scope": "mesh",
"identity": false
}
],
"data": {
"consumers": {
"nfs-share": {
"class": "none",
"why": "the shared folders are the operator's data (hq ADR 0051): what a client writes lands in them, and they are protected where the operator declares them, never by this module, which keeps nothing of a consumer's"
}
}
},
"claims": [
{
"name": "node-nfs-server",
"scope": "node",
"serves": [
"exports",
"clients",
"test",
"reload",
"adopt"
]
}
],
"state": [
{
"name": "exports",
"ttl-seconds": 120,
"per-machine": true
}
],
"tools": [
"nfs_health"
],
"listens": [
{
"name": "nfs",
"port": 2049,
"protocol": "tcp",
"from": "mesh",
"fixed": true,
"why": "the shares, to the mesh's machines only (hq ADR 0263): NFS version 4 alone, which needs no other port, and never the home network, where a device that is not a node could claim any user id"
}
],
"resources": [
{
"id": "package",
"type": "package",
"package": "nfs-utils"
},
{
"id": "nfs-conf",
"type": "file",
"path": "/etc/nfs.conf.d/50-mesh.conf",
"mode": "0644",
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263). Replaced on every push; a drop-in of\n# the operator's that sorts after this one overrides it, and is theirs.\n#\n# NFS version 4 only: a client needs port 2049 and nothing else, so the module opens nothing more\n# than that, to the private network. Version 3 needs rpcbind and mountd, on ports the mesh does not open.\n[nfsd]\nvers2=n\nvers3=n\nvers4=y\nvers4.0=n\nvers4.1=y\nvers4.2=y\n"
},
{
"id": "config-dir",
"type": "directory",
"path": "/etc/nfs-server",
"mode": "0755"
},
{
"id": "config",
"type": "file",
"path": "/etc/nfs-server/shares.conf",
"mode": "0644",
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The module's process exports each\n# to the private network's range below, every client mapped to the folder's owner.\nshares=${setting:shares}\nrange=${machine:mesh-range}\n"
},
{
"id": "run-dir",
"type": "directory",
"path": "/run/nfs-server",
"mode": "0755"
},
{
"id": "server",
"type": "service",
"unit": "nfs-server.service",
"state": "running",
"boot": "enabled",
"restart-on": [
"nfs-conf"
],
"health": {
"kind": "unit"
}
},
{
"id": "exports",
"type": "process",
"name": "nfs-server-exports",
"artifact": "tools",
"run": [
"./nfs-server",
"exports"
],
"restart-on": [
"config"
],
"health": {
"kind": "tool",
"tool": "nfs_health",
"interval": "60s",
"timeout": "10s",
"looks": 2,
"grace": "90s"
}
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/nfs-server",
"binary": "nfs-server",
"loads": [
"nfs-server"
]
}
]
}
}