mesh-files: the holder of node-files, reads and writes with trash, owned paths and an act log, so file work on a node needs no shell (issue 900, ADR 0394)
mesh/delivery-group group feat/900-node-files checking: composed check out of date: novox/mesh-controller@210f7cf5e64b was judged against trunk 611e2275cf7c, t…
mesh/merge-gate out of date: judged against trunk 3db85a0dde83, the trunk is now 8b53a3663244; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 3db85a0dde83, the trunk is now 8b53a3663244; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request

This commit is contained in:
2026-10-16 14:50:03 +02:00
parent 3db85a0dde
commit 5d98bcc0db
14 changed files with 4477 additions and 0 deletions
+2
View File
@@ -21,6 +21,8 @@ modules/mesh-knowledge/cmd/mesh-knowledge/mesh-knowledge
modules/mesh-knowledge/mesh-knowledge
modules/mesh-reads/mesh-reads
modules/mesh-reads/cmd/mesh-reads/mesh-reads
modules/mesh-files/mesh-files
modules/mesh-files/cmd/mesh-files/mesh-files
# The provisioner client built in place (issue 671).
modules/mesh-provisioner-client/cmd/mesh-provisioner-client/mesh-provisioner-client
+63
View File
@@ -0,0 +1,63 @@
# mesh-files
The holder of the node seat `node-files` (novox/hq ADR 0394, issue 900; to-be 41 WP9). It reads and manages
the files and accounts of the node it runs on, through the mesh, so nothing about a node's files goes through
a shell.
It replaces `mesh-reads` (ADR 0310), which stays beside it until `mesh-files` is assigned on every node (ADR 0394 §9).
## Verbs
The seat's verbs, with the descriptions the controller defines (`internal/catalogue/files.go`).
- **Reads:** `stat`, `readlink`, `list`, `read`, `search`, `account`. As in mesh-reads: bounded answers, a place a
secret is kept refused by name before any read, a line shaped like a secret withheld.
- **Writes:** `write` (a text file whole, with an optional mode), `copy`, `move`, `delete` (recursive on
request), `mkdir` (parents on request), `chmod` (octal or `u+x,go-w`, recursive on request), `chown` (owner,
optional group, recursive on request).
- **Trash:** `trash` lists it, `restore` puts an item back.
Every write verb takes `reason` (required, for the audit trail) and `override` (a reason; needed only for a
path a module owns).
## Rights
It runs as the tool runner's account, the node's operator account, and declares no account of its own. An act
is tried with that account's rights first. Where that fails with a permission error, the act is repeated
through `sudo -n` with a fixed binary (cat, cp, mv, stat, chmod, chown, mkdir, rm under `/usr/bin`) and an
argument list built from fixed words and one path, never a shell (`sudo.go`). Tests use a fake runner.
Not yet through sudo: `stat`, `list`, `readlink` and `search` (they answer what the account reaches and say what it
may not); `read` and every write do.
## Owners and modes
- `copy` and `move` keep owner and mode (`cp -a`); a copy refuses a target that exists.
- A new file or folder takes its folder's owner and group.
- An overwrite keeps the overwritten file's owner and mode.
## Trash
`delete`, and the overwrite of `write`, first move the old item to `<state>/trash/<id>/item` with a
`meta.json` (original path, owner, group, mode, caller, reason, time). `restore` puts it back and refuses when
the path is taken now. Entries older than 7 days are emptied on every write call and every `trash` list.
## Owned paths
The controller gives the holder `MESH_OWNED_PATHS`, a JSON object from an absolute path to `module (tool)`. A
write, move or delete of such a path, anything under it, or (for delete, move and recursive chmod and chown) a
folder that holds it, is refused naming the module and tool, unless `override` gives a reason. The override is
recorded in the act log and told to the operator, in words without a path, through `operator-channel.tell`.
Without the word, the module starts and owns nothing.
## The act log
Each write act that was done is one JSON line in `<state>/acts.log`: time, caller, verb, paths, reason, and for
an override the override and the owned paths it went through. The caller is the bus user the runtime proved
(`stdio.CallerOf`), or `unproven`; it is never read from the arguments. A refused act is not logged.
## What it refuses
- A place a secret is kept (`MESH_SECRET_PLACES`, and the known credential files in `refuse.go`) is refused by
name for `read`, `list` and `search`. Copying or moving such a place is allowed: nothing is printed.
- A copy or move to or from `<node>:/path` is refused: copying between nodes comes in step 2 (issue 903).
- A write without a reason; a path that is not absolute; a mode that is not a mode.
+913
View File
@@ -0,0 +1,913 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"syscall"
"time"
)
// OwnedWord is where the controller names the paths a module owns on this node (novox/hq ADR 0394 §5): a JSON
// object from each absolute path to "module (tool)".
const OwnedWord = "MESH_OWNED_PATHS"
// ownedFrom reads the controller's object. Absent is not an error: the controller part may roll later than this
// module, and until it does, nothing is owned.
func ownedFrom(raw string) (map[string]string, error) {
owned := map[string]string{}
if strings.TrimSpace(raw) == "" {
return owned, nil
}
if err := json.Unmarshal([]byte(raw), &owned); err != nil {
return nil, fmt.Errorf("%s is not an object from a path to its owner: %v", OwnedWord, err)
}
for p := range owned {
if !strings.HasPrefix(p, "/") {
return nil, fmt.Errorf("%s names %q, which is not an absolute path", OwnedWord, p)
}
}
return owned, nil
}
// Ident is an owner and a group, by number.
type Ident struct{ UID, GID int }
func ownerOnDisk(p string) (Ident, error) {
info, err := os.Lstat(p)
if err != nil {
return Ident{}, err
}
st, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return Ident{}, fmt.Errorf("%s: this system does not say who owns a file", p)
}
return Ident{int(st.Uid), int(st.Gid)}, nil
}
// Files reads and manages one node's files (novox/hq ADR 0394). It acts as the account it runs as, and repeats
// an act through Root where that account's rights do not reach.
type Files struct {
Reader
// State is the module's state directory: the trash and the act log.
State string
// Owned maps a path a module owns on this node to "module (tool)".
Owned map[string]string
// Node is this node's name, for the notice of an override.
Node string
// Tell tells the operator a plain notice (operator-channel.tell).
Tell func(headline string, lines []string, subject string) error
// OwnerOf says who owns a file; Lchown changes it; NewID makes a trash id. Tests replace each.
OwnerOf func(path string) (Ident, error)
Lchown func(path string, uid, gid int) error
NewID func() string
mu sync.Mutex
}
// Act is who asks a write, and why.
type Act struct {
// Caller is the bus user the runtime proved (never an argument).
Caller string
// Reason is why the file is changed; required.
Reason string
// Override is a reason to change a path a module owns anyway.
Override string
}
// ActAnswer is what a write verb answers.
type ActAnswer struct {
Done string `json:"done"`
Paths []string `json:"paths"`
Trashed string `json:"trashed,omitempty"`
Overrides []string `json:"overridden,omitempty"`
Notice string `json:"notice,omitempty"`
}
var remote = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]*:/`)
func notRemote(p string) error {
if remote.MatchString(strings.TrimSpace(p)) {
return fmt.Errorf("%s is on another node: copying between nodes comes in step 2 (issue 903)", p)
}
return nil
}
func (f *Files) ownerOf(p string) (Ident, error) {
if f.OwnerOf != nil {
return f.OwnerOf(p)
}
return ownerOnDisk(p)
}
func (f *Files) asRoot(bin string, args ...string) error {
if f.Root == nil {
return errors.New("this account may not do that, and sudo is not available to this holder")
}
_, err := f.Root.Run(bin, args...)
return err
}
func denied(err error) bool { return errors.Is(err, fs.ErrPermission) }
// lchown changes an owner (-1: leave as it is), through root where the account may not.
func (f *Files) lchown(path string, uid, gid int) error {
do := f.Lchown
if do == nil {
do = os.Lchown
}
err := do(path, uid, gid)
if err == nil || !denied(err) {
return err
}
spec := ""
if uid >= 0 {
spec = strconv.Itoa(uid)
}
if gid >= 0 {
spec += ":" + strconv.Itoa(gid)
}
if spec == "" {
return nil
}
return f.asRoot("/usr/bin/chown", spec, "--", path)
}
// chownTo gives a file an owner it has not got yet.
func (f *Files) chownTo(path string, id Ident) error {
if now, err := ownerOnDisk(path); err == nil && now == id {
return nil
}
return f.lchown(path, id.UID, id.GID)
}
func fileMode(perm uint32) fs.FileMode {
m := fs.FileMode(perm & 0o777)
if perm&0o4000 != 0 {
m |= fs.ModeSetuid
}
if perm&0o2000 != 0 {
m |= fs.ModeSetgid
}
if perm&0o1000 != 0 {
m |= fs.ModeSticky
}
return m
}
func permOf(m fs.FileMode) uint32 {
p := uint32(m.Perm())
if m&fs.ModeSetuid != 0 {
p |= 0o4000
}
if m&fs.ModeSetgid != 0 {
p |= 0o2000
}
if m&fs.ModeSticky != 0 {
p |= 0o1000
}
return p
}
// setMode sets a file's permission bits, through root where the account may not.
func (f *Files) setMode(path string, perm uint32) error {
err := os.Chmod(path, fileMode(perm))
if err == nil || !denied(err) {
return err
}
return f.asRoot("/usr/bin/chmod", fmt.Sprintf("%04o", perm), "--", path)
}
// ---- the owned paths (§5) and the act log (§6) ----
func within2(path, place string) bool { return within(filepath.Clean(path), filepath.Clean(place)) }
// guard refuses a change of what a module owns, unless the act gives an override. wide: the act also takes
// away or changes everything beneath the path, so a path that holds an owned one is owned too. It answers the
// owned paths the override went through.
func (f *Files) guard(a Act, wide bool, paths ...string) ([]hit, error) {
var hits []hit
for _, p := range paths {
for owned, owner := range f.Owned {
if within2(p, owned) || (wide && within2(owned, p)) {
if a.Override == "" {
return nil, fmt.Errorf("%s is owned by %s: change it with that, or give override with a reason "+
"(novox/hq ADR 0394 §5)", owned, owner)
}
hits = append(hits, hit{owned, owner})
}
}
}
return hits, nil
}
// hit is an owned path an override went through, and who owns it.
type hit struct{ Path, Owner string }
func (a Act) check() error {
if strings.TrimSpace(a.Reason) == "" {
return errors.New("reason is required: say why the file is changed, for the audit trail (novox/hq ADR 0394 §6)")
}
return nil
}
type actLine struct {
Time string `json:"time"`
Caller string `json:"caller"`
Verb string `json:"verb"`
Paths []string `json:"paths"`
Reason string `json:"reason"`
Override string `json:"override,omitempty"`
Owned []string `json:"owned,omitempty"`
Trashed string `json:"trashed,omitempty"`
}
// finish records a write act that was done: one line in the act log, and, for an override that was needed, a
// notice to the operator. A notice that cannot be sent is said in the answer, never swallowed.
func (f *Files) finish(a Act, verb string, hits []hit, ans ActAnswer) (ActAnswer, error) {
l := actLine{Time: f.Now().UTC().Format(time.RFC3339), Caller: a.Caller, Verb: verb, Paths: ans.Paths,
Reason: a.Reason, Trashed: ans.Trashed}
if len(hits) > 0 {
l.Override = a.Override
for _, h := range hits {
l.Owned = append(l.Owned, h.Path+" ("+h.Owner+")")
}
ans.Overrides = l.Owned
}
body, _ := json.Marshal(l)
if err := os.MkdirAll(f.State, 0o700); err != nil {
return ans, fmt.Errorf("%s was done, but the act log cannot be written: %v", verb, err)
}
log, err := os.OpenFile(filepath.Join(f.State, "acts.log"), os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0o600)
if err == nil {
_, err = log.Write(append(body, '\n'))
log.Close()
}
if err != nil {
return ans, fmt.Errorf("%s was done, but the act log cannot be written: %v", verb, err)
}
if len(hits) > 0 && f.Tell != nil {
// No path in the words: the operator channel holds its words to the content rule, which refuses one.
// The path is in the act log.
err := f.Tell("A module's file was changed by override",
[]string{fmt.Sprintf("%s owns it, and %s changed it anyway.", strings.Join(owners(hits), " and "), a.Caller),
"The reason given: " + a.Override},
"mesh-files on "+f.Node)
if err != nil {
ans.Notice = "the operator was not told: " + err.Error()
fmt.Fprintf(os.Stderr, "[mesh-files] %s\n", ans.Notice)
}
}
return ans, nil
}
func owners(hits []hit) []string {
seen := map[string]bool{}
var out []string
for _, h := range hits {
if !seen[h.Owner] {
seen[h.Owner] = true
out = append(out, h.Owner)
}
}
return out
}
// ---- the write verbs ----
func (f *Files) begin(a Act) error {
if err := a.check(); err != nil {
return err
}
f.sweep()
return nil
}
func parseMode(s string) (uint32, error) {
n, err := strconv.ParseUint(strings.TrimSpace(s), 8, 32)
if err != nil || n > 0o7777 {
return 0, fmt.Errorf("%q is not a mode such as 0644", s)
}
return uint32(n), nil
}
// Write writes one text file whole. The old file, if any, goes to the trash first; the new one keeps its owner
// and mode, or, when new, takes its folder's owner and group (ADR 0394 §3).
func (f *Files) Write(a Act, raw, content, mode string) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
path, err := absolute(raw)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, false, path)
if err != nil {
return ActAnswer{}, err
}
var perm uint32 = 0o644
asked := false
if strings.TrimSpace(mode) != "" {
if perm, err = parseMode(mode); err != nil {
return ActAnswer{}, err
}
asked = true
}
if _, err := os.Stat(filepath.Dir(path)); err != nil {
return ActAnswer{}, fmt.Errorf("the folder of %s cannot be reached: %s", path, why(err))
}
want, trashed := Ident{}, ""
if old, err := os.Lstat(path); err == nil {
if !old.Mode().IsRegular() {
return ActAnswer{}, fmt.Errorf("%s is a %s, not a file: write replaces a file whole", path, typeOf(old.Mode()))
}
if want, err = f.ownerOf(path); err != nil {
return ActAnswer{}, err
}
if !asked {
perm = permOf(old.Mode())
}
if trashed, err = f.toTrash(a, path); err != nil {
return ActAnswer{}, err
}
} else if want, err = f.ownerOf(filepath.Dir(path)); err != nil {
return ActAnswer{}, err
}
if err := f.create(path, content, perm); err != nil {
if trashed != "" {
if _, rerr := f.restore(trashed); rerr != nil {
err = fmt.Errorf("%v (and the old file could not be put back from the trash, id %s: %v)", err, trashed, rerr)
}
}
return ActAnswer{}, err
}
if err := f.chownTo(path, want); err != nil {
return ActAnswer{}, fmt.Errorf("%s was written, but its owner could not be set: %v", path, err)
}
if perm&0o6000 != 0 { // the chown cleared setuid and setgid
if err := f.setMode(path, perm); err != nil {
return ActAnswer{}, err
}
}
return f.finish(a, "write", hits, ActAnswer{Done: "wrote " + path, Paths: []string{path}, Trashed: trashed})
}
// create makes a new file with a content and a mode, as the account, or through root where the folder is not its.
func (f *Files) create(path, content string, perm uint32) error {
out, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err == nil {
_, err = io.WriteString(out, content)
if cerr := out.Close(); err == nil {
err = cerr
}
if err != nil {
os.Remove(path)
return err
}
return f.setMode(path, perm)
}
if !denied(err) {
return err
}
tmp := filepath.Join(f.State, "tmp-"+f.NewID())
if err := os.WriteFile(tmp, []byte(content), 0o600); err != nil {
return err
}
defer os.Remove(tmp)
if err := f.asRoot("/usr/bin/cp", "--", tmp, path); err != nil {
return err
}
return f.setMode(path, perm)
}
// copyTree copies a file, a link or a directory with what it holds, keeping modes, owners and times (cp -a).
func (f *Files) copyTree(from, to string) error {
info, err := os.Lstat(from)
if err != nil {
return err
}
id, err := f.ownerOf(from)
if err != nil {
return err
}
switch {
case info.Mode()&fs.ModeSymlink != 0:
target, err := os.Readlink(from)
if err != nil {
return err
}
if err := os.Symlink(target, to); err != nil {
return err
}
return f.chownTo(to, id)
case info.IsDir():
if err := os.Mkdir(to, 0o700); err != nil {
return err
}
entries, err := os.ReadDir(from)
if err != nil {
return err
}
for _, e := range entries {
if err := f.copyTree(filepath.Join(from, e.Name()), filepath.Join(to, e.Name())); err != nil {
return err
}
}
case info.Mode().IsRegular():
in, err := os.Open(from)
if err != nil {
return err
}
defer in.Close()
out, err := os.OpenFile(to, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err != nil {
return err
}
_, err = io.Copy(out, in)
if cerr := out.Close(); err == nil {
err = cerr
}
if err != nil {
return err
}
default:
return fmt.Errorf("%s is a %s, which is not copied", from, typeOf(info.Mode()))
}
// The owner first: a chown clears setuid and setgid, which the mode then sets again.
if err := f.chownTo(to, id); err != nil {
return err
}
if err := f.setMode(to, permOf(info.Mode())); err != nil {
return err
}
return os.Chtimes(to, info.ModTime(), info.ModTime())
}
// relocate moves a path: a rename, a copy and removal across file systems, and, where the account may not, mv
// through root. It keeps owner and mode.
func (f *Files) relocate(from, to string) error {
err := os.Rename(from, to)
switch {
case err == nil:
return nil
case errors.Is(err, syscall.EXDEV):
if err := f.copyTree(from, to); err != nil {
os.RemoveAll(to)
return err
}
return f.removeAll(from)
case denied(err):
return f.asRoot("/usr/bin/mv", "--", from, to)
}
return err
}
func (f *Files) removeAll(p string) error {
err := os.RemoveAll(p)
if err != nil && denied(err) {
return f.asRoot("/usr/bin/rm", "-rf", "--", p)
}
return err
}
// target checks the new path of a copy or a move: on this node, in a folder that is there, and not the source,
// inside it or holding it. An existing target is overwritten by the caller.
func target(rawFrom, from, raw string) (string, error) {
if err := notRemote(raw); err != nil {
return "", err
}
to, err := absolute(raw)
if err != nil {
return "", err
}
if within(to, from) || within(from, to) {
return "", fmt.Errorf("%s and %s are the same path, or one is inside the other", from, to)
}
if _, err := os.Stat(filepath.Dir(to)); err != nil {
return "", fmt.Errorf("the folder of %s cannot be reached: %s", to, why(err))
}
return to, nil
}
// overwritten puts an existing target in the trash and says whose it was, so the new one can take its owner and
// mode; it is nothing for a target that is not there.
type overwritten struct {
trashed string
id Ident
perm uint32
was bool
}
func (f *Files) makeRoom(a Act, to string) (overwritten, error) {
info, err := os.Lstat(to)
if err != nil {
return overwritten{}, nil
}
id, err := f.ownerOf(to)
if err != nil {
return overwritten{}, err
}
tid, err := f.toTrash(a, to)
if err != nil {
return overwritten{}, err
}
return overwritten{trashed: tid, id: id, perm: permOf(info.Mode()), was: true}, nil
}
// settle gives the new target the overwritten one's owner and mode, or, if the act failed, puts the old one back.
func (f *Files) settle(o overwritten, to string, actErr error) error {
if !o.was {
return actErr
}
if actErr != nil {
if _, rerr := f.restore(o.trashed); rerr != nil {
return fmt.Errorf("%v (and the old target could not be put back from the trash, id %s: %v)", actErr, o.trashed, rerr)
}
return actErr
}
if err := f.chownTo(to, o.id); err != nil {
return fmt.Errorf("%s is there, but its owner could not be set: %v", to, err)
}
if err := f.setMode(to, o.perm); err != nil {
return fmt.Errorf("%s is there, but its mode could not be set: %v", to, err)
}
return nil
}
// Copy copies a file or a directory to a new path on this node, keeping owner and mode.
func (f *Files) Copy(a Act, rawFrom, rawTo string) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
if err := notRemote(rawFrom); err != nil {
return ActAnswer{}, err
}
from, err := absolute(rawFrom)
if err != nil {
return ActAnswer{}, err
}
to, err := target(rawFrom, from, rawTo)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, true, to) // an existing target is replaced whole, with what it holds
if err != nil {
return ActAnswer{}, err
}
if _, err := os.Lstat(from); err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be copied: %s", from, why(err))
}
room, err := f.makeRoom(a, to)
if err != nil {
return ActAnswer{}, err
}
err = f.copyTree(from, to)
if err != nil {
os.RemoveAll(to)
if denied(err) {
err = f.asRoot("/usr/bin/cp", "-a", "--", from, to)
}
if err != nil {
err = fmt.Errorf("%s cannot be copied: %v", from, err)
}
}
if err := f.settle(room, to, err); err != nil {
return ActAnswer{}, err
}
return f.finish(a, "copy", hits, ActAnswer{Done: "copied " + from + " to " + to, Paths: []string{from, to}, Trashed: room.trashed})
}
// Move moves or renames a file or a directory on this node, keeping owner and mode.
func (f *Files) Move(a Act, rawFrom, rawTo string) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
if err := notRemote(rawFrom); err != nil {
return ActAnswer{}, err
}
from, err := absolute(rawFrom)
if err != nil {
return ActAnswer{}, err
}
to, err := target(rawFrom, from, rawTo)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, true, from)
if err != nil {
return ActAnswer{}, err
}
more, err := f.guard(a, true, to)
if err != nil {
return ActAnswer{}, err
}
hits = append(hits, more...)
if _, err := os.Lstat(from); err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be moved: %s", from, why(err))
}
room, err := f.makeRoom(a, to)
if err != nil {
return ActAnswer{}, err
}
err = f.relocate(from, to)
if err != nil {
err = fmt.Errorf("%s cannot be moved: %v", from, err)
}
if err := f.settle(room, to, err); err != nil {
return ActAnswer{}, err
}
return f.finish(a, "move", hits, ActAnswer{Done: "moved " + from + " to " + to, Paths: []string{from, to}, Trashed: room.trashed})
}
// Delete moves a file or a directory into the node's trash.
func (f *Files) Delete(a Act, raw string, recursive bool) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
path, err := absolute(raw)
if err != nil {
return ActAnswer{}, err
}
if path == "/" {
return ActAnswer{}, errors.New("/ is not deleted")
}
hits, err := f.guard(a, true, path)
if err != nil {
return ActAnswer{}, err
}
info, err := os.Lstat(path)
if err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be deleted: %s", path, why(err))
}
if info.IsDir() && !recursive {
return ActAnswer{}, fmt.Errorf("%s is a directory: give recursive \"true\" to delete it with what it holds", path)
}
id, err := f.toTrash(a, path)
if err != nil {
return ActAnswer{}, err
}
return f.finish(a, "delete", hits, ActAnswer{Done: "deleted " + path + " into the trash", Paths: []string{path}, Trashed: id})
}
// Mkdir makes a directory, and with parents the missing ones above it; each takes its folder's owner.
func (f *Files) Mkdir(a Act, raw string, parents bool) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
path, err := absolute(raw)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, false, path)
if err != nil {
return ActAnswer{}, err
}
if _, err := os.Lstat(path); err == nil {
return ActAnswer{}, fmt.Errorf("%s is there already", path)
}
var missing []string // from the first missing folder down
for p := path; ; p = filepath.Dir(p) {
if _, err := os.Lstat(p); err == nil {
break
}
missing = append([]string{p}, missing...)
if p == "/" {
break
}
}
if len(missing) > 1 && !parents {
return ActAnswer{}, fmt.Errorf("the folder of %s is not there: give parents \"true\" to make it too", path)
}
var made []string
for i, p := range missing {
err := os.Mkdir(p, 0o755)
if err == nil {
made = append(made, p)
continue
}
if !denied(err) {
return ActAnswer{}, err
}
// Root makes this folder and, with parents, every one below it in the same call.
args := []string{"--", p}
if parents {
args = []string{"-p", "--", path}
}
if err := f.asRoot("/usr/bin/mkdir", args...); err != nil {
return ActAnswer{}, err
}
made = append(made, missing[i:]...)
break
}
for _, p := range made {
if err := f.setMode(p, 0o755); err != nil {
return ActAnswer{}, err
}
if err := f.dirOwner(p); err != nil {
return ActAnswer{}, err
}
}
return f.finish(a, "mkdir", hits, ActAnswer{Done: "made " + path, Paths: []string{path}})
}
// dirOwner gives a new folder the owner and group of the folder it is in.
func (f *Files) dirOwner(p string) error {
id, err := f.ownerOf(filepath.Dir(p))
if err != nil {
return err
}
return f.chownTo(p, id)
}
var symbolic = regexp.MustCompile(`^([ugoa]*)([-+=])([rwxXst]*)$`)
// newMode applies a mode as chmod says it: octal, or comma-separated clauses such as u+x,go-w.
func newMode(cur uint32, isDir bool, spec string) (uint32, error) {
spec = strings.TrimSpace(spec)
if m, err := parseMode(spec); err == nil {
return m, nil
}
for _, clause := range strings.Split(spec, ",") {
m := symbolic.FindStringSubmatch(clause)
if m == nil {
return 0, fmt.Errorf("%q is not a mode such as 0644 or u+x", spec)
}
who := m[1]
if who == "" || strings.Contains(who, "a") {
who = "ugo"
}
var bits, mask uint32
for _, c := range m[3] {
switch c {
case 'r':
bits |= 0o444
case 'w':
bits |= 0o222
case 'x':
bits |= 0o111
case 'X':
if isDir || cur&0o111 != 0 {
bits |= 0o111
}
case 's':
bits |= 0o6000
case 't':
bits |= 0o1000
}
}
for _, w := range who {
switch w {
case 'u':
mask |= 0o4700
case 'g':
mask |= 0o2070
case 'o':
mask |= 0o1007
}
}
bits &= mask
switch m[2] {
case "+":
cur |= bits
case "-":
cur &^= bits
case "=":
cur = cur&^(mask&0o777) | bits&0o777
}
}
return cur, nil
}
// walk lists a path and, when recursive, everything under it (links are not followed), children before parents.
func walk(path string, recursive bool) ([]string, error) {
if !recursive {
return []string{path}, nil
}
var all []string
err := filepath.WalkDir(path, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
all = append(all, p)
return nil
})
for i, j := 0, len(all)-1; i < j; i, j = i+1, j-1 {
all[i], all[j] = all[j], all[i]
}
return all, err
}
// Chmod changes the mode of a path, and with recursive of everything under it.
func (f *Files) Chmod(a Act, raw, mode string, recursive bool) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
path, err := absolute(raw)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, recursive, path)
if err != nil {
return ActAnswer{}, err
}
if _, err := newMode(0, false, mode); err != nil {
return ActAnswer{}, err
}
all, err := walk(path, recursive)
if err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be changed: %v", path, err)
}
for _, p := range all {
info, err := os.Lstat(p)
if err != nil {
return ActAnswer{}, err
}
if info.Mode()&fs.ModeSymlink != 0 {
continue
}
m, _ := newMode(permOf(info.Mode()), info.IsDir(), mode)
if err := f.setMode(p, m); err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be changed: %v", p, err)
}
}
return f.finish(a, "chmod", hits, ActAnswer{Done: "set the mode of " + path + " to " + mode, Paths: []string{path}})
}
// number resolves an account or group, by name or number, through this node's account files.
func (f *Files) number(kind, v string) (int, error) {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
return n, nil
}
names := f.names()
if kind == "owner" {
for _, u := range names.users {
if u.Name == v {
return u.UID, nil
}
}
} else {
for _, g := range names.groups {
if g.Name == v {
return g.GID, nil
}
}
}
return 0, fmt.Errorf("%s %q is not an account or group of this node", kind, v)
}
// Chown changes the owner, and with it the group, of a path, and with recursive of everything under it.
func (f *Files) Chown(a Act, raw, owner, group string, recursive bool) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
path, err := absolute(raw)
if err != nil {
return ActAnswer{}, err
}
hits, err := f.guard(a, recursive, path)
if err != nil {
return ActAnswer{}, err
}
uid, err := f.number("owner", owner)
if err != nil {
return ActAnswer{}, err
}
gid := -1
if strings.TrimSpace(group) != "" {
if gid, err = f.number("group", group); err != nil {
return ActAnswer{}, err
}
}
all, err := walk(path, recursive)
if err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be changed: %v", path, err)
}
for _, p := range all {
if err := f.lchown(p, uid, gid); err != nil {
return ActAnswer{}, fmt.Errorf("%s cannot be changed: %v", p, err)
}
}
return f.finish(a, "chown", hits, ActAnswer{Done: "gave " + path + " to " + owner, Paths: []string{path}})
}
@@ -0,0 +1,819 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
"syscall"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// fakeRoot is the sudo -n runner of a test: it records each call, and answers a cat from a table. No test runs
// sudo (ADR 0391).
type fakeRoot struct {
mu sync.Mutex
calls []string
cat map[string]string
out map[string]string // the answer to a whole call, "bin arg arg"
err error
}
func (r *fakeRoot) Run(bin string, args ...string) ([]byte, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.calls = append(r.calls, bin+" "+strings.Join(args, " "))
if o, ok := r.out[bin+" "+strings.Join(args, " ")]; ok {
return []byte(o), nil
}
if bin == "/usr/bin/cat" {
if s, ok := r.cat[args[len(args)-1]]; ok {
return []byte(s), nil
}
}
if r.err != nil {
return nil, r.err
}
// What root would do in a folder the account may not write, done in the test's tree.
switch bin {
case "/usr/bin/cp":
dest := args[len(args)-1]
os.Chmod(filepath.Dir(dest), 0o700)
b, err := os.ReadFile(args[len(args)-2])
if err != nil {
return nil, err
}
return nil, os.WriteFile(dest, b, 0o600)
case "/usr/bin/mkdir":
dest := args[len(args)-1]
for p := filepath.Dir(dest); p != "/"; p = filepath.Dir(p) {
if _, err := os.Stat(p); err == nil {
os.Chmod(p, 0o700)
break
}
}
return nil, os.MkdirAll(dest, 0o755)
}
return nil, nil
}
// reach makes the fake answer, for a path, what root's readlink -e and stat would: a regular file that leads to itself.
func (r *fakeRoot) reach(p string) {
r.out["/usr/bin/readlink -e -- "+p] = p + "\n"
r.out["/usr/bin/stat -c "+statFormat+" -- "+p] = "regular file|1000|1000|600|-rw-------|1\n"
}
func (r *fakeRoot) called(prefix string) bool {
for _, c := range r.calls {
if strings.HasPrefix(c, prefix) {
return true
}
}
return false
}
type rig struct {
*Files
root *fakeRoot
dir string
owners map[string]Ident
told []string
}
// testFiles is a holder over a temporary tree: a fake root runner, a clock the test moves, owners the test
// names (a test cannot make a file of another account), and a chown that refuses what is not its own.
func testFiles(t *testing.T) *rig {
t.Helper()
r, dir := testReader(t)
g := &rig{root: &fakeRoot{cat: map[string]string{}, out: map[string]string{}}, dir: dir, owners: map[string]Ident{}}
r.Root = g.root
r.Now = func() time.Time { return time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC) }
seq := 0
g.Files = &Files{Reader: r, State: filepath.Join(dir, "state"), Owned: map[string]string{}, Node: "shanks",
OwnerOf: func(p string) (Ident, error) {
if id, ok := g.owners[p]; ok {
return id, nil
}
return ownerOnDisk(p)
},
Lchown: func(p string, uid, gid int) error { return &fs.PathError{Op: "lchown", Path: p, Err: syscall.EPERM} },
NewID: func() string { seq++; return fmt.Sprintf("t%02d", seq) },
Tell: func(headline string, lines []string, subject string) error {
g.told = append(g.told, subject+" | "+headline+" | "+strings.Join(lines, " | "))
return nil
}}
if err := os.MkdirAll(g.State, 0o700); err != nil {
t.Fatal(err)
}
return g
}
var by = Act{Caller: "person.jochen", Reason: "a test"}
func (g *rig) path(name string) string { return filepath.Join(g.dir, name) }
func mustRead(t *testing.T, p string) string {
t.Helper()
b, err := os.ReadFile(p)
if err != nil {
t.Fatal(err)
}
return string(b)
}
func modeOf(t *testing.T, p string) fs.FileMode {
t.Helper()
i, err := os.Lstat(p)
if err != nil {
t.Fatal(err)
}
return i.Mode().Perm()
}
func actLines(t *testing.T, g *rig) []map[string]any {
t.Helper()
raw, err := os.ReadFile(filepath.Join(g.State, "acts.log"))
if err != nil {
return nil
}
var out []map[string]any
for _, l := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
var m map[string]any
if err := json.Unmarshal([]byte(l), &m); err != nil {
t.Fatalf("a line of the act log is not JSON: %q", l)
}
out = append(out, m)
}
return out
}
// ADR 0394 §1: every write names why.
func TestAWriteWithoutAReasonIsRefused(t *testing.T) {
g := testFiles(t)
if _, err := g.Write(Act{Caller: "x"}, g.path("a"), "x", ""); err == nil || !strings.Contains(err.Error(), "reason") {
t.Errorf("no reason: %v", err)
}
if _, err := os.Stat(g.path("a")); err == nil {
t.Error("the file was written")
}
}
// ADR 0394 §3: a new file takes its folder's owner and group, and the mode asked.
func TestANewFileTakesItsFoldersOwner(t *testing.T) {
g := testFiles(t)
g.owners[g.dir] = Ident{4242, 4343}
if _, err := g.Write(by, g.path("new"), "hello\n", "0600"); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("new")) != "hello\n" || modeOf(t, g.path("new")) != 0o600 {
t.Errorf("content or mode wrong: %o", modeOf(t, g.path("new")))
}
if !g.root.called("/usr/bin/chown 4242:4343 -- " + g.path("new")) {
t.Errorf("the owner of the folder was not given: %v", g.root.calls)
}
}
// ADR 0394 §3, §4: an overwrite keeps the old file's owner and mode, and the old file goes to the trash.
func TestAnOverwriteKeepsOwnerAndModeAndTrashesTheOld(t *testing.T) {
g := testFiles(t)
write(t, g.path("f"), "old\n")
os.Chmod(g.path("f"), 0o640)
g.owners[g.path("f")] = Ident{4242, 4343}
if _, err := g.Write(by, g.path("f"), "new\n", ""); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("f")) != "new\n" || modeOf(t, g.path("f")) != 0o640 {
t.Errorf("content %q mode %o", mustRead(t, g.path("f")), modeOf(t, g.path("f")))
}
if !g.root.called("/usr/bin/chown 4242:4343 -- " + g.path("f")) {
t.Errorf("the old owner was not given back: %v", g.root.calls)
}
tr, err := g.Trash()
if err != nil || len(tr.Items) != 1 || tr.Items[0].Path != g.path("f") {
t.Fatalf("trash: %+v %v", tr, err)
}
if mustRead(t, filepath.Join(g.State, "trash", tr.Items[0].ID, "item")) != "old\n" {
t.Error("the old content is not in the trash")
}
}
// ADR 0394 §3: a copy keeps mode and owner, a directory with what it holds.
func TestACopyKeepsModeAndOwner(t *testing.T) {
g := testFiles(t)
write(t, g.path("src/a"), "A\n")
os.Chmod(g.path("src/a"), 0o750)
g.owners[g.path("src/a")] = Ident{4242, 4343}
if _, err := g.Copy(by, g.path("src"), g.path("dst")); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("dst/a")) != "A\n" || modeOf(t, g.path("dst/a")) != 0o750 {
t.Errorf("copy: %q %o", mustRead(t, g.path("dst/a")), modeOf(t, g.path("dst/a")))
}
if !g.root.called("/usr/bin/chown 4242:4343 -- " + g.path("dst/a")) {
t.Errorf("owner not kept: %v", g.root.calls)
}
if _, err := os.Stat(g.path("src/a")); err != nil {
t.Error("the source is gone")
}
}
// ADR 0394 §3: a move keeps mode and owner, and the source is gone.
func TestAMoveKeepsModeAndOwner(t *testing.T) {
g := testFiles(t)
write(t, g.path("a"), "A\n")
os.Chmod(g.path("a"), 0o700)
if _, err := g.Move(by, g.path("a"), g.path("b")); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("b")) != "A\n" || modeOf(t, g.path("b")) != 0o700 {
t.Error("the move lost content or mode")
}
if _, err := os.Stat(g.path("a")); err == nil {
t.Error("the source is still there")
}
}
// ADR 0394 §4: a delete goes to the trash, restore puts it back, and refuses when the path is taken now.
func TestADeleteGoesToTheTrashAndRestoreBringsItBack(t *testing.T) {
g := testFiles(t)
write(t, g.path("d/f"), "keep\n")
if _, err := g.Delete(by, g.path("d"), false); err == nil || !strings.Contains(err.Error(), "recursive") {
t.Errorf("a directory without recursive: %v", err)
}
if _, err := g.Delete(by, g.path("d"), true); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(g.path("d")); err == nil {
t.Fatal("not deleted")
}
tr, _ := g.Trash()
if len(tr.Items) != 1 {
t.Fatalf("trash %+v", tr)
}
id := tr.Items[0].ID
write(t, g.path("d/other"), "x")
if _, err := g.Restore(by, id); err == nil || !strings.Contains(err.Error(), "there now") {
t.Errorf("restore over a taken path: %v", err)
}
os.RemoveAll(g.path("d"))
if _, err := g.Restore(by, id); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("d/f")) != "keep\n" {
t.Error("not restored")
}
if tr, _ := g.Trash(); len(tr.Items) != 0 {
t.Errorf("the restored item is still in the trash: %+v", tr)
}
}
// ADR 0394 §4: an entry older than 7 days is emptied on a trash list and on a write call.
func TestATrashEntryOlderThanSevenDaysIsEmptied(t *testing.T) {
g := testFiles(t)
write(t, g.path("f"), "x")
if _, err := g.Delete(by, g.path("f"), false); err != nil {
t.Fatal(err)
}
now := g.Now()
g.Now = func() time.Time { return now.Add(6*24*time.Hour + 23*time.Hour) }
if tr, _ := g.Trash(); len(tr.Items) != 1 {
t.Fatalf("emptied too early: %+v", tr)
}
g.Now = func() time.Time { return now.Add(7*24*time.Hour + time.Minute) }
if tr, _ := g.Trash(); len(tr.Items) != 0 {
t.Fatalf("not emptied: %+v", tr)
}
entries, _ := os.ReadDir(filepath.Join(g.State, "trash"))
if len(entries) != 0 {
t.Errorf("the entry is still on disk: %v", entries)
}
// and on a write call
write(t, g.path("g"), "y")
g.Delete(by, g.path("g"), false)
g.Now = func() time.Time { return now.Add(30 * 24 * time.Hour) }
if _, err := g.Mkdir(by, g.path("m"), false); err != nil {
t.Fatal(err)
}
entries, _ = os.ReadDir(filepath.Join(g.State, "trash"))
if len(entries) != 0 {
t.Errorf("a write call left an old entry: %v", entries)
}
}
// ADR 0394 §5: an owned path is refused naming its module and tool; with an override it is changed, logged and
// told to the operator.
func TestAnOwnedPathIsRefusedNamingItsOwnerAndPassesWithAnOverride(t *testing.T) {
g := testFiles(t)
g.Owned[g.path("etc")] = "mesh-hosts (node-hostname)"
write(t, g.path("etc/hosts"), "old\n")
for name, run := range map[string]func() error{
"write": func() error { _, err := g.Write(by, g.path("etc/hosts"), "x", ""); return err },
"delete": func() error { _, err := g.Delete(by, g.path("etc/hosts"), false); return err },
"move": func() error { _, err := g.Move(by, g.path("etc/hosts"), g.path("elsewhere")); return err },
"into": func() error { _, err := g.Move(by, g.path("elsewhere"), g.path("etc/new")); return err },
"parent": func() error { _, err := g.Delete(by, g.dir, true); return err },
} {
err := run()
if err == nil || !strings.Contains(err.Error(), "mesh-hosts (node-hostname)") {
t.Errorf("%s: %v", name, err)
}
}
if mustRead(t, g.path("etc/hosts")) != "old\n" {
t.Fatal("an owned file was changed")
}
if len(actLines(t, g)) != 0 || len(g.told) != 0 {
t.Error("a refused act was logged or told")
}
ov := by
ov.Override = "the module is down and the node cannot resolve"
if _, err := g.Write(ov, g.path("etc/hosts"), "new\n", ""); err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("etc/hosts")) != "new\n" {
t.Error("the override did not write")
}
lines := actLines(t, g)
if len(lines) != 1 || lines[0]["override"] != ov.Override {
t.Errorf("log: %v", lines)
}
if len(g.told) != 1 || !strings.HasPrefix(g.told[0], "mesh-files on shanks | ") ||
!strings.Contains(g.told[0], ov.Override) || !strings.Contains(g.told[0], "mesh-hosts (node-hostname)") ||
!strings.Contains(g.told[0], "person.jochen") {
t.Errorf("told: %v", g.told)
}
if strings.Contains(g.told[0], "/") && strings.Contains(g.told[0], g.dir) {
t.Errorf("the notice carries a path, which the operator channel refuses: %v", g.told)
}
if l := actLines(t, g); !strings.Contains(fmt.Sprint(l[0]["owned"]), g.path("etc")) {
t.Errorf("the path is in the act log: %v", l[0])
}
// an override on a path nobody owns says nothing
g.told = nil
if _, err := g.Write(ov, g.path("free"), "x", ""); err != nil || len(g.told) != 0 {
t.Errorf("an override that was not needed: %v %v", err, g.told)
}
}
// ADR 0394 §7: a credential file is refused before it is read, but may be copied and moved.
func TestACredentialFileIsRefusedToReadButMayBeCopied(t *testing.T) {
g := testFiles(t)
write(t, g.path("home/.ssh/id_ed25519"), "key\n")
if _, err := g.Read(g.path("home/.ssh/id_ed25519"), ReadAsk{}); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("read: %v", err)
}
if g.root.called("/usr/bin/cat") {
t.Error("it went for the content with root")
}
if _, err := g.Copy(by, g.path("home/.ssh/id_ed25519"), g.path("backup_key")); err != nil {
t.Errorf("copy of a secret place: %v", err)
}
if _, err := g.Move(by, g.path("backup_key"), g.path("backup_key2")); err != nil {
t.Errorf("move of a secret place: %v", err)
}
}
// ADR 0394 §1: a target on another node is refused until step 2.
func TestATargetOnAnotherNodeIsRefused(t *testing.T) {
g := testFiles(t)
write(t, g.path("a"), "x")
for _, to := range []string{"ace:/tmp/a", "novox:/etc/hosts"} {
for _, run := range []func() error{
func() error { _, err := g.Copy(by, g.path("a"), to); return err },
func() error { _, err := g.Move(by, g.path("a"), to); return err },
func() error { _, err := g.Copy(by, to, g.path("b")); return err },
} {
if err := run(); err == nil || !strings.Contains(err.Error(), "issue 903") {
t.Errorf("%s: %v", to, err)
}
}
}
if _, err := os.Stat(g.path("a")); err != nil {
t.Error("the source was touched")
}
}
// ADR 0394 §2: a permission error repeats the act through sudo -n with a fixed binary and argument list.
func TestAPermissionErrorFallsBackToRoot(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("run as root, nothing is refused")
}
g := testFiles(t)
if err := os.MkdirAll(g.path("locked"), 0o500); err != nil {
t.Fatal(err)
}
defer os.Chmod(g.path("locked"), 0o700)
if _, err := g.Write(by, g.path("locked/f"), "x\n", ""); err != nil {
t.Fatal(err)
}
if !g.root.called("/usr/bin/cp -- ") || !g.root.called("/usr/bin/cp -- "+filepath.Join(g.State, "tmp")) {
t.Errorf("write did not go through root: %v", g.root.calls)
}
g.root.calls = nil
os.Chmod(g.path("locked"), 0o500)
if _, err := g.Mkdir(by, g.path("locked/sub"), true); err != nil {
t.Fatal(err)
}
if !g.root.called("/usr/bin/mkdir -p -- " + g.path("locked/sub")) {
t.Errorf("mkdir: %v", g.root.calls)
}
// a root that fails is said, not swallowed
os.Chmod(g.path("locked"), 0o500)
g.root.err = errors.New("sudo: a password is required")
if _, err := g.Mkdir(by, g.path("locked/sub2"), false); err == nil || !strings.Contains(err.Error(), "password") {
t.Errorf("a failing root: %v", err)
}
// a read the account may not do is done by cat
write(t, g.path("secretish"), "line one\nline two\n")
os.Chmod(g.path("secretish"), 0)
defer os.Chmod(g.path("secretish"), 0o600)
g.root.cat[g.path("secretish")] = "line one\nline two\n"
g.root.reach(g.path("secretish"))
a, err := g.Read(g.path("secretish"), ReadAsk{})
if err != nil || a.Text != "line one\nline two\n" {
t.Errorf("read through root: %+v %v", a, err)
}
if !g.root.called("/usr/bin/cat -- " + g.path("secretish")) {
t.Errorf("calls: %v", g.root.calls)
}
}
// The root runner is a fixed list of binaries, never a shell.
func TestTheRootRunnerRunsOnlyAFixedList(t *testing.T) {
for _, bin := range []string{"/bin/sh", "/usr/bin/bash", "/usr/bin/env", "cat", "/usr/bin/sudo", "/usr/bin/sh", "/usr/bin/grep"} {
if _, err := (sudoRoot{}).Run(bin, "-c", "id"); err == nil || !strings.Contains(err.Error(), "not one of") {
t.Errorf("%s: %v", bin, err)
}
}
}
// ADR 0394 §6: every write act appends one JSON line, with the caller the runtime proved.
func TestEveryWriteActAppendsOneLogLine(t *testing.T) {
g := testFiles(t)
write(t, g.path("a"), "x")
acts := []struct {
verb string
run func() error
}{
{"write", func() error { _, err := g.Write(by, g.path("w"), "x", ""); return err }},
{"mkdir", func() error { _, err := g.Mkdir(by, g.path("m"), false); return err }},
{"chmod", func() error { _, err := g.Chmod(by, g.path("a"), "0600", false); return err }},
{"copy", func() error { _, err := g.Copy(by, g.path("a"), g.path("c")); return err }},
{"move", func() error { _, err := g.Move(by, g.path("c"), g.path("d")); return err }},
{"delete", func() error { _, err := g.Delete(by, g.path("d"), false); return err }},
{"chown", func() error { _, err := g.Chown(by, g.path("a"), "ops", "", false); return err }},
}
for i, a := range acts {
if err := a.run(); err != nil {
t.Fatalf("%s: %v", a.verb, err)
}
lines := actLines(t, g)
if len(lines) != i+1 {
t.Fatalf("after %s: %d lines", a.verb, len(lines))
}
l := lines[i]
if l["verb"] != a.verb || l["caller"] != "person.jochen" || l["reason"] != "a test" || l["time"] == nil || l["paths"] == nil {
t.Errorf("%s: %v", a.verb, l)
}
}
// a refused act adds none
g.Mkdir(by, g.path("a/b/c"), false)
if len(actLines(t, g)) != len(acts) {
t.Error("a failed act was logged")
}
}
// ADR 0159 / issue 365: the tool takes the caller from the runtime, never from the arguments.
func TestTheToolRecordsTheCallerTheRuntimeNamed(t *testing.T) {
g := testFiles(t)
var tool stdio.Tool
for _, x := range tools(g.Files) {
if x.Name == Seat+".mkdir" {
tool = x
}
}
args := map[string]any{"path": g.path("m"), "reason": "why", "caller": "person.mallory"}
if _, err := tool.Run(args); err != nil {
t.Fatal(err)
}
if l := actLines(t, g); len(l) != 1 || l[0]["caller"] != stdio.Unproven {
t.Errorf("log: %v", l)
}
}
func TestMkdirChmodAndChown(t *testing.T) {
g := testFiles(t)
if _, err := g.Mkdir(by, g.path("a/b"), false); err == nil {
t.Error("a missing parent was made without parents")
}
g.owners[g.dir] = Ident{4242, 4343}
if _, err := g.Mkdir(by, g.path("a/b"), true); err != nil {
t.Fatal(err)
}
if !g.root.called("/usr/bin/chown 4242:4343 -- " + g.path("a")) {
t.Errorf("a made parent takes the owner of its folder: %v", g.root.calls)
}
write(t, g.path("a/b/f"), "x")
os.Chmod(g.path("a/b/f"), 0o600)
if _, err := g.Chmod(by, g.path("a"), "u+x,g+r", true); err != nil {
t.Fatal(err)
}
if modeOf(t, g.path("a/b/f")) != 0o740 {
t.Errorf("recursive symbolic chmod: %o", modeOf(t, g.path("a/b/f")))
}
if _, err := g.Chmod(by, g.path("a/b/f"), "0644", false); err != nil || modeOf(t, g.path("a/b/f")) != 0o644 {
t.Errorf("octal: %v %o", err, modeOf(t, g.path("a/b/f")))
}
if _, err := g.Chmod(by, g.path("a/b/f"), "banana", false); err == nil {
t.Error("a bad mode")
}
g.root.calls = nil
if _, err := g.Chown(by, g.path("a/b/f"), "ops", "wheel", false); err != nil {
t.Fatal(err)
}
if !g.root.called("/usr/bin/chown 1000:998 -- " + g.path("a/b/f")) {
t.Errorf("chown by name: %v", g.root.calls)
}
if _, err := g.Chown(by, g.path("a/b/f"), "nobody-here", "", false); err == nil {
t.Error("an unknown account")
}
}
// ADR 0394 §5: without the word the module owns nothing and starts; a bad word is refused.
func TestWithoutOwnedPathsItOwnsNothing(t *testing.T) {
if o, err := ownedFrom(""); err != nil || len(o) != 0 {
t.Errorf("none: %v %v", o, err)
}
if o, err := ownedFrom(`{"/etc/hosts":"mesh-hosts (node-hostname)"}`); err != nil || o["/etc/hosts"] == "" {
t.Errorf("a word: %v %v", o, err)
}
if _, err := ownedFrom(`["/etc/hosts"]`); err == nil || !strings.Contains(err.Error(), OwnedWord) {
t.Errorf("a list: %v", err)
}
}
func TestItServesTheWriteVerbsToo(t *testing.T) {
g := testFiles(t)
got := map[string]bool{}
for _, tool := range tools(g.Files) {
got[tool.Name] = true
}
for _, v := range []string{"stat", "readlink", "list", "read", "search", "account", "write", "copy", "move",
"delete", "mkdir", "chmod", "chown", "trash", "restore"} {
if !got[Seat+"."+v] {
t.Errorf("no %s", v)
}
}
}
// ADR 0394 §3, §4: a copy or a move onto an existing target overwrites it: the old one goes to the trash, and
// the new one keeps the old one's owner and mode.
func TestACopyOrMoveOntoAnExistingTargetOverwritesIt(t *testing.T) {
g := testFiles(t)
write(t, g.path("new"), "new\n")
write(t, g.path("old"), "old\n")
os.Chmod(g.path("old"), 0o640)
g.owners[g.path("old")] = Ident{4242, 4343}
a, err := g.Copy(by, g.path("new"), g.path("old"))
if err != nil {
t.Fatal(err)
}
if mustRead(t, g.path("old")) != "new\n" || modeOf(t, g.path("old")) != 0o640 {
t.Errorf("copy: %q %o", mustRead(t, g.path("old")), modeOf(t, g.path("old")))
}
if !g.root.called("/usr/bin/chown 4242:4343 -- " + g.path("old")) {
t.Errorf("the old owner was not kept: %v", g.root.calls)
}
if a.Trashed == "" || mustRead(t, filepath.Join(g.State, "trash", a.Trashed, "item")) != "old\n" {
t.Errorf("the old target is not in the trash: %+v", a)
}
write(t, g.path("m"), "moved\n")
os.Chmod(g.path("old"), 0o600)
b, err := g.Move(by, g.path("m"), g.path("old"))
if err != nil || mustRead(t, g.path("old")) != "moved\n" || b.Trashed == "" {
t.Fatalf("move: %v %+v", err, b)
}
if _, err := os.Stat(g.path("m")); err == nil {
t.Error("the source is still there")
}
if modeOf(t, g.path("old")) != 0o600 {
t.Errorf("the overwritten file's mode is kept: %o", modeOf(t, g.path("old")))
}
// a target inside the source, or a source inside the target, is not an overwrite
write(t, g.path("d/f"), "x")
if _, err := g.Copy(by, g.path("d"), g.path("d/in")); err == nil {
t.Error("a copy into itself")
}
if _, err := g.Move(by, g.path("d/f"), g.path("d")); err == nil {
t.Error("a move over its own folder")
}
if mustRead(t, g.path("d/f")) != "x" {
t.Error("the file was lost")
}
}
// ADR 0394 §7: only read and search refuse a place a secret is kept; the writes do not.
func TestTheWritesDoNotRefuseASecretPlace(t *testing.T) {
g := testFiles(t)
k := g.path("home/.ssh/id_ed25519")
write(t, k, "key\n")
for name, run := range map[string]func() error{
"write": func() error { _, err := g.Write(by, k, "new\n", ""); return err },
"chmod": func() error { _, err := g.Chmod(by, k, "0600", false); return err },
"mkdir": func() error { _, err := g.Mkdir(by, g.path("home/.ssh/sub"), false); return err },
"delete": func() error { _, err := g.Delete(by, k, false); return err },
} {
if err := run(); err != nil {
t.Errorf("%s: %v", name, err)
}
}
}
// ADR 0394 §2: where the account's own rights refuse a read, stat, readlink, list and search repeat it through
// root with a fixed binary; a secret place is refused by name first, and root is not asked.
func TestTheReadsFallBackToRoot(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("run as root, nothing is refused")
}
g := testFiles(t)
locked := g.path("locked")
write(t, locked+"/inner/f", "x\n")
os.Chmod(locked, 0)
defer os.Chmod(locked, 0o700)
in := locked + "/inner"
g.root.out["/usr/bin/stat -c %F|%u|%g|%a|%A|%s -- "+in] = "directory|1000|1000|700|drwx------|4096\n"
g.root.out["/usr/bin/stat -c %F|%u|%g|%a|%A|%s -- "+in+"/f"] = "regular file|1000|998|4640|-rwSr-----|2\n"
st, err := g.Stat([]string{in + "/f"}, false)
if err != nil {
t.Fatal(err)
}
cs := st.Paths[0].Components
last := cs[len(cs)-1]
if st.Paths[0].Stopped != "" || last.Type != "file" || last.Mode != "4640" || last.Owner != "ops" ||
last.Group != "wheel" || last.Size == nil || *last.Size != 2 || cs[len(cs)-2].Type != "directory" {
t.Errorf("stat through root: %+v", st.Paths[0])
}
// readlink
g.root.out["/usr/bin/readlink -- "+in+"/l"] = "f\n"
g.root.out["/usr/bin/readlink -e -- "+in+"/l"] = in + "/f\n"
rl, err := g.Readlink(in + "/l")
if err != nil || !rl.IsLink || rl.Link != "f" || rl.LeadsTo != in+"/f" || !rl.Exists {
t.Errorf("readlink through root: %+v %v", rl, err)
}
// list
g.root.out["/usr/bin/find -H "+in+" -mindepth 1 -maxdepth 1 -printf %y\t%U\t%G\t%m\t%s\t%l\t%f\x00"] =
"f\t1000\t998\t640\t2\t\tf\x00d\t1000\t1000\t755\t4096\t\tsub dir\x00l\t1000\t1000\t777\t1\tf\tl\x00"
ls, err := g.List(in)
if err != nil || ls.Count != 3 {
t.Fatalf("list through root: %+v %v", ls, err)
}
byName := map[string]Entry{}
for _, e := range ls.Entries {
byName[e.Name] = e
}
if e := byName["f"]; e.Type != "file" || e.Mode != "0640" || e.Group != "wheel" || e.Size == nil || *e.Size != 2 {
t.Errorf("entry f: %+v", e)
}
if byName["sub dir"].Type != "directory" || byName["l"].Type != "link" || byName["l"].Link != "f" {
t.Errorf("entries: %+v", ls.Entries)
}
// search: the files are listed by find, the secret ones skipped by name, the rest read through cat
g.root.out["/usr/bin/find -H "+locked+" -maxdepth 5 -xdev -type f -printf %p\x00"] =
in + "/f\x00" + locked + "/.ssh/id_rsa\x00"
g.root.cat[in+"/f"] = "one\nneedle here\n"
g.root.reach(in + "/f")
sr, err := g.Search(SearchAsk{Text: "needle", Path: locked})
if err != nil || len(sr.Matches) != 1 || sr.Matches[0].File != in+"/f" || sr.Matches[0].Line != 2 {
t.Fatalf("search through root: %+v %v", sr, err)
}
if len(sr.Skipped) != 1 || !strings.HasSuffix(sr.Skipped[0], "id_rsa") {
t.Errorf("a credential file was not skipped by name: %+v", sr.Skipped)
}
if g.root.called("/usr/bin/cat -- " + locked + "/.ssh/id_rsa") {
t.Error("root read a credential file")
}
// a secret place is refused by name, and root is never asked
g.root.calls = nil
if _, err := g.List(g.path("home/.ssh")); err == nil {
t.Error("a secret place was listed")
}
if _, err := g.Read(locked+"/.ssh/id_rsa", ReadAsk{}); err == nil || len(g.root.calls) != 0 {
t.Errorf("a credential file was read: %v %v", err, g.root.calls)
}
}
// Review 1, finding 3: before root reads a file, root says where it leads and what it is. A link into a refused
// place, and a file that is not a regular file, are refused before any cat.
func TestRootNeverCatsAThroughALinkIntoASecretPlaceOrANonFile(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("run as root, nothing is refused")
}
g := testFiles(t)
locked := g.path("locked")
write(t, locked+"/x", "x\n")
os.Chmod(locked, 0)
defer os.Chmod(locked, 0o700)
g.root.out["/usr/bin/readlink -e -- "+locked+"/link"] = "/etc/ssl/private/k\n"
g.root.cat["/etc/ssl/private/k"] = "KEY\n"
g.root.cat[locked+"/link"] = "KEY\n"
if _, err := g.Read(locked+"/link", ReadAsk{}); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("a link into a secret place: %v", err)
}
g.root.out["/usr/bin/readlink -e -- "+locked+"/fifo"] = locked + "/fifo\n"
g.root.out["/usr/bin/stat -c "+statFormat+" -- "+locked+"/fifo"] = "fifo|1000|1000|600|prw-------|0\n"
g.root.cat[locked+"/fifo"] = "never\n"
if _, err := g.Read(locked+"/fifo", ReadAsk{}); err == nil || !strings.Contains(err.Error(), "not a file") {
t.Errorf("a fifo: %v", err)
}
// a search meets the same two
g.root.out["/usr/bin/find -H "+locked+" -maxdepth 5 -xdev -type f -printf %p\x00"] = locked + "/link\x00" + locked + "/fifo\x00"
sr, err := g.Search(SearchAsk{Text: "KEY", Path: locked})
if err != nil || len(sr.Matches) != 0 {
t.Errorf("search: %+v %v", sr, err)
}
for _, c := range g.root.calls {
if strings.HasPrefix(c, "/usr/bin/cat") {
t.Errorf("root read: %s", c)
}
}
}
// Review 1, finding 4: a directory target that holds an owned path is replaced only with an override.
func TestOverwritingAFolderThatHoldsAnOwnedPathIsRefused(t *testing.T) {
g := testFiles(t)
g.Owned[g.path("dst/etc")] = "mesh-hosts (node-hostname)"
write(t, g.path("dst/etc/hosts"), "old\n")
write(t, g.path("src/a"), "A\n")
if _, err := g.Copy(by, g.path("src"), g.path("dst")); err == nil || !strings.Contains(err.Error(), "mesh-hosts") {
t.Errorf("copy: %v", err)
}
write(t, g.path("src2/a"), "A\n")
if _, err := g.Move(by, g.path("src2"), g.path("dst")); err == nil || !strings.Contains(err.Error(), "mesh-hosts") {
t.Errorf("move: %v", err)
}
if mustRead(t, g.path("dst/etc/hosts")) != "old\n" {
t.Error("the owned file was replaced")
}
}
// Review 1, finding 5: a copy keeps setuid and setgid: the owner is set before the mode, as a chown clears them.
func TestACopyKeepsSetuid(t *testing.T) {
g := testFiles(t)
g.Lchown = func(p string, uid, gid int) error { return os.Lchown(p, os.Getuid(), os.Getgid()) }
write(t, g.path("tool"), "x\n")
if err := os.Chmod(g.path("tool"), 0o755|fs.ModeSetuid|fs.ModeSetgid); err != nil {
t.Fatal(err)
}
g.owners[g.path("tool")] = Ident{4242, 4343}
if _, err := g.Copy(by, g.path("tool"), g.path("tool2")); err != nil {
t.Fatal(err)
}
info, _ := os.Lstat(g.path("tool2"))
if info.Mode()&fs.ModeSetuid == 0 || info.Mode()&fs.ModeSetgid == 0 {
t.Errorf("the copy lost setuid or setgid: %v", info.Mode())
}
}
// Review 1, finding 1: every seat verb the code asks is declared in invokes.
func TestEverySeatVerbTheCodeAsksIsDeclared(t *testing.T) {
raw, _ := os.ReadFile("../../module.json")
var m struct{ Invokes []string }
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
declared := map[string]bool{}
for _, i := range m.Invokes {
declared[i] = true
}
files, _ := filepath.Glob("*.go")
asked := map[string]bool{}
for _, f := range files {
if strings.HasSuffix(f, "_test.go") {
continue
}
src := mustRead(t, f)
for _, hit := range regexp.MustCompile(`"(seat:[a-z-]+\.[a-z_-]+)"`).FindAllStringSubmatch(src, -1) {
asked[hit[1]] = true
}
}
if len(asked) == 0 {
t.Fatal("found no seat verb asked")
}
for a := range asked {
if !declared[a] {
t.Errorf("%s is asked but not in invokes %v", a, m.Invokes)
}
}
for d := range declared {
if !asked[d] {
t.Errorf("%s is declared but never asked", d)
}
}
}
+328
View File
@@ -0,0 +1,328 @@
// mesh-files (novox/hq ADR 0394, to-be 41 WP9): the holder of node-files. A node's files and accounts, read and
// managed through the mesh — the owner and mode of each component of a path, a link, a directory, a text file,
// a search by fixed string, an account or a group; and the writes: write, copy, move, delete, mkdir, chmod,
// chown, and the trash that delete and overwrite use.
//
// It runs as the tool runner's account, the node's operator account, and acts with that account's own rights
// first; where that fails with a permission error it repeats the act through `sudo -n` with a fixed binary
// (sudo.go), never a shell. It refuses by name the places a secret is kept (refuse.go), withholds any piece of
// an answer shaped like a secret (mesh-sdk's outward), and refuses a change of a path a module owns unless a
// reason is given (files.go). Every write act is one line in the act log (files.go).
//
// stdout is the MCP channel; what this module says, it says on stderr.
package main
import (
"encoding/json"
"fmt"
"os"
"strconv"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Seat is the role this module holds.
const Seat = "node-files"
// PlacesWord is where the controller names this node's secret places (novox/hq ADR 0394 §5).
const PlacesWord = "MESH_SECRET_PLACES"
func main() {
places, err := placesFrom(os.Getenv(PlacesWord))
if err != nil {
fmt.Fprintf(os.Stderr, "[mesh-files] %v\n", err)
os.Exit(1)
}
f, err := newFiles(places)
if err != nil {
fmt.Fprintf(os.Stderr, "[mesh-files] %v\n", err)
os.Exit(1)
}
if err := stdio.Serve("", tools(f)); err != nil {
fmt.Fprintf(os.Stderr, "[mesh-files] %v\n", err)
os.Exit(1)
}
}
// StateWord is where the module's state directory (the trash and the act log) is named.
const StateWord = "MESH_FILES_STATE"
func newFiles(places []string) (*Files, error) {
state := os.Getenv(StateWord)
if strings.TrimSpace(state) == "" {
return nil, fmt.Errorf("%s is not given: the trash and the act log have no place", StateWord)
}
owned, err := ownedFrom(os.Getenv(OwnedWord))
if err != nil {
return nil, err
}
if len(owned) == 0 {
fmt.Fprintf(os.Stderr, "[mesh-files] %s is not given, so this node's owned paths are unknown and nothing is "+
"refused as owned (novox/hq ADR 0394 §5): a controller that knows node-files composes it\n", OwnedWord)
}
r := newReader(places)
r.Root = sudoRoot{}
node := os.Getenv("MESH_NODE")
return &Files{Reader: r, State: state, Owned: owned, Node: node, Tell: tellOperator,
OwnerOf: ownerOnDisk, Lchown: os.Lchown, NewID: newID}, nil
}
// tellOperator tells the operator a plain notice, through the operator channel's tell verb: no condition is
// opened, and nothing is kept open.
func tellOperator(headline string, lines []string, subject string) error {
_, err := stdio.Ask("seat:operator-channel.tell", map[string]any{"headline": headline, "lines": lines, "subject": subject})
return err
}
// placesFrom reads the controller's list. Absent is refused: without it, a module's own-secret directory
// would be read wherever the account's modes allow, and a refusal the decision promises would be silent.
func placesFrom(raw string) ([]string, error) {
if strings.TrimSpace(raw) == "" {
return nil, fmt.Errorf("%s is not given, so this node's secret places are unknown and nothing would be "+
"refused by them (novox/hq ADR 0394 §5): a controller that knows node-files composes it", PlacesWord)
}
var places []string
if err := json.Unmarshal([]byte(raw), &places); err != nil {
return nil, fmt.Errorf("%s is not a list of paths: %v", PlacesWord, err)
}
return places, nil
}
func str(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
func arg(a map[string]any, k string) string {
switch v := a[k].(type) {
case string:
return strings.TrimSpace(v)
case float64:
return strconv.FormatFloat(v, 'f', -1, 64)
case bool:
return strconv.FormatBool(v)
}
return ""
}
func number(a map[string]any, k string) (int, error) {
s := arg(a, k)
if s == "" {
return 0, nil
}
n, err := strconv.Atoi(s)
if err != nil {
return 0, fmt.Errorf("%s is a whole number, and %q was given", k, s)
}
return n, nil
}
// pathsArg is the paths given, one per line — or, from a caller that sends a list, the list.
func pathsArg(a map[string]any) []string {
var raw []string
switch v := a["paths"].(type) {
case string:
raw = strings.Split(v, "\n")
case []any:
for _, p := range v {
if s, ok := p.(string); ok {
raw = append(raw, s)
}
}
}
var out []string
for _, p := range raw {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
// bounded is the last bound (novox/hq ADR 0394): whatever a verb made, no answer is larger than MaxAnswer.
func bounded(answer any, err error) (any, error) {
if err != nil {
return nil, err
}
body, err := json.Marshal(answer)
if err != nil {
return nil, err
}
if len(body) > MaxAnswer {
return nil, fmt.Errorf("the answer is %d bytes, and an answer of node-files is at most %d: ask for less", len(body), MaxAnswer)
}
return answer, nil
}
// verb is one of the seat's verbs: listed as `<seat>.<verb>`, so the runtime serves it on the seat's subject.
// The descriptions are the controller's (internal/catalogue/reads.go), said again here for a caller of the
// module's own address.
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input,
Run: func(a map[string]any) (any, error) { return bounded(run(a)) }}
}
func object(props map[string]any, required ...string) map[string]any {
if required == nil {
required = []string{}
}
return map[string]any{"type": "object", "properties": props, "required": required}
}
// The descriptions of the reads are the controller's (internal/catalogue/files.go): the reads' own, with
// where they are read as restated for the operator's account.
const asAccount = "as the node's operator account, through sudo -n where its rights do not reach; a place a " +
"secret is kept, or a known credential file, is refused by name"
func boolArg(a map[string]any, k string) bool { return arg(a, k) == "true" }
func actOf(a map[string]any) Act {
return Act{Caller: stdio.CallerOf(a), Reason: arg(a, "reason"), Override: arg(a, "override")}
}
// write is one of the write verbs: it takes reason and override beside its own inputs.
func writeVerb(name, description string, props map[string]any, required []string,
run func(act Act, a map[string]any) (any, error)) stdio.Tool {
props["reason"] = str("why the file is changed, for the audit trail")
props["override"] = str("a reason, to change a path a module owns anyway (optional)")
return verb(name, description+" Refuses a path a module owns unless override names a reason; done as the node's "+
"operator account, through sudo -n where its rights do not reach.", object(props, append(required, "reason")...),
func(a map[string]any) (any, error) { return run(actOf(a), a) })
}
func tools(f *Files) []stdio.Tool {
r := f.Reader
return append(readTools(r),
writeVerb("write", "Write one text file whole, replacing what it holds; the old content goes to the node's trash.",
map[string]any{"path": str("an absolute path"), "content": str("the text to write"),
"mode": str("the file's mode, such as 0644 (optional)")}, []string{"path", "content"},
func(act Act, a map[string]any) (any, error) {
content, _ := a["content"].(string) // as given: the text is not trimmed
return f.Write(act, arg(a, "path"), content, arg(a, "mode"))
}),
writeVerb("copy", "Copy a file or directory to a path on this node. A target on another node (<node>:/path) is "+
"refused: copying between nodes comes in step 2 (issue 903). An existing target is overwritten, the old one "+
"kept in the node's trash for 7 days.",
map[string]any{"from": str("an absolute path"), "to": str("an absolute path on this node")},
[]string{"from", "to"},
func(act Act, a map[string]any) (any, error) { return f.Copy(act, arg(a, "from"), arg(a, "to")) }),
writeVerb("move", "Move or rename a file or directory. A target on another node is refused: copying between nodes "+
"comes in step 2 (issue 903). An existing target is overwritten, the old one kept in the node's trash for 7 days.",
map[string]any{"from": str("an absolute path"), "to": str("an absolute path")}, []string{"from", "to"},
func(act Act, a map[string]any) (any, error) { return f.Move(act, arg(a, "from"), arg(a, "to")) }),
writeVerb("delete", "Delete a file or directory into the node's trash, where restore can bring it back.",
map[string]any{"path": str("an absolute path"),
"recursive": str("\"true\": delete a directory with what it holds")}, []string{"path"},
func(act Act, a map[string]any) (any, error) {
return f.Delete(act, arg(a, "path"), boolArg(a, "recursive"))
}),
writeVerb("mkdir", "Make a directory.",
map[string]any{"path": str("an absolute path"),
"parents": str("\"true\": make the missing parents too")}, []string{"path"},
func(act Act, a map[string]any) (any, error) {
return f.Mkdir(act, arg(a, "path"), boolArg(a, "parents"))
}),
writeVerb("chmod", "Change the mode of a file or directory.",
map[string]any{"path": str("an absolute path"), "mode": str("the mode, such as 0644 or u+x"),
"recursive": str("\"true\": also everything under a directory")}, []string{"path", "mode"},
func(act Act, a map[string]any) (any, error) {
return f.Chmod(act, arg(a, "path"), arg(a, "mode"), boolArg(a, "recursive"))
}),
writeVerb("chown", "Change the owner and group of a file or directory.",
map[string]any{"path": str("an absolute path"), "owner": str("an account's name or number"),
"group": str("a group's name or number (optional)"),
"recursive": str("\"true\": also everything under a directory")}, []string{"path", "owner"},
func(act Act, a map[string]any) (any, error) {
return f.Chown(act, arg(a, "path"), arg(a, "owner"), arg(a, "group"), boolArg(a, "recursive"))
}),
verb("trash", "What the node's trash holds: each deleted or replaced item with its id, original path and when "+
"it went.", object(map[string]any{}),
func(map[string]any) (any, error) { return f.Trash() }),
verb("restore", "Put an item of the node's trash back where it was; refuses when something is there now.",
object(map[string]any{"id": str("an id from trash"),
"reason": str("why it is put back, for the audit trail"),
"override": str("a reason, to change a path a module owns anyway (optional)")}, "id", "reason"),
func(a map[string]any) (any, error) { return f.Restore(actOf(a), arg(a, "id")) }),
)
}
// readTools are the reads, each saying how it reads.
func readTools(r Reader) []stdio.Tool {
ts := rawReadTools(r)
for i := range ts {
if !strings.Contains(ts[i].Description, asAccount) {
ts[i].Description += " Read " + asAccount + "."
}
}
return ts
}
func rawReadTools(r Reader) []stdio.Tool {
return []stdio.Tool{
verb("stat", "Each component of one path or several, from / down: its type, owner and group (name and "+
"number), mode, whether it is a link and to what, and whether it is a mount point and of which file "+
"system. A link is not followed unless follow is \"true\". At most 200 paths in one call. Read "+asAccount+".",
object(map[string]any{"paths": str("one absolute path, or several separated by line breaks"),
"follow": str("\"true\": also say where the path leads once every link is followed")}, "paths"),
func(a map[string]any) (any, error) { return r.Stat(pathsArg(a), arg(a, "follow") == "true") }),
verb("readlink", "What one link points to, as written, and where the path leads once every link is "+
"followed, and whether that exists.",
object(map[string]any{"path": str("an absolute path")}, "path"),
func(a map[string]any) (any, error) { return r.Readlink(arg(a, "path")) }),
verb("list", "Every entry of one directory with its type, owner, group, mode, size and, for a link, its "+
"target. At most 2000 entries; a longer directory is cut and says so. A place a secret is kept is "+
"refused by name.",
object(map[string]any{"path": str("an absolute path to a directory")}, "path"),
func(a map[string]any) (any, error) { return r.List(arg(a, "path")) }),
verb("read", "One text file, or a part of it by line: at most 256 KiB in one answer (64 KiB unless more "+
"is asked), cut at a line and said so. A negative from counts from the end, as tail does. A binary file "+
"is refused; a place a secret is kept, or a known credential file, is refused by name; a line shaped "+
"like a secret is withheld and its number said. /proc and /sys are read as far as the account may. Read "+asAccount+".",
object(map[string]any{"path": str("an absolute path to a file"),
"from": str("the first line to answer, counted from 1; negative: that many lines from the end (optional)"),
"lines": str("how many lines to answer (optional)"),
"bytes": str("the most bytes to answer, up to 262144 (optional, 65536 unless given)")}, "path"),
func(a map[string]any) (any, error) {
var ask ReadAsk
var err error
if ask.From, err = number(a, "from"); err != nil {
return nil, err
}
if ask.Lines, err = number(a, "lines"); err != nil {
return nil, err
}
if ask.Bytes, err = number(a, "bytes"); err != nil {
return nil, err
}
return r.Read(arg(a, "path"), ask)
}),
verb("search", "Find a fixed string (never a pattern) in one file or under one directory: every matching "+
"line with its file and number. Bounded: at most 500 matches (100 unless more is asked), 8 directories "+
"deep (4 unless more is asked), 5000 files, 4 MiB per file and 20 seconds, on the file system of the path "+
"named; links are not followed and binary files are skipped. A place a secret is kept is skipped and named; a matching line shaped like "+
"a secret is withheld. The answer says which bound it met.",
object(map[string]any{"text": str("the fixed string to find"),
"path": str("an absolute path to a file or a directory"),
"depth": str("how many directories deep, up to 8 (optional)"),
"matches": str("the most matches to answer, up to 500 (optional)"),
"case": str("\"false\": match whatever the case (optional)")}, "text", "path"),
func(a map[string]any) (any, error) {
text, _ := a["text"].(string) // as given: a space at either end is part of what is found
ask := SearchAsk{Text: text, Path: arg(a, "path"), CaseInsensitive: arg(a, "case") == "false"}
var err error
if ask.Depth, err = number(a, "depth"); err != nil {
return nil, err
}
if ask.Matches, err = number(a, "matches"); err != nil {
return nil, err
}
return r.Search(ask)
}),
verb("account", "One account or group by name or number, as this node's account files say: an account's "+
"number, group, home, shell and every group it is in; a group's number and members. With neither, every "+
"account and group. Never a password field.",
object(map[string]any{"name": str("an account's name or number (optional)"),
"group": str("a group's name or number (optional)")}),
func(a map[string]any) (any, error) { return r.Account(arg(a, "name"), arg(a, "group")) }),
}
}
@@ -0,0 +1,64 @@
package main
import (
"encoding/json"
"os"
"testing"
)
// ADR 0394 §2: the holder runs as the tool runner's account, the node's operator account: it declares no
// runs-as and makes no account, claims node-files once on each node and serves every verb of the seat, keeps
// a state directory for the trash and the act log, and may tell the operator's channel of an override.
func TestTheHolderRunsAsTheToolRunnersAccount(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
RunsAs string `json:"runs-as"`
Invokes []string `json:"invokes"`
Claims []struct {
Name, Scope string
Serves []string
} `json:"claims"`
Resources []map[string]any `json:"resources"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
if m.RunsAs != "" {
t.Errorf("runs as %q", m.RunsAs)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || m.Claims[0].Scope != "node" {
t.Fatalf("claims %v", m.Claims)
}
served := map[string]bool{}
for _, v := range m.Claims[0].Serves {
served[v] = true
}
for _, tool := range tools(testFiles(t).Files) {
name := tool.Name[len(Seat)+1:]
if !served[name] {
t.Errorf("the manifest does not serve %s", name)
}
delete(served, name)
}
if len(served) > 0 {
t.Errorf("the manifest serves what the module does not: %v", served)
}
state := false
for _, r := range m.Resources {
if r["type"] == "user" {
t.Errorf("the module makes an account: %v", r)
}
if r["id"] == "state" && r["type"] == "directory" {
state = true
}
}
if !state {
t.Error("no state directory for the trash and the act log")
}
if len(m.Invokes) != 1 || m.Invokes[0] != "seat:operator-channel.tell" {
t.Errorf("invokes %v", m.Invokes)
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,534 @@
package main
import (
crand "crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/json"
"encoding/pem"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"syscall"
"testing"
"time"
)
// A token shaped as a secret, written into files the tests read.
const aToken = "ghp_abcdefghijklmnopqrstuvwxyz0123456789"
func testReader(t *testing.T, places ...string) (Reader, string) {
t.Helper()
dir := t.TempDir()
passwd := filepath.Join(dir, "passwd")
group := filepath.Join(dir, "group")
mountinfo := filepath.Join(dir, "mountinfo")
write(t, passwd, "root:x:0:0::/root:/usr/bin/bash\nops:x:1000:1000:The Operator:/home/ops:/usr/bin/zsh\n"+
"mesh-files:x:961:961::/var/lib/mesh-files:/usr/bin/nologin\n")
write(t, group, "root:x:0:\nwheel:x:998:ops\nops:x:1000:\nmesh-files:x:961:\ndocker:x:970:ops,builder\n")
data := filepath.Join(dir, "data")
write(t, mountinfo, "22 1 0:21 / / rw,relatime - btrfs /dev/nvme0n1p2 rw\n"+
"40 22 0:40 / "+strings.ReplaceAll(data, " ", `\040`)+" rw - nfs4 home-server:/tank/data rw\n")
r := Reader{Refuser: Refuser{Places: places}, Passwd: passwd, Group: group, MountInfo: mountinfo,
Now: time.Now}
return r, dir
}
func write(t *testing.T, path, content string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
// ADR 0394 §5: without the controller's list of secret places it does not start: a refusal it promises would be
// silent.
func TestItDoesNotStartWithoutTheNodesSecretPlaces(t *testing.T) {
if _, err := placesFrom(""); err == nil || !strings.Contains(err.Error(), PlacesWord) {
t.Errorf("no places: %v", err)
}
if p, err := placesFrom(`["/var/lib/mesh-vault"]`); err != nil || len(p) != 1 {
t.Errorf("a list: %v %v", p, err)
}
if p, err := placesFrom(`[]`); err != nil || len(p) != 0 {
t.Errorf("an empty list is a node with no secret place: %v %v", p, err)
}
}
// ADR 0394 §5: a path under the vault's directory, a module's own-secret directory and a known credential file
// are each refused by name before any read — though the account's modes would allow it — and a link to one is
// refused as the place is.
func TestSecretPlacesAreRefusedByNameBeforeAnyRead(t *testing.T) {
r0, dir := testReader(t)
vault := filepath.Join(dir, "var/lib/mesh-vault")
own := filepath.Join(dir, "var/lib/desk-channel")
r, _ := r0, dir
r.Places = []string{vault, own}
for _, p := range []string{
filepath.Join(vault, "ledger/entries.json"),
filepath.Join(own, "broker"),
filepath.Join(own, "settings.json"),
filepath.Join(dir, "home/ops/.claude/.credentials.json"),
filepath.Join(dir, "home/ops/.ssh/config"),
filepath.Join(dir, "home/ops/.pgpass"),
filepath.Join(dir, "etc/ssh/ssh_host_ed25519_key"),
filepath.Join(dir, "etc/shadow"),
filepath.Join(dir, "srv/tls/server.key"),
} {
write(t, p, "harmless words\n")
if _, err := r.Read(p, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("%s was read: %v", p, err)
}
}
if _, err := r.List(vault); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("the vault's directory was listed: %v", err)
}
link := filepath.Join(dir, "tmp/innocent")
write(t, filepath.Join(dir, "tmp/x"), "")
if err := os.Symlink(filepath.Join(own, "broker"), link); err != nil {
t.Fatal(err)
}
if _, err := r.Read(link, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "leads to") {
t.Errorf("a link to a secret place was read: %v", err)
}
// A search under a directory holding a secret place skips it and names it.
write(t, filepath.Join(dir, "home/ops/notes.txt"), "harmless words\n")
a, err := r.Search(SearchAsk{Text: "harmless", Path: filepath.Join(dir, "home"), Depth: 8})
if err != nil {
t.Fatal(err)
}
if len(a.Matches) != 1 || !strings.HasSuffix(a.Matches[0].File, "notes.txt") || len(a.Skipped) != 3 {
t.Errorf("search: matches %v, skipped %v", a.Matches, a.Skipped)
}
// What a public key or a certificate is, is not refused.
for _, p := range []string{filepath.Join(dir, "etc/ssh/ssh_host_ed25519_key.pub"), filepath.Join(dir, "etc/ssl/certs/ca.pem")} {
write(t, p, "harmless words\n")
if _, err := r.Read(p, ReadAsk{}); err != nil {
t.Errorf("%s was refused: %v", p, err)
}
}
}
// ADR 0394 §3: what the account's file modes refuse is refused — the first bound, below the names.
func TestAFileTheAccountMayNotReadIsRefusedByItsMode(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("run as root, a file's mode refuses nothing")
}
r, dir := testReader(t)
p := filepath.Join(dir, "operators-only")
write(t, p, "the operator's words\n")
if err := os.Chmod(p, 0o000); err != nil {
t.Fatal(err)
}
if _, err := r.Read(p, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "may not reach it") {
t.Errorf("a file of mode 0000: %v", err)
}
closed := filepath.Join(dir, "closed")
write(t, filepath.Join(closed, "inside"), "x")
if err := os.Chmod(closed, 0o700&^0o100); err != nil {
t.Fatal(err)
}
defer os.Chmod(closed, 0o755)
a, err := r.Stat([]string{filepath.Join(closed, "inside")}, false)
if err != nil {
t.Fatal(err)
}
if got := a.Paths[0]; !strings.Contains(got.Stopped, "may not reach") {
t.Errorf("a path through a directory it may not search: %+v", got)
}
}
// ADR 0394 §2 and the bounds: a read over the size bound is cut at a line and says so; more than the most is
// refused; lines and a tail are answered as asked.
func TestAReadIsBounded(t *testing.T) {
r, dir := testReader(t)
p := filepath.Join(dir, "big")
var b strings.Builder
for i := 1; i <= 20000; i++ {
fmt.Fprintf(&b, "line %d of the file\n", i)
}
write(t, p, b.String())
a, err := r.Read(p, ReadAsk{})
if err != nil {
t.Fatal(err)
}
if len(a.Text) > DefaultReadSize || !strings.Contains(a.Cut, fmt.Sprint(DefaultReadSize)) || a.From != 1 {
t.Errorf("a default read: %d bytes, cut %q", len(a.Text), a.Cut)
}
if _, err := r.Read(p, ReadAsk{Bytes: MaxReadSize + 1}); err == nil {
t.Error("more than the most a read answers was answered")
}
a, err = r.Read(p, ReadAsk{From: 10, Lines: 2})
if err != nil || a.Text != "line 10 of the file\nline 11 of the file\n" || a.From != 10 || a.Cut != "" {
t.Errorf("lines 10 and 11: %+v %v", a, err)
}
a, err = r.Read(p, ReadAsk{From: -2})
if err != nil || a.Text != "line 19999 of the file\nline 20000 of the file\n" || a.From != 19999 {
t.Errorf("the last two lines: %q from %d, %v", a.Text, a.From, err)
}
if _, err := r.Read(p, ReadAsk{From: 30000}); err == nil || !strings.Contains(err.Error(), "20000 lines") {
t.Errorf("a line past the end: %v", err)
}
}
// A binary file, a directory and a fifo are refused, and the fifo without waiting for a writer.
func TestOnlyATextFileIsRead(t *testing.T) {
r, dir := testReader(t)
bin := filepath.Join(dir, "bin")
write(t, bin, "ELF\x00\x01\x02")
if _, err := r.Read(bin, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "binary") {
t.Errorf("a binary file: %v", err)
}
if _, err := r.Read(dir, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "directory") {
t.Errorf("a directory: %v", err)
}
fifo := filepath.Join(dir, "fifo")
if err := syscall.Mkfifo(fifo, 0o644); err != nil {
t.Fatal(err)
}
done := make(chan error, 1)
go func() { _, err := r.Read(fifo, ReadAsk{}); done <- err }()
select {
case err := <-done:
if err == nil || !strings.Contains(err.Error(), "fifo") {
t.Errorf("a fifo: %v", err)
}
case <-time.After(5 * time.Second):
t.Fatal("a read of a fifo waits for a writer")
}
}
// ADR 0394 §4: a line shaped like a secret is withheld, its number said, and the rest is answered.
func TestALineShapedLikeASecretIsWithheld(t *testing.T) {
r, dir := testReader(t)
p := filepath.Join(dir, "app.conf")
write(t, p, "listen = /run/app.sock\ntoken = "+aToken+"\nuser = app\n")
a, err := r.Read(p, ReadAsk{})
if err != nil {
t.Fatal(err)
}
if strings.Contains(a.Text, aToken) || !strings.Contains(a.Text, "user = app") ||
!strings.Contains(a.Text, "listen = /run/app.sock") || fmt.Sprint(a.Withheld) != "[2]" {
t.Errorf("read: %q withheld %v", a.Text, a.Withheld)
}
m, err := r.Search(SearchAsk{Text: "token", Path: dir})
if err != nil {
t.Fatal(err)
}
body, _ := json.Marshal(m)
if strings.Contains(string(body), aToken) || len(m.Matches) != 1 || len(m.Withheld) != 1 {
t.Errorf("search: %s", body)
}
}
// The bounds of a search: matches, depth, the most asked, and time — each met is said.
func TestASearchIsBounded(t *testing.T) {
r, dir := testReader(t)
for i := 0; i < 30; i++ {
write(t, filepath.Join(dir, fmt.Sprintf("f%02d", i)), "needle\nneedle\n")
}
a, err := r.Search(SearchAsk{Text: "needle", Path: dir, Matches: 7})
if err != nil {
t.Fatal(err)
}
if len(a.Matches) != 7 || !strings.HasPrefix(a.Bound, "matches") {
t.Errorf("seven matches asked: %d, bound %q", len(a.Matches), a.Bound)
}
if _, err := r.Search(SearchAsk{Text: "needle", Path: dir, Matches: MaxMatches + 1}); err == nil {
t.Error("more matches than the most were asked and answered")
}
if _, err := r.Search(SearchAsk{Text: "needle", Path: dir, Depth: MaxDepth + 1}); err == nil {
t.Error("deeper than the most was asked and answered")
}
deep := t.TempDir()
write(t, filepath.Join(deep, "a/b/c/shallow"), "needle\n")
write(t, filepath.Join(deep, "a/b/c/d/e/deep"), "needle\n")
a, err = r.Search(SearchAsk{Text: "needle", Path: deep, Depth: 3})
if err != nil {
t.Fatal(err)
}
if len(a.Matches) != 1 || !strings.HasSuffix(a.Matches[0].File, "shallow") {
t.Errorf("three deep: %+v", a.Matches)
}
// Case: as asked.
write(t, filepath.Join(deep, "case"), "NEEDLE\n")
if a, _ := r.Search(SearchAsk{Text: "needle", Path: filepath.Join(deep, "case")}); len(a.Matches) != 0 {
t.Error("a match whatever the case, unasked")
}
if a, _ := r.Search(SearchAsk{Text: "needle", Path: filepath.Join(deep, "case"), CaseInsensitive: true}); len(a.Matches) != 1 {
t.Error("no match whatever the case, asked")
}
// Time: a clock past the bound stops the walk and says so.
start := time.Now()
calls := 0
r.Now = func() time.Time { calls++; return start.Add(time.Duration(calls) * SearchTime / 3) }
a, err = r.Search(SearchAsk{Text: "needle", Path: dir})
if err != nil || !strings.HasPrefix(a.Bound, "time") {
t.Errorf("the time bound: %q %v", a.Bound, err)
}
// A fixed string: a pattern's characters are themselves.
write(t, filepath.Join(deep, "pattern"), "a.c\nabc\n")
if a, _ := r.Search(SearchAsk{Text: "a.c", Path: filepath.Join(deep, "pattern")}); len(a.Matches) != 1 || a.Matches[0].Line != 1 {
t.Errorf("a.c as a fixed string: %+v", a.Matches)
}
}
// The survey issue 418 was asked for: each component of a path with its owner, group, mode, whether it is a
// link and to what, and whether it is a mount point — the link not followed unless asked.
func TestStatSaysEachComponent(t *testing.T) {
r, dir := testReader(t)
data := filepath.Join(dir, "data")
write(t, filepath.Join(data, "real/file"), "x")
if err := os.Chmod(filepath.Join(data, "real"), os.ModeSetgid|0o750); err != nil {
t.Fatal(err)
}
if err := os.Symlink("real", filepath.Join(data, "link")); err != nil {
t.Fatal(err)
}
a, err := r.Stat([]string{filepath.Join(data, "link"), filepath.Join(data, "real/file")}, false)
if err != nil {
t.Fatal(err)
}
link := a.Paths[0]
last := link.Components[len(link.Components)-1]
if link.Components[0].Path != "/" || last.Type != "link" || last.Link != "real" || link.LeadsTo != "" {
t.Errorf("the link, not followed: %+v", link)
}
var dataC, realC Component
for _, c := range a.Paths[1].Components {
switch c.Path {
case data:
dataC = c
case filepath.Join(data, "real"):
realC = c
}
}
if dataC.Mount == nil || dataC.Mount.FS != "nfs4" {
t.Errorf("the mount point is not said: %+v", dataC)
}
if realC.Mode != "2750" || realC.Type != "directory" || realC.UID == nil || realC.Perms == "" {
t.Errorf("a component's mode: %+v", realC)
}
if uid := *realC.UID; uid == 1000 && realC.Owner != "ops" {
t.Errorf("owner %d named %q", uid, realC.Owner)
}
a, err = r.Stat([]string{filepath.Join(data, "link")}, true)
if err != nil {
t.Fatal(err)
}
if a.Paths[0].LeadsTo != filepath.Join(data, "real") || a.Paths[0].Leads == nil || a.Paths[0].Leads.Type != "directory" {
t.Errorf("followed: %+v", a.Paths[0])
}
a, err = r.Stat([]string{filepath.Join(dir, "missing/deeper")}, false)
if err != nil || !strings.Contains(a.Paths[0].Stopped, "does not exist") {
t.Errorf("a missing path: %+v %v", a, err)
}
if _, err := r.Stat([]string{"relative/path"}, false); err == nil {
t.Error("a relative path was answered")
}
many := make([]string, MaxPaths+1)
for i := range many {
many[i] = "/"
}
if _, err := r.Stat(many, false); err == nil {
t.Error("more paths than the most were answered")
}
}
func TestReadlinkSaysTheTargetAndWhereItLeads(t *testing.T) {
r, dir := testReader(t)
write(t, filepath.Join(dir, "target"), "")
if err := os.Symlink("target", filepath.Join(dir, "l1")); err != nil {
t.Fatal(err)
}
if err := os.Symlink("l1", filepath.Join(dir, "l2")); err != nil {
t.Fatal(err)
}
if err := os.Symlink("nowhere", filepath.Join(dir, "dangling")); err != nil {
t.Fatal(err)
}
a, err := r.Readlink(filepath.Join(dir, "l2"))
if err != nil || !a.IsLink || a.Link != "l1" || a.LeadsTo != filepath.Join(dir, "target") || !a.Exists {
t.Errorf("l2: %+v %v", a, err)
}
a, err = r.Readlink(filepath.Join(dir, "dangling"))
if err != nil || !a.IsLink || a.Exists || !strings.Contains(a.Stopped, "does not exist") {
t.Errorf("dangling: %+v %v", a, err)
}
}
func TestListSaysEveryEntry(t *testing.T) {
r, dir := testReader(t)
write(t, filepath.Join(dir, "d/a"), "abc")
if err := os.Symlink("a", filepath.Join(dir, "d/b")); err != nil {
t.Fatal(err)
}
a, err := r.List(filepath.Join(dir, "d"))
if err != nil {
t.Fatal(err)
}
if a.Count != 2 || a.Entries[0].Name != "a" || *a.Entries[0].Size != 3 || a.Entries[1].Type != "link" || a.Entries[1].Link != "a" {
t.Errorf("list: %+v", a)
}
for i := 0; i < MaxEntries+5; i++ {
write(t, filepath.Join(dir, "many", fmt.Sprintf("e%05d", i)), "")
}
a, err = r.List(filepath.Join(dir, "many"))
if err != nil || len(a.Entries) != MaxEntries || a.Count != MaxEntries+5 || a.Cut == "" {
t.Errorf("a long directory: %d of %d, cut %q, %v", len(a.Entries), a.Count, a.Cut, err)
}
}
// getent passwd, getent group and id: an account with every group it is in, a group with its members, never a
// password field.
func TestAccountAndGroupLookup(t *testing.T) {
r, _ := testReader(t)
a, err := r.Account("ops", "")
if err != nil {
t.Fatal(err)
}
u := a.Accounts[0]
sort.Strings(u.Groups)
if u.UID != 1000 || u.Group != "ops" || u.Shell != "/usr/bin/zsh" || fmt.Sprint(u.Groups) != "[docker ops wheel]" {
t.Errorf("ops: %+v", u)
}
if a, err := r.Account("961", ""); err != nil || a.Accounts[0].Name != "mesh-files" {
t.Errorf("by number: %+v %v", a, err)
}
g, err := r.Account("", "docker")
if err != nil || g.Groups[0].GID != 970 || fmt.Sprint(g.Groups[0].Members) != "[ops builder]" {
t.Errorf("docker: %+v %v", g, err)
}
if _, err := r.Account("nobody-here", ""); err == nil {
t.Error("a missing account was answered")
}
all, err := r.Account("", "")
if err != nil || len(all.Accounts) != 3 || len(all.Groups) != 5 {
t.Errorf("all: %+v %v", all, err)
}
body, _ := json.Marshal(all)
if strings.Contains(string(body), `"x"`) || strings.Contains(string(body), "password") {
t.Errorf("a password field is answered: %s", body)
}
}
// No answer is larger than the bound, whatever a verb made.
func TestNoAnswerIsLargerThanTheBound(t *testing.T) {
if _, err := bounded(strings.Repeat("x", MaxAnswer+1), nil); err == nil {
t.Error("an answer over the bound was given")
}
if _, err := bounded("small", nil); err != nil {
t.Error(err)
}
}
// The review of #244 (required 1): a link flipped between a harmless file and a secret place, while it is read
// and searched, never answers the secret place's content — what was opened is judged, not the name.
func TestALinkFlippedDuringAReadNeverReadsASecretPlace(t *testing.T) {
r, dir := testReader(t)
secret := filepath.Join(dir, "var/lib/mesh-files")
r.Places = []string{secret}
write(t, filepath.Join(secret, "broker"), "THE-BROKER-CREDENTIAL\n")
write(t, filepath.Join(dir, "plain"), "THE-PLAIN-WORDS\n")
link := filepath.Join(dir, "shared/link")
if err := os.MkdirAll(filepath.Dir(link), 0o755); err != nil {
t.Fatal(err)
}
stop := make(chan struct{})
done := make(chan struct{})
go func() {
defer close(done)
targets := []string{filepath.Join(dir, "plain"), filepath.Join(secret, "broker")}
for i := 0; ; i++ {
select {
case <-stop:
return
default:
}
tmp := link + ".new"
os.Remove(tmp)
if os.Symlink(targets[i%2], tmp) == nil {
os.Rename(tmp, link)
}
}
}()
leaked := 0
for i := 0; i < 1000; i++ {
if a, err := r.Read(link, ReadAsk{}); err == nil && strings.Contains(a.Text, "BROKER") {
leaked++
}
if a, err := r.Search(SearchAsk{Text: "BROKER", Path: filepath.Dir(link)}); err == nil && len(a.Matches) > 0 {
leaked++
}
}
close(stop)
<-done
if leaked > 0 {
t.Errorf("a secret place was answered %d times through a flipped link", leaked)
}
}
// The review of #244 (required 2): every line of a private key's block is withheld, in a read and in a search,
// though the file's name is not refused.
func TestEveryLineOfAKeyBlockIsWithheld(t *testing.T) {
r, dir := testReader(t)
key, err := rsa.GenerateKey(crand.Reader, 2048)
if err != nil {
t.Fatal(err)
}
der, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
t.Fatal(err)
}
body := string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der}))
p := filepath.Join(dir, "server.pem")
write(t, p, "# the server's key\n"+body+"after = the block\n")
a, err := r.Read(p, ReadAsk{})
if err != nil {
t.Fatal(err)
}
lines := strings.Count(body, "\n")
for _, l := range strings.Split(strings.TrimSpace(body), "\n")[1:] {
if strings.Contains(a.Text, l) && !strings.HasPrefix(l, "-----END") {
t.Fatalf("a line of the key was answered: %q", l)
}
}
if len(a.Withheld) != lines || !strings.Contains(a.Text, "after = the block") || !strings.Contains(a.Text, "the server's key") {
t.Errorf("withheld %d of the block's %d lines; text:\n%s", len(a.Withheld), lines, a.Text)
}
// A search for a character every base64 line holds answers none of the key.
m, err := r.Search(SearchAsk{Text: "A", Path: p, Matches: MaxMatches})
if err != nil {
t.Fatal(err)
}
for _, x := range m.Matches {
if !strings.HasPrefix(x.Text, "‹withheld") {
t.Errorf("a search answered a line of the key: %q", x.Text)
}
}
}
// The review of #244 (required 3 and should): histories, browser profiles, a process's private entries and the
// cluster credentials are refused by name.
func TestHistoriesProfilesAndProcessesAreRefusedByName(t *testing.T) {
var r Refuser
for _, p := range []string{"/home/ops/.zsh_history", "/home/ops/.bash_history", "/root/.psql_history",
"/home/ops/.viminfo", "/home/ops/.mozilla/firefox/x.default/logins.json",
"/home/ops/.config/google-chrome/Default/Cookies", "/home/ops/.config/chromium/Default/Login Data",
"/proc/self/environ", "/proc/1234/mem", "/proc/self/fd/3", "/proc/thread-self/root/etc/shadow",
"/etc/rancher/k3s/k3s.yaml", "/etc/kubernetes/admin.conf"} {
if r.Refused(p) == "" {
t.Errorf("%s is not refused", p)
}
}
for _, p := range []string{"/proc/mdstat", "/proc/self/status", "/home/ops/.config/i3/config", "/etc/hosts"} {
if why := r.Refused(p); why != "" {
t.Errorf("%s is refused: %s", p, why)
}
}
}
+129
View File
@@ -0,0 +1,129 @@
package main
import (
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/outward"
)
// What is refused by name, before any read (novox/hq ADR 0394 §5), and what an answer withholds (§4).
//
// **Two lists of places.** The places the mesh knows a secret is kept on this node — every module's own-secret
// directory, every credential file and grant directory, the vault's directories — come from the controller,
// which composes them from the manifests (MESH_SECRET_PLACES). The known credential files of the programs
// on a machine are this module's list, below: the mesh does not write them, so nothing else names them. Both
// hold whatever the account's file modes would allow: the file modes are the first bound, the names the second.
// knownDirs are directories that hold nothing but credentials, wherever they are: an account's keys.
var knownDirs = map[string]bool{
".ssh": true, ".gnupg": true, ".password-store": true, ".aws": true, ".kube": true, ".azure": true,
".mozilla": true, ".thunderbird": true, // browser and mail profiles: saved logins and cookies
}
// knownNames are file names that are a credential whatever directory they are in.
var knownNames = map[string]bool{
".credentials.json": true, // the agent's credentials (Claude Code)
".netrc": true, ".pgpass": true, ".git-credentials": true, ".npmrc": true, ".pypirc": true,
".my.cnf": true, ".vault-token": true, "credentials": true, "credentials.json": true,
"id_rsa": true, "id_dsa": true, "id_ecdsa": true, "id_ed25519": true, "id_ecdsa_sk": true, "id_ed25519_sk": true,
"shadow": true, "gshadow": true, "shadow-": true, "gshadow-": true, "opasswd": true,
"broker": true, // the mesh's name for a module's bus credential
// Shell and program histories: where a token is typed.
".bash_history": true, ".zsh_history": true, ".histfile": true, ".python_history": true, ".psql_history": true,
".mysql_history": true, ".lesshst": true, ".viminfo": true, ".node_repl_history": true, ".sqlite_history": true,
".rediscli_history": true,
}
// knownSuffixes are file name endings a credential has.
var knownSuffixes = []string{".key", ".p12", ".pfx", ".jks", ".keystore", ".creds", ".nk", ".kdbx", "_key"}
// knownPaths are the places on a machine that hold credentials and are not a module's.
var knownPaths = []string{
"/etc/ssl/private", "/etc/wireguard", "/etc/NetworkManager/system-connections", "/etc/ipsec.d/private",
"/var/lib/iwd", "/etc/docker/key.json", "/root/.docker/config.json", "/var/lib/sss/secrets",
"/etc/rancher", "/etc/kubernetes/admin.conf",
}
// knownProfiles are directories under an account's .config that hold a browser's saved logins and cookies.
var knownProfiles = map[string]bool{"google-chrome": true, "chromium": true, "BraveSoftware": true, "vivaldi": true,
"microsoft-edge": true}
// procPrivate are the entries of a process under /proc that reach its memory, its environment or its files.
var procPrivate = map[string]bool{"environ": true, "mem": true, "fd": true, "map_files": true, "root": true,
"cwd": true}
// Refuser says whether a path is a place a secret is kept.
type Refuser struct {
// Places are the mesh's, from the controller: refused with everything beneath them.
Places []string
}
// Refused says why a path may not be read, or "" when it may. The path is clean and absolute.
func (r Refuser) Refused(path string) string {
path = filepath.Clean(path)
for _, p := range r.Places {
if within(path, p) {
return "it is a place the mesh keeps a secret (" + p + ")"
}
}
for _, p := range knownPaths {
if within(path, p) {
return "it is a place a machine keeps credentials (" + p + ")"
}
}
parts := strings.Split(strings.TrimPrefix(path, "/"), "/")
if len(parts) >= 3 && parts[0] == "proc" && procPrivate[parts[2]] {
return "it reaches a process's " + parts[2]
}
for i, part := range parts {
if i > 0 && parts[i-1] == ".config" && knownProfiles[part] {
return "it is " + part + "'s profile, which keeps saved logins"
}
if knownDirs[part] {
return "it is in " + part + ", which holds an account's credentials"
}
if i == len(parts)-1 {
if knownNames[part] {
return part + " is a known credential file"
}
for _, s := range knownSuffixes {
if strings.HasSuffix(part, s) && len(part) > len(s) {
return part + " is named as a key or a credential (" + s + ")"
}
}
if strings.HasSuffix(part, ".pem") && (strings.Contains(part, "key") || strings.Contains(part, "private")) {
return part + " is named as a private key"
}
if part == "config.json" && i > 0 && parts[i-1] == ".docker" {
return "a container registry login is kept there"
}
}
}
return ""
}
func within(path, place string) bool {
return path == place || strings.HasPrefix(path, strings.TrimSuffix(place, "/")+"/")
}
// Withheld is one piece of an answer the content rule kept back: where it was, and the class it carried.
type Withheld struct {
At string `json:"at"`
Class string `json:"why"`
}
// clean is the content rule's secret class (mesh-sdk outward, novox/hq ADR 0394 §4) over one piece of an
// answer: the piece, or a mark in its place when it is shaped like a secret.
//
// **What a file holds, not what the file system names.** It judges a file's lines, a matching line, a link's
// target, a mount's source and an account's words. A path and an entry's name are not judged: they are what
// the caller named or the file system calls a thing, and the rule's random-run test takes a temporary
// directory's or a container layer's name for a token, which would withhold the very survey asked for. A
// place where a secret sits by name is refused before any read instead (Refuser).
func clean(s string) (string, string) {
if refusal, ok := outward.CheckSecret(s); !ok {
return "‹withheld: " + refusal.What + "›", refusal.What
}
return s, ""
}
+74
View File
@@ -0,0 +1,74 @@
package main
import (
"bytes"
"context"
"fmt"
"io"
"os/exec"
"sort"
"strings"
"time"
)
// Root repeats an act with more rights than the account this runs as has (novox/hq ADR 0394 §2). It is an
// interface so a unit test uses a fake: no test runs sudo.
type Root interface {
// Run runs one fixed binary with the arguments given, never through a shell, and answers what it wrote.
Run(bin string, args ...string) ([]byte, error)
}
// rootBins are the only binaries that run as root. The argument list is the caller's, but each caller in this
// module builds it from fixed words and one path, and ends its options with `--`.
var rootBins = map[string]bool{
"/usr/bin/cat": true, "/usr/bin/cp": true, "/usr/bin/mv": true, "/usr/bin/stat": true, "/usr/bin/chmod": true,
"/usr/bin/chown": true, "/usr/bin/mkdir": true, "/usr/bin/rm": true, "/usr/bin/readlink": true, "/usr/bin/find": true,
}
func rootBinList() string {
var l []string
for b := range rootBins {
l = append(l, b)
}
sort.Strings(l)
return strings.Join(l, ", ")
}
// sudoRoot is the real runner: `sudo -n` (never a password prompt) with a binary from rootBins.
type sudoRoot struct{}
// capped keeps at most n bytes and drops the rest.
type capped struct {
buf bytes.Buffer
n int
}
func (c *capped) Write(p []byte) (int, error) {
if room := c.n - c.buf.Len(); room > 0 {
if len(p) < room {
room = len(p)
}
c.buf.Write(p[:room])
}
return len(p), nil
}
func (sudoRoot) Run(bin string, args ...string) ([]byte, error) {
if !rootBins[bin] {
return nil, fmt.Errorf("%s is not one of the binaries that run as root here (%s)", bin, rootBinList())
}
ctx, cancel := context.WithTimeout(context.Background(), SearchTime+10*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "/usr/bin/sudo", append([]string{"-n", "--", bin}, args...)...)
out, errs := &capped{n: MaxScan}, &capped{n: 4096}
cmd.Stdout, cmd.Stderr = out, errs
cmd.Stdin = io.LimitReader(strings.NewReader(""), 0)
if err := cmd.Run(); err != nil {
msg := strings.TrimSpace(errs.buf.String())
if msg == "" {
msg = err.Error()
}
return nil, fmt.Errorf("sudo -n %s: %s", bin, msg)
}
return out.buf.Bytes(), nil
}
+215
View File
@@ -0,0 +1,215 @@
package main
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"time"
)
// TrashKeep is how long a deleted or replaced item stays in the trash (novox/hq ADR 0394 §4).
const TrashKeep = 7 * 24 * time.Hour
// trashMeta is the small JSON beside a trashed item: what it was, who it belonged to, and who put it there.
type trashMeta struct {
ID string `json:"id"`
Path string `json:"path"`
UID int `json:"uid"`
GID int `json:"gid"`
Mode uint32 `json:"mode"`
Kind string `json:"kind"`
Caller string `json:"caller"`
Reason string `json:"reason"`
Time time.Time `json:"time"`
}
// TrashItem is one item of the trash, as trash answers it.
type TrashItem struct {
ID string `json:"id"`
Path string `json:"path"`
Kind string `json:"kind"`
Owner string `json:"owner"`
Mode string `json:"mode"`
Caller string `json:"caller"`
Reason string `json:"reason"`
Time string `json:"time"`
Until string `json:"emptied-after"`
}
// TrashAnswer is what trash answers.
type TrashAnswer struct {
Items []TrashItem `json:"items"`
}
func (f *Files) trashDir() string { return filepath.Join(f.State, "trash") }
func newID() string {
b := make([]byte, 5)
if _, err := rand.Read(b); err != nil {
panic(err)
}
return hex.EncodeToString(b)
}
func (f *Files) id() string {
if f.NewID != nil {
return f.NewID()
}
return newID()
}
// toTrash moves a path into a folder of its own in the trash, with a note of what it was.
func (f *Files) toTrash(a Act, path string) (string, error) {
info, err := os.Lstat(path)
if err != nil {
return "", fmt.Errorf("%s cannot be put in the trash: %s", path, why(err))
}
id, err := f.ownerOf(path)
if err != nil {
return "", err
}
tid := f.id()
dir := filepath.Join(f.trashDir(), tid)
if err := os.MkdirAll(dir, 0o700); err != nil {
return "", fmt.Errorf("the trash cannot be written: %v", err)
}
if err := f.relocate(path, filepath.Join(dir, "item")); err != nil {
os.RemoveAll(dir)
return "", fmt.Errorf("%s cannot be put in the trash: %v", path, err)
}
meta, _ := json.Marshal(trashMeta{ID: tid, Path: path, UID: id.UID, GID: id.GID, Mode: permOf(info.Mode()),
Kind: typeOf(info.Mode()), Caller: a.Caller, Reason: a.Reason, Time: f.Now().UTC()})
if err := os.WriteFile(filepath.Join(dir, "meta.json"), meta, 0o600); err != nil {
return "", fmt.Errorf("%s is in the trash, but its note cannot be written: %v", path, err)
}
return tid, nil
}
func (f *Files) metaOf(id string) (trashMeta, error) {
var m trashMeta
raw, err := os.ReadFile(filepath.Join(f.trashDir(), id, "meta.json"))
if err != nil {
return m, err
}
return m, json.Unmarshal(raw, &m)
}
// entries are the ids in the trash.
func (f *Files) entries() []string {
dirs, err := os.ReadDir(f.trashDir())
if err != nil {
return nil
}
var ids []string
for _, d := range dirs {
if d.IsDir() {
ids = append(ids, d.Name())
}
}
return ids
}
// sweep empties what is older than TrashKeep. It runs on every write call and every trash list; a failure is
// said on stderr and tried again at the next one.
func (f *Files) sweep() {
for _, id := range f.entries() {
m, err := f.metaOf(id)
if err != nil || f.Now().Sub(m.Time) <= TrashKeep {
continue
}
if err := f.removeAll(filepath.Join(f.trashDir(), id)); err != nil {
fmt.Fprintf(os.Stderr, "[mesh-files] the trash entry %s cannot be emptied: %v\n", id, err)
}
}
}
// Trash lists what the trash holds, oldest first.
func (f *Files) Trash() (TrashAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.sweep()
names := f.names()
ans := TrashAnswer{Items: []TrashItem{}}
var metas []trashMeta
for _, id := range f.entries() {
if m, err := f.metaOf(id); err == nil {
metas = append(metas, m)
}
}
sort.Slice(metas, func(i, j int) bool { return metas[i].Time.Before(metas[j].Time) })
for _, m := range metas {
owner := names.user(m.UID)
if owner == "" {
owner = fmt.Sprint(m.UID)
}
if g := names.group(m.GID); g != "" {
owner += ":" + g
}
ans.Items = append(ans.Items, TrashItem{ID: m.ID, Path: m.Path, Kind: m.Kind, Owner: owner,
Mode: fmt.Sprintf("%04o", m.Mode), Caller: m.Caller, Reason: m.Reason,
Time: m.Time.Format(time.RFC3339), Until: m.Time.Add(TrashKeep).Format(time.RFC3339)})
}
return ans, nil
}
var idShape = regexp.MustCompile(`^[A-Za-z0-9_.-]+$`)
// restore puts an item back where it was. It refuses when something is there now.
func (f *Files) restore(id string) (trashMeta, error) {
if !idShape.MatchString(id) || strings.HasPrefix(id, ".") {
return trashMeta{}, fmt.Errorf("%q is not an id from trash", id)
}
m, err := f.metaOf(id)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return m, fmt.Errorf("the trash holds no %s (it may have been emptied after %d days)", id, int(TrashKeep.Hours()/24))
}
return m, err
}
if _, err := os.Lstat(m.Path); err == nil {
return m, fmt.Errorf("%s is there now: delete or move it first, then restore", m.Path)
}
if _, err := os.Stat(filepath.Dir(m.Path)); err != nil {
return m, fmt.Errorf("the folder of %s cannot be reached: %s", m.Path, why(err))
}
dir := filepath.Join(f.trashDir(), id)
if err := f.relocate(filepath.Join(dir, "item"), m.Path); err != nil {
return m, fmt.Errorf("%s cannot be restored: %v", m.Path, err)
}
if err := f.chownTo(m.Path, Ident{m.UID, m.GID}); err != nil {
return m, fmt.Errorf("%s is back, but its owner could not be set: %v", m.Path, err)
}
return m, f.removeAll(dir)
}
// Restore puts an item of the trash back where it was.
func (f *Files) Restore(a Act, id string) (ActAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
if err := f.begin(a); err != nil {
return ActAnswer{}, err
}
id = strings.TrimSpace(id)
var hits []hit
if idShape.MatchString(id) {
if m, err := f.metaOf(id); err == nil {
var gerr error
if hits, gerr = f.guard(a, false, m.Path); gerr != nil {
return ActAnswer{}, gerr
}
}
}
m, err := f.restore(id)
if err != nil {
return ActAnswer{}, err
}
return f.finish(a, "restore", hits, ActAnswer{Done: "restored " + m.Path, Paths: []string{m.Path}})
}
+5
View File
@@ -0,0 +1,5 @@
module mesh-files
go 1.25.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44
+4
View File
@@ -0,0 +1,4 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012122749-67bfa22132f2 h1:1E5iapAO1o4vkwV/8CUBQvNfL3OjXRtiQGBaKHXJVW8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012122749-67bfa22132f2/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44 h1:r5xIo5qWOXQdCSMxvqJy8kV4U+P/QWXFK6971Ha6wU8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
+56
View File
@@ -0,0 +1,56 @@
{
"module": "mesh-files",
"version": "1",
"claims": [
{
"name": "node-files",
"scope": "node",
"serves": [
"stat",
"readlink",
"list",
"read",
"search",
"account",
"write",
"copy",
"move",
"delete",
"mkdir",
"chmod",
"chown",
"trash",
"restore"
]
}
],
"invokes": [
"seat:operator-channel.tell"
],
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-files",
"binary": "mesh-files",
"loads": [
"mesh-files"
],
"env": {
"MESH_FILES_STATE": "${dir:state}"
}
}
]
}
}