Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jochen
|
b1f2776693 |
@@ -11,9 +11,8 @@ Two directories, declared, so the mesh refuses a second module owning either:
|
||||
- `~/.claude` under the home of the account agents run as (the operator's, unless the machine names an agent
|
||||
account of its own), that account's, `0700`. The module owns the directory —
|
||||
that it exists, who owns it, its mode — and of what is inside only what it writes. Everything else
|
||||
in it (history, projects, local settings, a person's own rules and skills) is the person's
|
||||
and is never read or written (hq ADR 0182) — but for each project's memory folder, which the module
|
||||
carries between nodes (hq ADR 0343, below). Unassigned, the module leaves the directory: the host
|
||||
in it (memory, history, projects, local settings, a person's own rules and skills) is the person's
|
||||
and is never read or written (hq ADR 0182). Unassigned, the module leaves the directory: the host
|
||||
removes a directory only when it is empty.
|
||||
|
||||
And, for where the agent runs as an account of its own (hq ADR 0266):
|
||||
@@ -70,37 +69,8 @@ must see, a node that joins later included — kept, so it carries no secret eit
|
||||
| an MCP server registered through this module before registering was refused (hq issue 340) | a key in the module's `servers` state — `all.<server>` for every node, `<node>.<server>` for one; every node watches it and renders what applies to it, a node's own entry over the one for every node. A node that joins later, or was off, reads the whole current set at start; unregistering is a delete. Nothing adds a key any more: a new tool server is the operator's, in the `mcp_servers` setting at the controller's terminal |
|
||||
| a proposed instruction section (hq ADR 0274) | a key in the module's `proposals` state per proposal, the ask published on `operator-channel` under this module's name, and the router's record of it read under that name alone; what a node approved is its own file, never the bus |
|
||||
| a session's question to the operator (hq ADR 0332) | a key `q-…` in the module's `questions` state per question, the ask published on `operator-channel` under this module's name, and the router's record of it read under that name alone; the router's word that it ended without a choice is kept on the question; two slot keys `slot-1`, `slot-2` name the questions open |
|
||||
| a project's memory (hq ADR 0343) | a key in the module's `memory` state per file — `<kind>.<project and path, base64url>`, `operator` for the operator's own homes and `agent` for an agent account's — holding the text, its digest, the digest of the version it replaced, the node and when it changed; a removal is a value marked removed, never a deleted key. Written at the revision read, so of two nodes writing one file only one wins. Every node of the same kind watches it. No event and no tool answer carries memory text |
|
||||
| the agent's configuration (hq ADR 0216) | a key in the module's `config` state per registration — `mesh.<kind>.<name>` for every node, `node.<node>.<kind>.<name>` for one, `home.<node>.<kind>.<name>` for one account's own directory — the item and its files in one value, at most 256 KiB. Every node watches it and renders what applies to it, a node item over a mesh item of the same kind and name. A hook, settings, or an item that would run something by itself is never taken, whenever it was put there (hq issue 340) |
|
||||
|
||||
## A session's memory follows the operator (hq ADR 0343)
|
||||
|
||||
A session started on another node starts with what the sessions on every other node learned (hq issue 367). The
|
||||
module's long-running process carries each project's memory folder, `~/.claude/projects/<project>/memory/`, and
|
||||
nothing else of the home. A project is one entry of the projects directory and a path is relative, no part empty,
|
||||
`.` or `..`: a value in the state naming anything else is never taken, and nothing is touched for it.
|
||||
|
||||
- **Picked up** every ten seconds: a file whose text differs from the version last carried goes into the `memory`
|
||||
state. Not carried, and listed with why: a symbolic link, a file over 128 KiB or too large for one value once
|
||||
encoded, and a file holding a key block, a
|
||||
signed token, a known token prefix or a password in an address.
|
||||
- **Written** on every other node of the same home kind, only where its file is still the version the change
|
||||
replaced, or the version it last carried (a node that was away), or there is none; staged in the folder and renamed in, never through a symbolic link. The version
|
||||
replaced is kept first in `memory-copies/` in the module's state directory, 30 days.
|
||||
- **A file changed on two nodes** becomes one file holding both, the newer first, then a line naming each version's
|
||||
node and time, then the older (two versions too large for one file: the newer in it, the older beside it as
|
||||
`<name>.older-from-<node>`); what was written is carried, so every node ends on it, and the operator is warned through
|
||||
the operator channel's `notify` (a warning, cleared when the file next changes without a conflict).
|
||||
- **A removal** of a carried file, from a folder that still exists, is a marker in the state for 30 days: a node still
|
||||
holding the removed version moves it into the copies; a node that changed it keeps it and carries it again. A
|
||||
missing folder, a key the state lost and a deleted key remove nothing, and neither does a file or folder that
|
||||
cannot be read on a look: it is kept everywhere and listed as not carried.
|
||||
- **The two home kinds never mix**: the operator's own homes carry among themselves, an agent account's among
|
||||
themselves (hq ADR 0266), so a confined agent's notes never reach a session run as the operator.
|
||||
- **No tool writes memory** (hq issue 340): `claude_code_memory` only lists what is carried — fingerprints, never the
|
||||
text — the files not carried, the open conflicts and the copies kept. A value put straight into the state by a
|
||||
holder of a tool runner's credential is not yet told apart (hq issue 638).
|
||||
|
||||
## The mesh's tools first (hq ADR 0245)
|
||||
|
||||
The agent kept reaching for `ssh <machine> journalctl` while the service manager's `journal` verb existed. So:
|
||||
|
||||
@@ -285,25 +285,6 @@ func TestANewerGenerationFetchesTheTokenOnceByTheSeatsVerb(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The node's line names the generation it applied and why the manager moved it, and who asked (novox/hq issue 625).
|
||||
func TestAnAppliedBindingSaysWhyAndWho(t *testing.T) {
|
||||
p, w := node(t, "laptop")
|
||||
var asked []string
|
||||
done, err := OnBinding(p, &BindingState{Licence: "personal", Kind: "subscription", Generation: 7,
|
||||
Why: "the switch verb", By: "g14.node-tools"}, seat(t, "personal", "at-7", 7, &asked), writer(w))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{`"generation":7`, `"why":"the switch verb"`, `"by":"g14.node-tools"`} {
|
||||
if !strings.Contains(done, want) {
|
||||
t.Fatalf("the line %s lacks %s", done, want)
|
||||
}
|
||||
}
|
||||
if strings.Contains(done, "at-7") {
|
||||
t.Fatal("the line carries the token")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheTokenANodeIsHandedReplacesALoginsGrantAndLeavesNoRefreshToken(t *testing.T) {
|
||||
p, w := node(t, "laptop")
|
||||
writeFile(t, p.credentials(), `{"claudeAiOauth":{"accessToken":"at-old","refreshToken":"rt-spent","expiresAt":`+
|
||||
|
||||
@@ -152,36 +152,6 @@ func (s stateOf) Create(key string, value any) error {
|
||||
return err
|
||||
}
|
||||
func (s stateOf) Keys() ([]string, error) { return s.s.Keys() }
|
||||
|
||||
// memoryStateOf adapts the SDK's state to the memory carrier: every write at the revision read (ADR 0343).
|
||||
type memoryStateOf struct{ s stdio.KeptState }
|
||||
|
||||
func (s memoryStateOf) Get(key string) (json.RawMessage, uint64, bool, error) {
|
||||
e, err := s.s.Get(key)
|
||||
if err != nil || e == nil {
|
||||
return nil, 0, false, err
|
||||
}
|
||||
return e.Value, e.Revision, true, nil
|
||||
}
|
||||
func (s memoryStateOf) Create(key string, value any) error {
|
||||
_, err := s.s.Create(key, value)
|
||||
return err
|
||||
}
|
||||
func (s memoryStateOf) Update(key string, value any, revision uint64) error {
|
||||
_, err := s.s.Update(key, value, revision)
|
||||
return err
|
||||
}
|
||||
func (s memoryStateOf) Keys() ([]string, error) { return s.s.Keys() }
|
||||
func (s memoryStateOf) Delete(key string) error { return s.s.Delete(key) }
|
||||
|
||||
// memoryCarrier carries this node's memory folders (ADR 0343); nil until the long-running half starts it.
|
||||
var memoryCarrier *MemoryCarrier
|
||||
|
||||
// notifyOperator tells the operator through the operator channel's notify verb (ADR 0253).
|
||||
func notifyOperator(args map[string]any) error {
|
||||
_, err := ask("seat:operator-channel.notify", args)
|
||||
return err
|
||||
}
|
||||
func (s stateOf) Get(key string) (json.RawMessage, bool, error) {
|
||||
e, err := s.s.Get(key)
|
||||
if err != nil || e == nil {
|
||||
@@ -384,9 +354,6 @@ func tools(p Paths, servers ServerState, view *ServerView, config ConfigState, c
|
||||
out = append(out, stdio.Tool{Name: "claude_code_registrations",
|
||||
Description: "Read-only: everything registered through this module that a Claude Code session runs or obeys — the tool servers, the hooks, the settings and the skills, subagents, commands, output styles and instructions — by key, with who registered it, when, and whether it is still rendered (novox/hq issue 340), beside what the operator set for this node at the controller's terminal (managed_settings, mcp_servers). `by` is the node whose tool runner served the call, never the caller.",
|
||||
Run: func(map[string]any) (any, error) { return Registrations(p, servers, config) }})
|
||||
out = append(out, stdio.Tool{Name: "claude_code_memory",
|
||||
Description: "Read-only: what this node carries of the agent's memory between nodes (novox/hq ADR 0343) — the home kind it carries for (the operator's own homes, or an agent account's), each project's memory files with their fingerprint and the node and time of their last version, the files not carried and why, the conflicts still open (a file changed on two nodes, kept twice in one file, the newer first), and how many replaced or removed versions are kept. Fingerprints only, never the text; it changes nothing.",
|
||||
Run: func(map[string]any) (any, error) { return memoryCarrier.Status(), nil }})
|
||||
out = append(out, proposalTools(p)...)
|
||||
judge := NewJudge(p)
|
||||
out = append(out, judgeTool(judge), judgedTool(judge))
|
||||
@@ -463,12 +430,6 @@ func main() {
|
||||
proposing = &Proposing{Store: stateOf{stdio.State(ProposalsState)}, Bus: sdkAskBus{}, Record: recordWarrant,
|
||||
Now: time.Now, Write: writeManaged}
|
||||
go proposing.watch(p)
|
||||
memoryCarrier = NewMemoryCarrier(p, memoryStateOf{stdio.State(MemoryState)}, notifyOperator)
|
||||
go memoryCarrier.Run(func(handle func(key, op string, value json.RawMessage, revision uint64) error) error {
|
||||
return stdio.State(MemoryState).Watch("", func(c stdio.StateChange) error {
|
||||
return handle(c.Key, c.Op, c.Value, c.Revision)
|
||||
})
|
||||
})
|
||||
go run(p, view, configView)
|
||||
if err := stdio.Serve("", tools(p, servers, view, config, configView)); err != nil {
|
||||
say("%v", err)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,968 +0,0 @@
|
||||
package main
|
||||
|
||||
// A session's memory follows the operator (novox/hq ADR 0343, issue 367). Every test runs against temporary homes
|
||||
// and a state in memory with the bus's watch semantics: never the live memory folders.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// memState is the `memory` state in memory: revisions, create and compare-and-set, and every change handed to every
|
||||
// watcher, the writer included, as the bus does.
|
||||
type memState struct {
|
||||
mu sync.Mutex
|
||||
values map[string]json.RawMessage
|
||||
revs map[string]uint64
|
||||
rev uint64
|
||||
pending []change
|
||||
watchers []*MemoryCarrier
|
||||
}
|
||||
|
||||
type change struct {
|
||||
key, op string
|
||||
value json.RawMessage
|
||||
rev uint64
|
||||
}
|
||||
|
||||
func newMemState() *memState {
|
||||
return &memState{values: map[string]json.RawMessage{}, revs: map[string]uint64{}}
|
||||
}
|
||||
|
||||
var errChanged = errors.New("wrong last revision")
|
||||
|
||||
func (s *memState) Get(key string) (json.RawMessage, uint64, bool, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
v, ok := s.values[key]
|
||||
return v, s.revs[key], ok, nil
|
||||
}
|
||||
|
||||
func (s *memState) write(key string, value any) {
|
||||
raw, _ := json.Marshal(value)
|
||||
s.rev++
|
||||
s.values[key], s.revs[key] = raw, s.rev
|
||||
s.pending = append(s.pending, change{key, "put", raw, s.rev})
|
||||
}
|
||||
|
||||
func (s *memState) Create(key string, value any) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.values[key]; ok {
|
||||
return errChanged
|
||||
}
|
||||
s.write(key, value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *memState) Update(key string, value any, revision uint64) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.revs[key] != revision {
|
||||
return errChanged
|
||||
}
|
||||
s.write(key, value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *memState) Keys() ([]string, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
var keys []string
|
||||
for k := range s.values {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys, nil
|
||||
}
|
||||
|
||||
func (s *memState) Delete(key string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
delete(s.values, key)
|
||||
delete(s.revs, key)
|
||||
s.rev++
|
||||
s.pending = append(s.pending, change{key, "delete", nil, s.rev})
|
||||
return nil
|
||||
}
|
||||
|
||||
// pump hands every change so far to every watcher, until none is left.
|
||||
func (s *memState) pump(t *testing.T) {
|
||||
t.Helper()
|
||||
for i := 0; i < 50; i++ {
|
||||
s.mu.Lock()
|
||||
todo := s.pending
|
||||
s.pending = nil
|
||||
s.mu.Unlock()
|
||||
if len(todo) == 0 {
|
||||
return
|
||||
}
|
||||
for _, c := range todo {
|
||||
for _, w := range s.watchers {
|
||||
if err := w.Take(c.key, c.op, c.value, c.rev); err != nil {
|
||||
t.Logf("%s taking %s: %v", w.Node, c.key, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, w := range s.watchers {
|
||||
if err := w.Drain(); err != nil && !errors.Is(err, errChanged) {
|
||||
t.Logf("%s applying: %v", w.Node, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
t.Fatal("the state never settled")
|
||||
}
|
||||
|
||||
// clock is a clock a test moves.
|
||||
type clock struct{ now time.Time }
|
||||
|
||||
func (c *clock) Now() time.Time { return c.now }
|
||||
|
||||
// memNode is one node's carrier over its own temporary home and state directory.
|
||||
type memNode struct {
|
||||
*MemoryCarrier
|
||||
notices []map[string]any
|
||||
}
|
||||
|
||||
func newMemNode(t *testing.T, root, name, kind string, state *memState, c *clock) *memNode {
|
||||
t.Helper()
|
||||
n := &memNode{}
|
||||
home := filepath.Join(root, name, "home")
|
||||
stateDir := filepath.Join(root, name, "state")
|
||||
for _, d := range []string{filepath.Join(home, ".claude", "projects"), stateDir} {
|
||||
if err := os.MkdirAll(d, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
n.MemoryCarrier = &MemoryCarrier{Home: home, Node: name, State: stateDir, Kind: kind, Store: state, Now: c.Now,
|
||||
wake: make(chan struct{}, 1),
|
||||
Notify: func(a map[string]any) error { n.notices = append(n.notices, a); return nil }}
|
||||
state.watchers = append(state.watchers, n.MemoryCarrier)
|
||||
return n
|
||||
}
|
||||
|
||||
func (n *memNode) path(project, file string) string {
|
||||
return filepath.Join(n.Home, ".claude", "projects", project, "memory", filepath.FromSlash(file))
|
||||
}
|
||||
|
||||
func (n *memNode) write(t *testing.T, project, file, text string, at time.Time) {
|
||||
t.Helper()
|
||||
p := n.path(project, file)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(text), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = os.Chtimes(p, at, at)
|
||||
}
|
||||
|
||||
func (n *memNode) read(t *testing.T, project, file string) (string, bool) {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile(n.path(project, file))
|
||||
if os.IsNotExist(err) {
|
||||
return "", false
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(raw), true
|
||||
}
|
||||
|
||||
func (n *memNode) look(t *testing.T) {
|
||||
t.Helper()
|
||||
if err := n.Look(); err != nil && !errors.Is(err, errChanged) {
|
||||
t.Fatalf("%s's look: %v", n.Node, err)
|
||||
}
|
||||
}
|
||||
|
||||
// copies is every text kept in the node's copies.
|
||||
func (n *memNode) copies(t *testing.T) []string {
|
||||
t.Helper()
|
||||
var out []string
|
||||
_ = filepath.WalkDir(n.MemoryCarrier.copies(), func(p string, d os.DirEntry, err error) error {
|
||||
if err == nil && !d.IsDir() {
|
||||
raw, _ := os.ReadFile(p)
|
||||
out = append(out, string(raw))
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return out
|
||||
}
|
||||
|
||||
func holdsText(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
const project = "-home-op-projects-novox-hq"
|
||||
|
||||
func twoNodes(t *testing.T) (*memState, *clock, *memNode, *memNode) {
|
||||
state, c := newMemState(), &clock{now: time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)}
|
||||
root := t.TempDir()
|
||||
return state, c, newMemNode(t, root, "g14", HomeOperator, state, c), newMemNode(t, root, "shanks", HomeOperator, state, c)
|
||||
}
|
||||
|
||||
// carried writes a file on g14 and carries it to shanks.
|
||||
func carriedOnBoth(t *testing.T, state *memState, c *clock, a, b *memNode, file, text string) {
|
||||
t.Helper()
|
||||
a.write(t, project, file, text, c.now)
|
||||
a.look(t)
|
||||
state.pump(t)
|
||||
b.look(t)
|
||||
state.pump(t)
|
||||
if got, _ := b.read(t, project, file); got != text {
|
||||
t.Fatalf("not carried to %s: %q", b.Node, got)
|
||||
}
|
||||
}
|
||||
|
||||
// §5, §6: a memory file written on one node appears unchanged on another, at any depth, and is carried once.
|
||||
func TestAMemoryFileWrittenOnOneNodeAppearsUnchangedOnAnother(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
g14.write(t, project, "MEMORY.md", "- [state](state.md) — where things stand\n", c.now)
|
||||
g14.write(t, project, "notes/state.md", "ADR 0343 proposed.\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for file, want := range map[string]string{"MEMORY.md": "- [state](state.md) — where things stand\n", "notes/state.md": "ADR 0343 proposed.\n"} {
|
||||
if got, ok := shanks.read(t, project, file); !ok || got != want {
|
||||
t.Fatalf("%s on shanks: %q, %v", file, got, ok)
|
||||
}
|
||||
}
|
||||
before := state.rev
|
||||
shanks.look(t)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if state.rev != before {
|
||||
t.Fatalf("a carried file was carried again: %d changes", state.rev-before)
|
||||
}
|
||||
// A change on shanks comes back to g14, over the version it replaced, which g14 keeps.
|
||||
c.now = c.now.Add(time.Minute)
|
||||
shanks.write(t, project, "notes/state.md", "ADR 0343 accepted.\n", c.now)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
if got, _ := g14.read(t, project, "notes/state.md"); got != "ADR 0343 accepted.\n" {
|
||||
t.Fatalf("the change did not reach g14: %q", got)
|
||||
}
|
||||
if !holdsText(g14.copies(t), "ADR 0343 proposed.\n") {
|
||||
t.Fatal("the version replaced on g14 was not kept")
|
||||
}
|
||||
if len(g14.notices)+len(shanks.notices) != 0 {
|
||||
t.Fatalf("a plain change warned: %v %v", g14.notices, shanks.notices)
|
||||
}
|
||||
// No value in the state is anything but a version of a memory file, and no tool answer carries the text.
|
||||
raw, _ := json.Marshal(g14.Status())
|
||||
if strings.Contains(string(raw), "accepted") {
|
||||
t.Fatalf("the memory tool answered the text: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// §1, §2: only the memory folders are carried; a symbolic link, an oversized file and a secret's shape are not, and are
|
||||
// listed with why.
|
||||
func TestOnlyTheMemoryFoldersAreCarried(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
projectDir := filepath.Join(g14.Home, ".claude", "projects", project)
|
||||
g14.write(t, project, "MEMORY.md", "index\n", c.now)
|
||||
for path, text := range map[string]string{
|
||||
filepath.Join(projectDir, "session.jsonl"): "{}\n",
|
||||
filepath.Join(g14.Home, ".claude", "settings.json"): "{}\n",
|
||||
filepath.Join(g14.Home, ".claude", "history.jsonl"): "{}\n",
|
||||
} {
|
||||
if err := os.WriteFile(path, []byte(text), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
g14.write(t, project, "key.md", "-----BEGIN OPENSSH PRIVATE KEY-----\nabc\n", c.now)
|
||||
g14.write(t, project, "url.md", "https://op:hunter2@git.example.org/x\n", c.now)
|
||||
g14.write(t, project, "big.md", strings.Repeat("x", MemoryMostBytes+1), c.now)
|
||||
g14.write(t, project, "hash.md", "head 1fe6ea4ee47379007598e163784dbd7ac50b7f79\n", c.now)
|
||||
outside := filepath.Join(t.TempDir(), "elsewhere.md")
|
||||
_ = os.WriteFile(outside, []byte("not memory\n"), 0o600)
|
||||
if err := os.Symlink(outside, g14.path(project, "link.md")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for _, file := range []string{"key.md", "url.md", "big.md", "link.md"} {
|
||||
if _, ok := shanks.read(t, project, file); ok {
|
||||
t.Errorf("%s was carried", file)
|
||||
}
|
||||
}
|
||||
for _, file := range []string{"MEMORY.md", "hash.md"} {
|
||||
if _, ok := shanks.read(t, project, file); !ok {
|
||||
t.Errorf("%s was not carried", file)
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(shanks.Home, ".claude", "projects", project, "session.jsonl")); err == nil {
|
||||
t.Error("a session was carried")
|
||||
}
|
||||
for _, f := range []string{"settings.json", "history.jsonl"} {
|
||||
if _, err := os.Stat(filepath.Join(shanks.Home, ".claude", f)); err == nil {
|
||||
t.Errorf("%s was carried", f)
|
||||
}
|
||||
}
|
||||
keys, _ := state.Keys()
|
||||
if len(keys) != 2 {
|
||||
t.Fatalf("the state holds %d keys, not the two memory files: %v", len(keys), keys)
|
||||
}
|
||||
not, _ := g14.Status()["not carried"].(map[string]string)
|
||||
for _, file := range []string{"key.md", "url.md", "big.md", "link.md"} {
|
||||
if not[project+"/"+file] == "" {
|
||||
t.Errorf("%s is not listed as not carried: %v", file, not)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// §4: the operator's homes and an agent account's homes never take each other's memory.
|
||||
func TestHomesOfTheTwoKindsNeverTakeEachOthersMemory(t *testing.T) {
|
||||
state, c := newMemState(), &clock{now: time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)}
|
||||
root := t.TempDir()
|
||||
laptop := newMemNode(t, root, "g14", HomeOperator, state, c)
|
||||
control := newMemNode(t, root, "novox", HomeAgent, state, c)
|
||||
control.write(t, project, "MEMORY.md", "from a confined agent\n", c.now)
|
||||
control.look(t)
|
||||
state.pump(t)
|
||||
if _, ok := laptop.read(t, project, "MEMORY.md"); ok {
|
||||
t.Fatal("a confined agent's memory reached the operator's home")
|
||||
}
|
||||
laptop.write(t, project, "MEMORY.md", "the operator's\n", c.now)
|
||||
laptop.look(t)
|
||||
state.pump(t)
|
||||
if got, _ := control.read(t, project, "MEMORY.md"); got != "from a confined agent\n" {
|
||||
t.Fatalf("the operator's memory reached the agent account's home: %q", got)
|
||||
}
|
||||
// A value of the other kind put under this kind's key is refused too.
|
||||
key := MemoryKey(HomeOperator, project, "x.md")
|
||||
raw, _ := json.Marshal(MemoryVersion{Kind: HomeAgent, Project: project, Path: "x.md", Text: "x", Digest: digestOf([]byte("x")), Node: "novox"})
|
||||
if err := laptop.Take(key, "put", raw, 99); err == nil {
|
||||
t.Fatal("a value of another kind was taken")
|
||||
}
|
||||
// Which kind a node carries for.
|
||||
if k, _ := HomeKind(Paths{Agent: "jochen", Operator: "jochen", Home: "/home/jochen"}); k != HomeOperator {
|
||||
t.Fatalf("the operator's own home is %q", k)
|
||||
}
|
||||
if k, _ := HomeKind(Paths{Agent: "agent", Operator: "jochen", Home: "/home/agent", AgentHome: "/home/agent", OperatorHome: "/home/jochen"}); k != HomeAgent {
|
||||
t.Fatalf("an agent account's home is %q", k)
|
||||
}
|
||||
if k, why := HomeKind(Paths{Agent: "agent", Operator: "jochen", Home: "/home/jochen", AgentHome: "/home/agent", OperatorHome: "/home/jochen"}); k != "" || why == "" {
|
||||
t.Fatalf("a confined agent in the operator's home carries %q", k)
|
||||
}
|
||||
}
|
||||
|
||||
// §7, §8: a file changed on two nodes since it was carried is kept twice in one file on both, the newer first, with
|
||||
// one warning; it clears once the file is tidied.
|
||||
func TestAFileChangedOnTwoNodesIsKeptTwiceTheNewerFirstWithAWarning(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "- base\n")
|
||||
g14.write(t, project, "MEMORY.md", "- base\n- learned on g14\n", c.now.Add(time.Minute))
|
||||
shanks.write(t, project, "MEMORY.md", "- base\n- learned on shanks\n", c.now.Add(2*time.Minute))
|
||||
c.now = c.now.Add(3 * time.Minute)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
a, _ := g14.read(t, project, "MEMORY.md")
|
||||
b, _ := shanks.read(t, project, "MEMORY.md")
|
||||
if a != b {
|
||||
t.Fatalf("the nodes did not end on one text:\n%s\n----\n%s", a, b)
|
||||
}
|
||||
newer, older := strings.Index(a, "learned on shanks"), strings.Index(a, "learned on g14")
|
||||
marker := strings.Index(a, "<!-- novox mesh: this file was changed on two nodes")
|
||||
if newer < 0 || older < 0 || marker < 0 || !(newer < marker && marker < older) {
|
||||
t.Fatalf("not both versions, the newer first, with the line between:\n%s", a)
|
||||
}
|
||||
if !strings.Contains(a[marker:], "changed on shanks") || !strings.Contains(a[marker:], "changed on g14") {
|
||||
t.Fatalf("the line does not name both nodes:\n%s", a)
|
||||
}
|
||||
notices := append(append([]map[string]any{}, g14.notices...), shanks.notices...)
|
||||
if len(notices) != 1 || notices[0]["severity"] != "warning" || notices[0]["clear"] != nil {
|
||||
t.Fatalf("not one warning: %v", notices)
|
||||
}
|
||||
for _, field := range []string{"headline", "explanation", "summary", "resolved"} {
|
||||
s, _ := notices[0][field].(string)
|
||||
if s == "" || strings.ContainsAny(s, "/~:") && field != "explanation" || strings.Contains(s, ".md") {
|
||||
t.Errorf("the warning's %s is empty or names a path: %q", field, s)
|
||||
}
|
||||
// The plain rule (ADR 0253 §3): it never sends the operator elsewhere.
|
||||
if field != "summary" && (strings.Contains(strings.ToLower(s), "agent") || strings.Contains(strings.ToLower(s), "session")) {
|
||||
t.Errorf("the warning's %s sends the operator elsewhere: %q", field, s)
|
||||
}
|
||||
}
|
||||
if e, _ := notices[0]["explanation"].(string); !strings.HasPrefix(e, "Needs you: ") {
|
||||
t.Errorf("the explanation does not open with its verdict: %q", e)
|
||||
}
|
||||
if h, _ := notices[0]["headline"].(string); len([]rune(h)) > 60 {
|
||||
t.Errorf("the headline is longer than sixty characters: %q", h)
|
||||
}
|
||||
all := append(g14.copies(t), shanks.copies(t)...)
|
||||
for _, v := range []string{"- base\n- learned on g14\n", "- base\n- learned on shanks\n"} {
|
||||
if !holdsText(all, v) {
|
||||
t.Errorf("the version %q was not kept", v)
|
||||
}
|
||||
}
|
||||
if st := g14.Status(); len(st["conflicts"].([]conflict))+len(shanks.Status()["conflicts"].([]conflict)) != 1 {
|
||||
t.Fatalf("the conflict is not listed: %v", st)
|
||||
}
|
||||
// Tidied on either node, the warning clears.
|
||||
c.now = c.now.Add(time.Minute)
|
||||
g14.write(t, project, "MEMORY.md", "- base\n- learned on g14\n- learned on shanks\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
cleared := false
|
||||
for _, n := range append(g14.notices, shanks.notices...) {
|
||||
if n["clear"] == true {
|
||||
cleared = true
|
||||
}
|
||||
}
|
||||
if !cleared {
|
||||
t.Fatal("the warning did not clear once the file was tidied")
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "- base\n- learned on g14\n- learned on shanks\n" {
|
||||
t.Fatalf("the tidied file did not reach shanks: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The order of the two versions: the later change first, equal times by node name.
|
||||
func TestMergeVersionsPutsTheNewerFirst(t *testing.T) {
|
||||
at := time.Date(2026, 10, 13, 12, 0, 0, 0, time.UTC)
|
||||
m := string(MergeVersions([]byte("old\n"), "g14", at, []byte("new\n"), "shanks", at.Add(time.Second)))
|
||||
if strings.Index(m, "new") > strings.Index(m, "old") {
|
||||
t.Fatalf("the newer is not first: %s", m)
|
||||
}
|
||||
m = string(MergeVersions([]byte("from shanks\n"), "shanks", at, []byte("from ace\n"), "ace", at))
|
||||
if strings.Index(m, "from ace") > strings.Index(m, "from shanks") {
|
||||
t.Fatalf("equal times are not ordered by node name: %s", m)
|
||||
}
|
||||
}
|
||||
|
||||
// §9: a removal moves the other node's unchanged file into the copies.
|
||||
func TestARemovalMovesTheUnchangedFileIntoTheCopies(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "old.md", "an old note\n")
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "index\n")
|
||||
if err := os.Remove(g14.path(project, "old.md")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if _, ok := shanks.read(t, project, "old.md"); ok {
|
||||
t.Fatal("the removal did not reach shanks")
|
||||
}
|
||||
if !holdsText(shanks.copies(t), "an old note\n") {
|
||||
t.Fatal("the removed version was not kept on shanks")
|
||||
}
|
||||
raw, _, ok, _ := state.Get(MemoryKey(HomeOperator, project, "old.md"))
|
||||
var v MemoryVersion
|
||||
_ = json.Unmarshal(raw, &v)
|
||||
if !ok || !v.Removed || v.Text != "" {
|
||||
t.Fatalf("the removal is not a marker in the state: %s", raw)
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "index\n" {
|
||||
t.Fatal("another file was touched")
|
||||
}
|
||||
}
|
||||
|
||||
// §9: a change outlives a removal, whichever reaches the state first.
|
||||
func TestAChangeOutlivesARemoval(t *testing.T) {
|
||||
for _, removalFirst := range []bool{true, false} {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "note.md", "v1\n")
|
||||
_ = os.Remove(g14.path(project, "note.md"))
|
||||
shanks.write(t, project, "note.md", "v2\n", c.now.Add(time.Minute))
|
||||
if removalFirst {
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
shanks.look(t)
|
||||
} else {
|
||||
shanks.look(t)
|
||||
g14.look(t)
|
||||
}
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
for _, n := range []*memNode{g14, shanks} {
|
||||
if got, _ := n.read(t, project, "note.md"); got != "v2\n" {
|
||||
t.Fatalf("removal first %v: the change did not outlive the removal on %s: %q", removalFirst, n.Node, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// §9: a missing folder, a state that lost its keys and a deleted key remove nothing.
|
||||
func TestAMissingFolderOrAnAbsentKeyRemovesNothing(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "index\n")
|
||||
key := MemoryKey(HomeOperator, project, "MEMORY.md")
|
||||
// The folder gone on g14.
|
||||
if err := os.RemoveAll(filepath.Join(g14.Home, ".claude", "projects", project, "memory")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "index\n" {
|
||||
t.Fatal("a missing folder removed the file elsewhere")
|
||||
}
|
||||
// A deleted key.
|
||||
_ = state.Delete(key)
|
||||
state.pump(t)
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "index\n" {
|
||||
t.Fatal("a deleted key removed the file")
|
||||
}
|
||||
// The state lost: the file is carried again, not removed.
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
if _, _, ok, _ := state.Get(key); !ok {
|
||||
t.Fatal("a file whose key the state lost was not carried again")
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "index\n" {
|
||||
t.Fatalf("the file was touched on shanks: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// §10: on a first carrying, two nodes' different versions are both kept, and each node's own files reach the other.
|
||||
func TestTheFirstCarryingKeepsEveryVersion(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
g14.write(t, project, "MEMORY.md", "- laptop's index\n", c.now)
|
||||
g14.write(t, project, "laptop.md", "only on the laptop\n", c.now)
|
||||
shanks.write(t, project, "MEMORY.md", "- workstation's index\n", c.now.Add(time.Hour))
|
||||
shanks.write(t, project, "workstation.md", "only on the workstation\n", c.now)
|
||||
shanks.write(t, project, "same.md", "same\n", c.now)
|
||||
g14.write(t, project, "same.md", "same\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for _, n := range []*memNode{g14, shanks} {
|
||||
index, _ := n.read(t, project, "MEMORY.md")
|
||||
if !strings.Contains(index, "laptop's index") || !strings.Contains(index, "workstation's index") ||
|
||||
strings.Index(index, "workstation's index") > strings.Index(index, "laptop's index") {
|
||||
t.Fatalf("%s's index does not hold both, the newer first:\n%s", n.Node, index)
|
||||
}
|
||||
for file, want := range map[string]string{"laptop.md": "only on the laptop\n", "workstation.md": "only on the workstation\n", "same.md": "same\n"} {
|
||||
if got, _ := n.read(t, project, file); got != want {
|
||||
t.Fatalf("%s's %s: %q", n.Node, file, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
if n := len(g14.notices) + len(shanks.notices); n != 1 {
|
||||
t.Fatalf("%d warnings for one conflict", n)
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing is written through a symbolic link: a memory folder that is one is neither read nor written.
|
||||
func TestNothingIsWrittenThroughASymbolicLink(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
elsewhere := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(shanks.Home, ".claude", "projects", project), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(elsewhere, filepath.Join(shanks.Home, ".claude", "projects", project, "memory")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g14.write(t, project, "MEMORY.md", "index\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if entries, _ := os.ReadDir(elsewhere); len(entries) != 0 {
|
||||
t.Fatalf("written through a symbolic link: %v", entries)
|
||||
}
|
||||
}
|
||||
|
||||
// The manifest declares the state and the tool; the tool reads and never writes.
|
||||
func TestTheManifestDeclaresTheMemoryState(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m struct {
|
||||
State []string `json:"state"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &m)
|
||||
if !holdsText(m.State, MemoryState) {
|
||||
t.Fatalf("the manifest does not declare the %q state: %v", MemoryState, m.State)
|
||||
}
|
||||
var carrier *MemoryCarrier
|
||||
if st := carrier.Status(); st["carrying"] != false {
|
||||
t.Fatalf("a carrier not started says it carries: %v", st)
|
||||
}
|
||||
}
|
||||
|
||||
// replay hands one node the whole current set, as a watch begins when the node comes back.
|
||||
func (s *memState) replay(t *testing.T, n *memNode) {
|
||||
t.Helper()
|
||||
keys, _ := s.Keys()
|
||||
for _, k := range keys {
|
||||
raw, rev, _, _ := s.Get(k)
|
||||
if err := n.Take(k, "put", raw, rev); err != nil {
|
||||
t.Logf("%s taking %s: %v", n.Node, k, err)
|
||||
}
|
||||
}
|
||||
if err := n.Drain(); err != nil && !errors.Is(err, errChanged) {
|
||||
t.Logf("%s applying: %v", n.Node, err)
|
||||
}
|
||||
}
|
||||
|
||||
// §6: a node that was away while a file changed twice takes the newest version, whose base it never saw, because
|
||||
// its own file holds no change it did not carry; the version replaced is kept.
|
||||
func TestANodeThatWasAwayTakesTheNewestVersion(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "v1\n")
|
||||
state.watchers = []*MemoryCarrier{g14.MemoryCarrier}
|
||||
for _, v := range []string{"v2\n", "v3\n"} {
|
||||
c.now = c.now.Add(time.Minute)
|
||||
g14.write(t, project, "MEMORY.md", v, c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
}
|
||||
state.watchers = append(state.watchers, shanks.MemoryCarrier)
|
||||
state.replay(t, shanks)
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "v3\n" {
|
||||
t.Fatalf("the node that was away holds %q", got)
|
||||
}
|
||||
if !holdsText(shanks.copies(t), "v1\n") {
|
||||
t.Fatal("the version replaced was not kept")
|
||||
}
|
||||
if len(g14.notices)+len(shanks.notices) != 0 {
|
||||
t.Fatal("taking a newer version warned")
|
||||
}
|
||||
}
|
||||
|
||||
// §2, §7: two versions too large to share one file are kept in two: the newer in the file, the older beside it,
|
||||
// both carried; and a text that would not fit in one value of the state is not carried.
|
||||
func TestTwoVersionsTooLargeForOneFileAreKeptInTwo(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "big.md", "base\n")
|
||||
older := strings.Repeat("from g14 ", MemoryMostBytes/9-100) + "\n"
|
||||
newer := strings.Repeat("from shanks ", MemoryMostBytes/12-100) + "\n"
|
||||
g14.write(t, project, "big.md", older, c.now.Add(time.Minute))
|
||||
shanks.write(t, project, "big.md", newer, c.now.Add(2*time.Minute))
|
||||
c.now = c.now.Add(3 * time.Minute)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
beside := OlderBeside("big.md", "g14")
|
||||
if beside != "big.older-from-g14.md" {
|
||||
t.Fatalf("the older is named %q", beside)
|
||||
}
|
||||
for _, n := range []*memNode{g14, shanks} {
|
||||
if got, _ := n.read(t, project, "big.md"); got != newer {
|
||||
t.Fatalf("%s's file does not hold the newer version", n.Node)
|
||||
}
|
||||
if got, _ := n.read(t, project, beside); got != older {
|
||||
t.Fatalf("%s does not hold the older version beside it", n.Node)
|
||||
}
|
||||
}
|
||||
notices := append(append([]map[string]any{}, g14.notices...), shanks.notices...)
|
||||
if len(notices) != 1 || !strings.Contains(notices[0]["explanation"].(string), "second file beside it") {
|
||||
t.Fatalf("not one warning saying the older is beside it: %v", notices)
|
||||
}
|
||||
// Under the size limit, but too large once encoded: not carried, and said.
|
||||
g14.write(t, project, "angles.md", strings.Repeat("<", MemoryMostBytes-10), c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if _, ok := shanks.read(t, project, "angles.md"); ok {
|
||||
t.Fatal("a text too large for one value was carried")
|
||||
}
|
||||
not, _ := g14.Status()["not carried"].(map[string]string)
|
||||
if not[project+"/angles.md"] == "" {
|
||||
t.Fatalf("it is not listed as not carried: %v", not)
|
||||
}
|
||||
}
|
||||
|
||||
// stateHolds says whether any value of the state holds s.
|
||||
func (s *memState) holds(sub string) bool {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for _, v := range s.values {
|
||||
var mv MemoryVersion
|
||||
_ = json.Unmarshal(v, &mv)
|
||||
if strings.Contains(mv.Text, sub) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// putRaw puts a value straight into the state, as a holder of a tool runner's credential could.
|
||||
func (s *memState) putRaw(t *testing.T, v MemoryVersion) string {
|
||||
t.Helper()
|
||||
key := MemoryKey(v.Kind, v.Project, v.Path)
|
||||
s.mu.Lock()
|
||||
s.write(key, v)
|
||||
s.mu.Unlock()
|
||||
return key
|
||||
}
|
||||
|
||||
// Review of #276, 1: a value whose project or path leads out of a memory folder is never taken, and nothing is
|
||||
// written for it.
|
||||
func TestAValueLeadingOutOfAMemoryFolderIsNeverTaken(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
root := filepath.Dir(filepath.Dir(shanks.Home))
|
||||
for _, bad := range []struct{ project, path string }{
|
||||
{"../../../escaped", "x.md"}, {"..", "x.md"}, {"a/b", "x.md"}, {"", "x.md"},
|
||||
{project, "../x.md"}, {project, "/abs.md"}, {project, "a//x.md"}, {project, "./x.md"}, {project, "a/../../x.md"},
|
||||
} {
|
||||
text := "planted\n"
|
||||
state.putRaw(t, MemoryVersion{Kind: HomeOperator, Project: bad.project, Path: bad.path, Text: text,
|
||||
Digest: digestOf([]byte(text)), Node: "elsewhere", Changed: c.now})
|
||||
if NameProblem(bad.project, bad.path) == "" {
|
||||
t.Errorf("%q %q is taken as a name", bad.project, bad.path)
|
||||
}
|
||||
}
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
shanks.look(t)
|
||||
_ = filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
|
||||
if err == nil && !d.IsDir() {
|
||||
if raw, _ := os.ReadFile(p); string(raw) == "planted\n" {
|
||||
t.Errorf("a planted value was written at %s", p)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if NameProblem(project, "notes/a.md") != "" || NameProblem(project, "MEMORY.md") != "" {
|
||||
t.Fatal("a memory file's own name is refused")
|
||||
}
|
||||
}
|
||||
|
||||
// Review of #276, 2: a file or a folder that cannot be read is kept everywhere, never carried as a removal, and is
|
||||
// said as not carried.
|
||||
func TestWhatCannotBeReadIsNeverTakenForGone(t *testing.T) {
|
||||
if os.Geteuid() == 0 {
|
||||
t.Skip("root reads a file whatever its mode")
|
||||
}
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "index\n")
|
||||
carriedOnBoth(t, state, c, g14, shanks, "notes/deep.md", "deep\n")
|
||||
file, folder := g14.path(project, "MEMORY.md"), filepath.Dir(g14.path(project, "notes/deep.md"))
|
||||
_ = os.Chmod(file, 0)
|
||||
_ = os.Chmod(folder, 0)
|
||||
t.Cleanup(func() { _ = os.Chmod(file, 0o644); _ = os.Chmod(folder, 0o755) })
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for _, f := range []string{"MEMORY.md", "notes/deep.md"} {
|
||||
if _, ok := shanks.read(t, project, f); !ok {
|
||||
t.Errorf("%s, unreadable on g14, was removed on shanks", f)
|
||||
}
|
||||
}
|
||||
not, _ := g14.Status()["not carried"].(map[string]string)
|
||||
if len(not) == 0 {
|
||||
t.Fatalf("nothing is said as not carried: %v", not)
|
||||
}
|
||||
keys, _ := state.Keys()
|
||||
for _, k := range keys {
|
||||
raw, _, _, _ := state.Get(k)
|
||||
var v MemoryVersion
|
||||
_ = json.Unmarshal(raw, &v)
|
||||
if v.Removed {
|
||||
t.Errorf("%s/%s was carried as a removal", v.Project, v.Path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Review of #276, 3: no text reaches the state without being screened, on the removal and the conflict paths too.
|
||||
func TestNoUnscreenedTextReachesTheState(t *testing.T) {
|
||||
token := "ghp_" + strings.Repeat("A1b2C3d4", 5) + "\n"
|
||||
big := strings.Repeat("x", MemoryMostBytes+10) + "\n"
|
||||
for _, text := range []string{token, big} {
|
||||
// The removal path: removed on g14, changed on shanks to what is never carried.
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "note.md", "v1\n")
|
||||
_ = os.Remove(g14.path(project, "note.md"))
|
||||
shanks.write(t, project, "note.md", text, c.now.Add(time.Minute))
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if state.holds(text[:20]) {
|
||||
t.Errorf("the removal path put text that is never carried into the state")
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "note.md"); got != text {
|
||||
t.Errorf("the change on shanks was not kept")
|
||||
}
|
||||
// The conflict path: changed on g14, and on shanks to what is never carried.
|
||||
state, c, g14, shanks = twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "note.md", "v1\n")
|
||||
g14.write(t, project, "note.md", "v2\n", c.now.Add(time.Minute))
|
||||
shanks.write(t, project, "note.md", text, c.now.Add(2*time.Minute))
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
if state.holds(text[:20]) {
|
||||
t.Errorf("the conflict path put text that is never carried into the state")
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "note.md"); got != text {
|
||||
t.Errorf("shanks's own version was not kept as it was")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Review of #276, 4: a conflict whose file cannot be written leaves the state as it was, says so once, and merges
|
||||
// once when the folder is writable again.
|
||||
func TestAConflictThatCannotBeWrittenIsNotMergedAgainAndAgain(t *testing.T) {
|
||||
if os.Geteuid() == 0 {
|
||||
t.Skip("root writes a folder whatever its mode")
|
||||
}
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "MEMORY.md", "base\n")
|
||||
g14.write(t, project, "MEMORY.md", "base\nfrom g14\n", c.now.Add(time.Minute))
|
||||
shanks.write(t, project, "MEMORY.md", "base\nfrom shanks\n", c.now.Add(2*time.Minute))
|
||||
folder := filepath.Dir(shanks.path(project, "MEMORY.md"))
|
||||
_ = os.Chmod(folder, 0o500)
|
||||
t.Cleanup(func() { _ = os.Chmod(folder, 0o700) })
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for i := 0; i < 5; i++ {
|
||||
_ = shanks.Look()
|
||||
state.pump(t)
|
||||
}
|
||||
if state.holds("<!-- novox mesh") {
|
||||
t.Fatal("a merge that could not be written here reached the state")
|
||||
}
|
||||
unwritten := 0
|
||||
for _, n := range shanks.notices {
|
||||
if strings.HasSuffix(n["key"].(string), ".unwritten") {
|
||||
unwritten++
|
||||
}
|
||||
}
|
||||
if unwritten != 1 {
|
||||
t.Fatalf("%d warnings that the file could not be written", unwritten)
|
||||
}
|
||||
_ = os.Chmod(folder, 0o700)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
for _, n := range []*memNode{g14, shanks} {
|
||||
got, _ := n.read(t, project, "MEMORY.md")
|
||||
if strings.Count(got, "<!-- novox mesh") != 1 || !strings.Contains(got, "from g14") || !strings.Contains(got, "from shanks") {
|
||||
t.Fatalf("%s does not hold both versions once:\n%s", n.Node, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Review of #276, minor: a file that changed between its reading and its writing is not written over; text that is
|
||||
// not UTF-8 is not carried.
|
||||
func TestAFileChangedBeforeItsWritingIsNotWrittenOver(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
_ = state
|
||||
g14.write(t, project, "a.md", "read\n", c.now)
|
||||
g14.write(t, project, "a.md", "changed meanwhile\n", c.now)
|
||||
err := g14.writeLocal(project, "a.md", []byte("incoming\n"), "replaced", digestOf([]byte("read\n")))
|
||||
if !errors.Is(err, errChangedHere) {
|
||||
t.Fatalf("written over a change: %v", err)
|
||||
}
|
||||
if got, _ := g14.read(t, project, "a.md"); got != "changed meanwhile\n" {
|
||||
t.Fatalf("the change was replaced: %q", got)
|
||||
}
|
||||
g14.write(t, project, "latin1.md", "caf\xe9\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
if _, ok := shanks.read(t, project, "latin1.md"); ok {
|
||||
t.Fatal("text that is not UTF-8 was carried")
|
||||
}
|
||||
not, _ := g14.Status()["not carried"].(map[string]string)
|
||||
if !strings.Contains(not[project+"/latin1.md"], "UTF-8") {
|
||||
t.Fatalf("it is not said why: %v", not)
|
||||
}
|
||||
}
|
||||
|
||||
// §9: a node that was away takes a removal of a version newer than the one it holds, when it holds no change of
|
||||
// its own: the removed file does not come back.
|
||||
func TestARemovalReachesANodeThatWasAway(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
carriedOnBoth(t, state, c, g14, shanks, "old.md", "v1\n")
|
||||
state.watchers = []*MemoryCarrier{g14.MemoryCarrier}
|
||||
c.now = c.now.Add(time.Minute)
|
||||
g14.write(t, project, "old.md", "v2\n", c.now)
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
_ = os.Remove(g14.path(project, "old.md"))
|
||||
g14.look(t)
|
||||
state.pump(t)
|
||||
state.watchers = append(state.watchers, shanks.MemoryCarrier)
|
||||
state.replay(t, shanks)
|
||||
shanks.look(t)
|
||||
state.pump(t)
|
||||
g14.look(t)
|
||||
if _, ok := shanks.read(t, project, "old.md"); ok {
|
||||
t.Fatal("the removal did not reach the node that was away")
|
||||
}
|
||||
if _, ok := g14.read(t, project, "old.md"); ok {
|
||||
t.Fatal("the removed file came back")
|
||||
}
|
||||
if !holdsText(shanks.copies(t), "v1\n") {
|
||||
t.Fatal("the version removed on the node that was away was not kept")
|
||||
}
|
||||
}
|
||||
|
||||
// slowState answers every read only once it is let go, as a state a whole bus away does under load.
|
||||
type slowState struct {
|
||||
*memState
|
||||
gate chan struct{}
|
||||
}
|
||||
|
||||
func (s slowState) Get(key string) (json.RawMessage, uint64, bool, error) {
|
||||
<-s.gate
|
||||
return s.memState.Get(key)
|
||||
}
|
||||
|
||||
// Issue 367, seen live: the runtime hands a watch its current values one at a time and gives them all ten seconds;
|
||||
// shanks's handler read the state and waited for its look for each of a few hundred values, so the watch never
|
||||
// started and nothing written on g14 reached it. The handler now only notes a change: it returns at once while a
|
||||
// look holds the carrier and while the state is slow to answer, and Drain applies what it noted.
|
||||
func TestTheWatchHandlerReturnsAtOnceAndDrainAppliesWhatItNoted(t *testing.T) {
|
||||
state, c, g14, shanks := twoNodes(t)
|
||||
g14.write(t, project, "MEMORY.md", "index\n", c.now)
|
||||
g14.look(t)
|
||||
state.mu.Lock()
|
||||
state.pending = nil
|
||||
state.mu.Unlock()
|
||||
slow := slowState{memState: state, gate: make(chan struct{})}
|
||||
shanks.Store = slow
|
||||
key := MemoryKey(HomeOperator, project, "MEMORY.md")
|
||||
raw, rev, _, _ := state.Get(key)
|
||||
shanks.mu.Lock() // a look running
|
||||
done := make(chan error, 1)
|
||||
start := time.Now()
|
||||
for i := 0; i < 400; i++ {
|
||||
go func() { done <- shanks.Take(key, "put", raw, rev) }()
|
||||
}
|
||||
for i := 0; i < 400; i++ {
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatalf("the watch handler waited for the look or the state (%d of 400 returned)", i)
|
||||
}
|
||||
}
|
||||
if took := time.Since(start); took > time.Second {
|
||||
t.Fatalf("400 values took %s to hand over", took)
|
||||
}
|
||||
shanks.mu.Unlock()
|
||||
close(slow.gate)
|
||||
if err := shanks.Drain(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, _ := shanks.read(t, project, "MEMORY.md"); got != "index\n" {
|
||||
t.Fatalf("what the watch noted was not applied: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -298,9 +298,6 @@ type BindingState struct {
|
||||
Licence string `json:"licence"`
|
||||
Kind string `json:"kind"`
|
||||
Generation int64 `json:"generation"`
|
||||
// Why and By are what moved the binding and who asked, as the manager recorded it (novox/hq issue 625).
|
||||
Why string `json:"why,omitempty"`
|
||||
By string `json:"by,omitempty"`
|
||||
}
|
||||
|
||||
// Current is what the seat answers to `current`: the licence this node is bound to and its token, sealed.
|
||||
@@ -436,15 +433,6 @@ func OnBinding(p Paths, b *BindingState, ask Ask, write WriteManaged) (string, e
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
// The line says which binding this was and why the manager moved it, so a node's journal alone answers
|
||||
// what changed here and who asked (novox/hq issue 625).
|
||||
out["generation"] = b.Generation
|
||||
if b.Why != "" {
|
||||
out["why"] = b.Why
|
||||
}
|
||||
if b.By != "" {
|
||||
out["by"] = b.By
|
||||
}
|
||||
raw, _ := json.Marshal(out)
|
||||
return string(raw), nil
|
||||
}
|
||||
|
||||
@@ -19,8 +19,7 @@
|
||||
"holdings",
|
||||
"config",
|
||||
"proposals",
|
||||
"questions",
|
||||
"memory"
|
||||
"questions"
|
||||
],
|
||||
"reads": [
|
||||
"claude-licence-manager.bindings"
|
||||
@@ -33,7 +32,6 @@
|
||||
"claude_code_grant",
|
||||
"claude_code_add_api_key",
|
||||
"claude_code_registrations",
|
||||
"claude_code_memory",
|
||||
"claude_code_mcp_list",
|
||||
"claude_code_mcp_register",
|
||||
"claude_code_mcp_unregister",
|
||||
|
||||
@@ -253,7 +253,6 @@ func text(a map[string]any, k string) (string, error) {
|
||||
}
|
||||
|
||||
// verb is one of the seat's verbs: listed as `<seat>.<verb>`, so the runtime serves it on the seat's subject.
|
||||
// What it changes is recorded as asked by its caller, as the runtime proved it (novox/hq issues 365 and 625).
|
||||
func verb(name, description string, input map[string]any, run func(ctx context.Context, m *Manager, a map[string]any) (any, error)) stdio.Tool {
|
||||
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input,
|
||||
Run: func(a map[string]any) (any, error) {
|
||||
@@ -261,8 +260,7 @@ func verb(name, description string, input map[string]any, run func(ctx context.C
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx := WithCause(context.Background(), Cause{Why: "the " + name + " verb", By: stdio.CallerOf(a)})
|
||||
return run(ctx, m, a)
|
||||
return run(context.Background(), m, a)
|
||||
}}
|
||||
}
|
||||
|
||||
@@ -290,7 +288,7 @@ func tools() []stdio.Tool {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Bind(ctx, c, l)
|
||||
return m.Bind(ctx, c, l, "bind")
|
||||
}),
|
||||
verb("switch", "Move a consumer to another licence. Its node fetches the new licence's token at once and points the agent's account at it. A login on a node does this by itself (ADR 0209); this is for moving one without a login, or back.",
|
||||
map[string]any{"consumer": consumer, "licence": str("the licence to move to")},
|
||||
@@ -299,7 +297,7 @@ func tools() []stdio.Tool {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Bind(ctx, c, l)
|
||||
return m.Bind(ctx, c, l, "switch")
|
||||
}),
|
||||
verb("release", "Unbind a consumer. Its node keeps its last token, which expires within hours.",
|
||||
map[string]any{"consumer": consumer},
|
||||
@@ -308,7 +306,7 @@ func tools() []stdio.Tool {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Release(ctx, c)
|
||||
return m.Release(ctx, c, "release")
|
||||
}),
|
||||
verb("refresh", "Refresh a licence now, or every due licence when none is named; under each licence's lease, so it never races the daemon. Answers the outcome, never a token.",
|
||||
map[string]any{"licence": str("a licence; absent for every due one")},
|
||||
|
||||
@@ -59,54 +59,6 @@ type BindingState struct {
|
||||
Licence string `json:"licence"`
|
||||
Kind string `json:"kind"`
|
||||
Generation int64 `json:"generation"`
|
||||
// Why and By are what moved it and who asked (novox/hq issue 625), so the node can say so when it applies it.
|
||||
Why string `json:"why,omitempty"`
|
||||
By string `json:"by,omitempty"`
|
||||
}
|
||||
|
||||
// Cause is why a consumer's binding changed and who or what asked for it (novox/hq issue 625): Why is the verb
|
||||
// called or the rule that fired, By the caller the runtime proved (stdio.CallerOf) or this module itself.
|
||||
type Cause struct {
|
||||
Why string
|
||||
By string
|
||||
}
|
||||
|
||||
// Itself names this module where it moved a binding by a rule of its own, with no caller.
|
||||
const Itself = "claude-licence-manager"
|
||||
|
||||
type causeKey struct{}
|
||||
|
||||
// WithCause is ctx carrying why what runs in it changes a binding, and who asked.
|
||||
func WithCause(ctx context.Context, c Cause) context.Context {
|
||||
return context.WithValue(ctx, causeKey{}, c)
|
||||
}
|
||||
|
||||
// CauseOf is the cause ctx carries, or "a rule of this module's own" when none: never empty, so a record always
|
||||
// says something, and a change nobody named a cause for reads as this module's own.
|
||||
func CauseOf(ctx context.Context) Cause {
|
||||
if c, ok := ctx.Value(causeKey{}).(Cause); ok && c.By != "" {
|
||||
return c
|
||||
}
|
||||
return Cause{Why: "a rule of this module's own", By: Itself}
|
||||
}
|
||||
|
||||
// record says one change of what a consumer holds, the same fields in three places (novox/hq issue 625): a line
|
||||
// in the journal, a row in the audit table, and the event `licence.<what>` on the bus. Never a token.
|
||||
func (m *Manager) record(ctx context.Context, what, line string, detail map[string]any) {
|
||||
_ = m.Store.Audit(ctx, what, detail)
|
||||
m.Log("%s", line)
|
||||
_ = m.Emit("licence."+what, detail)
|
||||
}
|
||||
|
||||
// moved records a consumer bound or switched: from which licence to which, at which generation, why and by whom.
|
||||
func (m *Manager) moved(ctx context.Context, consumer, from, licence string, generation int64, c Cause) {
|
||||
what, line := "bound", fmt.Sprintf("bound %s to %s", consumer, licence)
|
||||
if from != "" {
|
||||
what, line = "switched", fmt.Sprintf("switched %s from %s to %s", consumer, from, licence)
|
||||
}
|
||||
m.record(ctx, what, fmt.Sprintf("%s (generation %d): %s; asked by %s", line, generation, c.Why, c.By),
|
||||
map[string]any{"consumer": consumer, "licence": licence, "from": from, "generation": generation,
|
||||
"why": c.Why, "by": c.By})
|
||||
}
|
||||
|
||||
// Settings are the manager's own, declared with defaults (design 39 §8).
|
||||
@@ -315,12 +267,11 @@ func (m *Manager) bindReporters(ctx context.Context, reports []Holdings) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c := Cause{Why: rep.Node + " reported the account of " + l.Name + " and was bound to nothing", By: Itself}
|
||||
m.moved(ctx, rep.Node, "", l.Name, b.Generation, c)
|
||||
if err := m.PutBinding(ctx, rep.Node, BindingState{Licence: l.Name, Kind: l.Kind, Generation: b.Generation,
|
||||
Why: c.Why, By: c.By}); err != nil {
|
||||
_ = m.Store.Audit(ctx, "bound", map[string]any{"consumer": rep.Node, "licence": l.Name, "by": "its account's report"})
|
||||
if err := m.PutBinding(ctx, rep.Node, BindingState{Licence: l.Name, Kind: l.Kind, Generation: b.Generation}); err != nil {
|
||||
return err
|
||||
}
|
||||
m.Log("bound %s to %s, the licence its account already has", rep.Node, l.Name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -400,21 +351,21 @@ func (m *Manager) adoptOne(ctx context.Context, c Candidate, reports []Holdings)
|
||||
|
||||
// A login moves its node (ADR 0209): the node this login came from is bound to its licence —
|
||||
// switched, if it was bound to another. Then every node reporting this account and bound to nothing.
|
||||
// Who asked is the person who logged in there; the mesh knows them only as that login (novox/hq issue 625).
|
||||
cause := Cause{Why: "a login to " + l.Name + " on " + c.Node + " (ADR 0209)", By: Itself}
|
||||
ctx = WithCause(ctx, cause)
|
||||
if b, err := m.Store.Binding(ctx, c.Node); err != nil {
|
||||
return "", err
|
||||
} else if b == nil || b.Licence != l.Name {
|
||||
moved, err := m.Store.Bind(ctx, c.Node, l.Name)
|
||||
if err != nil {
|
||||
if _, err := m.Store.Bind(ctx, c.Node, l.Name); err != nil {
|
||||
return "", err
|
||||
}
|
||||
from := ""
|
||||
if b != nil {
|
||||
from = b.Licence
|
||||
}
|
||||
m.moved(ctx, c.Node, from, l.Name, moved.Generation, cause)
|
||||
_ = m.Store.Audit(ctx, map[bool]string{true: "switched", false: "bound"}[b != nil],
|
||||
map[string]any{"consumer": c.Node, "licence": l.Name, "from": from, "by": "a login there"})
|
||||
if b != nil {
|
||||
m.Log("%s moved from %s to %s: a login there", c.Node, from, l.Name)
|
||||
}
|
||||
}
|
||||
for _, rep := range reports {
|
||||
if rep.Identity == nil || rep.Identity.AccountUUID != c.Identity.AccountUUID {
|
||||
@@ -423,9 +374,8 @@ func (m *Manager) adoptOne(ctx context.Context, c Candidate, reports []Holdings)
|
||||
if b, err := m.Store.Binding(ctx, rep.Node); err != nil || b != nil {
|
||||
continue
|
||||
}
|
||||
if b, err := m.Store.Bind(ctx, rep.Node, l.Name); err == nil {
|
||||
m.moved(ctx, rep.Node, "", l.Name, b.Generation,
|
||||
Cause{Why: rep.Node + " reports the account of " + l.Name + ", logged in on " + c.Node + ", and was bound to nothing", By: Itself})
|
||||
if _, err := m.Store.Bind(ctx, rep.Node, l.Name); err == nil {
|
||||
_ = m.Store.Audit(ctx, "bound", map[string]any{"consumer": rep.Node, "licence": l.Name, "by": "its login"})
|
||||
}
|
||||
}
|
||||
if err := m.publishAdvance(ctx, l); err != nil {
|
||||
@@ -449,17 +399,14 @@ func (m *Manager) keep(l *Licence, g FullGrant) {
|
||||
l.RotatedAt = m.Now().UnixMilli()
|
||||
}
|
||||
|
||||
// publishAdvance gives every consumer of a licence a new generation, and tells each in the state with the
|
||||
// cause ctx carries, so the node says why it fetched again (novox/hq issue 625).
|
||||
// publishAdvance gives every consumer of a licence a new generation, and tells each in the state.
|
||||
func (m *Manager) publishAdvance(ctx context.Context, l Licence) error {
|
||||
bindings, err := m.Store.Advance(ctx, l.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c := CauseOf(ctx)
|
||||
for _, b := range bindings {
|
||||
if err := m.PutBinding(ctx, b.Consumer, BindingState{Licence: b.Licence, Kind: l.Kind, Generation: b.Generation,
|
||||
Why: c.Why, By: c.By}); err != nil {
|
||||
if err := m.PutBinding(ctx, b.Consumer, BindingState{Licence: b.Licence, Kind: l.Kind, Generation: b.Generation}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
@@ -535,19 +482,7 @@ func (m *Manager) Rotate(ctx context.Context, name string, force bool) (map[stri
|
||||
if err := m.Store.SaveLicence(ctx, *l); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := CauseOf(ctx)
|
||||
if force {
|
||||
c.Why = "a refresh asked for: " + c.Why
|
||||
} else {
|
||||
c = Cause{Why: "due: older than its cadence or near its expiry", By: Itself}
|
||||
if by := CauseOf(ctx); by.By != Itself {
|
||||
c.By = by.By
|
||||
}
|
||||
}
|
||||
ctx = WithCause(ctx, Cause{Why: "a rotation of " + name + " (" + c.Why + ")", By: c.By})
|
||||
m.record(ctx, "rotated", fmt.Sprintf("rotated %s, its token now expiring %s: %s; asked by %s", name,
|
||||
time.UnixMilli(l.AccessExpiresAt).UTC().Format(time.RFC3339), c.Why, c.By),
|
||||
map[string]any{"licence": name, "expiresAt": l.AccessExpiresAt, "why": c.Why, "by": c.By})
|
||||
_ = m.Store.Audit(ctx, "rotated", map[string]any{"licence": name, "expiresAt": l.AccessExpiresAt})
|
||||
if err := m.publishAdvance(ctx, *l); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -659,9 +594,8 @@ func (m *Manager) Current(ctx context.Context, consumer, publicKey string) (map[
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Bind binds or switches a consumer: a person's act (ADR 0183), told to the consumer as a new generation, and
|
||||
// recorded with the cause ctx carries — the verb and its proven caller (novox/hq issue 625).
|
||||
func (m *Manager) Bind(ctx context.Context, consumer, licence string) (map[string]any, error) {
|
||||
// Bind binds or switches a consumer: a person's act (ADR 0183), told to the consumer as a new generation.
|
||||
func (m *Manager) Bind(ctx context.Context, consumer, licence, by string) (map[string]any, error) {
|
||||
l, err := m.Store.Licence(ctx, licence)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -677,21 +611,19 @@ func (m *Manager) Bind(ctx context.Context, consumer, licence string) (map[strin
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
from := ""
|
||||
from, what := "", "bound"
|
||||
if before != nil {
|
||||
from = before.Licence
|
||||
from, what = before.Licence, "switched"
|
||||
}
|
||||
c := CauseOf(ctx)
|
||||
m.moved(ctx, consumer, from, licence, b.Generation, c)
|
||||
if err := m.PutBinding(ctx, consumer, BindingState{Licence: licence, Kind: l.Kind, Generation: b.Generation,
|
||||
Why: c.Why, By: c.By}); err != nil {
|
||||
_ = m.Store.Audit(ctx, what, map[string]any{"consumer": consumer, "licence": licence, "from": from, "by": by})
|
||||
if err := m.PutBinding(ctx, consumer, BindingState{Licence: licence, Kind: l.Kind, Generation: b.Generation}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"consumer": consumer, "licence": licence, "generation": b.Generation, "from": from}, nil
|
||||
}
|
||||
|
||||
// Release unbinds a consumer; its node keeps its last token, which expires within hours.
|
||||
func (m *Manager) Release(ctx context.Context, consumer string) (map[string]any, error) {
|
||||
func (m *Manager) Release(ctx context.Context, consumer, by string) (map[string]any, error) {
|
||||
was, err := m.Store.Binding(ctx, consumer)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -702,9 +634,7 @@ func (m *Manager) Release(ctx context.Context, consumer string) (map[string]any,
|
||||
if err := m.DeleteBinding(ctx, consumer); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := CauseOf(ctx)
|
||||
m.record(ctx, "released", fmt.Sprintf("released %s from %s: %s; asked by %s", consumer, was.Licence, c.Why, c.By),
|
||||
map[string]any{"consumer": consumer, "licence": was.Licence, "generation": was.Generation, "why": c.Why, "by": c.By})
|
||||
_ = m.Store.Audit(ctx, "released", map[string]any{"consumer": consumer, "licence": was.Licence, "by": by})
|
||||
return map[string]any{"consumer": consumer, "released": true, "was": was.Licence}, nil
|
||||
}
|
||||
|
||||
@@ -755,10 +685,7 @@ func (m *Manager) adoptKey(ctx context.Context, name, key, from string) (map[str
|
||||
if err := m.Store.SaveLicence(ctx, l); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := CauseOf(ctx)
|
||||
_ = m.Store.Audit(ctx, "adopted", map[string]any{"licence": name, "kind": "api-key", "from": from, "why": c.Why, "by": c.By})
|
||||
m.Log("adopted the API key %s from %s; asked by %s", name, from, c.By)
|
||||
ctx = WithCause(ctx, Cause{Why: "the API key " + name + " adopted from " + from, By: c.By})
|
||||
_ = m.Store.Audit(ctx, "adopted", map[string]any{"licence": name, "kind": "api-key", "from": from})
|
||||
if err := m.publishAdvance(ctx, l); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -132,13 +132,8 @@ func TestAManagerWithNoLicenceAdoptsTheNewestLoginThatRefreshesAndNeverExchanges
|
||||
if len(bs) != 2 || mm.state["laptop"].Licence != licence1 || mm.state["server"].Licence != licence1 {
|
||||
t.Fatalf("bindings %v, state %v", bs, mm.state)
|
||||
}
|
||||
// The adoption is said, and so is each binding it made (novox/hq issue 625).
|
||||
events := strings.Join(mm.events, "|")
|
||||
for _, want := range []string{"licence.adopted ", `licence.bound {"by":"claude-licence-manager","consumer":"laptop"`,
|
||||
`licence.bound {"by":"claude-licence-manager","consumer":"server"`} {
|
||||
if !strings.Contains(events, want) {
|
||||
t.Fatalf("events lack %s: %v", want, mm.events)
|
||||
}
|
||||
if !strings.HasPrefix(strings.Join(mm.events, "|"), "licence.adopted") {
|
||||
t.Fatalf("events %v", mm.events)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -239,13 +234,13 @@ func TestARotationAndASwitchEachGiveANewerGeneration(t *testing.T) {
|
||||
if _, err := mm.m.AdoptKey(ctx, "api", file); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := mm.m.Bind(ctx, "laptop", "api"); err != nil {
|
||||
if _, err := mm.m.Bind(ctx, "laptop", "api", "test"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if mm.state["laptop"].Licence != "api" || mm.state["laptop"].Generation <= g1 {
|
||||
t.Fatalf("a switch to a licence rotated less often went backwards: %v", mm.state["laptop"])
|
||||
}
|
||||
if _, err := mm.m.Release(ctx, "laptop"); err != nil {
|
||||
if _, err := mm.m.Release(ctx, "laptop", "test"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := mm.state["laptop"]; ok {
|
||||
@@ -459,73 +454,3 @@ func TestWhatIsNotAnAPIKeyIsRefused(t *testing.T) {
|
||||
t.Fatal("nothing was kept")
|
||||
}
|
||||
}
|
||||
|
||||
// A change of what a consumer holds says what, why and who in the journal, the audit and an event (novox/hq
|
||||
// issue 625): a login, a switch through the verb by a proven caller, a rotation and a release.
|
||||
func TestEveryMoveSaysWhatWhyAndWho(t *testing.T) {
|
||||
mm := newMesh(t)
|
||||
var lines []string
|
||||
mm.m.Log = func(format string, args ...any) { lines = append(lines, fmt.Sprintf(format, args...)) }
|
||||
ctx := context.Background()
|
||||
_, _ = mm.m.Consider(ctx, []Holdings{mm.login("laptop", "rt-a", true, t0)})
|
||||
if s := mm.state["laptop"]; !strings.Contains(s.Why, "a login to "+licence1+" on laptop") || s.By != Itself {
|
||||
t.Fatalf("a login's binding said %+v", s)
|
||||
}
|
||||
file := filepath.Join(t.TempDir(), "key")
|
||||
_ = os.WriteFile(file, []byte("sk-ant-api03-key-0123456789abcdefghij\n"), 0o600)
|
||||
if _, err := mm.m.AdoptKey(ctx, "api", file); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
asked := WithCause(ctx, Cause{Why: "the switch verb", By: "g14.node-tools"})
|
||||
if _, err := mm.m.Bind(asked, "laptop", "api"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s := mm.state["laptop"]; s.Why != "the switch verb" || s.By != "g14.node-tools" {
|
||||
t.Fatalf("the node is told %+v", s)
|
||||
}
|
||||
said := strings.Join(lines, "\n")
|
||||
if !strings.Contains(said, "switched laptop from "+licence1+" to api (generation ") ||
|
||||
!strings.Contains(said, "the switch verb; asked by g14.node-tools") {
|
||||
t.Fatalf("the journal said:\n%s", said)
|
||||
}
|
||||
var switched string
|
||||
for _, e := range mm.events {
|
||||
if strings.HasPrefix(e, "licence.switched ") {
|
||||
switched = e
|
||||
}
|
||||
}
|
||||
for _, want := range []string{`"by":"g14.node-tools"`, `"from":"` + licence1 + `"`, `"licence":"api"`, `"consumer":"laptop"`, `"generation":`} {
|
||||
if !strings.Contains(switched, want) {
|
||||
t.Fatalf("the event %q lacks %s; events %v", switched, want, mm.events)
|
||||
}
|
||||
}
|
||||
last := mm.store.Audits[len(mm.store.Audits)-1]
|
||||
if raw, _ := json.Marshal(last); !strings.Contains(string(raw), `"by":"g14.node-tools"`) || !strings.Contains(string(raw), "switched") {
|
||||
t.Fatalf("the audit kept %s", raw)
|
||||
}
|
||||
|
||||
_, _ = mm.m.Rotate(WithCause(ctx, Cause{Why: "the refresh verb", By: "novox.node-tools"}), licence1, true)
|
||||
if !strings.Contains(strings.Join(mm.events, "|"), `licence.rotated {"by":"novox.node-tools"`) {
|
||||
t.Fatalf("a rotation asked for said no caller: %v", mm.events)
|
||||
}
|
||||
|
||||
if _, err := mm.m.Release(WithCause(ctx, Cause{Why: "the release verb", By: "person.operator"}), "laptop"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if said := strings.Join(lines, "\n"); !strings.Contains(said, "released laptop from api: the release verb; asked by person.operator") {
|
||||
t.Fatalf("the journal said:\n%s", said)
|
||||
}
|
||||
for _, e := range mm.events {
|
||||
if strings.Contains(e, "rt-a") || strings.Contains(e, "sk-ant-") || strings.Contains(e, "local-laptop") {
|
||||
t.Fatalf("an event carries a secret: %s", e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A call with no cause is said as this module's own, never as nobody.
|
||||
func TestNoCauseReadsAsTheModulesOwn(t *testing.T) {
|
||||
if c := CauseOf(context.Background()); c.By != Itself || c.Why == "" {
|
||||
t.Fatalf("no cause read as %+v", c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,7 +3,7 @@ module claude-licence-manager
|
||||
go 1.25.0
|
||||
|
||||
require (
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012082113-9747f3e1b1ac
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7
|
||||
github.com/jackc/pgx/v5 v5.11.0
|
||||
)
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012082113-9747f3e1b1ac h1:jnZ/n+fFMo7VQPNNXK1gjStHIJPw3SPbzPxsf/PpZqw=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012082113-9747f3e1b1ac/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
|
||||
@@ -8,11 +8,7 @@
|
||||
],
|
||||
"contributes": {
|
||||
"postgres-database": {
|
||||
"name": "claude_licences",
|
||||
"readable": [
|
||||
"audit",
|
||||
"binding"
|
||||
]
|
||||
"name": "claude_licences"
|
||||
}
|
||||
},
|
||||
"binds": {
|
||||
@@ -63,10 +59,6 @@
|
||||
"emits": [
|
||||
"licence.adopted",
|
||||
"licence.refused",
|
||||
"licence.bound",
|
||||
"licence.switched",
|
||||
"licence.released",
|
||||
"licence.rotated",
|
||||
"licence.failing",
|
||||
"usage.read"
|
||||
],
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
# dev-tools
|
||||
|
||||
The tools a machine needs to develop the mesh's own code (novox/hq issue 626). Every node is a development
|
||||
machine: an agent working on a change builds it and runs its tests there, with these tools, before it opens a
|
||||
pull request. The lab is the QA step after that: it tests the work in a sandbox, through its seat at the mesh
|
||||
scope, and is not where development happens.
|
||||
|
||||
Assign it to every node.
|
||||
|
||||
## Owns
|
||||
|
||||
| what | package | class (ADR 0182) |
|
||||
|---|---|---|
|
||||
| the Go toolchain, for `go build` and `go test` in every Go module | `go` | package |
|
||||
| a C compiler, for `go test -race` and cgo | `gcc` | package |
|
||||
| make | `make` | package |
|
||||
| Node.js and npm, for the TypeScript modules' `npm ci` and `npm test` | `nodejs`, `npm` | package |
|
||||
| Python, for the scripts some checks run | `python` | package |
|
||||
| git and jq | `git`, `jq` | package |
|
||||
|
||||
All are in the distribution's official repositories, so the node-engine's package manager installs them on any
|
||||
node. The versions are the distribution's: the toolchain images that run `mesh/repo-check` stay the judge of a
|
||||
pull request, and a local run is the developer's early answer, not the verdict.
|
||||
|
||||
Nothing else is placed: no file, no service, no environment. A module's own dependencies come from its lock
|
||||
file (`go.mod`, `package-lock.json`), as in the toolchain images.
|
||||
@@ -1,49 +0,0 @@
|
||||
{
|
||||
"module": "dev-tools",
|
||||
"version": "1",
|
||||
"capabilities": [
|
||||
"package-manager"
|
||||
],
|
||||
"resources": [
|
||||
{
|
||||
"id": "go",
|
||||
"type": "package",
|
||||
"package": "go"
|
||||
},
|
||||
{
|
||||
"id": "c-compiler",
|
||||
"type": "package",
|
||||
"package": "gcc"
|
||||
},
|
||||
{
|
||||
"id": "make",
|
||||
"type": "package",
|
||||
"package": "make"
|
||||
},
|
||||
{
|
||||
"id": "nodejs",
|
||||
"type": "package",
|
||||
"package": "nodejs"
|
||||
},
|
||||
{
|
||||
"id": "npm",
|
||||
"type": "package",
|
||||
"package": "npm"
|
||||
},
|
||||
{
|
||||
"id": "python",
|
||||
"type": "package",
|
||||
"package": "python"
|
||||
},
|
||||
{
|
||||
"id": "git",
|
||||
"type": "package",
|
||||
"package": "git"
|
||||
},
|
||||
{
|
||||
"id": "jq",
|
||||
"type": "package",
|
||||
"package": "jq"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -44,20 +44,6 @@ const (
|
||||
IgnoredBy = "file:" + ListFile
|
||||
)
|
||||
|
||||
// isOurList says whether an entry of a jail's ignore list, or the source the daemon's log names, is the list of
|
||||
// the operator's own addresses. fail2ban keeps a file entry in its own form, not as written: it reads
|
||||
// `file:/var/lib/…` and prints it back as `file://var/lib/…` (server/ipdns.py, FileIPAddrSet.__repr__, in 1.1),
|
||||
// in `get <jail> ignoreip` and in its `Ignore … by …` lines alike. So the path is compared, with fail2ban's own
|
||||
// reading of the slashes after `file:`, never the text (seen on ace on 2026-10-13: every jail held the list
|
||||
// and the watch said none did).
|
||||
func isOurList(word string) bool {
|
||||
path, ok := strings.CutPrefix(strings.TrimSpace(word), "file:")
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
return "/"+strings.TrimLeft(path, "/") == ListFile
|
||||
}
|
||||
|
||||
// Own is the setting read: its entries, and each one that is not an address or a range, with why.
|
||||
type Own struct {
|
||||
Entries []netip.Prefix `json:"-"`
|
||||
|
||||
@@ -15,15 +15,10 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// asPrinted is the list's entry as fail2ban 1.1 prints it back, in `get <jail> ignoreip` and in its log: it reads
|
||||
// `file:/var/lib/…` and keeps it as `file://var/lib/…` (FileIPAddrSet.__repr__), as ace's daemon printed it on
|
||||
// 2026-10-13.
|
||||
const asPrinted = "file://var/lib/mesh-fail2ban/own-addresses"
|
||||
|
||||
// ignoring is a jail's ignore list as fail2ban-client 1.1 prints it, with the base the module writes.
|
||||
const ignoring = "These IP addresses/networks are ignored:\n|- 127.0.0.0/8\n|- 10.10.0.0/24\n|- 10.0.0.0/8\n" +
|
||||
"|- 172.16.0.0/12\n|- 192.168.0.0/16\n|- 169.254.0.0/16\n|- fc00::/7\n|- fe80::/10\n" +
|
||||
"|- " + asPrinted + "\n`- ::1\n"
|
||||
"|- file:/var/lib/mesh-fail2ban/own-addresses\n`- ::1\n"
|
||||
|
||||
// home is the operator's home uplink in these tests (RFC 5737).
|
||||
const home = "203.0.113.5"
|
||||
@@ -158,7 +153,7 @@ func appendLog(t *testing.T, root, text string) {
|
||||
func daemon(missing bool) Runner {
|
||||
gitea := ignoring
|
||||
if missing {
|
||||
gitea = strings.Replace(ignoring, "|- "+asPrinted+"\n", "", 1)
|
||||
gitea = strings.Replace(ignoring, "|- file:/var/lib/mesh-fail2ban/own-addresses\n", "", 1)
|
||||
}
|
||||
return fake(map[string]string{"status": "Status\n|- Number of jail:\t2\n`- Jail list:\tgitea, recidive\n",
|
||||
"get gitea ignoreip": gitea, "get recidive ignoreip": ignoring}, nil)
|
||||
@@ -173,7 +168,7 @@ func TestAFailureIgnoredFromTheOperatorsOwnAddressIsSaidUntilADayAfterTheLast(t
|
||||
clock := now
|
||||
w := &Watcher{F: Fail2ban{Run: daemon(false)}, Root: root, Now: func() time.Time { return clock }}
|
||||
|
||||
appendLog(t, root, logAt(now.Add(-48*time.Hour), "gitea", home, asPrinted)+
|
||||
appendLog(t, root, logAt(now.Add(-48*time.Hour), "gitea", home, IgnoredBy)+
|
||||
logAt(now.Add(-time.Minute), "sshd", "192.168.1.1", "ip"))
|
||||
s := w.Look(context.Background())
|
||||
if err := WriteState(root, s); err != nil {
|
||||
@@ -188,7 +183,7 @@ func TestAFailureIgnoredFromTheOperatorsOwnAddressIsSaidUntilADayAfterTheLast(t
|
||||
|
||||
var loop strings.Builder
|
||||
for i := 0; i < 40; i++ {
|
||||
loop.WriteString(logAt(now.Add(time.Duration(i)*15*time.Second), "gitea", home, asPrinted))
|
||||
loop.WriteString(logAt(now.Add(time.Duration(i)*15*time.Second), "gitea", home, IgnoredBy))
|
||||
}
|
||||
appendLog(t, root, loop.String()[:loop.Len()-10]) // the daemon is still writing the last line
|
||||
clock = now.Add(10 * time.Minute)
|
||||
@@ -308,21 +303,3 @@ func TestTheManifestGivesEveryJailTheListAndTheWatchItsHealth(t *testing.T) {
|
||||
t.Errorf("tools %v", m.Tools)
|
||||
}
|
||||
}
|
||||
|
||||
// The list is known by its path, in the form the module writes it and in the form fail2ban prints it back.
|
||||
func TestTheListIsKnownInTheFormFail2banPrintsItBack(t *testing.T) {
|
||||
for _, word := range []string{IgnoredBy, asPrinted, "file:///var/lib/mesh-fail2ban/own-addresses"} {
|
||||
if !isOurList(word) {
|
||||
t.Errorf("%q is the list and was not known", word)
|
||||
}
|
||||
}
|
||||
for _, word := range []string{"ip", "/var/lib/mesh-fail2ban/own-addresses", "file://var/lib/mesh-fail2ban/other",
|
||||
"file:/var/lib/mesh-fail2ban/own-addresses.new", "192.168.0.0/16"} {
|
||||
if isOurList(word) {
|
||||
t.Errorf("%q was taken for the list", word)
|
||||
}
|
||||
}
|
||||
if !namesOurList(listed(ignoring)) {
|
||||
t.Errorf("the jail's list as the daemon prints it was read as lacking the list")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -103,7 +103,7 @@ func (w *Watcher) Look(ctx context.Context) State {
|
||||
s.Unread = append(s.Unread, fmt.Sprintf("the jail %s's ignore list cannot be read: %v", j, err))
|
||||
continue
|
||||
}
|
||||
if !namesOurList(listed(out)) {
|
||||
if !contains(listed(out), IgnoredBy) {
|
||||
s.Missing = append(s.Missing, j)
|
||||
}
|
||||
}
|
||||
@@ -115,15 +115,6 @@ func (w *Watcher) Look(ctx context.Context) State {
|
||||
return s
|
||||
}
|
||||
|
||||
func namesOurList(entries []string) bool {
|
||||
for _, e := range entries {
|
||||
if isOurList(e) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func contains(list []string, want string) bool {
|
||||
for _, l := range list {
|
||||
if l == want {
|
||||
@@ -171,7 +162,7 @@ func (w *Watcher) readLog(now time.Time) error {
|
||||
// take keeps one line of the log when it is a failure ignored because of the operator's own addresses.
|
||||
func (w *Watcher) take(line string, now time.Time) {
|
||||
m := ignoredLine.FindStringSubmatch(line)
|
||||
if m == nil || !isOurList(m[4]) {
|
||||
if m == nil || m[4] != IgnoredBy {
|
||||
return
|
||||
}
|
||||
at, err := time.ParseInLocation("2006-01-02 15:04:05", m[1], time.Local)
|
||||
|
||||
@@ -905,15 +905,6 @@ export class GiteaAdmin {
|
||||
GiteaAdmin.fail(`/teams/${teamId}/members/${username}`, res);
|
||||
}
|
||||
|
||||
/** Take a user off a team; one not on it is already off. */
|
||||
async removeUserFromTeam(teamId: number, username: string): Promise<void> {
|
||||
const res = await this.request(`/teams/${teamId}/members/${encodeURIComponent(username)}`, {
|
||||
method: "DELETE",
|
||||
});
|
||||
if (res.status === 204 || res.status === 200 || res.status === 404) return;
|
||||
GiteaAdmin.fail(`/teams/${teamId}/members/${username}`, res);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a consumer's user logs in with exactly this password and is still a member of the
|
||||
* package team. Read-only: the password is checked as the consumer presents it, basic auth on the
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
"type": "module",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"build": "tsc client.ts token.ts index.ts provisioner/index.ts provisioner/teams.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
|
||||
"build": "tsc client.ts token.ts index.ts provisioner/index.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
|
||||
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
|
||||
},
|
||||
"dependencies": {
|
||||
|
||||
@@ -11,8 +11,8 @@
|
||||
//
|
||||
// The `npm-package-registry` interface: a consumer authenticates to the npm registry at
|
||||
// `/api/packages/novox/npm/` with basic auth, as `as` with the password the mesh minted, and can
|
||||
// read packages under the `@novox` scope, and write them unless it asked for read only (teams.ts). The
|
||||
// registry's npm owner is the gitea org `novox`; a consumer is a gitea *user* on one of that org's teams.
|
||||
// read and write packages under the `@novox` scope. The registry's npm owner is the gitea org
|
||||
// `novox`; a consumer is a gitea *user* placed on that org's package team.
|
||||
//
|
||||
// **The user name and password are the mesh's, not the provisioner's (ADR 0048).** The mesh derives
|
||||
// the login and hands it to both ends, and mints the password. gitea creates a user under exactly
|
||||
@@ -24,9 +24,11 @@
|
||||
|
||||
import { runProvisioner, type Provision } from "@novox/mesh-sdk/provisioner";
|
||||
import { GiteaAdmin } from "../client.js";
|
||||
import { grant, holds } from "./teams.js";
|
||||
|
||||
// Which team a consumer joins — `packages` to publish, `readers` to read only — is teams.ts's.
|
||||
// The npm registry owner: a gitea org named `novox`, whose package team every consumer joins so it
|
||||
// can read and write packages under the `@novox` scope (ADR 0076).
|
||||
const ORG = "novox";
|
||||
const PACKAGE_TEAM = "packages";
|
||||
|
||||
const gitea = GiteaAdmin.fromEnv();
|
||||
|
||||
@@ -36,10 +38,19 @@ const gitea = GiteaAdmin.fromEnv();
|
||||
// module needs the mesh to say where each provision's file is — not yet possible, and not faked.
|
||||
runProvisioner("npm-package-registry", {
|
||||
async create(p: Provision): Promise<void> {
|
||||
// The org and its teams are ensured per create: idempotent, and no separate bootstrap step. The
|
||||
// user carries the consumer's login and the mesh's minted password, set every run so a rotation
|
||||
// takes; membership of its team is what grants it what it may do.
|
||||
await grant(gitea, p);
|
||||
// The org and its package team are the same for every consumer; ensuring them per-create is
|
||||
// idempotent and needs no separate bootstrap step.
|
||||
await gitea.ensureOrg(ORG);
|
||||
const teamId = await gitea.ensureTeam(ORG, PACKAGE_TEAM, true);
|
||||
// The user carries the consumer's login and the mesh's minted password, set every run so a
|
||||
// rotation takes. Membership of the package team is what grants read+write on packages.
|
||||
//
|
||||
// The address is gitea's own convention for one that is not real: its email validation
|
||||
// requires a dotted domain, so `@localhost` was refused at create — the fault that had this
|
||||
// grant retrying for a day — while `@noreply.localhost` is the shape gitea itself gives
|
||||
// hidden addresses.
|
||||
await gitea.ensureUser(p.as, p.password, `${p.as}@noreply.localhost`);
|
||||
await gitea.addUserToTeam(teamId, p.as);
|
||||
},
|
||||
|
||||
async remove(p: { as: string }): Promise<void> {
|
||||
@@ -49,6 +60,6 @@ runProvisioner("npm-package-registry", {
|
||||
// Asked every minute by the harness: whether the backend still holds this consumer exactly as
|
||||
// the mesh gave it, so a login lost behind the provisioner's back is made again (novox/hq issue 120).
|
||||
async holds(p: Provision): Promise<boolean> {
|
||||
return holds(gitea, p);
|
||||
return gitea.holdsTeamMember(ORG, PACKAGE_TEAM, p.as, p.password);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
// Which team of the forge's npm owner a consumer of `npm-package-registry` joins, and the per-consumer
|
||||
// half of the provisioner around it — pure of the forge's address, so it is tested with a stand-in.
|
||||
//
|
||||
// **Two teams, by what the consumer said it needs** (novox/hq issue 624). A consumer that publishes —
|
||||
// the build agent, whose builds publish the SDK and every TypeScript module's packages — joins `packages`:
|
||||
// code read on the org's repositories and write on its packages. A consumer that contributes
|
||||
// `{"access": "read"}` joins `readers`: code read and package read, and no write. The lab is the first: it
|
||||
// clones branches that are not yet reviewed and runs their code beside its credential, and code that is
|
||||
// not reviewed must not be able to publish the mesh's packages with it.
|
||||
|
||||
/** The npm registry's owner: the gitea org every consumer's packages live under (ADR 0076). */
|
||||
export const ORG = "novox";
|
||||
/** The team whose members read and write the org's packages. */
|
||||
export const PACKAGE_TEAM = "packages";
|
||||
/** The team whose members read the org's code and packages, and write nothing. */
|
||||
export const READ_TEAM = "readers";
|
||||
|
||||
/** The team a consumer joins, from what it contributed; an access it may not ask for is refused. */
|
||||
export function teamFor(values: Readonly<Record<string, unknown>> | undefined): { team: string; packageWrite: boolean } {
|
||||
const access = values?.access;
|
||||
if (access === undefined || access === "write") return { team: PACKAGE_TEAM, packageWrite: true };
|
||||
if (access === "read") return { team: READ_TEAM, packageWrite: false };
|
||||
throw new Error(`npm-package-registry: access ${JSON.stringify(access)} is not "read" or "write"`);
|
||||
}
|
||||
|
||||
/** What the adapter asks of the forge's admin API (client.ts GiteaAdmin). */
|
||||
export interface Forge {
|
||||
ensureOrg(org: string): Promise<void>;
|
||||
ensureTeam(org: string, team: string, packageWrite: boolean): Promise<number>;
|
||||
ensureUser(username: string, password: string, email: string): Promise<void>;
|
||||
addUserToTeam(teamId: number, username: string): Promise<void>;
|
||||
removeUserFromTeam(teamId: number, username: string): Promise<void>;
|
||||
holdsTeamMember(org: string, team: string, username: string, password: string): Promise<boolean>;
|
||||
}
|
||||
|
||||
export interface Consumer {
|
||||
readonly as: string;
|
||||
readonly password: string;
|
||||
readonly values: Readonly<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* grant makes the consumer's user, with exactly the mesh's password, a member of its one team. A consumer
|
||||
* that asks for read only is also taken out of the package team, so one that asked for write before and
|
||||
* read now loses the write at once.
|
||||
*/
|
||||
export async function grant(forge: Forge, p: Consumer): Promise<void> {
|
||||
const { team, packageWrite } = teamFor(p.values);
|
||||
await forge.ensureOrg(ORG);
|
||||
const teamId = await forge.ensureTeam(ORG, team, packageWrite);
|
||||
// The address is gitea's own convention for one that is not real: its email validation requires a
|
||||
// dotted domain, so `@localhost` was refused at create, while `@noreply.localhost` is the shape gitea
|
||||
// itself gives hidden addresses.
|
||||
await forge.ensureUser(p.as, p.password, `${p.as}@noreply.localhost`);
|
||||
await forge.addUserToTeam(teamId, p.as);
|
||||
if (!packageWrite) {
|
||||
await forge.removeUserFromTeam(await forge.ensureTeam(ORG, PACKAGE_TEAM, true), p.as);
|
||||
}
|
||||
}
|
||||
|
||||
/** holds is whether the forge still holds the consumer as the mesh gave it: its login, in its team. */
|
||||
export async function holds(forge: Forge, p: Consumer): Promise<boolean> {
|
||||
return forge.holdsTeamMember(ORG, teamFor(p.values).team, p.as, p.password);
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { grant, holds, PACKAGE_TEAM, READ_TEAM, teamFor, type Forge } from "../provisioner/teams.ts";
|
||||
|
||||
// Defends novox/hq issue 624: the lab reads the forge with a credential of its own that can write no
|
||||
// package. A consumer that contributes {"access": "read"} joins a team with code and package read only.
|
||||
|
||||
function aForge(): { forge: Forge; did: string[] } {
|
||||
const did: string[] = [];
|
||||
const teams: Record<string, number> = {};
|
||||
const forge: Forge = {
|
||||
async ensureOrg(org) { did.push(`org ${org}`); },
|
||||
async ensureTeam(org, team, packageWrite) {
|
||||
did.push(`team ${org}/${team} packages:${packageWrite ? "write" : "read"}`);
|
||||
return (teams[team] ??= Object.keys(teams).length + 1);
|
||||
},
|
||||
async ensureUser(username) { did.push(`user ${username}`); },
|
||||
async addUserToTeam(id, username) { did.push(`add ${username} to ${id}`); },
|
||||
async removeUserFromTeam(id, username) { did.push(`remove ${username} from ${id}`); },
|
||||
async holdsTeamMember(org, team, username) { did.push(`holds ${username} in ${org}/${team}`); return true; },
|
||||
};
|
||||
return { forge, did };
|
||||
}
|
||||
|
||||
test("a consumer that asks nothing, or write, publishes: the package team", () => {
|
||||
assert.deepEqual(teamFor({}), { team: PACKAGE_TEAM, packageWrite: true });
|
||||
assert.deepEqual(teamFor({ access: "write" }), { team: PACKAGE_TEAM, packageWrite: true });
|
||||
});
|
||||
|
||||
test("a consumer that asks to read joins the readers, who write no package", () => {
|
||||
assert.deepEqual(teamFor({ access: "read" }), { team: READ_TEAM, packageWrite: false });
|
||||
assert.throws(() => teamFor({ access: "admin" }), /not "read" or "write"/);
|
||||
});
|
||||
|
||||
test("a reader is made, put on the readers team, and taken off the package team", async () => {
|
||||
const { forge, did } = aForge();
|
||||
await grant(forge, { as: "lab-shanks", password: "pw", values: { access: "read" } });
|
||||
assert.deepEqual(did, [
|
||||
"org novox",
|
||||
"team novox/readers packages:read",
|
||||
"user lab-shanks",
|
||||
"add lab-shanks to 1",
|
||||
"team novox/packages packages:write",
|
||||
"remove lab-shanks from 2",
|
||||
]);
|
||||
did.length = 0;
|
||||
await holds(forge, { as: "lab-shanks", password: "pw", values: { access: "read" } });
|
||||
assert.deepEqual(did, ["holds lab-shanks in novox/readers"]);
|
||||
});
|
||||
|
||||
test("a publisher is made and put on the package team, as before", async () => {
|
||||
const { forge, did } = aForge();
|
||||
await grant(forge, { as: "build-agent-ace", password: "pw", values: {} });
|
||||
assert.deepEqual(did, ["org novox", "team novox/packages packages:write", "user build-agent-ace", "add build-agent-ace to 1"]);
|
||||
});
|
||||
@@ -8,5 +8,5 @@
|
||||
"skipLibCheck": true,
|
||||
"noEmit": true
|
||||
},
|
||||
"include": ["client.ts", "token.ts", "pulls.ts", "protection.ts", "index.ts", "provisioner/index.ts", "provisioner/teams.ts", "tools/index.ts"]
|
||||
"include": ["client.ts", "token.ts", "pulls.ts", "protection.ts", "index.ts", "provisioner/index.ts", "tools/index.ts"]
|
||||
}
|
||||
|
||||
+2
-18
@@ -5,20 +5,6 @@
|
||||
"container-runtime",
|
||||
"virtualisation"
|
||||
],
|
||||
"requires": [
|
||||
"npm-package-registry"
|
||||
],
|
||||
"contributes": {
|
||||
"npm-package-registry": {
|
||||
"access": "read"
|
||||
}
|
||||
},
|
||||
"binds": {
|
||||
"npm-package-registry": "${dir:state}/package-registry.json"
|
||||
},
|
||||
"secrets": {
|
||||
"npm-package-registry": "${dir:state}/package-registry.secret"
|
||||
},
|
||||
"data": {
|
||||
"own": [
|
||||
{
|
||||
@@ -47,7 +33,7 @@
|
||||
"type": "file",
|
||||
"path": "${dir:state}/lab.env",
|
||||
"mode": "0600",
|
||||
"content": "MESH_LAB_FORGE=${setting:forge}\nMESH_LAB_FORGE_BINDING=${dir:state}/package-registry.json\nMESH_LAB_FORGE_SECRET=${dir:state}/package-registry.secret\n"
|
||||
"content": "MESH_LAB_FORGE=${setting:forge}\n"
|
||||
},
|
||||
{
|
||||
"id": "git",
|
||||
@@ -104,9 +90,7 @@
|
||||
],
|
||||
"env": {
|
||||
"MESH_LAB_WORK": "${dir:work}",
|
||||
"MESH_LAB_ENV_FILE": "${dir:state}/lab.env",
|
||||
"MESH_LAB_FORGE_BINDING": "${dir:state}/package-registry.json",
|
||||
"MESH_LAB_FORGE_SECRET": "${dir:state}/package-registry.secret"
|
||||
"MESH_LAB_ENV_FILE": "${dir:state}/lab.env"
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
@@ -4,9 +4,6 @@
|
||||
"description": "lab — the lab, as a module: runs beds against the forge's branches when the mesh asks (novox/hq ADR 0172).",
|
||||
"type": "module",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
|
||||
},
|
||||
"dependencies": { "@novox/mesh-sdk": "^0.1.0" },
|
||||
"devDependencies": { "@types/node": "^22.0.0", "typescript": "^5.6.0" }
|
||||
}
|
||||
|
||||
@@ -1,321 +0,0 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { installPlan, installStep, NPM_REPOSITORIES, REPOSITORIES, script } from "../tools/runs.ts";
|
||||
|
||||
// Defends novox/hq issue 624: lab_run ran `npm ci` at the root of every checkout it installs, and
|
||||
// mesh-tools keeps its package in node-tools/ with no lock file at its root, so every run failed at its
|
||||
// setup, in 9 seconds, before anything was built.
|
||||
|
||||
function aCheckout(files: Record<string, string>): string {
|
||||
const root = mkdtempSync(join(tmpdir(), "lab-checkout-"));
|
||||
for (const [path, content] of Object.entries(files)) {
|
||||
mkdirSync(join(root, path, ".."), { recursive: true });
|
||||
writeFileSync(join(root, path), content);
|
||||
}
|
||||
return root;
|
||||
}
|
||||
|
||||
test("a checkout with a lock file at its root is installed there", () => {
|
||||
const root = aCheckout({ "package.json": "{}", "package-lock.json": "{}", "node-tools/package.json": "{}" });
|
||||
assert.deepEqual(installPlan(root), { install: [root], unlocked: [] });
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("a checkout whose package lives one directory down is installed there, as mesh-tools' node-tools", () => {
|
||||
const root = aCheckout({ "node-tools/package.json": "{}", "node-tools/package-lock.json": "{}", "Dockerfile": "" });
|
||||
assert.deepEqual(installPlan(root), { install: [join(root, "node-tools")], unlocked: [] });
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("a package.json with no lock file is named, never installed", () => {
|
||||
const root = aCheckout({ "package.json": "{}" });
|
||||
assert.deepEqual(installPlan(root), { install: [], unlocked: [root] });
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// The shell a run executes follows the same rule: npm is asked where installPlan says, and a checkout
|
||||
// with nothing installable fails the run loudly, naming it.
|
||||
function runStep(root: string, build: boolean): { ok: boolean; said: string; asked: string[] } {
|
||||
const bin = mkdtempSync(join(tmpdir(), "lab-npm-"));
|
||||
const record = join(bin, "asked");
|
||||
writeFileSync(join(bin, "npm"), `#!/bin/sh\necho "$(pwd) $*" >> ${JSON.stringify(record)}\n`);
|
||||
chmodSync(join(bin, "npm"), 0o755);
|
||||
const ran = spawnSync("bash", ["-c", `set -euo pipefail\n${installStep(root, build)}`], {
|
||||
encoding: "utf8", env: { ...process.env, PATH: `${bin}:${process.env.PATH}` },
|
||||
});
|
||||
let asked: string[] = [];
|
||||
try {
|
||||
asked = readFileSync(record, "utf8").trim().split("\n").filter(Boolean);
|
||||
} catch { /* nothing asked */ }
|
||||
rmSync(bin, { recursive: true, force: true });
|
||||
return { ok: ran.status === 0, said: `${ran.stdout}${ran.stderr}`, asked };
|
||||
}
|
||||
|
||||
test("a run installs and builds mesh-tools in node-tools, where its lock file is", () => {
|
||||
const root = aCheckout({ "node-tools/package.json": "{}", "node-tools/package-lock.json": "{}" });
|
||||
const r = runStep(root, true);
|
||||
assert.ok(r.ok, r.said);
|
||||
assert.deepEqual(r.asked, [
|
||||
`${join(root, "node-tools")} ci --no-audit --no-fund --loglevel=error`,
|
||||
`${join(root, "node-tools")} run build --if-present`,
|
||||
]);
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("a run installs a checkout with its lock file at the root there, and builds only when asked", () => {
|
||||
const root = aCheckout({ "package.json": "{}", "package-lock.json": "{}" });
|
||||
const r = runStep(root, false);
|
||||
assert.ok(r.ok, r.said);
|
||||
assert.deepEqual(r.asked, [`${root} ci --no-audit --no-fund --loglevel=error`]);
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("a run fails at its setup, naming the checkout, when nothing of it can be installed", () => {
|
||||
const layouts: Record<string, string>[] = [{ "package.json": "{}" }, { "Dockerfile": "" }, { "sub/package.json": "{}" }];
|
||||
for (const files of layouts) {
|
||||
const root = aCheckout(files);
|
||||
const r = runStep(root, false);
|
||||
assert.equal(r.ok, false, `installed ${JSON.stringify(files)}`);
|
||||
assert.match(r.said, /issue 624/);
|
||||
assert.deepEqual(r.asked, []);
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// A run checks out mesh-tools-go too (novox/hq issue 616): genesis builds the Go toolchain from the
|
||||
// repository beside mesh-tools, and a bed that raises genesis serves it. And it tells the beds where the
|
||||
// checkouts are.
|
||||
test("a run checks out mesh-tools-go and says where its checkouts are", () => {
|
||||
assert.ok(REPOSITORIES.includes("mesh-tools-go"));
|
||||
const s = script("/work", "run-x", "http://forge", ["test/integration/genesis-keeps-its-bus.test.ts"], {});
|
||||
assert.match(s, /novox\/mesh-tools-go\.git' '\/work\/run-x\/mesh-tools-go'/);
|
||||
assert.match(s, /export MESH_LAB_REPOS='\/work\/run-x'/);
|
||||
assert.doesNotMatch(s, /cd '\/work\/run-x'\/\$repo && npm ci/);
|
||||
for (const repo of NPM_REPOSITORIES) assert.ok(s.includes(`'/work/run-x/${repo}'/package-lock.json`), repo);
|
||||
});
|
||||
|
||||
// Defends novox/hq issue 624, its second half: mesh-tools-go is private, and the lab cloned over HTTP with
|
||||
// no credential ("could not read Username"). A run clones with the forge credential the mesh binds and
|
||||
// seals to the lab, through git's credential store, and the secret is in no script, URL or argument.
|
||||
import { credentialArgs, credentialPath, forgeCredential } from "../tools/runs.ts";
|
||||
|
||||
function aBinding(as: string, secret: string): { binding: string; secret: string } {
|
||||
const dir = mkdtempSync(join(tmpdir(), "lab-binding-"));
|
||||
writeFileSync(join(dir, "package-registry.json"), JSON.stringify({ provision: "npm-package-registry", at: "10.42.0.1", as,
|
||||
serves: { scheme: "http", port: 3000 } }));
|
||||
writeFileSync(join(dir, "package-registry.secret"), `${secret}\n`);
|
||||
return { binding: join(dir, "package-registry.json"), secret: join(dir, "package-registry.secret") };
|
||||
}
|
||||
|
||||
test("the forge credential is the bound account and its sealed secret, for the forge's own origin", () => {
|
||||
const b = aBinding("lab-shanks", "s3cret-of-forty-characters");
|
||||
assert.equal(forgeCredential("http://novox.internal:20000", b.binding, b.secret),
|
||||
"http://lab-shanks:s3cret-of-forty-characters@novox.internal:20000");
|
||||
assert.equal(forgeCredential("http://novox.internal:20000/", b.binding, ""), undefined);
|
||||
assert.equal(forgeCredential("", b.binding, b.secret), undefined);
|
||||
assert.equal(forgeCredential("http://novox.internal:20000", "/nowhere", b.secret), undefined);
|
||||
});
|
||||
|
||||
test("a run clones through git's credential store, and its script carries no secret", () => {
|
||||
const b = aBinding("lab-shanks", "s3cret-of-forty-characters");
|
||||
const credential = forgeCredential("http://novox.internal:20000", b.binding, b.secret);
|
||||
const path = credentialPath("/work", "run-x");
|
||||
const s = script("/work", "run-x", "http://novox.internal:20000", ["t.test.ts"], {}, path);
|
||||
assert.ok(credential);
|
||||
assert.doesNotMatch(s, /s3cret/);
|
||||
assert.match(s, /git -c credential\.helper= -c 'credential\.helper=store --file=\/work\/\.run-x\.git-credentials' clone --quiet --depth 50 --branch 'main' 'http:\/\/novox\.internal:20000\/novox\/mesh-tools-go\.git'/);
|
||||
assert.match(s, /export GIT_TERMINAL_PROMPT=0/);
|
||||
assert.equal(path.startsWith("/work/run-x/"), false, "the credential is kept out of the run's own directory");
|
||||
assert.equal(credentialArgs(undefined), "");
|
||||
});
|
||||
|
||||
// Held against git itself: the credential in the store is offered to the forge's origin when it asks, and
|
||||
// a clone without it fails at once rather than prompting.
|
||||
test("git presents the stored credential to the forge, and only when one is given", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "lab-git-"));
|
||||
const store = join(dir, "git-credentials");
|
||||
writeFileSync(store, "http://lab-shanks:pw@forge.invalid:20000\n", { mode: 0o600 });
|
||||
const fill = spawnSync("sh", ["-c", `printf 'protocol=http\\nhost=forge.invalid:20000\\n\\n' | git ${credentialArgs(store)}credential fill`],
|
||||
{ encoding: "utf8", env: { ...process.env, GIT_TERMINAL_PROMPT: "0" } });
|
||||
assert.match(fill.stdout, /username=lab-shanks/);
|
||||
assert.match(fill.stdout, /password=pw/);
|
||||
const other = spawnSync("sh", ["-c", `printf 'protocol=http\\nhost=elsewhere.invalid\\n\\n' | git ${credentialArgs(store)}credential fill`],
|
||||
{ encoding: "utf8", env: { ...process.env, GIT_TERMINAL_PROMPT: "0" } });
|
||||
assert.doesNotMatch(other.stdout, /password=pw/);
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// Review 196 on mesh-catalog #272: the credential file outlived a stop, because SIGTERM to the run's
|
||||
// process group killed the wrapper before its cleanup. It is now removed right after the clones, on
|
||||
// every way the script ends, and at the next run's start and in lab_check for what a crash left.
|
||||
import { cleanCredentials, credentialGuard } from "../tools/runs.ts";
|
||||
|
||||
test("a stopped script removes the credential, whatever ends it", () => {
|
||||
for (const signal of ["SIGTERM", "SIGINT", "SIGHUP"] as const) {
|
||||
const dir = mkdtempSync(join(tmpdir(), "lab-guard-"));
|
||||
const file = join(dir, ".run-x.git-credentials");
|
||||
writeFileSync(file, "http://u:p@h\n", { mode: 0o600 });
|
||||
const ran = spawnSync("bash", ["-c", `${credentialGuard(file)}\nkill -${signal.slice(3)} $$\nsleep 5`], { encoding: "utf8", timeout: 10_000 });
|
||||
assert.notEqual(ran.status, 0, `${signal}: the script was not stopped`);
|
||||
assert.equal(existsSync(file), false, `${signal} left the credential on disk`);
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("the script removes the credential as soon as its clones are done", () => {
|
||||
const s = script("/work", "run-x", "http://forge", ["t.test.ts"], {}, "/work/.run-x.git-credentials");
|
||||
const lastClone = s.lastIndexOf("clone --quiet");
|
||||
const removed = s.indexOf("rm -f '/work/.run-x.git-credentials'", lastClone);
|
||||
const building = s.indexOf("npm ci");
|
||||
assert.ok(lastClone > 0 && removed > lastClone && removed < building, "not removed between the clones and the build");
|
||||
assert.match(s, /trap 'rm -f '\\''\/work\/\.run-x\.git-credentials'\\''' EXIT/);
|
||||
});
|
||||
|
||||
test("credentials a crashed run left are removed, but not the running one's", () => {
|
||||
const work = mkdtempSync(join(tmpdir(), "lab-work-"));
|
||||
for (const n of [".run-a.git-credentials", ".run-b.git-credentials", "run-a", "other"]) writeFileSync(join(work, n), "x");
|
||||
const removed = cleanCredentials(work, join(work, ".run-b.git-credentials"));
|
||||
assert.deepEqual(removed, [join(work, ".run-a.git-credentials")]);
|
||||
assert.ok(existsSync(join(work, ".run-b.git-credentials")) && existsSync(join(work, "run-a")) && existsSync(join(work, "other")));
|
||||
rmSync(work, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// The lab asks the forge for read only (review 196): its credential is for clones of branches nobody has
|
||||
// reviewed yet, and must not be able to publish the mesh's packages.
|
||||
test("the lab asks the package registry for read only", () => {
|
||||
const m = JSON.parse(readFileSync(join(import.meta.dirname, "..", "module.json"), "utf8")) as {
|
||||
requires: unknown[]; contributes?: Record<string, Record<string, unknown>>;
|
||||
};
|
||||
assert.ok(m.requires.includes("npm-package-registry"));
|
||||
assert.equal(m.contributes?.["npm-package-registry"]?.access, "read");
|
||||
});
|
||||
|
||||
// lab_check failed for ever on "destination path '<work>/check' already exists": it cloned into one fixed
|
||||
// path and ignored whether removing the last one worked (novox/hq issue 624). Each check now works in a
|
||||
// fresh directory, and what earlier checks left is cleared or named.
|
||||
import { checkDir, clearChecks } from "../tools/runs.ts";
|
||||
|
||||
test("each check gets a fresh directory of its own", () => {
|
||||
const work = mkdtempSync(join(tmpdir(), "lab-work-"));
|
||||
const a = checkDir(work);
|
||||
const b = checkDir(work);
|
||||
assert.notEqual(a, b);
|
||||
assert.ok(a.startsWith(join(work, "check-")) && existsSync(a) && existsSync(b));
|
||||
rmSync(work, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// As root nothing is locked against removal, so the unremovable tree cannot be made: skipped there, before
|
||||
// it makes anything (the merge check runs as root).
|
||||
test("what earlier checks left is cleared, and what cannot be is named rather than tripped over", {
|
||||
skip: process.getuid?.() === 0 ? "running as root, which no directory mode stops from removing anything" : false,
|
||||
}, () => {
|
||||
const work = mkdtempSync(join(tmpdir(), "lab-work-"));
|
||||
for (const d of ["check/src", "check-setup/mesh-lab", "check-abc", "run-x", "keep"]) mkdirSync(join(work, d), { recursive: true });
|
||||
// A tree this account cannot empty: a directory it may not write, with something in it.
|
||||
mkdirSync(join(work, "check-stuck", "locked"), { recursive: true });
|
||||
writeFileSync(join(work, "check-stuck", "locked", "file"), "x");
|
||||
chmodSync(join(work, "check-stuck", "locked"), 0o555);
|
||||
const said = clearChecks(work);
|
||||
chmodSync(join(work, "check-stuck", "locked"), 0o755);
|
||||
for (const gone of ["check", "check-setup", "check-abc"]) assert.equal(existsSync(join(work, gone)), false, gone);
|
||||
assert.ok(existsSync(join(work, "run-x")) && existsSync(join(work, "keep")));
|
||||
assert.equal(said.length, 1);
|
||||
assert.match(said[0] as string, /check-stuck, left by an earlier check, could not be removed/);
|
||||
rmSync(work, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// And as root too: what earlier checks left is cleared.
|
||||
test("leftover checks are cleared, whoever runs the check", () => {
|
||||
const work = mkdtempSync(join(tmpdir(), "lab-work-"));
|
||||
for (const d of ["check/src", "check-setup/mesh-lab", "check-abc", "run-x"]) mkdirSync(join(work, d), { recursive: true });
|
||||
assert.deepEqual(clearChecks(work), []);
|
||||
for (const gone of ["check", "check-setup", "check-abc"]) assert.equal(existsSync(join(work, gone)), false, gone);
|
||||
assert.ok(existsSync(join(work, "run-x")));
|
||||
rmSync(work, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// The lab's tools read the bound credential, so the runtime is given that file by the tools bundle's env
|
||||
// (ADR 0192), as gitea and umami are given theirs: the runtime's account may then read exactly it, and no
|
||||
// other credential changes hands (issue 624).
|
||||
test("the lab's tools bundle names its bound credential and binding in its env", () => {
|
||||
const m = JSON.parse(readFileSync(join(import.meta.dirname, "..", "module.json"), "utf8")) as {
|
||||
secrets: Record<string, string>; binds: Record<string, string>;
|
||||
build: { artifacts: { name: string; env?: Record<string, string> }[] };
|
||||
};
|
||||
const env = m.build.artifacts.find((a) => a.name === "code")?.env ?? {};
|
||||
assert.equal(env.MESH_LAB_FORGE_SECRET, m.secrets["npm-package-registry"]);
|
||||
assert.equal(env.MESH_LAB_FORGE_BINDING, m.binds["npm-package-registry"]);
|
||||
});
|
||||
|
||||
// The next setup failure of novox/hq issue 624 (run-2026-10-13T13-15-26-634Z on shanks): npm 12 refused
|
||||
// mesh-tools' locked @novox/mesh-sdk with EALLOWREMOTE. The lab named the registry by its forge setting's
|
||||
// address (http://novox.internal:20000/…), the lock file holds the address the forge writes every tarball
|
||||
// URL at (https://git.novox.be/…), and npm 12 installs a locked tarball only from its configured registry.
|
||||
// A run names the registry by the address the registry names itself by.
|
||||
import { npmRegistry } from "../tools/runs.ts";
|
||||
|
||||
function aRegistry(answer: () => Response | Promise<Response>): { get: typeof fetch; asked: string[] } {
|
||||
const asked: string[] = [];
|
||||
const get = (async (url: string | URL | Request) => {
|
||||
asked.push(String(url));
|
||||
return answer();
|
||||
}) as typeof fetch;
|
||||
return { get, asked };
|
||||
}
|
||||
|
||||
const packument = (tarball: string) => new Response(JSON.stringify({
|
||||
name: "@novox/mesh-sdk", "dist-tags": { latest: "0.1.14" },
|
||||
versions: { "0.1.1": { dist: { tarball: "http://old.example/api/packages/novox/npm/x.tgz" } }, "0.1.14": { dist: { tarball } } },
|
||||
}), { status: 200, headers: { "content-type": "application/json" } });
|
||||
|
||||
test("a run names the package registry by the address the registry gives its tarballs", async () => {
|
||||
const r = aRegistry(() => packument("https://forge.example.org/api/packages/novox/npm/%40novox%2Fmesh-sdk/-/0.1.14/mesh-sdk-0.1.14.tgz"));
|
||||
const got = await npmRegistry("http://forge.example.org:3000/", r.get);
|
||||
assert.deepEqual(got, { registry: "https://forge.example.org/api/packages/novox/npm/" });
|
||||
assert.deepEqual(r.asked, ["http://forge.example.org:3000/api/packages/novox/npm/@novox%2fmesh-sdk"]);
|
||||
// The forge's own scheme is taken too, with its port.
|
||||
const plain = await npmRegistry("http://forge.example.org:3000",
|
||||
aRegistry(() => packument("http://forge.example.org:3000/api/packages/novox/npm/%40novox%2Fmesh-sdk/-/0.1.14/mesh-sdk-0.1.14.tgz")).get);
|
||||
assert.deepEqual(plain, { registry: "http://forge.example.org:3000/api/packages/novox/npm/" });
|
||||
});
|
||||
|
||||
// What the registry answers may come over plain HTTP, so it never decides where packages come from: only
|
||||
// the forge's own host, at https or the forge's scheme, under the npm path, with nothing after it
|
||||
// (review of mesh-catalog #275).
|
||||
test("a registry that does not give an address the lab takes leaves the forge's, and the reason is said", async () => {
|
||||
const sdk = "/api/packages/novox/npm/%40novox%2Fmesh-sdk/-/0.1.14/mesh-sdk-0.1.14.tgz";
|
||||
for (const [answer, why] of [
|
||||
[() => new Response("not found", { status: 404 }), /it answered 404/],
|
||||
[() => new Response("{}", { status: 200 }), /no tarball/],
|
||||
[() => { throw new Error("connect ECONNREFUSED"); }, /ECONNREFUSED/],
|
||||
[() => packument(`https://forge.example.org/elsewhere/mesh-sdk-0.1.14.tgz`), /not under \/api\/packages\/novox\/npm\//],
|
||||
[() => packument(`https://attacker.example.net${sdk}`), /attacker\.example\.net, which is not the forge's host novox\.internal/],
|
||||
[() => packument(`https://attacker.example.net/x?u=https://novox.internal${sdk}`), /query or a fragment/],
|
||||
[() => packument(`https://novox.internal${sdk}?x=1`), /query or a fragment/],
|
||||
[() => packument(`https://novox.internal${sdk}#frag`), /query or a fragment/],
|
||||
[() => packument(`https://attacker.example.net/#https://novox.internal${sdk}`), /query or a fragment/],
|
||||
[() => packument(`ftp://novox.internal${sdk}`), /ftp: — neither https: nor the forge's http:/],
|
||||
[() => packument(`file:///api/packages/novox/npm/x.tgz`), /file: — neither/],
|
||||
[() => packument(`https://novox.internal/api/packages/novox/npm/\nregistry=https://attacker.example.net/\nallow-remote=all`), /whitespace or a control character/],
|
||||
[() => packument(`https://user:pw@novox.internal${sdk}`), /user or password/],
|
||||
] as [() => Response, RegExp][]) {
|
||||
const got = await npmRegistry("http://novox.internal:20000", aRegistry(answer).get);
|
||||
assert.equal(got.registry, "http://novox.internal:20000/api/packages/novox/npm/", String(why));
|
||||
assert.match(got.problem ?? "", /did not give an address the lab takes/);
|
||||
assert.match(got.problem ?? "", why);
|
||||
}
|
||||
});
|
||||
|
||||
test("a run's npm configuration names the registry it was given, and the forge's own address only without one", () => {
|
||||
const given = script("/work", "run-x", "http://novox.internal:20000", ["t.test.ts"], {}, undefined,
|
||||
"https://forge.example.org/api/packages/novox/npm/");
|
||||
assert.match(given, / '@novox:registry=https:\/\/forge\.example\.org\/api\/packages\/novox\/npm\/' > '\/work\/run-x\/\.npmrc'\n/);
|
||||
assert.doesNotMatch(given, /@novox:registry=http:\/\/novox\.internal/);
|
||||
const none = script("/work", "run-x", "http://novox.internal:20000", ["t.test.ts"], {});
|
||||
assert.match(none, /'@novox:registry=http:\/\/novox\.internal:20000\/api\/packages\/novox\/npm\/'/);
|
||||
});
|
||||
@@ -4,13 +4,7 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
checkDir, cleanCredentials, clearChecks, credentialArgs, forgeCredential, installPlan, listRuns, npmRegistry, NPM_REPOSITORIES, readStatus, REPOSITORIES, running, start,
|
||||
stop, tail, writeCredential,
|
||||
} from "./runs.js";
|
||||
import { listRuns, readStatus, REPOSITORIES, running, start, stop, tail } from "./runs.js";
|
||||
|
||||
export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
|
||||
const work = env.MESH_LAB_WORK ?? "/var/lib/mesh-lab-runs";
|
||||
@@ -18,10 +12,6 @@ export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
|
||||
// ADR 0192): read from the env-file the mesh fills, at each call, so a changed setting is used
|
||||
// without restarting the runtime. MESH_LAB_FORGE itself still wins, for a hand-run instance.
|
||||
const forgeOf = (): string => (env.MESH_LAB_FORGE ?? wordIn(env.MESH_LAB_ENV_FILE, "MESH_LAB_FORGE")).replace(/\/+$/, "");
|
||||
// The forge credential the mesh binds and seals to the lab (novox/hq issue 624), read at each call.
|
||||
const credentialOf = (forge: string): string | undefined => forgeCredential(forge,
|
||||
env.MESH_LAB_FORGE_BINDING ?? wordIn(env.MESH_LAB_ENV_FILE, "MESH_LAB_FORGE_BINDING"),
|
||||
env.MESH_LAB_FORGE_SECRET ?? wordIn(env.MESH_LAB_ENV_FILE, "MESH_LAB_FORGE_SECRET"));
|
||||
return [
|
||||
{
|
||||
name: "lab_check",
|
||||
@@ -30,34 +20,13 @@ export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
|
||||
run: async () => {
|
||||
const forge = forgeOf();
|
||||
if (!forge) return { ok: false, output: "the lab's forge is not set: settings for lab, {\"forge\": \"<url>\"}" };
|
||||
// What earlier checks left is cleared, and what could not be is said; this check works in a fresh
|
||||
// directory of its own either way, and removes it when it is done.
|
||||
const leftovers = clearChecks(work);
|
||||
const dir = checkDir(work);
|
||||
let check: ReturnType<typeof spawnSync>;
|
||||
let problems: string[];
|
||||
try {
|
||||
const clone = spawnSync("git", ["clone", "--quiet", "--depth", "1", `${forge}/novox/mesh-lab.git`, dir], { encoding: "utf8" });
|
||||
if (clone.status !== 0) return { ok: false, output: [String(clone.stderr).trim(), ...leftovers].join("\n") };
|
||||
spawnSync("npm", ["ci", "--no-audit", "--no-fund", "--loglevel=error"], { cwd: dir, encoding: "utf8" });
|
||||
check = spawnSync("node", ["--experimental-strip-types", "src/cli.ts", "check"], { cwd: dir, encoding: "utf8" });
|
||||
problems = setupProblems(forge, `${dir}/setup`, credentialOf(forge));
|
||||
// And the address a run names the package registry by (npmRegistry): said when the registry does not say it.
|
||||
const registry = await npmRegistry(forge);
|
||||
if (registry.problem) problems.push(registry.problem);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
// **And what a run would check out and install, before a run finds out** (novox/hq issue 624): every
|
||||
// repository it clones answers on the forge, and every one it installs has a lock file where the run
|
||||
// installs from.
|
||||
// A credential a stopped, crashed or rebooted run left behind is removed, and said; one still in use
|
||||
// by a running run is not.
|
||||
const busy = running(work);
|
||||
const left = cleanCredentials(work, busy ? `${work}/.${busy.id}.git-credentials` : undefined);
|
||||
const removed = left.length > 0 ? `removed ${left.length} forge credential(s) a run left behind: ${left.join(", ")}` : "";
|
||||
const output = [`${check.stdout}${check.stderr}`.trim(), removed, ...leftovers, ...problems].filter(Boolean).join("\n");
|
||||
return { ok: check.status === 0 && problems.length === 0, output };
|
||||
const dir = `${work}/check`;
|
||||
spawnSync("rm", ["-rf", dir]);
|
||||
const clone = spawnSync("git", ["clone", "--quiet", "--depth", "1", `${forge}/novox/mesh-lab.git`, dir], { encoding: "utf8" });
|
||||
if (clone.status !== 0) return { ok: false, output: clone.stderr };
|
||||
spawnSync("npm", ["ci", "--no-audit", "--no-fund", "--loglevel=error"], { cwd: dir, encoding: "utf8" });
|
||||
const check = spawnSync("node", ["--experimental-strip-types", "src/cli.ts", "check"], { cwd: dir, encoding: "utf8" });
|
||||
return { ok: check.status === 0, output: `${check.stdout}${check.stderr}`.trim() };
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -90,9 +59,7 @@ export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
|
||||
if (stranger.length > 0) return { started: false, reason: `the lab does not build ${stranger.join(", ")}` };
|
||||
const busy = running(work);
|
||||
if (busy) return { started: false, reason: `${busy.id} is still ${busy.state}; one run at a time`, running: busy };
|
||||
const registry = await npmRegistry(forge);
|
||||
return { started: true, run: start(work, forge, tests, refs, credentialOf(forge), registry.registry),
|
||||
registry: registry.registry, ...(registry.problem ? { problem: registry.problem } : {}) };
|
||||
return { started: true, run: start(work, forge, tests, refs) };
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -122,52 +89,6 @@ export function getLabTools(env: NodeJS.ProcessEnv): ToolDefinition[] {
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* What would stop a run at its setup, in sentences: a repository the run clones that the forge does not
|
||||
* answer for, and a checkout it installs with nothing installable (installPlan). Empty when a run's setup
|
||||
* would go through.
|
||||
*/
|
||||
export function setupProblems(forge: string, dir: string, credential?: string): string[] {
|
||||
const problems: string[] = [];
|
||||
spawnSync("rm", ["-rf", dir]);
|
||||
// The credential a run would offer, in a file of its own outside the check's clones, removed after.
|
||||
const keep = mkdtempSync(join(tmpdir(), "lab-check-"));
|
||||
const file = writeCredential(join(keep, "git-credentials"), credential);
|
||||
const git = (args: string[]) =>
|
||||
spawnSync("sh", ["-c", `git ${credentialArgs(file)}${args.map((a) => `'${a.replace(/'/g, `'\\''`)}'`).join(" ")}`], {
|
||||
encoding: "utf8", env: { ...process.env, GIT_TERMINAL_PROMPT: "0" },
|
||||
});
|
||||
try {
|
||||
for (const repo of REPOSITORIES) {
|
||||
const url = `${forge}/novox/${repo}.git`;
|
||||
const remote = git(["ls-remote", "--heads", url, "main"]);
|
||||
if (remote.status !== 0 || !remote.stdout.includes("refs/heads/main")) {
|
||||
const said = remote.stderr.trim().split("\n")[0] ?? "";
|
||||
problems.push(`a run checks out ${repo}, and ${url} could not be read` +
|
||||
(said ? `: ${said}` : ": no main branch") +
|
||||
(credential ? "" : " — the lab holds no forge credential (its npm-package-registry binding and secret)"));
|
||||
continue;
|
||||
}
|
||||
if (!NPM_REPOSITORIES.includes(repo)) continue;
|
||||
const at = `${dir}/${repo}`;
|
||||
const clone = git(["clone", "--quiet", "--depth", "1", url, at]);
|
||||
if (clone.status !== 0) {
|
||||
problems.push(`${repo} could not be cloned from ${url}: ${clone.stderr.trim()}`);
|
||||
continue;
|
||||
}
|
||||
const plan = installPlan(at);
|
||||
for (const d of plan.unlocked) problems.push(`${repo}: ${d.slice(at.length) || "/"} has a package.json and no package-lock.json, so a run's npm ci cannot install it`);
|
||||
if (plan.install.length === 0 && plan.unlocked.length === 0) {
|
||||
problems.push(`${repo}: no package-lock.json at its root or one directory down, so a run installs nothing of it`);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
rmSync(keep, { recursive: true, force: true });
|
||||
spawnSync("rm", ["-rf", dir]);
|
||||
}
|
||||
return problems;
|
||||
}
|
||||
|
||||
/** One word from an env-file (`KEY=value` lines), or "" when the file or the word is absent. */
|
||||
export function wordIn(file: string | undefined, word: string): string {
|
||||
if (!file) return "";
|
||||
|
||||
+13
-248
@@ -7,236 +7,11 @@
|
||||
// for it, and says it was lost rather than pretending it is still going.
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
/**
|
||||
* The repositories a lab run checks out, side by side, as the lab expects its siblings. mesh-tools-go is
|
||||
* one since genesis builds the Go toolchain at its base step, from the repository beside mesh-tools
|
||||
* (novox/hq issue 616): a bed that raises genesis serves it from its forge.
|
||||
*/
|
||||
export const REPOSITORIES = ["mesh-lab", "mesh-controller", "mesh-host", "mesh-catalog", "mesh-tools", "mesh-tools-go", "mesh-sdk"];
|
||||
|
||||
/** The checkouts whose JavaScript dependencies a run installs: the lab, the SDK and the base. */
|
||||
export const NPM_REPOSITORIES = ["mesh-sdk", "mesh-tools", "mesh-lab"];
|
||||
|
||||
/**
|
||||
* Where a checkout's JavaScript dependencies are installed (novox/hq issue 624): its root when the root
|
||||
* holds a lock file, otherwise each directory one level down that holds one. `npm ci` installs only from a
|
||||
* lock file, and a repository that moved its package into a directory of its own (mesh-tools' node-tools)
|
||||
* has none at its root; the run asked for one there and failed before it built anything. A package.json
|
||||
* with no lock file beside it cannot be installed reproducibly, and is named rather than installed.
|
||||
*/
|
||||
export function installPlan(root: string): { install: string[]; unlocked: string[] } {
|
||||
const has = (dir: string, file: string) => existsSync(join(dir, file));
|
||||
if (has(root, "package-lock.json")) return { install: [root], unlocked: [] };
|
||||
const install: string[] = [];
|
||||
const unlocked: string[] = has(root, "package.json") ? [root] : [];
|
||||
for (const entry of readdirSync(root, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory() || entry.name.startsWith(".") || entry.name === "node_modules") continue;
|
||||
const dir = join(root, entry.name);
|
||||
if (has(dir, "package-lock.json")) install.push(dir);
|
||||
else if (has(dir, "package.json")) unlocked.push(dir);
|
||||
}
|
||||
return { install: install.sort(), unlocked: unlocked.sort() };
|
||||
}
|
||||
|
||||
/**
|
||||
* installStep is the shell a run installs one checkout with: the same rule as installPlan, so what
|
||||
* lab_check says a run would install is what it installs. Building runs where the package says how.
|
||||
*/
|
||||
export function installStep(root: string, build: boolean): string {
|
||||
const q = shellQuote(root);
|
||||
const install = `(cd "$d" && npm ci --no-audit --no-fund --loglevel=error${build ? " && npm run build --if-present" : ""})`;
|
||||
return `if [ -f ${q}/package-lock.json ]; then d=${q}; ${install}; else
|
||||
[ ! -f ${q}/package.json ] || { echo "${root}: a package.json and no package-lock.json, so npm ci cannot install it (novox/hq issue 624)" >&2; exit 1; }
|
||||
found=0
|
||||
for d in ${q}/*/; do
|
||||
d="\${d%/}"; [ "$(basename "$d")" = node_modules ] && continue
|
||||
if [ -f "$d/package-lock.json" ]; then found=1; ${install}
|
||||
elif [ -f "$d/package.json" ]; then echo "$d: a package.json and no package-lock.json, so npm ci cannot install it (novox/hq issue 624)" >&2; exit 1; fi
|
||||
done
|
||||
[ "$found" = 1 ] || { echo "${root}: no package-lock.json at its root or one directory down, so nothing of it can be installed (novox/hq issue 624)" >&2; exit 1; }
|
||||
fi`;
|
||||
}
|
||||
|
||||
/** Where the forge serves the mesh's npm packages, under its own address (gitea's npm registry for the novox org). */
|
||||
export const NPM_PATH = "/api/packages/novox/npm/";
|
||||
|
||||
/**
|
||||
* The address a run names the @novox registry by: the one the registry names itself by, when that is the
|
||||
* forge's own host (novox/hq issue 624).
|
||||
*
|
||||
* **npm 12 installs a locked tarball only from the registry it is configured with.** Its `allow-remote`
|
||||
* defaults to `none`, and a lock file's `resolved` URL counts as the registry's only when its origin and
|
||||
* path are the configured scope registry's; any other is a "remote" tarball and refused (EALLOWREMOTE).
|
||||
* The forge writes every tarball URL — in what it answers, and so in every lock file made against it — at
|
||||
* its own public address, whichever address it was asked at. So the lab asks the registry, at the forge,
|
||||
* for the SDK's newest tarball and names the registry by the origin of that URL.
|
||||
*
|
||||
* **What the registry answers is not trusted for where packages come from** (review of mesh-catalog #275):
|
||||
* it may be fetched over plain HTTP. The tarball URL is parsed, and taken only when it is https (or the
|
||||
* forge's own scheme), its path is under the npm path, it carries no query or fragment, and its host is
|
||||
* the forge's own host. The registry is then built from
|
||||
* the parsed origin and the npm path alone, never from the string answered, so nothing the registry says
|
||||
* reaches the run's npm configuration but a scheme, a host and a port. Anything else falls back to the
|
||||
* forge's own address and says why, so a run still starts and lab_check names it.
|
||||
*
|
||||
* So the lab's forge setting is the address the forge names itself by (its public https name), not a
|
||||
* second name for it: a forge set as another name of the same server is refused here, named by lab_check,
|
||||
* and its runs stop at the first locked @novox package.
|
||||
*/
|
||||
export async function npmRegistry(forge: string, get: typeof fetch = fetch): Promise<{ registry: string; problem?: string }> {
|
||||
const asked = `${forge.replace(/\/+$/, "")}${NPM_PATH}`;
|
||||
const url = `${asked}@novox%2fmesh-sdk`;
|
||||
const fallback = (why: string) => ({
|
||||
registry: asked,
|
||||
problem: `the package registry at ${url} did not give an address the lab takes (${why}), so a run names it ${asked}, and npm 12 refuses a locked @novox tarball at any other`,
|
||||
});
|
||||
let own: URL;
|
||||
try {
|
||||
own = new URL(forge);
|
||||
} catch {
|
||||
return fallback(`the forge ${JSON.stringify(forge)} is not a URL`);
|
||||
}
|
||||
try {
|
||||
const res = await get(url, { signal: AbortSignal.timeout(15_000) });
|
||||
if (!res.ok) return fallback(`it answered ${res.status}`);
|
||||
const doc = await res.json() as { "dist-tags"?: { latest?: string }; versions?: Record<string, { dist?: { tarball?: unknown } }> };
|
||||
const latest = doc["dist-tags"]?.latest ?? "";
|
||||
const tarball = doc.versions?.[latest]?.dist?.tarball;
|
||||
if (typeof tarball !== "string" || !tarball) return fallback("no tarball for its latest version");
|
||||
if (/[\s\u0000-\u001f\u007f]/.test(tarball)) return fallback("its tarball URL holds whitespace or a control character");
|
||||
let t: URL;
|
||||
try {
|
||||
t = new URL(tarball);
|
||||
} catch {
|
||||
return fallback("its tarball is not a URL");
|
||||
}
|
||||
if (t.protocol !== "https:" && t.protocol !== own.protocol) return fallback(`its tarball is ${t.protocol} — neither https: nor the forge's ${own.protocol}`);
|
||||
if (t.username || t.password) return fallback("its tarball URL carries a user or password");
|
||||
if (t.search || t.hash) return fallback("its tarball URL carries a query or a fragment");
|
||||
if (!t.pathname.startsWith(NPM_PATH)) return fallback(`its tarball is not under ${NPM_PATH}`);
|
||||
if (t.hostname !== own.hostname) return fallback(`its tarball is at ${t.hostname}, which is not the forge's host ${own.hostname}`);
|
||||
return { registry: `${t.origin}${NPM_PATH}` };
|
||||
} catch (e) {
|
||||
return fallback(e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The forge credential a run's clones may present, as a git credential-store line —
|
||||
* scheme://user:password@host[:port] — for the forge's own origin, or undefined when the lab holds none.
|
||||
*
|
||||
* **The mesh's credential for the forge, the way the build agent holds it** (novox/hq issue 624): the lab
|
||||
* requires the package registry, the forge's provisioner binds it an account (`as`) and seals that
|
||||
* account's password, and the forge that answers npm is the forge that hosts the repositories. Private
|
||||
* repositories (mesh-tools-go) are cloned with it; public ones clone as before. Written to a file git reads
|
||||
* through its credential store, never into a URL, an argument, the run's script or its log, and offered
|
||||
* only to the forge's own scheme, host and port.
|
||||
*/
|
||||
export function forgeCredential(forge: string, bindingFile: string, secretFile: string): string | undefined {
|
||||
if (!forge || !bindingFile || !secretFile) return undefined;
|
||||
let as = "";
|
||||
let secret = "";
|
||||
try {
|
||||
as = String((JSON.parse(readFileSync(bindingFile, "utf8")) as { as?: string }).as ?? "");
|
||||
secret = readFileSync(secretFile, "utf8").trim();
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
if (!as || !secret) return undefined;
|
||||
let origin: URL;
|
||||
try {
|
||||
origin = new URL(forge);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
const line = new URL(`${origin.protocol}//${origin.host}`);
|
||||
line.username = as;
|
||||
line.password = secret;
|
||||
return line.toString().replace(/\/$/, "");
|
||||
}
|
||||
|
||||
/** Where a run keeps its forge credential: beside its directory, never in it, and removed when it ends. */
|
||||
export function credentialPath(work: string, id: string): string {
|
||||
return join(work, `.${id}.git-credentials`);
|
||||
}
|
||||
|
||||
/** writeCredential puts a credential-store line where git reads it, readable by this account alone. */
|
||||
export function writeCredential(path: string, line: string | undefined): string | undefined {
|
||||
if (!line) return undefined;
|
||||
writeFileSync(path, `${line}\n`, { mode: 0o600 });
|
||||
return path;
|
||||
}
|
||||
|
||||
/**
|
||||
* credentialGuard is the shell that removes the credential however the script ends — its end, an error,
|
||||
* or a stop's SIGTERM to the run's process group, which kills the wrapper before its own cleanup runs
|
||||
* (review 196 on mesh-catalog #272). The script also removes it as soon as its clones are done.
|
||||
*/
|
||||
export function credentialGuard(path: string | undefined): string {
|
||||
if (!path) return "";
|
||||
const rm = `rm -f ${shellQuote(path)}`;
|
||||
return `trap ${shellQuote(rm)} EXIT
|
||||
trap ${shellQuote(`${rm}; exit 143`)} TERM
|
||||
trap ${shellQuote(`${rm}; exit 130`)} INT
|
||||
trap ${shellQuote(`${rm}; exit 129`)} HUP`;
|
||||
}
|
||||
|
||||
/**
|
||||
* cleanCredentials removes every run's credential file left in the work directory but the one named — what
|
||||
* a crash or a reboot left behind, which no trap could remove. Answers what it removed.
|
||||
*/
|
||||
export function cleanCredentials(work: string, keep?: string): string[] {
|
||||
if (!existsSync(work)) return [];
|
||||
const removed: string[] = [];
|
||||
for (const name of readdirSync(work)) {
|
||||
if (!/^\.run-.*\.git-credentials$/.test(name)) continue;
|
||||
const path = join(work, name);
|
||||
if (path === keep) continue;
|
||||
rmSync(path, { force: true });
|
||||
removed.push(path);
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
|
||||
/**
|
||||
* checkDir is a directory of lab_check's own for one check, fresh each time (novox/hq issue 624). The check
|
||||
* cloned into the one fixed path `<work>/check`, removing it first and ignoring whether that worked; a tree
|
||||
* an earlier check left that this account could not remove — one cloned as another account before the
|
||||
* work directory was the operator's — made every check fail on "destination path already exists".
|
||||
*/
|
||||
export function checkDir(work: string): string {
|
||||
mkdirSync(work, { recursive: true });
|
||||
return mkdtempSync(join(work, "check-"));
|
||||
}
|
||||
|
||||
/**
|
||||
* clearChecks removes what earlier checks left in the work directory — the fixed `check` and `check-setup`
|
||||
* of before, and any `check-*` — and says, for each it could not remove, which and why, so a check reports
|
||||
* it rather than tripping over it.
|
||||
*/
|
||||
export function clearChecks(work: string): string[] {
|
||||
if (!existsSync(work)) return [];
|
||||
const said: string[] = [];
|
||||
for (const name of readdirSync(work)) {
|
||||
if (name !== "check" && name !== "check-setup" && !name.startsWith("check-")) continue;
|
||||
const path = join(work, name);
|
||||
try {
|
||||
rmSync(path, { recursive: true, force: true });
|
||||
} catch (err) {
|
||||
said.push(`${path}, left by an earlier check, could not be removed: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
return said;
|
||||
}
|
||||
|
||||
/** The arguments that have git offer the credential in that file, and only that one. */
|
||||
export function credentialArgs(path: string | undefined): string {
|
||||
if (!path) return "";
|
||||
return `-c credential.helper= -c ${shellQuote(`credential.helper=store --file=${path}`)} `;
|
||||
}
|
||||
/** The repositories a lab run checks out, side by side, as the lab expects its siblings. */
|
||||
export const REPOSITORIES = ["mesh-lab", "mesh-controller", "mesh-host", "mesh-catalog", "mesh-tools", "mesh-sdk"];
|
||||
|
||||
export interface RunStatus {
|
||||
id: string;
|
||||
@@ -303,8 +78,7 @@ const shellQuote = (s: string) => `'${s.replace(/'/g, `'\\''`)}'`;
|
||||
* The environment is the one the lab's README describes for a run against sibling checkouts, pointed
|
||||
* at this run's own tree, so what is built and claimed is exactly what was checked out.
|
||||
*/
|
||||
export function script(work: string, id: string, forge: string, tests: string[], refs: Record<string, string>,
|
||||
credential?: string, registry = `${forge}${NPM_PATH}`): string {
|
||||
export function script(work: string, id: string, forge: string, tests: string[], refs: Record<string, string>): string {
|
||||
const dir = runDir(work, id);
|
||||
const setState = (state: string) =>
|
||||
`node -e ${shellQuote(
|
||||
@@ -313,28 +87,26 @@ export function script(work: string, id: string, forge: string, tests: string[],
|
||||
const clones = REPOSITORIES.map((repo) => {
|
||||
const ref = refs[repo] ?? "main";
|
||||
return [
|
||||
`git ${credentialArgs(credential)}clone --quiet --depth 50 --branch ${shellQuote(ref)} ${shellQuote(`${forge}/novox/${repo}.git`)} ${shellQuote(join(dir, repo))}`,
|
||||
`git clone --quiet --depth 50 --branch ${shellQuote(ref)} ${shellQuote(`${forge}/novox/${repo}.git`)} ${shellQuote(join(dir, repo))}`,
|
||||
`echo "${repo} $(git -C ${shellQuote(join(dir, repo))} rev-parse HEAD)" >> ${shellQuote(join(dir, "commits.txt"))}`,
|
||||
].join("\n");
|
||||
}).join("\n");
|
||||
const bin = join(dir, "bin");
|
||||
return `set -euo pipefail
|
||||
# A repository the forge will not show fails its clone at once, rather than waiting on a prompt.
|
||||
export GIT_TERMINAL_PROMPT=0
|
||||
${credentialGuard(credential)}
|
||||
cd ${shellQuote(dir)}
|
||||
${setState("checking-out")}
|
||||
${clones}
|
||||
${credential ? `# The forge credential is for the clones, and nothing after them runs with it on disk.\nrm -f ${shellQuote(credential)}` : ""}
|
||||
node -e ${shellQuote(
|
||||
`const fs=require("fs");const f=${JSON.stringify(join(dir, "status.json"))};const s=JSON.parse(fs.readFileSync(f,"utf8"));s.commits=Object.fromEntries(fs.readFileSync(${JSON.stringify(join(dir, "commits.txt"))},"utf8").trim().split("\\n").map(l=>l.split(" ")));fs.writeFileSync(f,JSON.stringify(s,null,2))`,
|
||||
)}
|
||||
${setState("building")}
|
||||
# The @novox scope resolves from the mesh's own package registry on the forge, named by the address it
|
||||
# names itself by (npmRegistry), which is the address its lock files hold; nothing else is asked of it.
|
||||
printf '%s\n' ${shellQuote(`@novox:registry=${registry}`)} > ${shellQuote(join(dir, ".npmrc"))}
|
||||
# The @novox scope resolves from the mesh's own package registry on the forge, as the builder
|
||||
# resolves it; nothing else is asked of it.
|
||||
printf '%s\n' ${shellQuote(`@novox:registry=${forge}/api/packages/novox/npm/`)} > ${shellQuote(join(dir, ".npmrc"))}
|
||||
export NPM_CONFIG_USERCONFIG=${shellQuote(join(dir, ".npmrc"))}
|
||||
${NPM_REPOSITORIES.map((repo) => installStep(join(dir, repo), repo !== "mesh-lab")).join("\n")}
|
||||
for repo in mesh-sdk mesh-tools mesh-lab; do (cd ${shellQuote(dir)}/$repo && npm ci --no-audit --no-fund --loglevel=error); done
|
||||
(cd ${shellQuote(dir)}/mesh-sdk && npm run build --if-present)
|
||||
(cd ${shellQuote(dir)}/mesh-tools && npm run build --if-present)
|
||||
mkdir -p ${shellQuote(bin)}
|
||||
for p in postgres-provisioner objectstore-provisioner route-proxy; do
|
||||
(cd ${shellQuote(dir)}/mesh-controller && CGO_ENABLED=0 go build -o ${shellQuote(bin)}/$p ./examples/$p)
|
||||
@@ -345,8 +117,6 @@ export MESH_LAB_MODULES=${shellQuote(join(dir, "mesh-controller", "examples", "m
|
||||
export MESH_LAB_BUILDER=${shellQuote(join(dir, "mesh-controller", "build", "mesh-builder"))}
|
||||
export MESH_LAB_BOOTSTRAP_BINARY=${shellQuote(join(dir, "mesh-host", "mesh-bootstrap"))}
|
||||
export MESH_LAB_CATALOG=${shellQuote(join(dir, "mesh-catalog", "modules"))}
|
||||
# Where a bed that serves repositories from its own forge finds them: this run's checkouts, side by side.
|
||||
export MESH_LAB_REPOS=${shellQuote(dir)}
|
||||
export MESH_LAB_PROVISIONER=${shellQuote(join(bin, "postgres-provisioner"))}
|
||||
export MESH_LAB_OBJECTSTORE_PROVISIONER=${shellQuote(join(bin, "objectstore-provisioner"))}
|
||||
export MESH_LAB_ROUTE_PROXY=${shellQuote(join(bin, "route-proxy"))}
|
||||
@@ -357,17 +127,13 @@ node --experimental-strip-types src/cli.ts suite ${tests.map(shellQuote).join("
|
||||
}
|
||||
|
||||
/** start begins a run and returns at once with its status. */
|
||||
export function start(work: string, forge: string, tests: string[], refs: Record<string, string>,
|
||||
credential?: string, registry?: string): RunStatus {
|
||||
export function start(work: string, forge: string, tests: string[], refs: Record<string, string>): RunStatus {
|
||||
const id = `run-${new Date().toISOString().replace(/[:.]/g, "-")}`;
|
||||
const dir = runDir(work, id);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
const status: RunStatus = { id, state: "checking-out", started: new Date().toISOString(), tests, refs };
|
||||
writeFileSync(statusPath(work, id), JSON.stringify(status, null, 2));
|
||||
// What a crashed or rebooted run left behind goes before this run writes its own.
|
||||
cleanCredentials(work);
|
||||
const credentials = writeCredential(credentialPath(work, id), credential);
|
||||
writeFileSync(join(dir, "run.sh"), script(work, id, forge, tests, refs, credentials, registry), { mode: 0o700 });
|
||||
writeFileSync(join(dir, "run.sh"), script(work, id, forge, tests, refs), { mode: 0o700 });
|
||||
|
||||
// The wrapper records how the run ended, then removes the checkouts and keeps the log and status: a
|
||||
// run's tree is its own, and the next run starts from fresh ones (novox/hq ADR 0172).
|
||||
@@ -375,7 +141,6 @@ export function start(work: string, forge: string, tests: string[], refs: Record
|
||||
node -e ${shellQuote(
|
||||
`const f=${JSON.stringify(statusPath(work, id))};const s=JSON.parse(require("fs").readFileSync(f,"utf8"));if(s.state!=="stopped"){s.state=process.argv[1]==="0"?"passed":"failed"};s.exit=Number(process.argv[1]);s.ended=new Date().toISOString();require("fs").writeFileSync(f,JSON.stringify(s,null,2))`,
|
||||
)} "$code"
|
||||
rm -f ${shellQuote(credentialPath(work, id))}
|
||||
cd ${shellQuote(dir)} && rm -rf ${REPOSITORIES.map(shellQuote).join(" ")} bin`;
|
||||
const child = spawn("bash", ["-c", wrapper], { detached: true, stdio: "ignore" });
|
||||
child.unref();
|
||||
|
||||
@@ -17,7 +17,20 @@ another node requires.
|
||||
(`/etc/nfs-server/machines`, rendered by the mesh), never one a consumer gives, and it must lie inside
|
||||
the private network's range. The control node, or any node, gets a share only when both name it. A
|
||||
want that is not granted, and a grant nobody wants, are said by `exports`. Port 2049 is opened to the
|
||||
mesh only; the home network is never named.
|
||||
mesh by default. The machine's local network is reached only by a grant to `local-network` (below).
|
||||
- **The local network, only when granted and admitted** (novox/hq ADR 0341, after ADR 0340). `grants` may
|
||||
name `local-network` in a node's place: `library=ws:rw,local-network:ro`. The process then reads the
|
||||
machine's links (`ip -j`) and takes the subnets on the links carrying its default route that lie wholly
|
||||
in a private range (RFC 1918, fc00::/7); a public subnet, the private network's link and container
|
||||
bridges are never counted, and no range is ever typed in a setting. It exports to such a subnet only
|
||||
when the mesh's packet filter (table `inet mesh`, chain `input`, read with `nft -j list ruleset`)
|
||||
admits 2049/tcp from all of it, which it does once the assignment binds 2049 with the reach
|
||||
`local-network`. A grant the filter does not admit is not exported, said by `exports`, and makes the
|
||||
module unhealthy, naming that act. Every device on the local network then acts as the folder's owner,
|
||||
as the hand-written export it replaces already allowed.
|
||||
- **A dataset's `sharenfs` is cleared only when nobody loses the folder.** `adopt` refuses while the
|
||||
property, by its value or the file ZFS writes for it, exports to a client the mesh's live export does
|
||||
not cover: for a property to the home network, the module's `local-network` export.
|
||||
- **Every client is the folder's owner.** NFS with `sec=sys` believes the user id a client sends, and one
|
||||
id names different accounts on different machines. Every line is `all_squash` with `anonuid` and
|
||||
`anongid` read from the folder's owner on this machine, so a file written from any node lands owned by
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
|
||||
func TestStrayFindsAnExportOfTheModulesThatNoFileCarries(t *testing.T) {
|
||||
live := []Entry{{Path: "/srv/library", Client: "10.0.0.4"}}
|
||||
if got := Stray(live, nil, nil, meshRange); len(got) != 1 {
|
||||
if got := Stray(live, nil, nil, meshRange, nil); len(got) != 1 {
|
||||
t.Errorf("an export inside the mesh range that no file carries is the module's leftover: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -21,7 +21,7 @@ func TestStrayLeavesAnExportAnotherFileCarriesEvenInsideTheMeshRange(t *testing.
|
||||
// ZFS's own file exports the library to a client inside the mesh's range.
|
||||
foreign := ParseExports("/srv/library 10.0.0.9(rw,no_subtree_check)\n", "/etc/exports.d/zfs.exports")
|
||||
live := []Entry{{Path: "/srv/library", Client: "10.0.0.9", Options: "rw"}}
|
||||
if got := Stray(live, nil, foreign, meshRange); len(got) != 0 {
|
||||
if got := Stray(live, nil, foreign, meshRange, nil); len(got) != 0 {
|
||||
t.Errorf("what ZFS's file exports is not the module's, whatever its client: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -30,7 +30,7 @@ func TestStrayLeavesAnExportAnotherFileCarriesEvenInsideTheMeshRange(t *testing.
|
||||
func TestStrayLeavesAClientOutsideTheMeshRange(t *testing.T) {
|
||||
live := []Entry{{Path: "/srv/library", Client: "192.0.2.0/24"}, {Path: "/srv/library", Client: "192.0.2.7"},
|
||||
{Path: "/srv/library", Client: "10.0.0.0/16"}}
|
||||
if got := Stray(live, nil, nil, meshRange); len(got) != 0 {
|
||||
if got := Stray(live, nil, nil, meshRange, nil); len(got) != 0 {
|
||||
t.Errorf("a client the module never exports to (outside or wider than the mesh's range) is not its leftover: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,417 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// A share granted to the machine's local network (novox/hq ADR 0341 clauses 8 and 9, after ADR 0340).
|
||||
//
|
||||
// **The grant names `local-network`, never a range.** `grants` may name `local-network` for a share beside
|
||||
// node names: `library=ws:rw,local-network:ro`. The export goes to the subnets this machine's own local
|
||||
// network is, by ADR 0340's rules: the networks on the links that carry its default route (the uplink),
|
||||
// each wholly inside a private range (RFC 1918, or fc00::/7), never a public or link-local one, never the
|
||||
// private network's own link or a container bridge. A range typed in a setting is never used.
|
||||
//
|
||||
// **A grant the filter does not admit is not exported, and is said.** ADR 0340 lets an assignment bind a
|
||||
// port with the reach `local-network`, and the filter then admits it from those subnets on that port's
|
||||
// rule alone. The module does not take the reach on trust: every pass it reads the mesh's filter (the
|
||||
// table `inet mesh`, chain `input`) and exports to a local subnet only when 2049/tcp is admitted from all
|
||||
// of it. Otherwise the share is not exported there, and the status and the health say so, naming the act
|
||||
// (bind 2049 with the reach local-network) — a grant never reads as served while the filter drops it.
|
||||
//
|
||||
// **What this does not promise.** NFS with sec=sys trusts the client, so every device on the local
|
||||
// network acts as the share's owner (all_squash, as for a node). That is what the hand-written export it
|
||||
// replaces already allowed.
|
||||
|
||||
// LocalNetwork is the name a grant gives the machine's local network.
|
||||
const LocalNetwork = "local-network"
|
||||
|
||||
// NFSPort is the port the filter must admit from a local subnet before a share is exported to it.
|
||||
const NFSPort = 2049
|
||||
|
||||
// private are the ranges a local network must lie wholly inside (ADR 0340 §2).
|
||||
var private = []netip.Prefix{
|
||||
netip.MustParsePrefix("10.0.0.0/8"),
|
||||
netip.MustParsePrefix("172.16.0.0/12"),
|
||||
netip.MustParsePrefix("192.168.0.0/16"),
|
||||
netip.MustParsePrefix("fc00::/7"),
|
||||
}
|
||||
|
||||
// wholly says whether a network lies wholly inside one of the private ranges.
|
||||
func wholly(p netip.Prefix) bool {
|
||||
for _, r := range private {
|
||||
if p.Bits() >= r.Bits() && r.Contains(p.Addr()) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ipRoute is one route as `ip -j route show default` answers it.
|
||||
type ipRoute struct {
|
||||
Dev string `json:"dev"`
|
||||
}
|
||||
|
||||
// ipLink is one link as `ip -j addr show` answers it.
|
||||
type ipLink struct {
|
||||
Name string `json:"ifname"`
|
||||
AddrInfo []struct {
|
||||
Family string `json:"family"`
|
||||
Local string `json:"local"`
|
||||
Prefixlen int `json:"prefixlen"`
|
||||
Scope string `json:"scope"`
|
||||
} `json:"addr_info"`
|
||||
}
|
||||
|
||||
// LocalNetworksFrom is the machine's local networks from what `ip` answered: the links carrying a default
|
||||
// route, and their addresses. A network not wholly private, or not global in scope, is left out, and why.
|
||||
func LocalNetworksFrom(routes4, routes6, addrs string) ([]string, []string) {
|
||||
uplinks := map[string]bool{}
|
||||
for _, text := range []string{routes4, routes6} {
|
||||
var rs []ipRoute
|
||||
if strings.TrimSpace(text) == "" {
|
||||
continue
|
||||
}
|
||||
if err := json.Unmarshal([]byte(text), &rs); err != nil {
|
||||
continue
|
||||
}
|
||||
for _, r := range rs {
|
||||
if r.Dev != "" {
|
||||
uplinks[r.Dev] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
var links []ipLink
|
||||
if strings.TrimSpace(addrs) != "" {
|
||||
_ = json.Unmarshal([]byte(addrs), &links)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
var out, left []string
|
||||
for _, l := range links {
|
||||
if !uplinks[l.Name] {
|
||||
continue
|
||||
}
|
||||
for _, a := range l.AddrInfo {
|
||||
if a.Scope != "global" {
|
||||
continue
|
||||
}
|
||||
addr, err := netip.ParseAddr(a.Local)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
p, err := addr.Prefix(a.Prefixlen)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if !wholly(p) {
|
||||
left = append(left, fmt.Sprintf("%s on %s is not wholly inside a private range, so it is not the local network", p, l.Name))
|
||||
continue
|
||||
}
|
||||
if !seen[p.String()] {
|
||||
seen[p.String()] = true
|
||||
out = append(out, p.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, left
|
||||
}
|
||||
|
||||
// LocalNetworks is this machine's local networks now, read with `ip`, and what was left out or could not
|
||||
// be read.
|
||||
func (m *Machine) LocalNetworks(ctx context.Context) ([]string, []string) {
|
||||
r4 := m.Run(ctx, "ip", "-j", "-4", "route", "show", "default")
|
||||
r6 := m.Run(ctx, "ip", "-j", "-6", "route", "show", "default")
|
||||
ad := m.Run(ctx, "ip", "-j", "addr", "show")
|
||||
var notes []string
|
||||
if r4.Code != 0 || ad.Code != 0 {
|
||||
notes = append(notes, "this machine's links could not be read with ip: "+firstLine(r4.Err+" "+ad.Err))
|
||||
}
|
||||
nets, left := LocalNetworksFrom(r4.Out, r6.Out, ad.Out)
|
||||
return nets, append(notes, left...)
|
||||
}
|
||||
|
||||
// nftRule is one rule of `nft -j list ruleset`.
|
||||
type nftRule struct {
|
||||
Family string `json:"family"`
|
||||
Table string `json:"table"`
|
||||
Chain string `json:"chain"`
|
||||
Expr []json.RawMessage `json:"expr"`
|
||||
}
|
||||
|
||||
// nftChain is one chain of `nft -j list ruleset`.
|
||||
type nftChain struct {
|
||||
Family string `json:"family"`
|
||||
Table string `json:"table"`
|
||||
Name string `json:"name"`
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
|
||||
// Admitted says, for each subnet, whether the mesh's filter (table inet mesh, chain input) accepts tcp to
|
||||
// the port from all of it, read from `nft -j list ruleset`. A rule counts only when every condition it has
|
||||
// is one this reads — the protocol, the destination port and the source address — and its verdict is
|
||||
// accept; a rule with any other condition (an interface, a connection state) is left out, so nothing is
|
||||
// called admitted that may not be. With no such table, nothing is admitted, and why.
|
||||
func Admitted(ruleset string, port int, subnets []string) (map[string]bool, string) {
|
||||
out := map[string]bool{}
|
||||
var doc struct {
|
||||
Nftables []map[string]json.RawMessage `json:"nftables"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(ruleset), &doc); err != nil {
|
||||
return out, "the packet filter's rules could not be read (nft -j list ruleset): " + err.Error()
|
||||
}
|
||||
var rules []nftRule
|
||||
found := false
|
||||
policy := ""
|
||||
for _, item := range doc.Nftables {
|
||||
if raw, ok := item["chain"]; ok {
|
||||
var c nftChain
|
||||
if json.Unmarshal(raw, &c) == nil && c.Family == "inet" && c.Table == "mesh" && c.Name == "input" {
|
||||
found, policy = true, c.Policy
|
||||
}
|
||||
}
|
||||
if raw, ok := item["rule"]; ok {
|
||||
var r nftRule
|
||||
if json.Unmarshal(raw, &r) == nil && r.Family == "inet" && r.Table == "mesh" && r.Chain == "input" {
|
||||
rules = append(rules, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return out, "the mesh's packet filter (table inet mesh, chain input) is not there, so no local network is admitted"
|
||||
}
|
||||
if policy == "accept" {
|
||||
for _, s := range subnets {
|
||||
out[s] = true
|
||||
}
|
||||
return out, ""
|
||||
}
|
||||
for _, r := range rules {
|
||||
sources, ok := admitsPort(r.Expr, port)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for _, s := range subnets {
|
||||
p, err := netip.ParsePrefix(s)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if sources == nil {
|
||||
out[s] = true
|
||||
continue
|
||||
}
|
||||
for _, src := range sources {
|
||||
if src.Bits() <= p.Bits() && src.Contains(p.Addr()) {
|
||||
out[s] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out, ""
|
||||
}
|
||||
|
||||
// admitsPort reads one rule: whether it accepts tcp to the port, and from which sources (nil: from
|
||||
// anywhere). ok is false for a rule that does not accept that port, or that has a condition this does not
|
||||
// read.
|
||||
func admitsPort(exprs []json.RawMessage, port int) (sources []netip.Prefix, ok bool) {
|
||||
accepts, portMatched, anywhere := false, false, true
|
||||
for _, raw := range exprs {
|
||||
var e map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &e) != nil {
|
||||
return nil, false
|
||||
}
|
||||
if _, is := e["accept"]; is {
|
||||
accepts = true
|
||||
continue
|
||||
}
|
||||
if _, is := e["counter"]; is {
|
||||
continue
|
||||
}
|
||||
m, is := e["match"]
|
||||
if !is {
|
||||
return nil, false
|
||||
}
|
||||
var match struct {
|
||||
Op string `json:"op"`
|
||||
Left json.RawMessage `json:"left"`
|
||||
Right json.RawMessage `json:"right"`
|
||||
}
|
||||
if json.Unmarshal(m, &match) != nil || (match.Op != "==" && match.Op != "in") {
|
||||
return nil, false
|
||||
}
|
||||
var left struct {
|
||||
Payload *struct {
|
||||
Protocol string `json:"protocol"`
|
||||
Field string `json:"field"`
|
||||
} `json:"payload"`
|
||||
Meta *struct {
|
||||
Key string `json:"key"`
|
||||
} `json:"meta"`
|
||||
}
|
||||
if json.Unmarshal(match.Left, &left) != nil {
|
||||
return nil, false
|
||||
}
|
||||
switch {
|
||||
case left.Meta != nil && left.Meta.Key == "l4proto":
|
||||
var proto string
|
||||
if json.Unmarshal(match.Right, &proto) != nil || proto != "tcp" {
|
||||
return nil, false
|
||||
}
|
||||
case left.Payload != nil && left.Payload.Protocol == "tcp" && left.Payload.Field == "dport":
|
||||
if !hasPort(match.Right, port) {
|
||||
return nil, false
|
||||
}
|
||||
portMatched = true
|
||||
case left.Payload != nil && (left.Payload.Protocol == "ip" || left.Payload.Protocol == "ip6") && left.Payload.Field == "saddr":
|
||||
anywhere = false
|
||||
sources = append(sources, prefixesOf(match.Right)...)
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
if !accepts || !portMatched {
|
||||
return nil, false
|
||||
}
|
||||
if anywhere {
|
||||
return nil, true
|
||||
}
|
||||
if sources == nil {
|
||||
sources = []netip.Prefix{}
|
||||
}
|
||||
return sources, true
|
||||
}
|
||||
|
||||
// hasPort reads a match's right side: a number, or a set holding it.
|
||||
func hasPort(raw json.RawMessage, port int) bool {
|
||||
var n int
|
||||
if json.Unmarshal(raw, &n) == nil {
|
||||
return n == port
|
||||
}
|
||||
var set struct {
|
||||
Set []json.RawMessage `json:"set"`
|
||||
}
|
||||
if json.Unmarshal(raw, &set) == nil {
|
||||
for _, item := range set.Set {
|
||||
if json.Unmarshal(item, &n) == nil && n == port {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// prefixesOf reads a match's right side as sources: an address, a prefix, or a set of either.
|
||||
func prefixesOf(raw json.RawMessage) []netip.Prefix {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
if a, err := netip.ParseAddr(s); err == nil {
|
||||
return []netip.Prefix{netip.PrefixFrom(a, a.BitLen())}
|
||||
}
|
||||
if p, err := netip.ParsePrefix(s); err == nil {
|
||||
return []netip.Prefix{p.Masked()}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var pre struct {
|
||||
Prefix *struct {
|
||||
Addr string `json:"addr"`
|
||||
Len int `json:"len"`
|
||||
} `json:"prefix"`
|
||||
Set []json.RawMessage `json:"set"`
|
||||
}
|
||||
if json.Unmarshal(raw, &pre) != nil {
|
||||
return nil
|
||||
}
|
||||
if pre.Prefix != nil {
|
||||
if a, err := netip.ParseAddr(pre.Prefix.Addr); err == nil {
|
||||
if p, err := a.Prefix(pre.Prefix.Len); err == nil {
|
||||
return []netip.Prefix{p}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var out []netip.Prefix
|
||||
for _, item := range pre.Set {
|
||||
out = append(out, prefixesOf(item)...)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// LocalNetworkAdmitted is what the process learns each pass: the machine's local subnets, which of them the
|
||||
// filter admits 2049 from, and what it could not learn.
|
||||
func (m *Machine) LocalNetworkAdmitted(ctx context.Context) (nets []string, admitted map[string]bool, notes []string) {
|
||||
nets, notes = m.LocalNetworks(ctx)
|
||||
if len(nets) == 0 {
|
||||
return nets, map[string]bool{}, notes
|
||||
}
|
||||
r := m.Run(ctx, "nft", "-j", "list", "ruleset")
|
||||
if r.Code != 0 {
|
||||
return nets, map[string]bool{}, append(notes, "the packet filter's rules could not be read (nft -j list ruleset): "+firstLine(r.Err))
|
||||
}
|
||||
admitted, note := Admitted(r.Out, NFSPort, nets)
|
||||
if note != "" {
|
||||
notes = append(notes, note)
|
||||
}
|
||||
return nets, admitted, notes
|
||||
}
|
||||
|
||||
// Covered says whether a client a hand-made export names is covered by the mesh's live export of the same
|
||||
// path: the same address, or a network inside one the mesh exports to. "*" and a name are never covered.
|
||||
func Covered(client string, mine []Grantee) bool {
|
||||
var want netip.Prefix
|
||||
if a, err := netip.ParseAddr(client); err == nil {
|
||||
want = netip.PrefixFrom(a, a.BitLen())
|
||||
} else if p, err := netip.ParsePrefix(client); err == nil {
|
||||
want = p.Masked()
|
||||
} else {
|
||||
return false
|
||||
}
|
||||
for _, g := range mine {
|
||||
var have netip.Prefix
|
||||
if a, err := netip.ParseAddr(g.Address); err == nil {
|
||||
have = netip.PrefixFrom(a, a.BitLen())
|
||||
} else if p, err := netip.ParsePrefix(g.Address); err == nil {
|
||||
have = p.Masked()
|
||||
} else {
|
||||
continue
|
||||
}
|
||||
if have.Bits() <= want.Bits() && have.Contains(want.Addr()) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// PropertyClients is who a dataset's sharenfs property exports to: every `@network` or address its value
|
||||
// names, and "*" for a bare `on` or a value naming nobody, which nothing the mesh exports ever covers.
|
||||
func PropertyClients(sharenfs string) []string {
|
||||
v := strings.TrimSpace(sharenfs)
|
||||
if v == "on" || v == "" {
|
||||
return []string{"*"}
|
||||
}
|
||||
var out []string
|
||||
for _, opt := range strings.Split(v, ",") {
|
||||
key, list, ok := strings.Cut(strings.TrimSpace(opt), "=")
|
||||
if key != "rw" && key != "ro" {
|
||||
continue
|
||||
}
|
||||
if !ok {
|
||||
return []string{"*"} // rw or ro alone: to everyone
|
||||
}
|
||||
for _, c := range strings.Split(list, ":") {
|
||||
c = strings.TrimPrefix(strings.TrimSpace(c), "@")
|
||||
if c != "" {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return []string{"*"}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Defends novox/hq ADR 0341 clauses 8 to 10: a share granted to local-network is exported to the machine's
|
||||
// own private subnets on its uplink, only those the mesh's filter admits 2049 from; a grant the filter does
|
||||
// not admit is said and unhealthy; and sharenfs is cleared only once the mesh's export covers every client
|
||||
// the property exported to.
|
||||
|
||||
const homeNet = "192.168.1.0/24"
|
||||
|
||||
// The home server's links, as `ip -j` answers them: the uplink with a private and a public address, the
|
||||
// private network's link, and a container bridge.
|
||||
const (
|
||||
routes4 = `[{"dst":"default","gateway":"192.168.1.1","dev":"enp6s0"}]`
|
||||
addrs = `[
|
||||
{"ifname":"lo","addr_info":[{"family":"inet","local":"127.0.0.1","prefixlen":8,"scope":"host"}]},
|
||||
{"ifname":"enp6s0","addr_info":[
|
||||
{"family":"inet","local":"192.168.1.135","prefixlen":24,"scope":"global"},
|
||||
{"family":"inet","local":"198.51.100.7","prefixlen":24,"scope":"global"},
|
||||
{"family":"inet6","local":"fe80::1","prefixlen":64,"scope":"link"}]},
|
||||
{"ifname":"mesh0","addr_info":[{"family":"inet","local":"10.0.0.2","prefixlen":32,"scope":"global"}]},
|
||||
{"ifname":"docker0","addr_info":[{"family":"inet","local":"172.17.0.1","prefixlen":16,"scope":"global"}]}]`
|
||||
)
|
||||
|
||||
// rulesetAdmitting is the mesh's filter with 2049 admitted from the nodes and, when local is set, from it.
|
||||
func rulesetAdmitting(local string) string {
|
||||
src := `{"set":["10.0.0.1","10.0.0.3"]}`
|
||||
if local != "" {
|
||||
addr, bits, _ := strings.Cut(local, "/")
|
||||
src = `{"set":["10.0.0.1","10.0.0.3",{"prefix":{"addr":"` + addr + `","len":` + bits + `}}]}`
|
||||
}
|
||||
return `{"nftables":[{"metainfo":{"version":"1.1"}},
|
||||
{"table":{"family":"inet","name":"mesh"}},
|
||||
{"chain":{"family":"inet","table":"mesh","name":"input","type":"filter","hook":"input","prio":0,"policy":"drop"}},
|
||||
{"rule":{"family":"inet","table":"mesh","chain":"input","expr":[{"match":{"op":"!=","left":{"meta":{"key":"iifname"}},"right":{"set":["enp6s0","mesh0"]}}},{"accept":null}]}},
|
||||
{"rule":{"family":"inet","table":"mesh","chain":"input","expr":[{"match":{"op":"==","left":{"payload":{"protocol":"ip","field":"saddr"}},"right":` + src + `}},{"match":{"op":"==","left":{"payload":{"protocol":"tcp","field":"dport"}},"right":2049}},{"accept":null}]}},
|
||||
{"rule":{"family":"inet","table":"mesh","chain":"input","expr":[{"match":{"op":"==","left":{"payload":{"protocol":"tcp","field":"dport"}},"right":445}},{"accept":null}]}}]}`
|
||||
}
|
||||
|
||||
func (f *fake) homeLinks(ruleset string) {
|
||||
f.answers["ip -j -4 route show default"] = Ran{Out: routes4}
|
||||
f.answers["ip -j -6 route show default"] = Ran{Out: "[]"}
|
||||
f.answers["ip -j addr show"] = Ran{Out: addrs}
|
||||
f.answers["nft -j list ruleset"] = Ran{Out: ruleset}
|
||||
}
|
||||
|
||||
func TestTheLocalNetworkIsTheUplinksPrivateSubnetsOnly(t *testing.T) {
|
||||
nets, left := LocalNetworksFrom(routes4, "", addrs)
|
||||
if !reflect.DeepEqual(nets, []string{homeNet}) {
|
||||
t.Errorf("local networks %v; the uplink's private subnet alone, never the public one, the mesh's link or a bridge", nets)
|
||||
}
|
||||
if len(left) != 1 || !strings.Contains(left[0], "198.51.100.0/24") {
|
||||
t.Errorf("the public subnet is not said as left out: %v", left)
|
||||
}
|
||||
if nets, _ := LocalNetworksFrom("", "", addrs); len(nets) != 0 {
|
||||
t.Errorf("with no default route there is no uplink and no local network: %v", nets)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdmittedReadsThePortsOwnRuleOnly(t *testing.T) {
|
||||
got, note := Admitted(rulesetAdmitting(homeNet), 2049, []string{homeNet})
|
||||
if note != "" || !got[homeNet] {
|
||||
t.Errorf("2049 admitted from the home network is not read as admitted: %v %q", got, note)
|
||||
}
|
||||
got, _ = Admitted(rulesetAdmitting(""), 2049, []string{homeNet})
|
||||
if got[homeNet] {
|
||||
t.Error("2049 admitted from the nodes only is read as admitted from the home network")
|
||||
}
|
||||
// A rule open to every source on 445 says nothing of 2049, and the interface rule is not read as one.
|
||||
if got, _ := Admitted(rulesetAdmitting(""), 445, []string{homeNet}); !got[homeNet] {
|
||||
t.Error("a port open from anywhere is not admitted")
|
||||
}
|
||||
if got, note := Admitted(`{"nftables":[]}`, 2049, []string{homeNet}); got[homeNet] || note == "" {
|
||||
t.Errorf("with no mesh filter a subnet is admitted, or nothing is said: %v %q", got, note)
|
||||
}
|
||||
}
|
||||
|
||||
// grantedLocal is the library granted to ws and to the local network, ws wanting it.
|
||||
func grantedLocal(f *fake, grant string) {
|
||||
f.mountinfo("/", "/srv/library")
|
||||
f.dir("/srv/library")
|
||||
f.file(ConfigFile, grantsConfig("library=/srv/library", "library=ws:rw,"+grant))
|
||||
f.serve("library")
|
||||
f.kernelReads()
|
||||
}
|
||||
|
||||
func TestAShareGrantedToTheLocalNetworkGoesToItsSubnetWhenTheFilterAdmitsIt(t *testing.T) {
|
||||
f := newFake(t)
|
||||
grantedLocal(f, "local-network:ro")
|
||||
f.homeLinks(rulesetAdmitting(homeNet))
|
||||
s := f.machine().Pass(context.Background(), nil)
|
||||
text := f.read(ExportFile)
|
||||
if !strings.Contains(text, " "+homeNet+"(ro,mp,sync,secure,no_subtree_check,all_squash,") {
|
||||
t.Errorf("the library is not exported read-only to the home network, squashed:\n%s", text)
|
||||
}
|
||||
if strings.Contains(text, "198.51.100") || strings.Contains(text, "172.17") {
|
||||
t.Errorf("a public subnet or a bridge is exported to:\n%s", text)
|
||||
}
|
||||
if len(s.Unreached) != 0 || !s.Shares[0].Live || len(s.Shares[0].To) != 2 {
|
||||
t.Errorf("status: %+v", s)
|
||||
}
|
||||
if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "local-network ("+homeNet+", ro)") {
|
||||
t.Errorf("health: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAGrantTheFilterDoesNotAdmitIsNotExportedAndIsUnhealthy(t *testing.T) {
|
||||
f := newFake(t)
|
||||
grantedLocal(f, "local-network:rw")
|
||||
f.homeLinks(rulesetAdmitting(""))
|
||||
s := f.machine().Pass(context.Background(), nil)
|
||||
if strings.Contains(f.read(ExportFile), homeNet) {
|
||||
t.Errorf("the library is exported to a subnet the filter drops:\n%s", f.read(ExportFile))
|
||||
}
|
||||
if len(s.Unreached) != 1 || !strings.Contains(s.Unreached[0], "reach local-network") {
|
||||
t.Errorf("the unreached grant is not said with its act: %v", s.Unreached)
|
||||
}
|
||||
if h := f.machine().Health(); h.Healthy || !strings.Contains(h.Why, "does not admit 2049/tcp") {
|
||||
t.Errorf("health: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWithNoPrivateNetworkOnTheUplinkALocalGrantGoesNowhereAndIsSaid(t *testing.T) {
|
||||
f := newFake(t)
|
||||
grantedLocal(f, "local-network:rw")
|
||||
s := f.machine().Pass(context.Background(), nil) // ip answers nothing
|
||||
if strings.Contains(f.read(ExportFile), "192.168") {
|
||||
t.Errorf("exported to a local network that was not found:\n%s", f.read(ExportFile))
|
||||
}
|
||||
if !strings.Contains(strings.Join(s.Said, "\n"), "reports no private network on its uplink") {
|
||||
t.Errorf("not said: %v", s.Said)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdoptWaitsUntilTheLocalNetworkExportCoversTheProperty(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.uid = 1000
|
||||
zfsWith(f, "rw=@"+homeNet)
|
||||
f.file("/etc/exports.d/zfs.exports", "/srv/library "+homeNet+"(sec=sys,rw,no_subtree_check,mountpoint)\n")
|
||||
grantedLocal(f, "local-network:rw")
|
||||
f.homeLinks(rulesetAdmitting("")) // the filter does not admit the home network yet
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
if _, err := f.machine().Adopt("/srv/library", false, ""); err == nil || !strings.Contains(err.Error(), "does not cover") {
|
||||
t.Fatalf("adopt while the home network is not covered: %v", err)
|
||||
}
|
||||
f.homeLinks(rulesetAdmitting(homeNet))
|
||||
f.machine().Pass(context.Background(), nil)
|
||||
f.ran = nil
|
||||
dry, err := f.machine().Adopt("/srv/library", false, "")
|
||||
if err != nil || !strings.Contains(dry.Said, "would set sharenfs=off") {
|
||||
t.Fatalf("dry run once covered: %+v %v", dry, err)
|
||||
}
|
||||
for _, r := range f.ran {
|
||||
if strings.Contains(r, "zfs set") {
|
||||
t.Fatalf("a dry run changed something: %v", f.ran)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPropertyClientsAndCovered(t *testing.T) {
|
||||
for v, want := range map[string]string{
|
||||
"rw=@192.168.1.0/24": "192.168.1.0/24",
|
||||
"rw=@192.168.1.0/24:@10.0.0.3": "192.168.1.0/24 10.0.0.3",
|
||||
"on": "*",
|
||||
"rw,no_root_squash": "*",
|
||||
"sec=sys,ro=@10.0.0.3": "10.0.0.3",
|
||||
} {
|
||||
if got := strings.Join(PropertyClients(v), " "); got != want {
|
||||
t.Errorf("PropertyClients(%q) = %q, not %q", v, got, want)
|
||||
}
|
||||
}
|
||||
mine := []Grantee{{Node: "ws", Address: "10.0.0.3"}, {Node: LocalNetwork, Address: homeNet}}
|
||||
for client, want := range map[string]bool{"10.0.0.3": true, homeNet: true, "192.168.1.7": true, "192.168.0.0/16": false, "*": false, "10.0.0.4": false} {
|
||||
if Covered(client, mine) != want {
|
||||
t.Errorf("Covered(%q) is not %v", client, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -259,7 +259,9 @@ func TestAdoptIsADryRunWithoutConfirmAndRefusesWhileTheMeshExportIsNotLive(t *te
|
||||
f := newFake(t)
|
||||
f.uid = 1000
|
||||
f.dir("/srv/library")
|
||||
zfsWith(f, "rw=@192.0.2.0/24")
|
||||
// A property the mesh's export covers once live: the one node. One to the home network is
|
||||
// localnet_test.go's.
|
||||
zfsWith(f, "rw=@"+wsAddress)
|
||||
f.file(ConfigFile, config("library=/srv/library"))
|
||||
f.serve("library")
|
||||
// The kernel has not read the mesh's file: no exportfs effect.
|
||||
|
||||
@@ -31,6 +31,11 @@ type Status struct {
|
||||
// Warnings make the module no less healthy but are said in its health: what it could not learn, and
|
||||
// how old what it relies on is.
|
||||
Warnings []string `json:"warnings,omitempty"`
|
||||
// Local is the machine's local subnets this pass (ADR 0341): what a grant to local-network may reach.
|
||||
Local []string `json:"local_networks,omitempty"`
|
||||
// Unreached is each grant to the local network the packet filter does not admit: not exported, and the
|
||||
// module is unhealthy until it is.
|
||||
Unreached []string `json:"unreached,omitempty"`
|
||||
// Reloaded is when the process last had the kernel read the export files again, and why.
|
||||
Reloaded *time.Time `json:"reloaded,omitempty"`
|
||||
ReloadWhy string `json:"reload_why,omitempty"`
|
||||
@@ -61,8 +66,17 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status {
|
||||
if _, note := m.Backup(); note != "" {
|
||||
s.Warnings = append(s.Warnings, note)
|
||||
}
|
||||
if grantsLocal(c) {
|
||||
nets, admitted, notes := m.LocalNetworkAdmitted(ctx)
|
||||
c.Local = &Local{Nets: nets, Admitted: admitted}
|
||||
s.Local = nets
|
||||
s.Said = append(s.Said, notes...)
|
||||
}
|
||||
m.setWalk(ctx)
|
||||
exports, refused, said := m.Judge(c, roster, wants)
|
||||
if c.Local != nil {
|
||||
s.Unreached = c.Local.Unreached
|
||||
}
|
||||
m.setWalk(nil)
|
||||
if ctx.Err() != nil {
|
||||
// Stopping: write nothing and reload nothing, so Stopped starts at once.
|
||||
@@ -100,7 +114,7 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status {
|
||||
}
|
||||
}
|
||||
if reload == "" {
|
||||
if stray := Stray(live, mine, foreign, c.Range); len(stray) > 0 {
|
||||
if stray := Stray(live, mine, foreign, c.Range, s.Local); len(stray) > 0 {
|
||||
reload = fmt.Sprintf("the kernel exports %s to %s, which the export file no longer carries", stray[0].Path, stray[0].Client)
|
||||
}
|
||||
}
|
||||
@@ -116,7 +130,7 @@ func (m *Machine) Pass(ctx context.Context, last *Status) Status {
|
||||
if note != "" && len(exports) > 0 {
|
||||
s.Problems = append(s.Problems, note)
|
||||
}
|
||||
for _, e := range Stray(live, mine, foreign, c.Range) {
|
||||
for _, e := range Stray(live, mine, foreign, c.Range, s.Local) {
|
||||
s.Problems = append(s.Problems, fmt.Sprintf("the kernel still exports %s to %s, which the module's export file "+
|
||||
"no longer carries: a reload has to take it away (exportfs -ra as root)", e.Path, e.Client))
|
||||
}
|
||||
@@ -256,21 +270,34 @@ func loud(what string) {
|
||||
}
|
||||
|
||||
// Stray is what the kernel exports that is the module's and that its file no longer carries: an entry to a
|
||||
// client inside the private network's range — the only clients the module ever exports to — that neither
|
||||
// the module's file nor any other export file (/etc/exports, ZFS's own) has. What another file exports is
|
||||
// not the module's, whatever its client.
|
||||
func Stray(live, mine, foreign []Entry, rng string) []Entry {
|
||||
// client inside the private network's range, or one of the machine's local subnets — the only clients the
|
||||
// module ever exports to — that neither the module's file nor any other export file (/etc/exports, ZFS's
|
||||
// own) has. What another file exports is not the module's, whatever its client.
|
||||
func Stray(live, mine, foreign []Entry, rng string, local []string) []Entry {
|
||||
prefix, err := netip.ParsePrefix(rng)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
ours := []netip.Prefix{prefix}
|
||||
for _, n := range local {
|
||||
if p, err := netip.ParsePrefix(n); err == nil {
|
||||
ours = append(ours, p.Masked())
|
||||
}
|
||||
}
|
||||
has := map[string]bool{}
|
||||
for _, e := range append(append([]Entry{}, mine...), foreign...) {
|
||||
has[path.Clean(e.Path)+" "+e.Client] = true
|
||||
}
|
||||
var out []Entry
|
||||
for _, e := range live {
|
||||
if has[path.Clean(e.Path)+" "+e.Client] || !insideRange(e.Client, prefix) {
|
||||
if has[path.Clean(e.Path)+" "+e.Client] {
|
||||
continue
|
||||
}
|
||||
inside := false
|
||||
for _, p := range ours {
|
||||
inside = inside || insideRange(e.Client, p)
|
||||
}
|
||||
if !inside {
|
||||
continue
|
||||
}
|
||||
out = append(out, e)
|
||||
@@ -288,3 +315,14 @@ func insideRange(client string, prefix netip.Prefix) bool {
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// grantsLocal says whether any share is granted to the local network, so the process looks at the links
|
||||
// and the filter only when a grant asks for them.
|
||||
func grantsLocal(c Config) bool {
|
||||
for _, nodes := range c.Grants {
|
||||
if _, ok := nodes[LocalNetwork]; ok {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -28,7 +28,8 @@ import (
|
||||
// write to WantsFile). The access is the lower of the two: a grant of ro is never widened by a want. The
|
||||
// address is the mesh's own, from the roster the mesh renders (RosterFile), never one a consumer gave,
|
||||
// and it must lie in the private network's range. A share nobody is granted and wants is exported to
|
||||
// nobody. The home network is never named.
|
||||
// nobody. The local network is named only by a grant to `local-network`, and then only the subnets the
|
||||
// packet filter admits 2049 from (localnet.go, novox/hq ADR 0341).
|
||||
//
|
||||
// **Never the backups, never root's.** A share is judged at its real path (symlinks resolved). It is
|
||||
// refused when it is the machine's root, is not there, is not a folder, is owned by root or root's group
|
||||
@@ -86,6 +87,17 @@ type Config struct {
|
||||
// Grants is, per share, the nodes this machine lets mount it and with which access.
|
||||
Grants map[string]map[string]string
|
||||
Range string
|
||||
// Local is the machine's local network as the process found it this pass (localnet.go): nil when it was
|
||||
// not looked at, and a grant to local-network then goes nowhere.
|
||||
Local *Local
|
||||
}
|
||||
|
||||
// Local is the machine's local subnets, which of them the filter admits 2049 from, and — written by Judge —
|
||||
// each local-network grant the filter does not admit.
|
||||
type Local struct {
|
||||
Nets []string
|
||||
Admitted map[string]bool
|
||||
Unreached []string
|
||||
}
|
||||
|
||||
// Want is one share a node's mounts module wants of this machine, as it put it on the bus.
|
||||
@@ -172,7 +184,8 @@ func ParseShares(value string) ([]Share, []string) {
|
||||
}
|
||||
|
||||
// ParseGrants reads the `grants` setting: `share=node:rw|ro[,node:rw|ro]`, entries separated by spaces;
|
||||
// `none` grants nothing.
|
||||
// `none` grants nothing. `local-network` in a node's place grants the machine's local network
|
||||
// (localnet.go).
|
||||
func ParseGrants(value string) (map[string]map[string]string, []string) {
|
||||
out := map[string]map[string]string{}
|
||||
seen := map[string]bool{}
|
||||
@@ -457,8 +470,25 @@ func (m *Machine) Judge(c Config, roster map[string]string, wants map[string][]W
|
||||
e.To = append(e.To, Grantee{Node: node, Address: addr, Access: access})
|
||||
}
|
||||
}
|
||||
if g, ok := granted[LocalNetwork]; ok {
|
||||
switch {
|
||||
case c.Local == nil || len(c.Local.Nets) == 0:
|
||||
said = append(said, fmt.Sprintf("%s is granted to the local network, and this machine reports no private network on its uplink, so it is exported to no local network", s.Name))
|
||||
default:
|
||||
for _, n := range c.Local.Nets {
|
||||
if !c.Local.Admitted[n] {
|
||||
c.Local.Unreached = append(c.Local.Unreached, fmt.Sprintf("the share %s is granted to the local network %s, and the packet filter does not admit %d/tcp from it, so it is not exported there: bind nfs-server's %d with the reach local-network (novox/hq ADR 0340, ADR 0341)", s.Name, n, NFSPort, NFSPort))
|
||||
continue
|
||||
}
|
||||
e.To = append(e.To, Grantee{Node: LocalNetwork, Address: n, Access: lower(g, s.Access)})
|
||||
}
|
||||
}
|
||||
}
|
||||
grantedNodes := make([]string, 0, len(granted))
|
||||
for n := range granted {
|
||||
if n == LocalNetwork {
|
||||
continue
|
||||
}
|
||||
if !wanting[n] {
|
||||
grantedNodes = append(grantedNodes, n)
|
||||
}
|
||||
@@ -474,7 +504,7 @@ func (m *Machine) Judge(c Config, roster map[string]string, wants map[string][]W
|
||||
e.Clients = strings.Join(addrs, ",")
|
||||
e.Line = ExportLine(e)
|
||||
if len(e.To) == 0 {
|
||||
said = append(said, fmt.Sprintf("%s is exported to no node: none is both granted it and wants it", s.Name))
|
||||
said = append(said, fmt.Sprintf("%s is exported to no node: none is both granted it and wants it, and no local network is granted and admitted", s.Name))
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
@@ -517,6 +547,7 @@ func ExportFileText(exports []Exported) string {
|
||||
b.WriteString("# Written by the mesh's module nfs-server (novox/hq ADR 0263). Rendered by the module's own process\n")
|
||||
b.WriteString("# from its assignment's `shares` and `grants` and the nodes that want each share; edit the settings,\n")
|
||||
b.WriteString("# never this file. One share per folder, to each granted and wanting node's private address only,\n")
|
||||
b.WriteString("# and to a local subnet only when local-network is granted and the filter admits 2049 from it;\n")
|
||||
b.WriteString("# every client squashed to the folder's owner.\n")
|
||||
for _, e := range exports {
|
||||
if e.Line == "" {
|
||||
|
||||
@@ -127,7 +127,7 @@ func (m *Machine) Exports() ExportsAnswer {
|
||||
}
|
||||
a.Notes = append(a.Notes, st.Problems...)
|
||||
a.Notes = append(a.Notes, st.Warnings...)
|
||||
a.Unmatched = st.Said
|
||||
a.Unmatched = append(append([]string{}, st.Said...), st.Unreached...)
|
||||
}
|
||||
entries, sets, notes := m.Foreign()
|
||||
a.Notes = append(a.Notes, notes...)
|
||||
@@ -380,6 +380,31 @@ func (m *Machine) Adopt(p string, confirm bool, why string) (AdoptAnswer, error)
|
||||
if lnote != "" || !SharedLive(live, *mine) {
|
||||
return a, fmt.Errorf("the mesh's export of %s to %s is not live in the kernel yet, so taking over would unexport it; wait for the module's process, or call reload", p, grantees(mine.To))
|
||||
}
|
||||
// Every client the property exported to must be covered by the mesh's live export, so no client loses
|
||||
// the folder in the take-over (novox/hq ADR 0341 clause 10): for a property to the home network, the
|
||||
// module's local-network export of it. Who the property exports to is read from its value and from the
|
||||
// file ZFS writes for it.
|
||||
clients := PropertyClients(ds.ShareNFS)
|
||||
entries, _, _ := m.Foreign()
|
||||
for _, e := range entries {
|
||||
if strings.Contains(e.From, "zfs") && path.Clean(e.Path) == p {
|
||||
clients = append(clients, e.Client)
|
||||
}
|
||||
}
|
||||
var uncovered []string
|
||||
seenClient := map[string]bool{}
|
||||
for _, c := range clients {
|
||||
if seenClient[c] {
|
||||
continue
|
||||
}
|
||||
seenClient[c] = true
|
||||
if !Covered(c, mine.To) {
|
||||
uncovered = append(uncovered, c)
|
||||
}
|
||||
}
|
||||
if len(uncovered) > 0 {
|
||||
return a, fmt.Errorf("the dataset's sharenfs exports %s to %s, which the mesh's export (to %s) does not cover, so taking over would unexport it for them: grant the share to local-network (and bind %d with the reach local-network) until the mesh's export covers them, or have a person decide they lose it", p, strings.Join(uncovered, ", "), grantees(mine.To), NFSPort)
|
||||
}
|
||||
plan := fmt.Sprintf("set sharenfs=off on the dataset %s (now %q) and run exportfs -ra; %s stays exported as the share %s to %s, every client as %d:%d",
|
||||
ds.Name, ds.ShareNFS, p, mine.Name, grantees(mine.To), mine.UID, mine.GID)
|
||||
if !confirm {
|
||||
@@ -428,6 +453,9 @@ func (m *Machine) Health() Health {
|
||||
for _, r := range st.Refused {
|
||||
found = append(found, r.Why)
|
||||
}
|
||||
for _, u := range st.Unreached {
|
||||
found = append(found, sentence(u))
|
||||
}
|
||||
if st.ReloadError != "" {
|
||||
found = append(found, sentence("the kernel could not read the export files again: "+st.ReloadError))
|
||||
}
|
||||
|
||||
@@ -55,7 +55,7 @@
|
||||
"grants": {
|
||||
"kind": "preference",
|
||||
"default": "none",
|
||||
"why": "which nodes may mount which share, and how, is this machine's to say (hq ADR 0263, review of mesh-catalog #136): share=node:rw|ro[,node:rw|ro], entries separated by spaces; none exports to no node, whatever a node wants"
|
||||
"why": "which nodes may mount which share, and how, is this machine's to say (hq ADR 0263, review of mesh-catalog #136): share=node:rw|ro[,node:rw|ro], entries separated by spaces; none exports to no node, whatever a node wants. local-network in a node's place grants this machine's local network, the private subnets on its uplink (hq ADR 0341)"
|
||||
}
|
||||
},
|
||||
"facts": {
|
||||
@@ -74,7 +74,7 @@
|
||||
"protocol": "tcp",
|
||||
"from": "mesh",
|
||||
"fixed": true,
|
||||
"why": "the shares, to the mesh's machines only (hq ADR 0263): NFS version 4 alone, which needs no other port, and never the home network, where a device that is not a node could claim any user id"
|
||||
"why": "the shares, to the mesh's machines (hq ADR 0263): NFS version 4 alone, which needs no other port. A machine whose grants name local-network binds this port with the reach local-network (hq ADR 0340, ADR 0341), and the module exports to a local subnet only while the filter admits it from there"
|
||||
}
|
||||
],
|
||||
"resources": [
|
||||
@@ -154,7 +154,7 @@
|
||||
"type": "file",
|
||||
"path": "/etc/nfs-server/shares.conf",
|
||||
"mode": "0644",
|
||||
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The grants: share=node:rw|ro[,…],\n# which nodes may mount each and how. The module's process exports a share to a node's private address\n# only when it is granted here and that node's mounts module wants it, every client mapped to the\n# folder's owner, and only to addresses inside the range below.\nshares=${setting:shares}\ngrants=${setting:grants}\nrange=${machine:mesh-range}\n"
|
||||
"content": "# Written by the mesh (module nfs-server, novox/hq ADR 0263) from this machine's assignment.\n# Replaced on every push; change the `shares` setting, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The grants: share=node:rw|ro[,…],\n# which nodes may mount each and how; local-network in a node's place grants this machine's local\n# network (novox/hq ADR 0341). The module's process exports a share to a node's private address only\n# when it is granted here and that node's mounts module wants it, and to a local subnet only when the\n# packet filter admits 2049 from it; every client mapped to the folder's owner.\nshares=${setting:shares}\ngrants=${setting:grants}\nrange=${machine:mesh-range}\n"
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
|
||||
@@ -0,0 +1,288 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The shares and logins, from the assignment, the hand-written configuration as found, and the module's
|
||||
// rendering (novox/hq ADR 0341 clauses 2 to 4).
|
||||
//
|
||||
// **One share per folder, every client as its owner.** A share is a name and a path in the `shares`
|
||||
// setting (`library=/srv/library`, `:ro` for read-only). It is served with `force user` and `force group`
|
||||
// read from the real folder's owner, never written as numbers in a setting, so a file written by any client
|
||||
// lands owned by the folder's owner. The refusals are ADR 0263's: the machine's root, a folder not there or
|
||||
// not a folder, a folder owned by root or root's group, and a folder inside or holding a backup repository.
|
||||
//
|
||||
// **SMB 2.1 and later, on 445 alone.** NetBIOS is off (`disable netbios`, `smb ports = 445`), so 137, 138 and
|
||||
// 139 are never served; the minimum protocol is SMB2_10. No guest: `map to guest = never`.
|
||||
|
||||
// Share is one share the operator asked for.
|
||||
type Share struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Access string `json:"access"`
|
||||
}
|
||||
|
||||
// Config is what the mesh rendered from the assignment.
|
||||
type Config struct {
|
||||
Shares []Share
|
||||
Users []string
|
||||
// Passwords is the folder the mesh places the logins' secrets in (smb-password-<user>.secret).
|
||||
Passwords string
|
||||
}
|
||||
|
||||
// ParseConfig reads the config file: `shares=`, `users=` and `passwords=` lines.
|
||||
func ParseConfig(text string) (Config, []string) {
|
||||
var c Config
|
||||
var problems []string
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
key, value, ok := strings.Cut(line, "=")
|
||||
if !ok {
|
||||
problems = append(problems, fmt.Sprintf("the line %q of %s says nothing this module reads", line, ConfigFile))
|
||||
continue
|
||||
}
|
||||
switch strings.TrimSpace(key) {
|
||||
case "shares":
|
||||
shares, bad := ParseShares(value)
|
||||
c.Shares, problems = shares, append(problems, bad...)
|
||||
case "users":
|
||||
users, bad := ParseUsers(value)
|
||||
c.Users, problems = users, append(problems, bad...)
|
||||
case "passwords":
|
||||
c.Passwords = strings.TrimSpace(value)
|
||||
default:
|
||||
problems = append(problems, fmt.Sprintf("%s names %q, which this module does not read", ConfigFile, key))
|
||||
}
|
||||
}
|
||||
return c, problems
|
||||
}
|
||||
|
||||
// ParseShares reads the `shares` setting: `name=path` or `name=path:ro`, separated by spaces.
|
||||
func ParseShares(value string) ([]Share, []string) {
|
||||
var shares []Share
|
||||
var problems []string
|
||||
seen := map[string]bool{}
|
||||
for _, item := range strings.Fields(value) {
|
||||
if item == "none" {
|
||||
continue
|
||||
}
|
||||
name, p, ok := strings.Cut(item, "=")
|
||||
if !ok || name == "" || p == "" {
|
||||
problems = append(problems, fmt.Sprintf("the share %q is not name=path", item))
|
||||
continue
|
||||
}
|
||||
if !validName(name) {
|
||||
problems = append(problems, fmt.Sprintf("the share name %q is not lower-case letters, digits and dashes", name))
|
||||
continue
|
||||
}
|
||||
if seen[name] {
|
||||
problems = append(problems, fmt.Sprintf("the share %q is named twice", name))
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
s := Share{Name: name, Path: p, Access: "rw"}
|
||||
if strings.HasSuffix(p, ":ro") {
|
||||
s.Path, s.Access = strings.TrimSuffix(p, ":ro"), "ro"
|
||||
} else if strings.HasSuffix(p, ":rw") {
|
||||
s.Path = strings.TrimSuffix(p, ":rw")
|
||||
}
|
||||
shares = append(shares, s)
|
||||
}
|
||||
sort.Slice(shares, func(i, j int) bool { return shares[i].Name < shares[j].Name })
|
||||
return shares, problems
|
||||
}
|
||||
|
||||
// ParseUsers reads the `users` setting: login names separated by spaces; `none` is none.
|
||||
func ParseUsers(value string) ([]string, []string) {
|
||||
var out, problems []string
|
||||
seen := map[string]bool{}
|
||||
for _, u := range strings.Fields(value) {
|
||||
if u == "none" {
|
||||
continue
|
||||
}
|
||||
if !validName(u) {
|
||||
problems = append(problems, fmt.Sprintf("the login %q is not lower-case letters, digits and dashes", u))
|
||||
continue
|
||||
}
|
||||
if !seen[u] {
|
||||
seen[u] = true
|
||||
out = append(out, u)
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, problems
|
||||
}
|
||||
|
||||
func validName(n string) bool {
|
||||
if n == "" || n[0] == '-' {
|
||||
return false
|
||||
}
|
||||
for _, r := range n {
|
||||
if !(r >= 'a' && r <= 'z' || r >= '0' && r <= '9' || r == '-') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Section is one section of a Samba configuration: its name and its parameters, keys lower-cased with their
|
||||
// spaces folded.
|
||||
type Section struct {
|
||||
Name string `json:"name"`
|
||||
Params map[string]string `json:"params"`
|
||||
}
|
||||
|
||||
// ParseSamba reads a Samba configuration's sections, in order. Comments (# and ;) and continuation lines are
|
||||
// handled as Samba does.
|
||||
func ParseSamba(text string) []Section {
|
||||
var out []Section
|
||||
var cur *Section
|
||||
text = strings.ReplaceAll(text, "\\\n", " ")
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || line[0] == '#' || line[0] == ';' {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
|
||||
out = append(out, Section{Name: strings.TrimSpace(line[1 : len(line)-1]), Params: map[string]string{}})
|
||||
cur = &out[len(out)-1]
|
||||
continue
|
||||
}
|
||||
k, v, ok := strings.Cut(line, "=")
|
||||
if !ok || cur == nil {
|
||||
continue
|
||||
}
|
||||
cur.Params[foldKey(k)] = strings.TrimSpace(v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func foldKey(k string) string { return strings.Join(strings.Fields(strings.ToLower(k)), " ") }
|
||||
|
||||
// special are the sections that are not shares.
|
||||
var special = map[string]bool{"global": true, "homes": true, "printers": true, "print$": true}
|
||||
|
||||
// HandShare is one share of the hand-written configuration, and the setting's share that takes its place.
|
||||
type HandShare struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
// ValidUsers is who may log in to it, as the hand-written file said.
|
||||
ValidUsers string `json:"valid_users,omitempty"`
|
||||
ReadOnly bool `json:"read_only"`
|
||||
// Counterpart is the setting's share of the same name and folder; empty when the settings name none.
|
||||
Counterpart string `json:"counterpart,omitempty"`
|
||||
Why string `json:"why,omitempty"`
|
||||
}
|
||||
|
||||
// HandShares are the hand-written configuration's shares, each matched to the setting's share of the same
|
||||
// name serving the same folder (a trailing slash aside).
|
||||
func HandShares(sections []Section, shares []Share) []HandShare {
|
||||
byName := map[string]Share{}
|
||||
for _, s := range shares {
|
||||
byName[strings.ToLower(s.Name)] = s
|
||||
}
|
||||
var out []HandShare
|
||||
for _, sec := range sections {
|
||||
if special[strings.ToLower(sec.Name)] {
|
||||
continue
|
||||
}
|
||||
h := HandShare{Name: sec.Name, Path: sec.Params["path"], ValidUsers: sec.Params["valid users"], ReadOnly: readOnly(sec.Params)}
|
||||
s, ok := byName[strings.ToLower(sec.Name)]
|
||||
switch {
|
||||
case !ok:
|
||||
h.Why = fmt.Sprintf("the setting shares names no share %s", sec.Name)
|
||||
case cleanPath(s.Path) != cleanPath(h.Path):
|
||||
h.Why = fmt.Sprintf("the setting's share %s serves %s, and the hand-written one serves %s", s.Name, s.Path, h.Path)
|
||||
default:
|
||||
h.Counterpart = s.Name
|
||||
}
|
||||
out = append(out, h)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func cleanPath(p string) string {
|
||||
if p == "" {
|
||||
return ""
|
||||
}
|
||||
return path.Clean(p)
|
||||
}
|
||||
|
||||
// readOnly is Samba's reading of a share's access: read only = yes unless writable/writeable/write ok says
|
||||
// otherwise, as Samba defaults it.
|
||||
func readOnly(p map[string]string) bool {
|
||||
yes := func(v string) bool {
|
||||
v = strings.ToLower(strings.TrimSpace(v))
|
||||
return v == "yes" || v == "true" || v == "1"
|
||||
}
|
||||
if v, ok := p["read only"]; ok {
|
||||
return yes(v)
|
||||
}
|
||||
for _, k := range []string{"writable", "writeable", "write ok"} {
|
||||
if v, ok := p[k]; ok {
|
||||
return !yes(v)
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Served is a share as the module serves it.
|
||||
type Served struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Access string `json:"access"`
|
||||
UID int `json:"uid"`
|
||||
GID int `json:"gid"`
|
||||
User string `json:"force_user"`
|
||||
Group string `json:"force_group"`
|
||||
Users []string `json:"valid_users"`
|
||||
// Live is whether Samba serves it now, as testparm reads the configuration in use.
|
||||
Live bool `json:"live"`
|
||||
}
|
||||
|
||||
// Refused is a share that is not served, and why.
|
||||
type Refused struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
|
||||
// Render is Samba's whole configuration from the shares the module serves.
|
||||
func Render(shares []Served) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(Marker + "\n")
|
||||
b.WriteString("# Rendered by the module's own process from its assignment's settings shares and users; change the\n")
|
||||
b.WriteString("# settings, never this file. The hand-written configuration it replaced is kept beside it, dated.\n")
|
||||
b.WriteString("[global]\n")
|
||||
for _, kv := range [][2]string{
|
||||
{"server role", "standalone server"},
|
||||
{"security", "user"},
|
||||
{"passdb backend", "tdbsam"},
|
||||
{"map to guest", "never"},
|
||||
{"server min protocol", "SMB2_10"},
|
||||
{"disable netbios", "yes"},
|
||||
{"smb ports", "445"},
|
||||
{"load printers", "no"},
|
||||
{"printing", "bsd"},
|
||||
{"printcap name", "/dev/null"},
|
||||
{"disable spoolss", "yes"},
|
||||
} {
|
||||
fmt.Fprintf(&b, " %s = %s\n", kv[0], kv[1])
|
||||
}
|
||||
for _, s := range shares {
|
||||
ro := "no"
|
||||
if s.Access == "ro" {
|
||||
ro = "yes"
|
||||
}
|
||||
fmt.Fprintf(&b, "\n[%s]\n path = %s\n read only = %s\n browseable = yes\n guest ok = no\n", s.Name, s.Path, ro)
|
||||
fmt.Fprintf(&b, " valid users = %s\n force user = %s\n force group = %s\n", strings.Join(s.Users, " "), s.User, s.Group)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
@@ -0,0 +1,279 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The machine as the module sees it (novox/hq ADR 0341).
|
||||
//
|
||||
// **Two processes, two accounts.** The tools run as the operator account (ADR 0175 §4) and read; what only
|
||||
// root may do (`smbcontrol`, `smbstatus`) goes through `sudo -n`. The module's long-running process runs as
|
||||
// root: it judges the shares, and once a person adopted the configuration it writes Samba's configuration
|
||||
// file whole, sets the logins' passwords from their secrets, and stops NetBIOS.
|
||||
//
|
||||
// **It writes its own files and nothing else.** Samba's configuration (only after adoption), its dated copy
|
||||
// of the hand-written one, the status, and the record of which passwords it set. Nothing here can create,
|
||||
// chown, chmod or remove anything under a shared path: the machine offers no such call, and Write refuses
|
||||
// every path but the module's (novox/hq ADR 0051).
|
||||
|
||||
// The module's files on the machine.
|
||||
const (
|
||||
// ConfigFile is what the mesh renders from the assignment: the shares, the logins, where the secrets are.
|
||||
ConfigFile = "/etc/smb-server/shares.conf"
|
||||
// SambaConf is Samba's configuration: the hand-written one until adoption, the module's after.
|
||||
SambaConf = "/etc/samba/smb.conf"
|
||||
// Candidate is the module's rendering, checked with testparm before it takes SambaConf's place.
|
||||
Candidate = "/etc/samba/smb.conf.mesh-new"
|
||||
// HandCopyPrefix begins the dated copy of the hand-written configuration kept at adoption.
|
||||
HandCopyPrefix = "/etc/samba/smb.conf.hand-written-"
|
||||
// StatusFile is the process's last look, for the tools.
|
||||
StatusFile = "/run/smb-server/status.json"
|
||||
// AdoptDir is where the adopt verb, as the operator account, leaves a person's word for the process.
|
||||
AdoptDir = "/var/lib/smb-server/adopt"
|
||||
// AdoptFile is that word: when, and why.
|
||||
AdoptFile = AdoptDir + "/adopt.json"
|
||||
// SetFile is which password the process last set for each login, as a digest, readable by root only.
|
||||
SetFile = "/var/lib/smb-server/passwords.json"
|
||||
// Unit is Samba's service; NMBUnit its NetBIOS name service, stopped once the module takes over.
|
||||
Unit = "smb.service"
|
||||
NMBUnit = "nmb.service"
|
||||
// Port is the one port SMB is served on.
|
||||
Port = 445
|
||||
)
|
||||
|
||||
// Marker is the first line of a configuration the module wrote: how it tells its own from a person's.
|
||||
const Marker = "# Written by the mesh (module smb-server, novox/hq ADR 0341)."
|
||||
|
||||
// writable is every file the module may write.
|
||||
func writable(p string) bool {
|
||||
switch p {
|
||||
case SambaConf, Candidate, StatusFile, AdoptFile, SetFile:
|
||||
return true
|
||||
}
|
||||
return strings.HasPrefix(p, HandCopyPrefix) && !strings.Contains(strings.TrimPrefix(p, HandCopyPrefix), "/")
|
||||
}
|
||||
|
||||
// CommandTimeout bounds every command.
|
||||
const CommandTimeout = 20 * time.Second
|
||||
|
||||
// Ran is what one command did.
|
||||
type Ran struct {
|
||||
Out string
|
||||
Err string
|
||||
Code int
|
||||
}
|
||||
|
||||
// Runner runs one command, with what it reads on stdin; injected, so everything is tested over recorded
|
||||
// answers.
|
||||
type Runner func(ctx context.Context, input, name string, args ...string) Ran
|
||||
|
||||
// ExecRunner runs a command on the machine, bounded by CommandTimeout.
|
||||
func ExecRunner(ctx context.Context, input, name string, args ...string) Ran {
|
||||
ctx, cancel := context.WithTimeout(ctx, CommandTimeout)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, name, args...)
|
||||
if input != "" {
|
||||
cmd.Stdin = strings.NewReader(input)
|
||||
}
|
||||
var stdout, stderr bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = &stdout, &stderr
|
||||
err := cmd.Run()
|
||||
r := Ran{Out: stdout.String(), Err: strings.TrimSpace(stderr.String())}
|
||||
if err != nil {
|
||||
var exit *exec.ExitError
|
||||
if errors.As(err, &exit) {
|
||||
r.Code = exit.ExitCode()
|
||||
} else {
|
||||
r.Code = -1
|
||||
if r.Err == "" {
|
||||
r.Err = err.Error()
|
||||
}
|
||||
}
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// Owner is who owns a path, and whether it is a directory.
|
||||
type Owner struct {
|
||||
UID, GID int
|
||||
Dir bool
|
||||
}
|
||||
|
||||
// Machine is a filesystem root (the real one, or a test's tree), a runner, who this process is, and a clock.
|
||||
type Machine struct {
|
||||
Root string
|
||||
Run Runner
|
||||
UID int
|
||||
Now func() time.Time
|
||||
// ownerOf reads a file's owner from its inode; tests set it so their folders stand for an owner.
|
||||
ownerOf func(info os.FileInfo) (uid, gid int)
|
||||
}
|
||||
|
||||
// Here is this machine.
|
||||
func Here() *Machine {
|
||||
return &Machine{Root: "/", Run: ExecRunner, UID: os.Getuid(), Now: time.Now, ownerOf: inodeOwner}
|
||||
}
|
||||
|
||||
func (m *Machine) at(p string) string { return filepath.Join(m.Root, p) }
|
||||
|
||||
// Read reads a file.
|
||||
func (m *Machine) Read(p string) (string, error) {
|
||||
b, err := os.ReadFile(m.at(p))
|
||||
return string(b), err
|
||||
}
|
||||
|
||||
// Exists says whether a path is there.
|
||||
func (m *Machine) Exists(p string) bool {
|
||||
_, err := os.Lstat(m.at(p))
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// Stat is a path's owner and whether it is a folder.
|
||||
func (m *Machine) Stat(p string) (Owner, error) {
|
||||
info, err := os.Stat(m.at(p))
|
||||
if err != nil {
|
||||
return Owner{}, err
|
||||
}
|
||||
of := m.ownerOf
|
||||
if of == nil {
|
||||
of = inodeOwner
|
||||
}
|
||||
uid, gid := of(info)
|
||||
return Owner{UID: uid, GID: gid, Dir: info.IsDir()}, nil
|
||||
}
|
||||
|
||||
// Real is a path with every symbolic link followed, as the machine sees it.
|
||||
func (m *Machine) Real(p string) (string, error) {
|
||||
real, err := filepath.EvalSymlinks(m.at(p))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
rel, err := filepath.Rel(m.Root, real)
|
||||
if err != nil || strings.HasPrefix(rel, "..") {
|
||||
return "", fmt.Errorf("%s leads outside the machine", p)
|
||||
}
|
||||
return path.Join("/", filepath.ToSlash(rel)), nil
|
||||
}
|
||||
|
||||
// Children are a folder's subfolders, by name; symbolic links are not followed.
|
||||
func (m *Machine) Children(p string) []string {
|
||||
entries, err := os.ReadDir(m.at(p))
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
for _, e := range entries {
|
||||
if e.Type()&fs.ModeSymlink == 0 && e.IsDir() {
|
||||
out = append(out, e.Name())
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Write writes one of the module's own files, and refuses every other path.
|
||||
func (m *Machine) Write(p, content string, mode os.FileMode) error {
|
||||
if !writable(p) {
|
||||
return fmt.Errorf("%s is not one of the module's files; it writes nothing else", p)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(m.at(p)), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp := m.at(p) + ".tmp"
|
||||
if err := os.WriteFile(tmp, []byte(content), mode); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, m.at(p))
|
||||
}
|
||||
|
||||
// Place moves the checked candidate into Samba's configuration's place.
|
||||
func (m *Machine) Place() error { return os.Rename(m.at(Candidate), m.at(SambaConf)) }
|
||||
|
||||
// NameOf is the account or group name for an id, read from the machine's passwd or group file.
|
||||
func (m *Machine) NameOf(file string, id int) string {
|
||||
text, _ := m.Read(file)
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
f := strings.Split(line, ":")
|
||||
if len(f) >= 3 && f[2] == strconv.Itoa(id) {
|
||||
return f[0]
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// HasAccount says whether the machine has an account of that name.
|
||||
func (m *Machine) HasAccount(name string) bool {
|
||||
text, _ := m.Read("/etc/passwd")
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
if f := strings.SplitN(line, ":", 2); len(f) == 2 && f[0] == name {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Escalated is a command as the operator account runs what only root may: through sudo without a prompt.
|
||||
func (m *Machine) Escalated(name string, args ...string) (string, []string) {
|
||||
if m.UID == 0 {
|
||||
return name, args
|
||||
}
|
||||
return "sudo", append([]string{"-n", name}, args...)
|
||||
}
|
||||
|
||||
// AsRoot runs what only root may do, and names a refusal by how it failed.
|
||||
func (m *Machine) AsRoot(ctx context.Context, input, name string, args ...string) (Ran, error) {
|
||||
program, argv := m.Escalated(name, args...)
|
||||
r := m.Run(ctx, input, program, argv...)
|
||||
if r.Code != 0 {
|
||||
why := r.Err
|
||||
if why == "" {
|
||||
why = strings.TrimSpace(r.Out)
|
||||
}
|
||||
return r, fmt.Errorf("%s %s: exit %d: %s", name, strings.Join(args, " "), r.Code, why)
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
// UnitState is what the service manager says of a unit: active, inactive, failed, …
|
||||
func (m *Machine) UnitState(unit string) string {
|
||||
r := m.Run(context.Background(), "", "systemctl", "is-active", unit)
|
||||
if s := strings.TrimSpace(r.Out); s != "" {
|
||||
return s
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
// UnitEnabled is what the service manager says of a unit's boot: enabled, disabled, …
|
||||
func (m *Machine) UnitEnabled(unit string) string {
|
||||
r := m.Run(context.Background(), "", "systemctl", "is-enabled", unit)
|
||||
if s := strings.TrimSpace(r.Out); s != "" {
|
||||
return s
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
|
||||
func inodeOwner(info os.FileInfo) (uid, gid int) {
|
||||
if st, ok := info.Sys().(*syscall.Stat_t); ok {
|
||||
return int(st.Uid), int(st.Gid)
|
||||
}
|
||||
return -1, -1
|
||||
}
|
||||
|
||||
func firstLine(s string) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if i := strings.IndexByte(s, '\n'); i >= 0 {
|
||||
return s[:i]
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
// The smb-server module's Go bundle (novox/hq ADR 0341): a node serves folders over SMB to the devices on its
|
||||
// own network, one share per folder, every client as the folder's owner, SMB 2.1 and later on 445 alone.
|
||||
//
|
||||
// Started with no arguments it is served by the node's runtime as the operator account, over MCP on stdio
|
||||
// (ADR 0188, ADR 0193): the node-smb-server seat's verbs and the module's health. Started as
|
||||
// `smb-server config` it is the module's long-running process, as root (process.go). stdout is the MCP
|
||||
// channel; everything else is said on stderr.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 {
|
||||
if os.Args[1] != "config" || len(os.Args) != 2 {
|
||||
fmt.Fprintln(os.Stderr, "usage: smb-server [config]")
|
||||
os.Exit(2)
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, os.Interrupt)
|
||||
defer stop()
|
||||
if err := Here().RunConfig(ctx); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err := stdio.Serve("", Tools(Here())); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,397 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// The module's process, as root: every Every it judges the shares, and — only once a person adopted the
|
||||
// configuration — writes Samba's configuration whole, checks it with testparm before Samba reads it, sets the
|
||||
// logins' passwords from their secrets and stops NetBIOS (novox/hq ADR 0341 clauses 3 to 7). Before adoption
|
||||
// it reads the hand-written configuration, reports it, and writes nothing but its status.
|
||||
|
||||
// Every is how often the process looks.
|
||||
const Every = 30 * time.Second
|
||||
|
||||
// Adoption is a person's word to take over, left by the adopt verb.
|
||||
type Adoption struct {
|
||||
At string `json:"at"`
|
||||
Why string `json:"why"`
|
||||
// Copy is the dated copy of the hand-written configuration, once the process kept it.
|
||||
Copy string `json:"copy,omitempty"`
|
||||
}
|
||||
|
||||
// Login is one login of the `users` setting, as the process found it.
|
||||
type Login struct {
|
||||
Name string `json:"name"`
|
||||
// Password is "set" (from its secret), "kept as found" (in Samba's database, no secret given),
|
||||
// "waits" (no secret, not in Samba's database) or "refused" (why says).
|
||||
Password string `json:"password"`
|
||||
Why string `json:"why,omitempty"`
|
||||
}
|
||||
|
||||
// Status is the process's last look, as the tools read it.
|
||||
type Status struct {
|
||||
At string `json:"at"`
|
||||
// Mode is "found" (the hand-written configuration serves, kept as found) or "adopted" (the module's).
|
||||
Mode string `json:"mode"`
|
||||
Shares []Served `json:"shares"`
|
||||
Refused []Refused `json:"refused,omitempty"`
|
||||
Hand []HandShare `json:"hand_written,omitempty"`
|
||||
Logins []Login `json:"logins,omitempty"`
|
||||
Problems []string `json:"problems,omitempty"`
|
||||
Waits []health.Wait `json:"waits,omitempty"`
|
||||
Adopted *Adoption `json:"adopted,omitempty"`
|
||||
// Written is when the process last wrote Samba's configuration, and Reloaded when Samba last read it.
|
||||
Written string `json:"written,omitempty"`
|
||||
Reloaded string `json:"reloaded,omitempty"`
|
||||
NetBIOS string `json:"netbios,omitempty"`
|
||||
}
|
||||
|
||||
// WalkDepth and WalkLimit bound the look below a share for a backup repository (ADR 0263 §3).
|
||||
const (
|
||||
WalkDepth = 4
|
||||
WalkLimit = 200000
|
||||
)
|
||||
|
||||
// isRepository says whether a folder is a restic repository: `config` beside `keys`, `snapshots`, `data`.
|
||||
func (m *Machine) isRepository(dir string) bool {
|
||||
if o, err := m.Stat(path.Join(dir, "config")); err != nil || o.Dir {
|
||||
return false
|
||||
}
|
||||
for _, sub := range []string{"keys", "snapshots", "data"} {
|
||||
if o, err := m.Stat(path.Join(dir, sub)); err != nil || !o.Dir {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// repositoryNear finds a backup repository at, above or within WalkDepth levels below a folder; complete is
|
||||
// false when the look below stopped at its bound.
|
||||
func (m *Machine) repositoryNear(real string) (found string, complete bool) {
|
||||
for p := real; ; p = path.Dir(p) {
|
||||
if m.isRepository(p) {
|
||||
return p, true
|
||||
}
|
||||
if p == "/" {
|
||||
break
|
||||
}
|
||||
}
|
||||
seen, level := 0, []string{real}
|
||||
for depth := 1; depth <= WalkDepth && len(level) > 0; depth++ {
|
||||
var next []string
|
||||
for _, d := range level {
|
||||
for _, c := range m.Children(d) {
|
||||
if seen++; seen > WalkLimit {
|
||||
return "", false
|
||||
}
|
||||
p := path.Join(d, c)
|
||||
if m.isRepository(p) {
|
||||
return p, true
|
||||
}
|
||||
next = append(next, p)
|
||||
}
|
||||
}
|
||||
level = next
|
||||
}
|
||||
return "", true
|
||||
}
|
||||
|
||||
// Judge turns the asked-for shares into served shares, or refusals.
|
||||
func (m *Machine) Judge(c Config) ([]Served, []Refused) {
|
||||
var out []Served
|
||||
var refused []Refused
|
||||
for _, s := range c.Shares {
|
||||
no := func(why string) { refused = append(refused, Refused{Name: s.Name, Path: s.Path, Why: why}) }
|
||||
if !path.IsAbs(s.Path) || strings.ContainsAny(s.Path, "\n\r\"\\") {
|
||||
no(fmt.Sprintf("The share %s names %q, which is not an absolute path a configuration can carry.", s.Name, s.Path))
|
||||
continue
|
||||
}
|
||||
real, err := m.Real(path.Clean(s.Path))
|
||||
if err != nil {
|
||||
no(fmt.Sprintf("The share %s's folder %s is not there (%v); the module never creates it.", s.Name, s.Path, err))
|
||||
continue
|
||||
}
|
||||
if real == "/" {
|
||||
no(fmt.Sprintf("The share %s is the machine's root, which is never shared.", s.Name))
|
||||
continue
|
||||
}
|
||||
o, err := m.Stat(real)
|
||||
if err != nil || !o.Dir {
|
||||
no(fmt.Sprintf("The share %s's path %s is not a folder.", s.Name, real))
|
||||
continue
|
||||
}
|
||||
if o.UID == 0 || o.GID == 0 {
|
||||
no(fmt.Sprintf("The share %s's folder %s is owned by root (%d:%d), and every client would act as root there: give it an owner of its own.", s.Name, real, o.UID, o.GID))
|
||||
continue
|
||||
}
|
||||
if p, complete := m.repositoryNear(real); p != "" {
|
||||
no(fmt.Sprintf("The share %s's folder %s is at, inside or above the backup repository %s, and backups are never shared.", s.Name, real, p))
|
||||
continue
|
||||
} else if !complete {
|
||||
no(fmt.Sprintf("The share %s's folder %s holds more folders than the module looks through (%d levels, at most %d), so it could not make sure no backup repository is under it.", s.Name, real, WalkDepth, WalkLimit))
|
||||
continue
|
||||
}
|
||||
user, group := m.NameOf("/etc/passwd", o.UID), m.NameOf("/etc/group", o.GID)
|
||||
if user == "" || group == "" {
|
||||
no(fmt.Sprintf("The share %s's folder %s is owned by %d:%d, which this machine names no account or group for, so no client can be made its owner.", s.Name, real, o.UID, o.GID))
|
||||
continue
|
||||
}
|
||||
if len(c.Users) == 0 {
|
||||
no(fmt.Sprintf("The share %s is not served: the setting users names no login.", s.Name))
|
||||
continue
|
||||
}
|
||||
out = append(out, Served{Name: s.Name, Path: real, Access: s.Access, UID: o.UID, GID: o.GID, User: user, Group: group, Users: c.Users})
|
||||
}
|
||||
return out, refused
|
||||
}
|
||||
|
||||
// PasswordSecret is the secret holding a login's password.
|
||||
func PasswordSecret(user string) string { return "smb-password-" + user }
|
||||
|
||||
// readSecret reads a login's password from its secret: one line; absent is not given.
|
||||
func (m *Machine) readSecret(dir, user string) (string, bool, error) {
|
||||
if dir == "" || !path.IsAbs(dir) {
|
||||
return "", false, fmt.Errorf("the mesh gave no folder for the logins' secrets")
|
||||
}
|
||||
raw, err := m.Read(path.Join(dir, PasswordSecret(user)+".secret"))
|
||||
if err != nil {
|
||||
if !m.Exists(path.Join(dir, PasswordSecret(user)+".secret")) {
|
||||
return "", false, nil
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
pw := strings.TrimRight(raw, "\r\n")
|
||||
if pw == "" || strings.ContainsAny(pw, "\r\n") {
|
||||
return "", false, fmt.Errorf("its secret is not one line")
|
||||
}
|
||||
return pw, true, nil
|
||||
}
|
||||
|
||||
func digest(user, pw string) string {
|
||||
h := sha256.Sum256([]byte(user + "\x00" + pw))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
|
||||
// Logins judges every login of the setting and, when adopted, sets a password whose secret is new or changed
|
||||
// (clause 5). A login in Samba's database with no secret given keeps its password; the module never removes
|
||||
// or resets an entry by itself.
|
||||
func (m *Machine) Logins(ctx context.Context, c Config, adopted bool) ([]Login, []health.Wait, []string) {
|
||||
var logins []Login
|
||||
var waits []health.Wait
|
||||
var problems []string
|
||||
set := map[string]string{}
|
||||
if raw, err := m.Read(SetFile); err == nil {
|
||||
_ = json.Unmarshal([]byte(raw), &set)
|
||||
}
|
||||
changed := false
|
||||
for _, u := range c.Users {
|
||||
l := Login{Name: u}
|
||||
inDB := m.Run(ctx, "", "pdbedit", "-L", "-u", u).Code == 0
|
||||
pw, given, err := m.readSecret(c.Passwords, u)
|
||||
switch {
|
||||
case err != nil:
|
||||
l.Password, l.Why = "refused", "its password "+PasswordSecret(u)+" could not be read: "+err.Error()
|
||||
problems = append(problems, fmt.Sprintf("the login %s: %s", u, l.Why))
|
||||
case !given && inDB:
|
||||
l.Password, l.Why = "kept as found", "no secret "+PasswordSecret(u)+" is given, so the password in Samba's database stands"
|
||||
case !given:
|
||||
l.Password, l.Why = "waits", "no secret "+PasswordSecret(u)+" is given and Samba's database has no such login"
|
||||
waits = append(waits, health.Wait{Part: "the login " + u, Secret: PasswordSecret(u), What: "the password of the login " + u})
|
||||
case !m.HasAccount(u):
|
||||
l.Password, l.Why = "refused", "this machine has no account "+u+"; the module creates no account"
|
||||
problems = append(problems, fmt.Sprintf("the login %s: %s", u, l.Why))
|
||||
case !adopted:
|
||||
l.Password, l.Why = "kept as found", "its secret is given and is set once the configuration is adopted"
|
||||
case set[u] == digest(u, pw) && inDB:
|
||||
l.Password = "set"
|
||||
default:
|
||||
// smbpasswd -s reads the new password twice from stdin; -a adds the login when it is not there.
|
||||
args := []string{"-s", "-a", u}
|
||||
if r := m.Run(ctx, pw+"\n"+pw+"\n", "smbpasswd", args...); r.Code != 0 {
|
||||
l.Password, l.Why = "refused", "smbpasswd refused it: "+firstLine(r.Err)
|
||||
problems = append(problems, fmt.Sprintf("the login %s: %s", u, l.Why))
|
||||
break
|
||||
}
|
||||
set[u], changed, l.Password = digest(u, pw), true, "set"
|
||||
}
|
||||
logins = append(logins, l)
|
||||
}
|
||||
if changed {
|
||||
raw, _ := json.MarshalIndent(set, "", " ")
|
||||
if err := m.Write(SetFile, string(raw)+"\n", 0o600); err != nil {
|
||||
problems = append(problems, "which passwords were set could not be kept: "+err.Error())
|
||||
}
|
||||
}
|
||||
return logins, waits, problems
|
||||
}
|
||||
|
||||
// Pass is one look; what it did is in the status it answers, which it has written.
|
||||
func (m *Machine) Pass(ctx context.Context) Status {
|
||||
s := Status{At: m.Now().UTC().Format(time.RFC3339), Mode: "found", Shares: []Served{}}
|
||||
text, err := m.Read(ConfigFile)
|
||||
if err != nil {
|
||||
s.Problems = append(s.Problems, fmt.Sprintf("the assignment's shares (%s) could not be read: %v", ConfigFile, err))
|
||||
}
|
||||
c, problems := ParseConfig(text)
|
||||
s.Problems = append(s.Problems, problems...)
|
||||
served, refused := m.Judge(c)
|
||||
s.Refused = refused
|
||||
conf, _ := m.Read(SambaConf)
|
||||
ours := strings.HasPrefix(conf, Marker)
|
||||
if !ours {
|
||||
s.Hand = HandShares(ParseSamba(conf), c.Shares)
|
||||
}
|
||||
var adoption *Adoption
|
||||
if raw, err := m.Read(AdoptFile); err == nil {
|
||||
var a Adoption
|
||||
if json.Unmarshal([]byte(raw), &a) == nil && a.At != "" {
|
||||
adoption = &a
|
||||
}
|
||||
}
|
||||
if ours && adoption == nil {
|
||||
// Written by the module before, and the word is gone: what it wrote stands, and is the module's.
|
||||
adoption = &Adoption{At: "before", Why: "the configuration is already the module's"}
|
||||
}
|
||||
s.Adopted = adoption
|
||||
if adoption != nil {
|
||||
if !ours {
|
||||
if missing := withoutCounterpart(s.Hand); len(missing) > 0 {
|
||||
s.Problems = append(s.Problems, "the configuration is not taken over: "+strings.Join(missing, "; "))
|
||||
adoption = nil
|
||||
} else if served == nil && len(c.Shares) > 0 {
|
||||
s.Problems = append(s.Problems, "the configuration is not taken over: no share the settings name can be served")
|
||||
adoption = nil
|
||||
}
|
||||
}
|
||||
}
|
||||
if adoption != nil {
|
||||
s.Mode = "adopted"
|
||||
m.takeOver(ctx, &s, conf, ours, served, adoption)
|
||||
} else {
|
||||
for i := range served {
|
||||
served[i].Live = false
|
||||
}
|
||||
}
|
||||
logins, waits, lp := m.Logins(ctx, c, s.Mode == "adopted")
|
||||
s.Logins, s.Waits = logins, waits
|
||||
s.Problems = append(s.Problems, lp...)
|
||||
if s.Mode == "adopted" {
|
||||
s.Shares = m.markLive(ctx, served)
|
||||
} else {
|
||||
s.Shares = served
|
||||
}
|
||||
raw, _ := json.MarshalIndent(s, "", " ")
|
||||
if err := m.Write(StatusFile, string(raw)+"\n", 0o644); err != nil {
|
||||
s.Problems = append(s.Problems, "the status file could not be written: "+err.Error())
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// withoutCounterpart is each hand-written share the settings do not take the place of, in words.
|
||||
func withoutCounterpart(hand []HandShare) []string {
|
||||
var out []string
|
||||
for _, h := range hand {
|
||||
if h.Counterpart == "" {
|
||||
out = append(out, fmt.Sprintf("the hand-written share %s (%s) has no counterpart: %s", h.Name, h.Path, h.Why))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// takeOver writes the module's configuration when it differs from what is there: the first time, a dated
|
||||
// copy of the hand-written one is kept; every time, testparm checks the candidate before it takes the
|
||||
// configuration's place and Samba reads it. Then NetBIOS is stopped.
|
||||
func (m *Machine) takeOver(ctx context.Context, s *Status, conf string, ours bool, served []Served, a *Adoption) {
|
||||
want := Render(served)
|
||||
if conf != want {
|
||||
if err := m.Write(Candidate, want, 0o644); err != nil {
|
||||
s.Problems = append(s.Problems, "the module's configuration could not be written: "+err.Error())
|
||||
return
|
||||
}
|
||||
if r := m.Run(ctx, "", "testparm", "-s", m.at(Candidate)); r.Code != 0 {
|
||||
s.Problems = append(s.Problems, "the module's configuration fails Samba's own check and is kept out of use: "+firstLine(r.Err))
|
||||
return
|
||||
}
|
||||
if !ours {
|
||||
copyAt := HandCopyPrefix + m.Now().UTC().Format("20060102T150405Z")
|
||||
if err := m.Write(copyAt, conf, 0o644); err != nil {
|
||||
s.Problems = append(s.Problems, "the hand-written configuration could not be kept, so it is not replaced: "+err.Error())
|
||||
return
|
||||
}
|
||||
a.Copy = copyAt
|
||||
if a.At != "before" {
|
||||
raw, _ := json.MarshalIndent(a, "", " ")
|
||||
_ = m.Write(AdoptFile, string(raw)+"\n", 0o644)
|
||||
}
|
||||
}
|
||||
if err := m.Place(); err != nil {
|
||||
s.Problems = append(s.Problems, "the module's configuration could not take its place: "+err.Error())
|
||||
return
|
||||
}
|
||||
s.Written = m.Now().UTC().Format(time.RFC3339)
|
||||
if r := m.Run(ctx, "", "smbcontrol", "smbd", "reload-config"); r.Code != 0 {
|
||||
s.Problems = append(s.Problems, "Samba did not read its configuration again: "+firstLine(r.Err))
|
||||
} else {
|
||||
s.Reloaded = s.Written
|
||||
}
|
||||
}
|
||||
if m.UnitState(NMBUnit) == "active" || m.UnitEnabled(NMBUnit) == "enabled" {
|
||||
if r := m.Run(ctx, "", "systemctl", "disable", "--now", NMBUnit); r.Code != 0 {
|
||||
s.Problems = append(s.Problems, "NetBIOS ("+NMBUnit+") could not be stopped: "+firstLine(r.Err))
|
||||
}
|
||||
}
|
||||
s.NetBIOS = m.UnitState(NMBUnit)
|
||||
}
|
||||
|
||||
// markLive marks each share Samba serves now, as testparm reads the configuration in use.
|
||||
func (m *Machine) markLive(ctx context.Context, served []Served) []Served {
|
||||
r := m.Run(ctx, "", "testparm", "-s", m.at(SambaConf))
|
||||
live := map[string]bool{}
|
||||
if r.Code == 0 {
|
||||
for _, sec := range ParseSamba(r.Out) {
|
||||
live[strings.ToLower(sec.Name)] = true
|
||||
}
|
||||
}
|
||||
for i := range served {
|
||||
served[i].Live = live[strings.ToLower(served[i].Name)] && m.UnitState(Unit) == "active"
|
||||
}
|
||||
return served
|
||||
}
|
||||
|
||||
// RunConfig is the process: a pass now and every Every until the context ends.
|
||||
func (m *Machine) RunConfig(ctx context.Context) error {
|
||||
m.Pass(ctx)
|
||||
t := time.NewTicker(Every)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case <-t.C:
|
||||
m.Pass(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// LastStatus is the process's last look, or why there is none.
|
||||
func (m *Machine) LastStatus() (*Status, error) {
|
||||
text, err := m.Read(StatusFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the module's process has written no status (%s): %v", StatusFile, err)
|
||||
}
|
||||
var s Status
|
||||
if err := json.Unmarshal([]byte(text), &s); err != nil {
|
||||
return nil, fmt.Errorf("the module's status file is not one: %v", err)
|
||||
}
|
||||
return &s, nil
|
||||
}
|
||||
@@ -0,0 +1,301 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Defends novox/hq ADR 0341 clauses 2 to 7 and 11 to 13: an SMB share is served as its folder's owner, SMB
|
||||
// 2.1 and later on 445 alone with NetBIOS off; before a person adopts it the hand-written configuration is
|
||||
// kept as found and nothing is written; adoption keeps a dated copy, writes the module's configuration only
|
||||
// once testparm passes it, and refuses while a hand-written share has no counterpart; a login without its
|
||||
// secret keeps its password, and one with it is set once.
|
||||
|
||||
type fake struct {
|
||||
t *testing.T
|
||||
root string
|
||||
answers map[string]Ran
|
||||
ran []string
|
||||
inputs map[string]string
|
||||
now time.Time
|
||||
}
|
||||
|
||||
func newFake(t *testing.T) *fake {
|
||||
f := &fake{t: t, root: t.TempDir(), answers: map[string]Ran{}, inputs: map[string]string{}, now: time.Date(2026, 10, 13, 9, 30, 0, 0, time.UTC)}
|
||||
f.answers["systemctl is-active smb.service"] = Ran{Out: "active\n"}
|
||||
f.answers["systemctl is-active nmb.service"] = Ran{Out: "active\n"}
|
||||
f.answers["systemctl is-enabled nmb.service"] = Ran{Out: "enabled\n"}
|
||||
f.answers["pdbedit -L -u media"] = Ran{Out: "media:1001:\n"}
|
||||
f.file("/etc/passwd", "root:x:0:0::/root:/bin/sh\nmedia:x:1001:2000::/home/media:/bin/sh\n")
|
||||
f.file("/etc/group", "root:x:0:\nmedia:x:2000:\n")
|
||||
return f
|
||||
}
|
||||
|
||||
func (f *fake) machine() *Machine {
|
||||
return &Machine{Root: f.root, UID: 0, Now: func() time.Time { return f.now },
|
||||
ownerOf: func(info os.FileInfo) (int, int) {
|
||||
if strings.HasPrefix(info.Name(), "rootowned") {
|
||||
return 0, 0
|
||||
}
|
||||
return 1001, 2000
|
||||
},
|
||||
Run: func(_ context.Context, input, name string, args ...string) Ran {
|
||||
line := strings.Join(append([]string{name}, args...), " ")
|
||||
line = strings.ReplaceAll(line, f.root, "")
|
||||
f.ran = append(f.ran, line)
|
||||
if input != "" {
|
||||
f.inputs[line] = input
|
||||
}
|
||||
if line == "systemctl disable --now nmb.service" {
|
||||
f.answers["systemctl is-active nmb.service"] = Ran{Out: "inactive\n"}
|
||||
f.answers["systemctl is-enabled nmb.service"] = Ran{Out: "disabled\n"}
|
||||
}
|
||||
if strings.HasPrefix(line, "testparm -s ") {
|
||||
if r, ok := f.answers["testparm"]; ok {
|
||||
return r
|
||||
}
|
||||
return Ran{Out: f.read(strings.TrimPrefix(line, "testparm -s "))}
|
||||
}
|
||||
return f.answers[line]
|
||||
}}
|
||||
}
|
||||
|
||||
func (f *fake) dir(p string) {
|
||||
if err := os.MkdirAll(filepath.Join(f.root, p), 0o755); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *fake) file(p, content string) {
|
||||
f.dir(filepath.Dir(p))
|
||||
if err := os.WriteFile(filepath.Join(f.root, p), []byte(content), 0o644); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *fake) read(p string) string {
|
||||
b, _ := os.ReadFile(filepath.Join(f.root, p))
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func (f *fake) ran1(prefix string) []string {
|
||||
var out []string
|
||||
for _, r := range f.ran {
|
||||
if strings.HasPrefix(r, prefix) {
|
||||
out = append(out, r)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// handWritten is the home server's configuration as issue 169's survey found it.
|
||||
const handWritten = "[global]\n workgroup = WORKGROUP\n server string = home\n\n[media]\n path = /storage/media/\n valid users = media\n writable = yes\n"
|
||||
|
||||
func (f *fake) home(shares, users string) {
|
||||
f.dir("/storage/media")
|
||||
f.file(ConfigFile, "shares="+shares+"\nusers="+users+"\npasswords=/var/lib/mesh/smb-server/state\n")
|
||||
f.file(SambaConf, handWritten)
|
||||
}
|
||||
|
||||
func TestTheRenderingServesAsTheOwnerOnFourFortyFiveAloneWithoutNetBIOS(t *testing.T) {
|
||||
text := Render([]Served{{Name: "media", Path: "/storage/media", Access: "rw", User: "media", Group: "media", Users: []string{"media"}}})
|
||||
for _, want := range []string{Marker, "server min protocol = SMB2_10", "disable netbios = yes", "smb ports = 445", "map to guest = never",
|
||||
"[media]", "path = /storage/media", "read only = no", "valid users = media", "force user = media", "force group = media"} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("the rendering lacks %q:\n%s", want, text)
|
||||
}
|
||||
}
|
||||
if strings.Contains(text, "139") || strings.Contains(text, "1001") {
|
||||
t.Errorf("the rendering names 139, or an owner as a number:\n%s", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAShareIsRefusedAtRootRootOwnedMissingOrNearABackupAndNothingIsCreated(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.dir("/srv/rootowned")
|
||||
f.dir("/srv/pool/library")
|
||||
f.file("/srv/pool/library/deep/repo/config", "x")
|
||||
for _, d := range []string{"keys", "snapshots", "data"} {
|
||||
f.dir("/srv/pool/library/deep/repo/" + d)
|
||||
}
|
||||
f.dir("/srv/ok")
|
||||
c, _ := ParseConfig("shares=root=/ owned=/srv/rootowned gone=/srv/gone lib=/srv/pool/library ok=/srv/ok:ro\nusers=media\n")
|
||||
served, refused := f.machine().Judge(c)
|
||||
why := map[string]string{}
|
||||
for _, r := range refused {
|
||||
why[r.Name] = r.Why
|
||||
}
|
||||
for name, says := range map[string]string{"root": "machine's root", "owned": "owned by root", "gone": "not there", "lib": "backup repository"} {
|
||||
if !strings.Contains(why[name], says) {
|
||||
t.Errorf("%s is not refused as %q: %q", name, says, why[name])
|
||||
}
|
||||
}
|
||||
if len(served) != 1 || served[0].Name != "ok" || served[0].Access != "ro" || served[0].User != "media" {
|
||||
t.Errorf("served: %+v", served)
|
||||
}
|
||||
if f.machine().Exists("/srv/gone") {
|
||||
t.Error("the module created a missing folder")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBeforeAdoptionTheHandWrittenConfigurationIsKeptAndNothingIsWritten(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.home("media=/storage/media", "media")
|
||||
s := f.machine().Pass(context.Background())
|
||||
if s.Mode != "found" || f.read(SambaConf) != handWritten {
|
||||
t.Fatalf("mode %s; the hand-written configuration changed:\n%s", s.Mode, f.read(SambaConf))
|
||||
}
|
||||
if len(s.Hand) != 1 || s.Hand[0].Counterpart != "media" || s.Hand[0].ReadOnly {
|
||||
t.Errorf("the hand-written share is not matched: %+v", s.Hand)
|
||||
}
|
||||
for _, r := range f.ran {
|
||||
if strings.HasPrefix(r, "smbcontrol") || strings.HasPrefix(r, "systemctl disable") || strings.HasPrefix(r, "smbpasswd") {
|
||||
t.Errorf("before adoption the process ran %q", r)
|
||||
}
|
||||
}
|
||||
if s.Logins[0].Password != "kept as found" {
|
||||
t.Errorf("the login: %+v", s.Logins)
|
||||
}
|
||||
if h := f.machine().Health(); !h.Healthy || !strings.Contains(h.Why, "adopt") {
|
||||
t.Errorf("health before adoption: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdoptRefusesWhileAHandWrittenShareHasNoCounterpart(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.home("library=/storage/media", "media")
|
||||
f.machine().Pass(context.Background())
|
||||
if _, err := f.machine().Adopt(true, "take it over"); err == nil || !strings.Contains(err.Error(), "no share disappears") {
|
||||
t.Fatalf("adopt with a hand-written share the settings do not name: %v", err)
|
||||
}
|
||||
if f.machine().Exists(AdoptFile) {
|
||||
t.Error("a refused adopt left the word for the process")
|
||||
}
|
||||
f.file(ConfigFile, "shares=media=/storage/media\nusers=media\npasswords=/x\n")
|
||||
f.machine().Pass(context.Background())
|
||||
dry, err := f.machine().Adopt(false, "")
|
||||
if err != nil || !dry.DryRun || len(dry.Beside) != 1 || !strings.Contains(dry.Beside[0], "every client as media:media") {
|
||||
t.Fatalf("dry run: %+v %v", dry, err)
|
||||
}
|
||||
if f.machine().Exists(AdoptFile) {
|
||||
t.Error("a dry run left the word for the process")
|
||||
}
|
||||
if _, err := f.machine().Adopt(true, ""); err == nil {
|
||||
t.Error("adopt with confirm and no why was done")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdoptionKeepsADatedCopyChecksAndThenStopsNetBIOS(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.home("media=/storage/media", "media")
|
||||
f.machine().Pass(context.Background())
|
||||
f.file(AdoptFile, `{"at":"2026-10-13T09:31:00Z","why":"the operator keeps the share"}`)
|
||||
f.ran = nil
|
||||
s := f.machine().Pass(context.Background())
|
||||
if s.Mode != "adopted" || len(s.Problems) != 0 {
|
||||
t.Fatalf("status: %+v", s)
|
||||
}
|
||||
copyAt := HandCopyPrefix + "20261013T093000Z"
|
||||
if f.read(copyAt) != handWritten {
|
||||
t.Errorf("the dated copy is not the hand-written configuration: %q", f.read(copyAt))
|
||||
}
|
||||
if !strings.HasPrefix(f.read(SambaConf), Marker) || !strings.Contains(f.read(SambaConf), "force user = media") {
|
||||
t.Errorf("the module's configuration is not in place:\n%s", f.read(SambaConf))
|
||||
}
|
||||
want := []string{"testparm -s " + Candidate, "smbcontrol smbd reload-config", "systemctl disable --now nmb.service"}
|
||||
for _, w := range want {
|
||||
if len(f.ran1(w)) != 1 {
|
||||
t.Errorf("%q ran %d times: %v", w, len(f.ran1(w)), f.ran)
|
||||
}
|
||||
}
|
||||
if !s.Shares[0].Live || s.NetBIOS != "inactive" {
|
||||
t.Errorf("served: %+v netbios %s", s.Shares, s.NetBIOS)
|
||||
}
|
||||
// The next pass finds its own configuration and writes nothing again.
|
||||
f.ran = nil
|
||||
f.machine().Pass(context.Background())
|
||||
if len(f.ran1("smbcontrol")) != 0 || len(f.ran1("systemctl disable")) != 0 {
|
||||
t.Errorf("an unchanged configuration was written or reloaded again: %v", f.ran)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAConfigurationFailingTestparmIsKeptOutOfUse(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.home("media=/storage/media", "media")
|
||||
f.file(AdoptFile, `{"at":"2026-10-13T09:31:00Z","why":"take over"}`)
|
||||
f.answers["testparm"] = Ran{Code: 1, Err: "Unknown parameter encountered"}
|
||||
s := f.machine().Pass(context.Background())
|
||||
if f.read(SambaConf) != handWritten {
|
||||
t.Errorf("a configuration testparm refused took the hand-written one's place")
|
||||
}
|
||||
if len(f.ran1("smbcontrol")) != 0 || !strings.Contains(strings.Join(s.Problems, " "), "fails Samba's own check") {
|
||||
t.Errorf("not said, or reloaded: %v %v", s.Problems, f.ran)
|
||||
}
|
||||
if h := f.machine().Health(); h.Healthy {
|
||||
t.Errorf("healthy with a refused configuration: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestALoginKeepsItsPasswordWithoutASecretWaitsWhenNewAndIsSetOnce(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.home("media=/storage/media", "media guest2")
|
||||
f.answers["pdbedit -L -u guest2"] = Ran{Code: 255}
|
||||
f.file(AdoptFile, `{"at":"2026-10-13T09:31:00Z","why":"take over"}`)
|
||||
s := f.machine().Pass(context.Background())
|
||||
got := map[string]string{}
|
||||
for _, l := range s.Logins {
|
||||
got[l.Name] = l.Password
|
||||
}
|
||||
if got["media"] != "kept as found" || got["guest2"] != "waits" || len(f.ran1("smbpasswd")) != 0 {
|
||||
t.Fatalf("logins %v; smbpasswd ran %v", got, f.ran1("smbpasswd"))
|
||||
}
|
||||
h := f.machine().Health()
|
||||
if h.State != "waits" || len(h.Waits) != 1 || h.Waits[0].Secret != "smb-password-guest2" {
|
||||
t.Errorf("health: %+v", h)
|
||||
}
|
||||
// The secret given: set once from stdin, never on the command line, and not again while unchanged.
|
||||
f.file("/etc/passwd", f.read("/etc/passwd")+"guest2:x:1002:2000::/:/bin/sh\n")
|
||||
f.file("/var/lib/mesh/smb-server/state/smb-password-guest2.secret", "s3cret\n")
|
||||
f.machine().Pass(context.Background())
|
||||
runs := f.ran1("smbpasswd")
|
||||
if len(runs) != 1 || runs[0] != "smbpasswd -s -a guest2" || f.inputs[runs[0]] != "s3cret\ns3cret\n" {
|
||||
t.Fatalf("smbpasswd: %v %q", runs, f.inputs)
|
||||
}
|
||||
f.answers["pdbedit -L -u guest2"] = Ran{Out: "guest2:1002:\n"}
|
||||
f.machine().Pass(context.Background())
|
||||
if len(f.ran1("smbpasswd")) != 1 {
|
||||
t.Errorf("an unchanged password was set again: %v", f.ran1("smbpasswd"))
|
||||
}
|
||||
var set map[string]string
|
||||
_ = json.Unmarshal([]byte(f.read(SetFile)), &set)
|
||||
if strings.Contains(f.read(SetFile), "s3cret") || set["guest2"] == "" {
|
||||
t.Errorf("the record of set passwords holds the password, or nothing: %s", f.read(SetFile))
|
||||
}
|
||||
if info, err := os.Stat(filepath.Join(f.root, SetFile)); err != nil || info.Mode().Perm() != 0o600 {
|
||||
t.Errorf("the record of set passwords is not root's alone: %v", info.Mode())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheModuleWritesOnlyItsOwnFiles(t *testing.T) {
|
||||
m := newFake(t).machine()
|
||||
for _, p := range []string{"/etc/passwd", "/storage/media/x", "/etc/samba/other.conf", HandCopyPrefix + "x/../../evil"} {
|
||||
if err := m.Write(p, "x", 0o644); err == nil {
|
||||
t.Errorf("wrote %s", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseSambaReadsSectionsAndAccess(t *testing.T) {
|
||||
secs := ParseSamba("; c\n[global]\nworkgroup = W\n[a]\n path = /x\n read only = no\n[b]\n path = /y\n[c]\n path = /z\n writeable = yes\n")
|
||||
if len(secs) != 4 || secs[1].Params["path"] != "/x" {
|
||||
t.Fatalf("%+v", secs)
|
||||
}
|
||||
if readOnly(secs[1].Params) || !readOnly(secs[2].Params) || readOnly(secs[3].Params) {
|
||||
t.Error("read only is not read as Samba reads it")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,366 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/health"
|
||||
)
|
||||
|
||||
// Seat is the role this module holds on its machine (novox/hq ADR 0341).
|
||||
const Seat = "node-smb-server"
|
||||
|
||||
// Stale is how old the process's last look may be before the tools say it stopped looking.
|
||||
const Stale = 5 * time.Minute
|
||||
|
||||
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
|
||||
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run}
|
||||
}
|
||||
|
||||
func str(description string) map[string]any {
|
||||
return map[string]any{"type": "string", "description": description}
|
||||
}
|
||||
|
||||
func arg(a map[string]any, k string) string {
|
||||
v, _ := a[k].(string)
|
||||
return strings.TrimSpace(v)
|
||||
}
|
||||
|
||||
// confirmed says whether an act was confirmed: the boolean true, or the string "true" the seat's schema passes.
|
||||
func confirmed(a map[string]any) bool {
|
||||
switch v := a["confirm"].(type) {
|
||||
case bool:
|
||||
return v
|
||||
case string:
|
||||
return strings.EqualFold(strings.TrimSpace(v), "true")
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Tools are the seat's verbs and the module's health.
|
||||
func Tools(m *Machine) []stdio.Tool {
|
||||
return []stdio.Tool{
|
||||
verb("shares", "Every SMB share this machine serves: its name, folder, rw or ro, the owner every client acts as "+
|
||||
"(uid, gid, names), the logins allowed and whether each login's password was set from its secret, is kept "+
|
||||
"as found or waits for it, and whether Samba serves it now; whether the configuration is the mesh's or still "+
|
||||
"the hand-written one, and each hand-written share with whether the settings take its place; the shares "+
|
||||
"refused, and why. A password is never shown. Replaces testparm -s, cat /etc/samba/smb.conf and pdbedit -L. (r)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Shares(), nil }),
|
||||
verb("sessions", "Which devices are connected to which share now, with the login and protocol each uses, as "+
|
||||
"Samba knows them (smbstatus, as root). Replaces smbstatus. (r)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Sessions(context.Background()) }),
|
||||
verb("test", "Whether this machine serves one share now: asked for and not refused, served by Samba, Samba "+
|
||||
"active, and 445 listening. Replaces smbclient -L. (r)",
|
||||
map[string]any{"share": str("the share's name")},
|
||||
func(a map[string]any) (any, error) { return m.Test(context.Background(), arg(a, "share")) }),
|
||||
verb("reload", "Have Samba read its configuration again now (smbcontrol smbd reload-config, as root). The "+
|
||||
"module's process does this itself whenever it writes the configuration; this is for after a change made "+
|
||||
"outside it. Changes no shared folder. Replaces smbcontrol smbd reload-config. (a)",
|
||||
map[string]any{}, func(map[string]any) (any, error) { return m.Reload(context.Background()) }),
|
||||
verb("adopt", "Take over Samba's hand-written configuration: the module's process keeps a dated copy of it "+
|
||||
"beside it and writes the mesh's own in its place, checked with testparm, then stops NetBIOS. Only when every "+
|
||||
"hand-written share has its counterpart in the setting shares (same name, same folder), so no share "+
|
||||
"disappears. Without confirm, a dry run: each hand-written share beside the module's rendering of it. A "+
|
||||
"person's act, with why. Replaces edit /etc/samba/smb.conf. (a)",
|
||||
map[string]any{
|
||||
"confirm": str("\"true\" to do it; anything else, a dry run"),
|
||||
"why": str("why, in a sentence (required with confirm)"),
|
||||
},
|
||||
func(a map[string]any) (any, error) { return m.Adopt(confirmed(a), arg(a, "why")) }),
|
||||
{
|
||||
Name: "smb_health",
|
||||
Description: "Whether this machine's SMB shares are served, as the module's health check (novox/hq ADR 0240, " +
|
||||
"ADR 0341): unhealthy when Samba is not active, the module's process has not looked for five minutes, a " +
|
||||
"share is refused or not served, or the configuration fails Samba's check; waits when a login's password " +
|
||||
"is its secret not yet given; with why. (r)",
|
||||
Run: func(map[string]any) (any, error) { return m.Health(), nil },
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// SharesAnswer is what `shares` answers.
|
||||
type SharesAnswer struct {
|
||||
Mode string `json:"configuration"`
|
||||
Shares []Served `json:"shares"`
|
||||
Refused []Refused `json:"refused,omitempty"`
|
||||
Hand []HandShare `json:"hand_written,omitempty"`
|
||||
Logins []Login `json:"logins,omitempty"`
|
||||
Service string `json:"service"`
|
||||
Looked string `json:"looked,omitempty"`
|
||||
Said []string `json:"said"`
|
||||
Notes []string `json:"notes,omitempty"`
|
||||
}
|
||||
|
||||
// Shares is the machine's SMB shares now.
|
||||
func (m *Machine) Shares() SharesAnswer {
|
||||
a := SharesAnswer{Shares: []Served{}, Service: m.UnitState(Unit)}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
a.Notes = append(a.Notes, err.Error())
|
||||
a.Said = append(a.Said, "the module's process has not looked yet")
|
||||
return a
|
||||
}
|
||||
a.Mode, a.Shares, a.Refused, a.Hand, a.Logins, a.Looked = st.Mode, st.Shares, st.Refused, st.Hand, st.Logins, st.At
|
||||
a.Notes = st.Problems
|
||||
if st.Mode == "found" {
|
||||
a.Said = append(a.Said, "the configuration is the hand-written one, kept as found; the module takes it over by adopt, a person's act")
|
||||
} else {
|
||||
a.Said = append(a.Said, "the configuration is the module's")
|
||||
}
|
||||
for _, s := range st.Shares {
|
||||
state := "served"
|
||||
if !s.Live {
|
||||
state = "NOT served"
|
||||
}
|
||||
a.Said = append(a.Said, fmt.Sprintf("%s: %s, %s, every client as %s:%s, for %s — %s", s.Name, s.Path, s.Access, s.User, s.Group, strings.Join(s.Users, " "), state))
|
||||
}
|
||||
for _, h := range st.Hand {
|
||||
if h.Counterpart != "" {
|
||||
a.Said = append(a.Said, fmt.Sprintf("hand-written %s (%s): the setting's share %s takes its place", h.Name, h.Path, h.Counterpart))
|
||||
} else {
|
||||
a.Said = append(a.Said, fmt.Sprintf("hand-written %s (%s): %s", h.Name, h.Path, h.Why))
|
||||
}
|
||||
}
|
||||
for _, l := range st.Logins {
|
||||
a.Said = append(a.Said, fmt.Sprintf("login %s: password %s", l.Name, l.Password))
|
||||
}
|
||||
for _, r := range st.Refused {
|
||||
a.Said = append(a.Said, "refused: "+r.Why)
|
||||
}
|
||||
a.Said = append(a.Said, "Samba is "+a.Service)
|
||||
return a
|
||||
}
|
||||
|
||||
// Sessions is what smbstatus says of the connected devices.
|
||||
func (m *Machine) Sessions(ctx context.Context) (any, error) {
|
||||
r, err := m.AsRoot(ctx, "", "smbstatus", "--json")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var v struct {
|
||||
Sessions map[string]struct {
|
||||
Machine string `json:"remote_machine"`
|
||||
Username string `json:"username"`
|
||||
Dialect string `json:"session_dialect"`
|
||||
} `json:"sessions"`
|
||||
Tcons map[string]struct {
|
||||
Service string `json:"service"`
|
||||
Machine string `json:"machine"`
|
||||
} `json:"tcons"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(r.Out), &v); err != nil {
|
||||
return nil, fmt.Errorf("smbstatus answered what this cannot read: %v", err)
|
||||
}
|
||||
type session struct {
|
||||
Device string `json:"device"`
|
||||
Login string `json:"login"`
|
||||
Protocol string `json:"protocol"`
|
||||
Shares []string `json:"shares"`
|
||||
}
|
||||
var out []session
|
||||
for _, s := range v.Sessions {
|
||||
se := session{Device: s.Machine, Login: s.Username, Protocol: s.Dialect}
|
||||
for _, t := range v.Tcons {
|
||||
if t.Machine == s.Machine {
|
||||
se.Shares = append(se.Shares, t.Service)
|
||||
}
|
||||
}
|
||||
out = append(out, se)
|
||||
}
|
||||
return map[string]any{"sessions": out}, nil
|
||||
}
|
||||
|
||||
// TestAnswer is what `test` answers.
|
||||
type TestAnswer struct {
|
||||
Share string `json:"share"`
|
||||
Served bool `json:"served"`
|
||||
Service string `json:"service"`
|
||||
Listening bool `json:"listening_445"`
|
||||
Said string `json:"said"`
|
||||
}
|
||||
|
||||
// Test says whether one share is served now.
|
||||
func (m *Machine) Test(ctx context.Context, share string) (TestAnswer, error) {
|
||||
if share == "" {
|
||||
return TestAnswer{}, fmt.Errorf("name the share")
|
||||
}
|
||||
a := TestAnswer{Share: share, Service: m.UnitState(Unit)}
|
||||
r := m.Run(ctx, "", "ss", "-ltnH", "sport", "=", ":445")
|
||||
a.Listening = strings.TrimSpace(r.Out) != ""
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
for _, s := range st.Shares {
|
||||
if s.Name == share {
|
||||
a.Served = s.Live && a.Service == "active" && a.Listening
|
||||
}
|
||||
}
|
||||
for _, rf := range st.Refused {
|
||||
if rf.Name == share {
|
||||
a.Said = rf.Why
|
||||
return a, nil
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case a.Served:
|
||||
a.Said = fmt.Sprintf("%s is served on 445", share)
|
||||
case st.Mode == "found":
|
||||
a.Said = fmt.Sprintf("%s is not served by the module: the configuration is still the hand-written one", share)
|
||||
default:
|
||||
a.Said = fmt.Sprintf("%s is not served now: Samba is %s, 445 listening: %v", share, a.Service, a.Listening)
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// Reload has Samba read its configuration again.
|
||||
func (m *Machine) Reload(ctx context.Context) (any, error) {
|
||||
if _, err := m.AsRoot(ctx, "", "smbcontrol", "smbd", "reload-config"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Shares(), nil
|
||||
}
|
||||
|
||||
// AdoptAnswer is what `adopt` answers.
|
||||
type AdoptAnswer struct {
|
||||
DryRun bool `json:"dry_run"`
|
||||
Done bool `json:"done"`
|
||||
Copy string `json:"copy,omitempty"`
|
||||
Beside []string `json:"beside"`
|
||||
Said string `json:"said"`
|
||||
}
|
||||
|
||||
// AdoptWait is how long a confirmed adopt waits for the process to take over.
|
||||
var AdoptWait = 45 * time.Second
|
||||
|
||||
// Adopt leaves a person's word for the process to take over, after a dry run that shows each hand-written share
|
||||
// beside its rendering. It refuses while a hand-written share has no counterpart.
|
||||
func (m *Machine) Adopt(confirm bool, why string) (AdoptAnswer, error) {
|
||||
a := AdoptAnswer{DryRun: !confirm}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
if st.Mode == "adopted" {
|
||||
return a, fmt.Errorf("the configuration is already the module's; nothing to take over")
|
||||
}
|
||||
served := map[string]Served{}
|
||||
for _, s := range st.Shares {
|
||||
served[s.Name] = s
|
||||
}
|
||||
var missing []string
|
||||
for _, h := range st.Hand {
|
||||
if h.Counterpart == "" {
|
||||
missing = append(missing, fmt.Sprintf("%s (%s): %s", h.Name, h.Path, h.Why))
|
||||
a.Beside = append(a.Beside, fmt.Sprintf("hand-written %s, %s, valid users %s → no counterpart", h.Name, h.Path, h.ValidUsers))
|
||||
continue
|
||||
}
|
||||
s, ok := served[h.Counterpart]
|
||||
if !ok {
|
||||
missing = append(missing, fmt.Sprintf("%s: the setting's share %s is refused", h.Name, h.Counterpart))
|
||||
continue
|
||||
}
|
||||
a.Beside = append(a.Beside, fmt.Sprintf("hand-written %s, %s, valid users %s, read only %v → the mesh's %s, %s, %s, valid users %s, every client as %s:%s",
|
||||
h.Name, h.Path, h.ValidUsers, h.ReadOnly, s.Name, s.Path, s.Access, strings.Join(s.Users, " "), s.User, s.Group))
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
return a, fmt.Errorf("not taken over, so no share disappears: %s", strings.Join(missing, "; "))
|
||||
}
|
||||
plan := "keep a dated copy of the hand-written configuration beside it, write the mesh's own in its place once testparm passes it, have Samba read it, and stop NetBIOS"
|
||||
if !confirm {
|
||||
a.Said = "Dry run: would " + plan + "."
|
||||
return a, nil
|
||||
}
|
||||
if why == "" {
|
||||
return a, fmt.Errorf("say why: adopt is a person's act, and the why is kept with it")
|
||||
}
|
||||
raw, _ := json.MarshalIndent(Adoption{At: m.Now().UTC().Format(time.RFC3339), Why: why}, "", " ")
|
||||
if err := m.Write(AdoptFile, string(raw)+"\n", 0o644); err != nil {
|
||||
return a, err
|
||||
}
|
||||
deadline := m.Now().Add(AdoptWait)
|
||||
for {
|
||||
if st, err := m.LastStatus(); err == nil && st.Mode == "adopted" && st.Written != "" {
|
||||
a.Done = true
|
||||
if st.Adopted != nil {
|
||||
a.Copy = st.Adopted.Copy
|
||||
}
|
||||
a.Said = fmt.Sprintf("Did: %s. Why: %s", plan, why)
|
||||
return a, nil
|
||||
}
|
||||
if !m.Now().Before(deadline) {
|
||||
a.Said = "The word is left for the module's process, which has not taken over yet; read shares for why."
|
||||
return a, nil
|
||||
}
|
||||
time.Sleep(2 * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
// HealthAnswer is the module's health: mesh-sdk's answer, with what was found.
|
||||
type HealthAnswer struct {
|
||||
health.Answer
|
||||
Found []string `json:"found,omitempty"`
|
||||
}
|
||||
|
||||
// Health judges the shares.
|
||||
func (m *Machine) Health() HealthAnswer {
|
||||
var found []string
|
||||
if s := m.UnitState(Unit); s != "active" {
|
||||
found = append(found, fmt.Sprintf("Samba (%s) is %s.", Unit, s))
|
||||
}
|
||||
st, err := m.LastStatus()
|
||||
if err != nil {
|
||||
found = append(found, err.Error()+".")
|
||||
return HealthAnswer{Answer: health.Unhealthy(strings.Join(found, " ")), Found: found}
|
||||
}
|
||||
if at, err := time.Parse(time.RFC3339, st.At); err != nil || m.Now().Sub(at) > Stale {
|
||||
found = append(found, "The module's process has not looked for five minutes.")
|
||||
}
|
||||
for _, p := range st.Problems {
|
||||
found = append(found, sentence(p))
|
||||
}
|
||||
for _, r := range st.Refused {
|
||||
found = append(found, r.Why)
|
||||
}
|
||||
if st.Mode == "adopted" {
|
||||
for _, s := range st.Shares {
|
||||
if !s.Live {
|
||||
found = append(found, fmt.Sprintf("The share %s (%s) is not served by Samba.", s.Name, s.Path))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(found) > 0 {
|
||||
return HealthAnswer{Answer: health.Unhealthy(strings.Join(found, " "), st.Waits...), Found: found}
|
||||
}
|
||||
if len(st.Waits) > 0 {
|
||||
var why []string
|
||||
for _, w := range st.Waits {
|
||||
why = append(why, fmt.Sprintf("%s waits for %s.", sentence(w.Part), w.What))
|
||||
}
|
||||
return HealthAnswer{Answer: health.Waiting(strings.Join(why, " "), st.Waits...)}
|
||||
}
|
||||
if st.Mode == "found" {
|
||||
return HealthAnswer{Answer: health.Healthy("Samba serves the hand-written configuration, kept as found; the module takes it over by node-smb-server.adopt, a person's act.")}
|
||||
}
|
||||
var parts []string
|
||||
for _, s := range st.Shares {
|
||||
parts = append(parts, s.Name)
|
||||
}
|
||||
return HealthAnswer{Answer: health.Healthy(fmt.Sprintf("Samba is active and serves %s on 445.", strings.Join(parts, ", ")))}
|
||||
}
|
||||
|
||||
func sentence(s string) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if s == "" {
|
||||
return s
|
||||
}
|
||||
s = strings.ToUpper(s[:1]) + s[1:]
|
||||
if !strings.HasSuffix(s, ".") {
|
||||
s += "."
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
module smbserver
|
||||
|
||||
go 1.22
|
||||
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
|
||||
@@ -0,0 +1,8 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.6 h1:9qzdYONYbJdWcu6sxQcq9v1LI0JxcfkiKYkMUzJSkVQ=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.6/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d h1:BR8uwv4BqyLaBwITuLylrd9DRUnu0D5DjJw0RuQSXbk=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
@@ -0,0 +1,144 @@
|
||||
{
|
||||
"module": "smb-server",
|
||||
"version": "1",
|
||||
"upgrade": {
|
||||
"policy": "record",
|
||||
"why": "the folders the devices on the home network open: a build that breaks Samba's configuration leaves every device's share refused, and the gate on this one machine does not see the devices (hq ADR 0236, ADR 0341)"
|
||||
},
|
||||
"capabilities": [
|
||||
"package-manager",
|
||||
"service-manager"
|
||||
],
|
||||
"claims": [
|
||||
{
|
||||
"name": "node-smb-server",
|
||||
"scope": "node",
|
||||
"serves": [
|
||||
"shares",
|
||||
"sessions",
|
||||
"test",
|
||||
"reload",
|
||||
"adopt"
|
||||
]
|
||||
}
|
||||
],
|
||||
"tools": [
|
||||
"smb_health"
|
||||
],
|
||||
"settings": {
|
||||
"shares": {
|
||||
"kind": "preference",
|
||||
"default": "none",
|
||||
"why": "a machine serves no SMB share until its assignment names one: space-separated name=folder, each optionally :ro, one share per folder, every client acting as the folder's owner (hq ADR 0341)"
|
||||
},
|
||||
"users": {
|
||||
"kind": "preference",
|
||||
"default": "none",
|
||||
"why": "the logins allowed on every share, space-separated; each login's password is the secret smb-password-<login>, and a login already in Samba's database whose secret is not given keeps its password (hq ADR 0341)"
|
||||
}
|
||||
},
|
||||
"own-secrets": {
|
||||
"smb-password-*": {
|
||||
"path": "${dir:state}/smb-password-*.secret",
|
||||
"issued-by": "outside"
|
||||
}
|
||||
},
|
||||
"listens": [
|
||||
{
|
||||
"name": "smb",
|
||||
"port": 445,
|
||||
"protocol": "tcp",
|
||||
"from": "mesh",
|
||||
"fixed": true,
|
||||
"why": "SMB 2.1 and later, on 445 alone: NetBIOS is off, so 137, 138 and 139 are never opened. The home server's assignment binds it with the reach local-network (hq ADR 0340, ADR 0341), so the devices on its own network reach it and nothing beyond"
|
||||
}
|
||||
],
|
||||
"resources": [
|
||||
{
|
||||
"id": "package",
|
||||
"type": "package",
|
||||
"package": "samba"
|
||||
},
|
||||
{
|
||||
"id": "state",
|
||||
"type": "directory",
|
||||
"mode": "0700",
|
||||
"place": "."
|
||||
},
|
||||
{
|
||||
"id": "config-dir",
|
||||
"type": "directory",
|
||||
"path": "/etc/smb-server",
|
||||
"mode": "0755"
|
||||
},
|
||||
{
|
||||
"id": "run-dir",
|
||||
"type": "directory",
|
||||
"path": "/run/smb-server",
|
||||
"mode": "0755"
|
||||
},
|
||||
{
|
||||
"id": "lib-dir",
|
||||
"type": "directory",
|
||||
"path": "/var/lib/smb-server",
|
||||
"mode": "0755"
|
||||
},
|
||||
{
|
||||
"id": "adopt-dir",
|
||||
"type": "directory",
|
||||
"path": "/var/lib/smb-server/adopt",
|
||||
"mode": "0755",
|
||||
"owner": "${machine:account}"
|
||||
},
|
||||
{
|
||||
"id": "config",
|
||||
"type": "file",
|
||||
"path": "/etc/smb-server/shares.conf",
|
||||
"mode": "0644",
|
||||
"content": "# Written by the mesh (module smb-server, novox/hq ADR 0341) from this machine's assignment.\n# Replaced on every push; change the settings, never this file.\n#\n# The shares: name=folder, or name=folder:ro, one share per folder. The users: the logins allowed. The\n# module's process serves each share with every client as the folder's owner, and writes Samba's\n# configuration only once a person adopted it (node-smb-server.adopt).\nshares=${setting:shares}\nusers=${setting:users}\npasswords=${dir:state}\n"
|
||||
},
|
||||
{
|
||||
"id": "server",
|
||||
"type": "service",
|
||||
"unit": "smb.service",
|
||||
"state": "running",
|
||||
"boot": "enabled",
|
||||
"health": {
|
||||
"kind": "unit"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "config-process",
|
||||
"type": "process",
|
||||
"name": "smb-server-config",
|
||||
"artifact": "tools",
|
||||
"run": [
|
||||
"./smb-server",
|
||||
"config"
|
||||
],
|
||||
"health": {
|
||||
"kind": "tool",
|
||||
"tool": "smb_health",
|
||||
"interval": "60s",
|
||||
"timeout": "10s",
|
||||
"looks": 2,
|
||||
"grace": "90s"
|
||||
}
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "tools",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/smb-server",
|
||||
"binary": "smb-server",
|
||||
"loads": [
|
||||
"smb-server"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user