novox/mesh-host
18
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Compare commits

base: novox/mesh-host:4065b45ae84f93a4a5dd7f01724fa1feffcdd8a8
Branches Tags
novox/mesh-host:main novox/mesh-host:feat/774-l3-reland novox/mesh-host:revert/774-l3 novox/mesh-host:fix/1126-924-held-and-judged novox/mesh-host:fix/924-judge-by-have novox/mesh-host:fix/1126-fold-watch novox/mesh-host:fix/1945-adopt-hashes novox/mesh-host:feat/r4g-finish-1935 novox/mesh-host:fix/1695-bucket-per-node novox/mesh-host:replay/r92-before novox/mesh-host:fix/567-an-access-is-judged-by-who-writes-it novox/mesh-host:fix/inspect-by-kind-not-the-ambiguous-bare-form
..
compare: novox/mesh-host:replay/r92-before
Branches Tags
novox/mesh-host:feat/774-l3-reland novox/mesh-host:main novox/mesh-host:revert/774-l3 novox/mesh-host:fix/1126-924-held-and-judged novox/mesh-host:fix/924-judge-by-have novox/mesh-host:fix/1126-fold-watch novox/mesh-host:fix/1945-adopt-hashes novox/mesh-host:feat/r4g-finish-1935 novox/mesh-host:fix/1695-bucket-per-node novox/mesh-host:replay/r92-before novox/mesh-host:fix/567-an-access-is-judged-by-who-writes-it novox/mesh-host:fix/inspect-by-kind-not-the-ambiguous-bare-form
1 Commits
4065b45ae8 .. replay/r92-before
Author SHA1 Message Date
jochen 9e233e1d87 Replay R92's commit before its fix: mesh-host main without genesis's registry trust (#91 reverted) 
Never to be merged. The replay of hq issue 92 runs genesis at the commit before its fix, and the fix's
own first parent stops at step 6 on issue 146; this is today's genesis without #91, so the bed
reaches the push it replays (mesh-lab replays/register.go R92).
 2026-10-13 02:57:43 +02:00
214 changed files with 4692 additions and 10708 deletions
Expand all files Collapse all files
README.md
+1 -14
View File
@@ -102,19 +102,6 @@ what it did not understand would apply most of a declaration and report success.
]}
```
**A directory or archive says how it treats files it did not write** (issue 1024,
[`novox/hq` ADR 0422](https://git.novox.be/novox/hq)). `unknown` is `move-out`, `module` or `program`;
absent, nothing about unknown paths is done. `knows` lists paths inside it, relative to its `path`, that
are not declared but belong there; a trailing `/` names a folder and everything in it. `knows` without
`unknown` is refused. For `move-out` the node-engine, after every apply and so on every reconcile, moves
each path inside the directory that nothing declared, no archive unpacked and `knows` does not name,
whole, into `<home>/.local/state/mesh-moved/<module>/<UTC time>/` beside a `<UTC time>.json` list. The
list names each move before it is made. Nothing is deleted. A path younger than a minute waits for the
next look; a socket, pipe or device is left and said. It is only for a directory under an account's
home on one file system with the kept folder, and otherwise the resource fails and nothing moves.
`module` and `program` are read by the module and the program that owns the path, and the node-engine
does nothing for them.
**It converges rather than executes.** Applying twice changes nothing the second time; applying
to a drifted machine returns it. A mode is *maintained*, not merely set — a permission applied
at creation is not a permission held.
@@ -257,7 +244,7 @@ without its neighbour checked out is a repository nobody can build.
```
mesh-controller: ./build/mesh-controller plan <node> --json > /tmp/d.json
mesh-sdk: MESH_EMITTED=/tmp/d.json go test ./declaration/ -v (run in mesh-sdk's go/ directory)
mesh-host: MESH_EMITTED=/tmp/d.json go test ./internal/declaration/ -v
```
**What this node says when it joins, read by the mesh:**
RELEASE-57.md
-6
View File
@@ -1,6 +0,0 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
cmd/mesh-bootstrap/main.go
-6
View File
@@ -116,8 +116,6 @@ const usage = `mesh-bootstrap — make a bare machine into a mesh
--tools-source the repository the shared base is built from
--tools-ref what of it to build (default main)
--tools-go-source the repository the Go toolchain is built from (default: mesh-tools-go beside --tools-source)
--tools-go-ref what of it to build (default main)
--catalog-source the catalogue REPOSITORY, for building its modules;
--catalog is the checkout that says what they are
--catalog-ref what of it to build (default main)
@@ -310,10 +308,6 @@ func newFlagSet(opts *bootstrap.Options, jsonOut *bool) *flag.FlagSet {
"the repository the shared base is built from")
set.StringVar(&opts.ToolsSource.Ref, "tools-ref", opts.ToolsSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.ToolsGoSource.Repository, "tools-go-source", opts.ToolsGoSource.Repository,
"the repository the Go toolchain is built from (default: mesh-tools-go beside --tools-source)")
set.StringVar(&opts.ToolsGoSource.Ref, "tools-go-ref", opts.ToolsGoSource.Ref,
"what of it to build (default main)")
set.StringVar(&opts.CatalogSource.Repository, "catalog-source", opts.CatalogSource.Repository,
"the catalogue REPOSITORY, for building its modules — --catalog is the checkout that says what they are")
set.StringVar(&opts.CatalogSource.Ref, "catalog-ref", opts.CatalogSource.Ref,
cmd/mesh-host/busy_test.go
-66
View File
@@ -1,66 +0,0 @@
package main
import (
"context"
"sync/atomic"
"testing"
"time"
"github.com/novox/mesh-host/internal/link"
)
// A container waiting because it said busy is news when it starts waiting and when it is replaced, never on
// its output alone (novox/hq ADR 0335): the walk waiting on this machine starts its gate at the report that
// says it replaced, and a reconcile is otherwise silent.
func TestABusyWaitIsSaidWhenItStartsAndWhenItEnds(t *testing.T) {
quiet := link.Report{Outward: []string{"eth0"}}
waiting := link.Report{Outward: []string{"eth0"}, Busy: []link.Busy{{ID: "build-agent.server",
Container: "mesh-build-agent", State: "waiting", Said: "building a"}}}
still := link.Report{Outward: []string{"eth0"}, Busy: []link.Busy{{ID: "build-agent.server",
Container: "mesh-build-agent", State: "waiting", Said: "building b, a minute on"}}}
replaced := link.Report{Outward: []string{"eth0"}, Busy: []link.Busy{{ID: "build-agent.server",
Container: "mesh-build-agent", State: "replaced-when-idle"}}}
if adoptionFingerprint(waiting) == adoptionFingerprint(quiet) {
t.Error("a container starting to wait is not news")
}
if adoptionFingerprint(still) != adoptionFingerprint(waiting) {
t.Error("a wait whose output changed reads as news on every ask")
}
if adoptionFingerprint(replaced) == adoptionFingerprint(waiting) {
t.Error("a container replaced after it waited is not news")
}
if !worthSaying(link.Report{Busy: replaced.Busy}) {
t.Error("a replacement after a wait is not worth saying")
}
}
// While a container waits, a pass is asked for every so often, not at the reconcile's five minutes; with
// nothing waiting, none (novox/hq ADR 0335 §2).
func TestAWaitIsAskedAgainSoonerThanTheReconcile(t *testing.T) {
if BusyAskEvery >= ReconcileEvery || BusyAskEvery > time.Minute {
t.Fatalf("a waiting container is asked every %s", BusyAskEvery)
}
var reconciles atomic.Int32
queue := &link.Queue{Reconcile: func(context.Context) (link.Report, bool) {
reconciles.Add(1)
return link.Report{}, false
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go queue.Run(ctx)
busyWaiting.Store(false)
t.Cleanup(func() { busyWaiting.Store(false) })
go busyPasses(ctx, queue, 10*time.Millisecond)
time.Sleep(100 * time.Millisecond)
if n := reconciles.Load(); n != 0 {
t.Fatalf("%d passes were asked with nothing waiting", n)
}
busyWaiting.Store(true)
deadline := time.Now().Add(2 * time.Second)
for reconciles.Load() == 0 && time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
}
if reconciles.Load() == 0 {
t.Fatal("no pass was asked while a container waits")
}
}
cmd/mesh-host/file_references_test.go
-92
View File
@@ -1,92 +0,0 @@
package main
import (
"context"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/link"
)
// aFilesLink is a link that serves objects by name.
type aFilesLink struct{ objects map[string][]byte }
func (a aFilesLink) FileObject(_ context.Context, name string, _ int64) ([]byte, error) {
if b, ok := a.objects[name]; ok {
return b, nil
}
return nil, os.ErrNotExist
}
func referenceDeclaration(path string, body []byte) []byte {
return []byte(`{"declaration":1,"resources":[{"id":"m.big","type":"file","path":"` + path + `","digest":"` +
declaration.DigestOf(body) + `","size":` + strconv.Itoa(len(body)) + `}]}`)
}
// 1695/AC1 and AC2, through the whole apply: a reference is fetched, verified and written; a differing object
// refuses the declaration whole and writes nothing.
func TestADeclarationWithAFileByDigestAppliesFromTheObject(t *testing.T) {
unansweringPacman(t)
body := []byte(strings.Repeat("a line of a big file\n", 4000))
name, _ := declaration.ObjectName(declaration.DigestOf(body))
dir := t.TempDir()
target := filepath.Join(dir, "big")
opts := options{state: filepath.Join(dir, "state.json"), out: &strings.Builder{}}
raw := referenceDeclaration(target, body)
bad := append([]byte(nil), body...)
bad[10] ^= 1
ctx := link.WithFiles(context.Background(), aFilesLink{map[string][]byte{name: bad}})
report := applyAndKeep(ctx, opts, raw, nil, nil, nil)
if report.Refused == "" || !strings.Contains(report.Refused, "m.big") {
t.Errorf("1695/AC2: a differing object did not refuse the declaration: %+v", report)
}
if _, err := os.Stat(target); err == nil {
t.Error("1695/AC2: a file was written from an object that differs")
}
ctx = link.WithFiles(context.Background(), aFilesLink{map[string][]byte{name: body}})
report = applyAndKeep(ctx, opts, raw, nil, nil, nil)
if report.Refused != "" {
t.Fatalf("1695/AC1: refused: %s", report.Refused)
}
got, err := os.ReadFile(target)
if err != nil || string(got) != string(body) {
t.Errorf("1695/AC1: the file holds %d bytes (%v), want %d", len(got), err, len(body))
}
}
// 1695/AC8: the hand-run `apply <file>` and `reconcile` have no bus: a reference whose file here verifies is applied
// as it is, any other refuses and writes nothing — never an empty file.
func TestTheHandApplyNeverWritesAReferenceEmpty(t *testing.T) {
unansweringPacman(t)
body := []byte(strings.Repeat("a line of a big file\n", 4000))
dir := t.TempDir()
target := filepath.Join(dir, "big")
opts := options{state: filepath.Join(dir, "state.json"), out: &strings.Builder{}}
raw := referenceDeclaration(target, body)
d, err := declaration.ParseFileTrusted(raw)
if err != nil {
t.Fatal(err)
}
if err := runApply(context.Background(), opts, d, raw, fromFile); err == nil || !strings.Contains(err.Error(), "digest") {
t.Errorf("1695/AC8: a reference with no file and no bus was not refused: %v", err)
}
if _, err := os.Stat(target); err == nil {
t.Error("1695/AC8: a file was written for a reference that could not be had")
}
if err := os.WriteFile(target, body, 0o644); err != nil {
t.Fatal(err)
}
d, _ = declaration.ParseFileTrusted(raw)
if err := runApply(context.Background(), opts, d, raw, fromFile); err != nil && !strings.Contains(err.Error(), "applied only in part") {
t.Fatalf("1695/AC8: %v", err)
}
if got, _ := os.ReadFile(target); string(got) != string(body) {
t.Errorf("1695/AC8: the file holds %d bytes, want %d", len(got), len(body))
}
}
cmd/mesh-host/filling.go
-74
View File
@@ -1,74 +0,0 @@
package main
import (
"context"
"crypto/ed25519"
"errors"
"path/filepath"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/filling"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// serveFilling writes filled files on the provisioner client's request for as long as this engine serves (novox/hq
// ADR 0349 §4). A socket that cannot be served is said once, and this engine goes on applying.
func serveFilling(ctx context.Context, opts options, signer ed25519.PublicKey, say func(string)) {
if err := filling.Serve(ctx, filling.Socket, fillingEngine(opts, signer, say), say); err != nil {
say("filled files are not written on request on this machine: " + err.Error())
}
}
// fillingEngine is the write of a filled file as this engine does it: the declaration it keeps, verified, read under
// the apply's lock — the same two locks every apply takes, the process's and the one beside the state — and the file
// written through the apply (apply.WriteFilled), its outcome recorded in the node's state as an apply's is.
func fillingEngine(opts options, signer ed25519.PublicKey, say func(string)) *filling.Engine {
return &filling.Engine{
Declared: func() (*declaration.Declaration, error) {
raw, err := store.LoadDeclared(store.DeclaredPath(opts.state), signer)
if err != nil {
return nil, err
}
return declaration.Parse(raw)
},
UIDOf: filling.UIDOf,
Lock: func() (func(), error) {
applying.Lock()
unlock, err := store.Lock(opts.state, nil)
if err != nil {
applying.Unlock()
return nil, err
}
return func() { unlock(); applying.Unlock() }, nil
},
Place: func(ctx context.Context, d *declaration.Declaration, path string, content []byte) (string, string, error) {
built, err := system.For(builtFor)
if err != nil {
return "", "", err
}
known, err := store.Load(opts.state)
if err != nil {
return "", "", err
}
report, updated, writeErr := apply.WriteFilled(ctx, built, d, known, path, content, apply.ExecRunner,
announceOr(say), apply.KeepIn(filepath.Dir(opts.state)), apply.WindowsIn(filepath.Dir(opts.state)))
// Saved whichever way it went, as an apply's: what was written is on the machine either way.
if err := store.Save(opts.state, updated); err != nil {
return "", "", errors.Join(writeErr, errors.New("written, and the node's state could not be saved: "+
err.Error()))
}
action, digest := "", ""
for _, o := range report.Outcomes {
if o.Target == path && o.Type == string(declaration.TypeFile) {
action = o.Action
}
}
if rec, ok := updated.At(string(declaration.TypeFile), path); ok {
digest = rec.Wrote
}
return action, digest, writeErr
},
}
}
cmd/mesh-host/filling_test.go
-105
View File
@@ -1,105 +0,0 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"os"
"os/user"
"path/filepath"
"testing"
"time"
"github.com/novox/mesh-host/internal/filling"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §4 (issue 668): a write of a filled file and an apply are one at a time — the write
// takes the lock the apply holds, and writes only once the apply let it go — and the write is recorded in the
// node's state as an apply's outcome is.
func TestAWriteAndAnApplyOfTheSameFileSerialise(t *testing.T) {
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
me, err := user.Current()
if err != nil {
t.Fatal(err)
}
if me.Uid == "0" {
t.Skip("run as root, whose account is never the provisioner's")
}
dir := t.TempDir()
opts := options{state: filepath.Join(dir, "state.json")}
public, private, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "m.conf")
raw, err := json.Marshal(map[string]any{"declaration": 1, "provisioner_account": me.Username,
"filled_files": []string{path}, "resources": []any{map[string]any{"id": "m.conf", "type": "file",
"path": path, "content": "from the declaration\n", "owner-at-runtime": "owned", "owner": me.Username}}})
if err != nil {
t.Fatal(err)
}
if err := store.SaveDeclared(store.DeclaredPath(opts.state),
store.Declared{Declaration: raw, Signature: ed25519.Sign(private, raw)}); err != nil {
t.Fatal(err)
}
e := fillingEngine(opts, public, func(string) {})
uid := uint32(os.Getuid())
if err := e.Accepts(uid); err != nil {
t.Fatal(err)
}
// An apply holds the lock.
applying.Lock()
done := make(chan filling.Answer)
go func() {
done <- e.Write(context.Background(), uid, filling.Request{Path: path, Content: []byte("filled\n")})
}()
select {
case a := <-done:
applying.Unlock()
t.Fatalf("written while an apply held the lock: %+v", a)
case <-time.After(200 * time.Millisecond):
}
if _, err := os.Stat(path); err == nil {
applying.Unlock()
t.Fatal("the file was written while an apply held the lock")
}
applying.Unlock()
a := <-done
if a.Refused != "" || a.Failed != "" || a.Action != "created" {
t.Fatalf("the write after the apply: %+v", a)
}
got, err := os.ReadFile(path)
if err != nil || string(got) != "filled\n" {
t.Fatalf("the file holds %q (%v)", got, err)
}
known, err := store.Load(opts.state)
if err != nil {
t.Fatal(err)
}
if rec, ok := known.Find("m.conf"); !ok || rec.Wrote != a.Digest || rec.Target != path {
t.Errorf("the write is not recorded in the node's state: %+v, answered %q", rec, a.Digest)
}
// And the cross-process lock beside the state too: a write waits for a `reconcile` run by hand.
unlock, err := store.Lock(opts.state, nil)
if err != nil {
t.Fatal(err)
}
go func() {
done <- e.Write(context.Background(), uid, filling.Request{Path: path, Content: []byte("again\n")})
}()
select {
case a := <-done:
unlock()
t.Fatalf("written while another process held the apply's lock: %+v", a)
case <-time.After(200 * time.Millisecond):
}
unlock()
if a := <-done; a.Action != "updated" {
t.Fatalf("the second write: %+v", a)
}
}
cmd/mesh-host/judge_directories.go
+1 -1
View File
@@ -4,8 +4,8 @@ import (
"errors"
"fmt"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
cmd/mesh-host/libraries.go
+1 -1
View File
@@ -11,8 +11,8 @@ import (
"sync/atomic"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
)
cmd/mesh-host/libraries_test.go
+1 -1
View File
@@ -8,8 +8,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
)
cmd/mesh-host/local_network_test.go
-29
View File
@@ -1,29 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/outward"
"github.com/novox/mesh-host/internal/profile"
)
func reportOn(networks ...outward.Network) link.Report {
return link.Report{Outward: []string{"eth0"},
Profile: profileAsReported(profile.Profile{Machine: &profile.Machine{Networks: networks}})}
}
// **A change of the networks a machine is attached to is said unasked** (novox/hq ADR 0340 §7): a port
// reached from the local network is admitted from them, so a laptop that joins another network is said at
// the next reconcile, not at the next push — and the same networks again are not news.
func TestAChangedNetworkIsSaidUnasked(t *testing.T) {
w := &adoptionWatch{}
home := reportOn(outward.Network{Link: "eth0", Network: "192.168.1.0/24"})
w.said(home)
if w.differs(reportOn(outward.Network{Link: "eth0", Network: "192.168.1.0/24"})) {
t.Fatal("the same networks read as a change, so the machine would speak on every reconcile")
}
if !w.differs(reportOn(outward.Network{Link: "eth0", Network: "10.20.0.0/16"})) {
t.Fatal("a machine that moved to another network does not say so until it is pushed to")
}
}
cmd/mesh-host/main.go
+9 -156
View File
@@ -24,15 +24,14 @@ import (
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"text/tabwriter"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bundle"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/inventory"
@@ -608,12 +607,6 @@ func runApply(ctx context.Context, opts options, d *declaration.Declaration, raw
return errors.New(olderGenerationWords(*refused))
}
// A file given by digest is had before the plan is drawn from it (novox/hq ADR 0486 §1, issue 1695). This command has
// no link to the bus, so a file already here that verifies is used, and any other is refused: never written empty.
if err := apply.ResolveFileReferences(ctx, d, link.FilesFrom(ctx)); err != nil {
return fmt.Errorf("a file of this declaration could not be had by its digest: %w", err)
}
// Said before anything is done.
steps := apply.Plan(d, known, origin)
if opts.json && opts.dryRun {
@@ -1136,7 +1129,7 @@ func runLink(ctx context.Context, opts options) error {
// The one liveness judge on this machine (novox/hq ADR 0240), made before anything applies: every
// report this host makes carries what it says.
if j, err := liveness.Open(filepath.Join(filepath.Dir(opts.state), liveness.FileName),
&liveness.Exec{Run: apply.ExecRunner, System: builtFor}); j != nil {
&liveness.Exec{Run: apply.ExecRunner}); j != nil {
if err != nil {
say(err.Error())
}
@@ -1248,16 +1241,6 @@ func runLink(ctx context.Context, opts options) error {
Say: say,
Timeout: opts.timeout,
}
// And the move of a value sealed to this machine into the vault, asked by the controller (design 57's V3,
// issue 676): the vault is asked over this queue's link, so the queue is named once it exists.
queue.MoveSecret = moveSecretFrom(opts.state, queue)
// And this machine's sealing key stated to the vault in its own name, at start and whenever it changes (ADR 0372,
// issue 786), whether or not any value was ever moved in.
go keepNodeKeyStated(ctx, func() (identity.SealingKey, error) {
return identity.LoadSealingKey(identity.SealingKeyPath(opts.state))
}, func(verb string, args any) (json.RawMessage, error) {
return queue.AskVault(ctx, verb, args)
}, say, nodeKeyRetryFrom, nodeKeyRetryUpTo)
worker := make(chan struct{})
go func() {
defer close(worker)
@@ -1276,7 +1259,6 @@ func runLink(ctx context.Context, opts options) error {
queue.ReconcileDue()
}
go holdTheMachine(ctx, queue)
go busyPasses(ctx, queue, BusyAskEvery)
// And whether what it runs stays up, looked at on its own clock and said when it changes (novox/hq
// ADR 0240) — between applies, which is when a container crash-loops.
if judging != nil {
@@ -1296,9 +1278,6 @@ func runLink(ctx context.Context, opts options) error {
// And mesh-cli is answered on this machine: the account from the kernel, the line asked of the controller
// (novox/hq ADR 0272).
go serveMeshCLI(aside, queue, say)
// And a filled file is written on the provisioner client's request, one at a time with the apply (novox/hq ADR
// 0349 §4).
go serveFilling(aside, opts, mine.Membership.Signer, say)
held := link.HoldRoused(aside, membership, queue, say, opts.timeout, rousedBySignal(ctx))
// The act in hand finishes and is kept before this process exits: an apply that stood aside with
@@ -1418,18 +1397,6 @@ func adoptionFingerprint(r link.Report) string {
// it could only speak when a declaration arrived, and a declaration cannot be composed until it
// has spoken: a machine waiting for a push that is waiting for the machine.
parts := []string{"firewall=" + r.Firewall, "outward=" + strings.Join(r.Outward, ",")}
// **And the networks those links are on** (novox/hq ADR 0340 §7): a port reached from the local
// network is admitted from them, so a laptop that joins another network, or a lease that moves, is
// said at the next reconcile rather than at the next push or the next day.
if networks := attachedNetworksOf(r); networks != "" {
parts = append(parts, "networks="+networks)
}
// And a container waiting because it said busy, or replaced after it waited (novox/hq ADR 0335): the walk
// waiting on this machine starts its gate at the report that says it replaced. Not its output, which
// may change on every ask.
for _, b := range r.Busy {
parts = append(parts, "busy "+b.ID+"="+b.State)
}
for _, h := range r.Held {
parts = append(parts, "held "+h.ID+"="+h.Changed)
}
@@ -1461,26 +1428,6 @@ func adoptionFingerprint(r link.Report) string {
return strings.Join(parts, "\n")
}
// attachedNetworksOf is the networks a report's profile says the machine is attached to, as one line: the
// part of what the machine is made of that the filter is composed from (novox/hq ADR 0340).
func attachedNetworksOf(r link.Report) string {
machine, ok := r.Profile["machine"].(map[string]any)
if !ok {
return ""
}
networks, ok := machine["networks"].([]any)
if !ok {
return ""
}
said := make([]string, 0, len(networks))
for _, n := range networks {
if entry, ok := n.(map[string]any); ok {
said = append(said, fmt.Sprintf("%v %v", entry["link"], entry["network"]))
}
}
return strings.Join(said, ",")
}
// applyState is how an apply went, as the fingerprint keeps it: refused, clean, or failed with what it did not
// apply and which keys its failure is said under.
func applyState(r link.Report) string {
@@ -1666,10 +1613,10 @@ func declaredResolvConf(d *declaration.Declaration) (string, string, bool) {
}
for _, r := range d.Resources {
f, ok := r.(*declaration.File)
if !ok || f.Path != network.ResolvConf || f.Seeded() || f.Into != "" {
if !ok || f.Path != network.ResolvConf || f.CreateOnce || f.Into != "" {
continue
}
module, ok := d.ModuleOfID(f.ID)
module, ok := liveness.ModuleOf(f.ID)
if !ok {
return "", "", false
}
@@ -1788,13 +1735,8 @@ func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Queue, sa
func healthAsReported(st liveness.Statement, ns *network.Statement) *link.Health {
// LibraryContract: this engine reads a resource's declared `health` and judges it (ADR 0240 Phase B), judges a
// user's declared `root` (novox/hq ADR 0266), and reads a declaration's identities, groups and libraries
// (novox/hq ADR 0328), a container's busy (ADR 0335) and a file's owner-at-runtime and the filled files (ADR
// 0349 §5) and the provisioner client's account (ADR 0349 §4), which is what tells the controller it may be sent
// any of those fields; and it writes a filled file only on the client's word (HandedContract, design 57 P7),
// which is what tells it it may hand a moved module's files to the provisioner here; and it opens the vault's box
// for a core credential (VaultBoxContract, ADR 0349 §16), which is what tells it it may send that box here; and it states its sealing key to the vault (NodeKeyContract, ADR
// 0372, issue 786).
h := &link.Health{Contract: link.NodeKeyContract, At: st.At.UTC(), Resources: []link.ResourceHealth{}}
// (novox/hq ADR 0328), which is what tells the controller it may be sent any of those fields.
h := &link.Health{Contract: link.OwnGroupContract, At: st.At.UTC(), Resources: []link.ResourceHealth{}}
for _, r := range st.Resources {
h.Resources = append(h.Resources, link.ResourceHealth{Module: r.Module, Resource: r.ID, Kind: r.Kind,
Target: r.Target, State: r.State, Reason: r.Reason, Since: r.Since.UTC(), Streak: r.Streak,
@@ -1818,8 +1760,7 @@ func withUnits(h *link.Health, us *units.Statement) *link.Health {
if us == nil || us.State == "" {
return h
}
h.Units = &link.UnitsHealth{State: us.State, Failed: []link.FailedUnit{}, Unread: us.Unread,
Away: us.Away}
h.Units = &link.UnitsHealth{State: us.State, Failed: []link.FailedUnit{}, Unread: us.Unread}
for _, f := range us.Failed {
if f.Module == "" {
h.Units.Failed = append(h.Units.Failed, link.FailedUnit{Unit: f.Unit, Scope: f.Scope, Load: f.Load,
@@ -1927,32 +1868,6 @@ func holdTheMachine(ctx context.Context, queue *link.Queue) {
}
}
// busyWaiting is whether the last apply left a container running because it said busy (novox/hq ADR 0335).
var busyWaiting atomic.Bool
// BusyAskEvery is how often a container waiting because it said busy is asked again (novox/hq ADR 0335 §2):
// each ask is a pass of its own, so a build that ends is followed by its replacement within this, not
// within the reconcile's five minutes — and the bound of 15 minutes is met within this of its end.
const BusyAskEvery = 20 * time.Second
// busyPasses asks for a pass every so often while a container waits because it said busy. **It asks; it
// does not apply**, as holdTheMachine: the queue's worker does, when its turn comes, so a pass never holds
// the machine for the wait, and a delivery waiting is that pass's apply.
func busyPasses(ctx context.Context, queue *link.Queue, every time.Duration) {
ticker := time.NewTicker(every)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if busyWaiting.Load() {
queue.ReconcileDue()
}
}
}
}
// reconcileKept is the reconcile's act, as the queue's worker runs it: hold the machine to what it
// was last told, read once it is this act's turn, and say what did not go.
//
@@ -2004,7 +1919,7 @@ func worthSaying(report link.Report) bool {
}
return len(report.Held) > 0 || report.Firewall != "" || len(report.Outward) > 0 ||
len(report.Filters) > 0 || report.FoundFirewall != nil || len(report.Windows) > 0 ||
len(report.Rollbacks) > 0 || len(report.Busy) > 0
len(report.Rollbacks) > 0
}
// announceOr is what the apply writes its detail with, given what the caller has to say things with.
@@ -2120,14 +2035,6 @@ func applyAndKeepHeld(ctx context.Context, opts options, raw []byte, signed *sto
known.Mode = apply.ModeOf(declared)
}
// **A file given by digest is fetched and verified before anything of the declaration is applied** (novox/hq ADR
// 0486 §1, issue 1695): one object missing or differing refuses the declaration whole. The signed bytes are what is
// kept; only what is applied has the content in place of the reference.
if err := apply.ResolveFileReferences(ctx, declared, link.FilesFrom(ctx)); err != nil {
return link.Report{Declared: digestOf(raw), Order: order,
Refused: "a file of this declaration could not be had by its digest: " + err.Error()}
}
built, err := system.For(builtFor)
if err != nil {
return link.Report{Refused: err.Error()}
@@ -2261,20 +2168,6 @@ func applyAndKeepHeld(ctx context.Context, opts options, raw []byte, signed *sto
for _, w := range outcome.Windows {
report.Windows = append(report.Windows, link.Window{Step: w.Step, Holds: w.Holds, Since: w.Opened, Until: w.Until})
}
// And a container left running because it said busy, or replaced after it waited (novox/hq ADR 0335):
// the walk waiting on this machine reads it. While one waits, this machine asks it again sooner than its
// reconcile (busyPasses).
for _, b := range outcome.Busy {
report.Busy = append(report.Busy, link.Busy{ID: b.ID, Container: b.Container, State: b.State,
Since: b.Since, Until: b.Until, Said: b.Said})
}
for _, m := range outcome.MovedOut {
report.MovedOut = append(report.MovedOut, link.MovedOut{Resource: m.Resource, From: m.From, To: m.To})
}
for _, l := range outcome.Left {
report.Left = append(report.Left, link.Left{Resource: l.Resource, Path: l.Path})
}
busyWaiting.Store(outcome.Waiting())
// And what runs here that nobody asked for (novox/hq ADR 0163).
if strays, err := apply.Strays(ctx, apply.ExecRunner, updated); err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: applied, and could not list what else runs here: %v\n", err)
@@ -2328,17 +2221,7 @@ func applyAndKeepHeld(ctx context.Context, opts options, raw []byte, signed *sto
}
// Kept whichever way it went, so a node that is disconnected next minute still knows what it
// was told. Saved after applying rather than before: what is kept is what this node acted on.
var before []string
if signed != nil {
// The modules the declaration kept until now held, so a have key written for one the new declaration no longer
// holds is written removed even by a node-engine that started since (issue 1369).
if kept, keptErr := store.ReadDeclared(store.DeclaredPath(opts.state)); keptErr == nil {
if old, parseErr := declaration.ParseTrusted(kept.Declaration); parseErr == nil {
for _, f := range moduleFacts(old, apply.Report{}, nil) {
before = append(before, f.Module)
}
}
}
if err := store.SaveDeclared(store.DeclaredPath(opts.state), *signed); err != nil {
report.Failed = map[string]string{"keeping the declaration": err.Error()}
}
@@ -2347,10 +2230,6 @@ func applyAndKeepHeld(ctx context.Context, opts options, raw []byte, signed *sto
report.Failed = map[string]string{"apply": applyErr.Error()}
}
report.Failed = withUnanswered(report.Failed, probe)
// And how each module of the declaration came out, for the have keys written beside this report (novox/hq ADR 0429
// §4, issue 1132): not on the wire.
report.Modules = moduleFacts(declared, outcome, applyErr)
report.Before = before
// Which resources did not apply, for whether a reconcile's account is news (issue 233): not said on the wire.
report.Unapplied = unapplied(declared, outcome, applyErr)
return report
@@ -2401,7 +2280,7 @@ func sealOpener(statePath string) apply.Unseal {
if err != nil {
return nil, err
}
return key.Open(sealed)
return key.Unseal(sealed)
}
}
@@ -2603,29 +2482,3 @@ func unhealthyWords(r liveness.State) string {
return fmt.Sprintf("%s (%s %s) is unhealthy: %s, %d automatic restart(s) since this engine began watching it",
r.ID, r.Kind, r.Target, r.Reason, r.Restarts)
}
// moduleFacts is how each module of a declaration came out of an apply, with the build the controller sent for it
// (novox/hq ADR 0429 §4, issue 1132): sorted by module so the facts do not move between applies. A module an adopted
// node has not taken is held as found and not applied, so it is no fact.
func moduleFacts(d *declaration.Declaration, outcome apply.Report, applyErr error) []link.ModuleFact {
if d == nil {
return nil
}
results := apply.ModuleResults(d, outcome, applyErr)
facts := make([]link.ModuleFact, 0, len(results))
for module, result := range results {
if _, untaken := untakenModules(d)[module]; untaken {
continue
}
facts = append(facts, link.ModuleFact{Module: module, Build: d.Builds[module], Failed: result.Failed, Detail: result.Detail})
}
sort.Slice(facts, func(i, j int) bool { return facts[i].Module < facts[j].Module })
return facts
}
func untakenModules(d *declaration.Declaration) map[string][]string {
if d.Adoption == nil {
return nil
}
return d.Adoption.Untaken
}
cmd/mesh-host/main_test.go
+1 -1
View File
@@ -12,9 +12,9 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bundle"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
cmd/mesh-host/module_facts_test.go
-38
View File
@@ -1,38 +0,0 @@
package main
import (
"errors"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
)
// 1132/AC1, AC3: the report carries one fact per module the declaration names, with the build the controller sent for
// it; a module not yet taken on an adopted node is not one.
func TestTheReportCarriesOneFactPerModuleWithItsBuild(t *testing.T) {
d, err := declaration.Parse([]byte(`{"declaration":1,"builds":{"a":"b-1","b":"b-2"},"resources":[
{"id":"a.dir","type":"directory","path":"/srv/a","module":"a"},
{"id":"b.dir","type":"directory","path":"/srv/b","module":"b"}]}`))
if err != nil {
t.Fatal(err)
}
failure := &apply.Error{Resource: "b.dir", Err: errors.New("no space"), Failures: []apply.Failure{{Resource: "b.dir", Err: errors.New("no space")}}}
facts := moduleFacts(d, apply.Report{}, failure)
if len(facts) != 2 || facts[0].Module != "a" || facts[0].Build != "b-1" || facts[0].Failed ||
facts[1].Module != "b" || facts[1].Build != "b-2" || !facts[1].Failed || facts[1].Detail == "" {
t.Fatalf("facts %+v", facts)
}
adopted, err := declaration.Parse([]byte(`{"declaration":1,"adoption":{"taken":["a"],"untaken":{"b":["b.dir"]}},"resources":[
{"id":"a.dir","type":"directory","path":"/srv/a","module":"a"},
{"id":"b.dir","type":"directory","path":"/srv/b","module":"b"}]}`))
if err != nil {
t.Fatal(err)
}
if facts := moduleFacts(adopted, apply.Report{}, nil); len(facts) != 1 || facts[0].Module != "a" {
t.Fatalf("an untaken module is a fact: %+v", facts)
}
if moduleFacts(nil, apply.Report{}, nil) != nil {
t.Fatal("facts without a declaration")
}
}
cmd/mesh-host/move_out_test.go
-113
View File
@@ -1,113 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"os"
osuser "os/user"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
"github.com/novox/mesh-host/internal/apply"
)
// TestAMoveOutDirectoryKeepsOnlyWhatItKnows drives the node-engine whole through its command (issue 1024, novox/hq
// ADR 0422): a directory under an account's home that says move-out keeps the file the mesh declared and moves a
// file nobody declared into the kept folder, named in a list, and the run says it moved it. It needs no bus: the
// apply is the one a delivery and the five-minute reconcile both run.
func TestAMoveOutDirectoryKeepsOnlyWhatItKnows(t *testing.T) {
me, err := osuser.Current()
if err != nil {
t.Fatal(err)
}
// The user database lists an account as a person's only at uid 1000 or more; root, as the forge's merge check
// runs, is listed under a number of its own (it can give files to it), by the same name.
listedUID := me.Uid
if uid, _ := strconv.Atoi(me.Uid); uid < 1000 || uid == 65534 {
listedUID = "4242"
}
root := t.TempDir()
home := filepath.Join(root, "home", "agent")
dir := filepath.Join(home, ".ssh")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
passwd := filepath.Join(root, "passwd")
line := fmt.Sprintf("%s:x:%s:%s::%s:/bin/sh\n", me.Username, listedUID, me.Gid, home)
if err := os.WriteFile(passwd, []byte(line), 0o644); err != nil {
t.Fatal(err)
}
t.Cleanup(apply.UseAccountsFileForTest(passwd))
// Independent of the machine: an arch host with a pacman on PATH that answers and holds nothing, as
// TestReplay233 puts one there (it answers with an error); nothing here declares a package.
bin := t.TempDir()
if err := os.WriteFile(filepath.Join(bin, "pacman"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
wasBuilt := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = wasBuilt })
stray := filepath.Join(dir, "id_stray")
if err := os.WriteFile(stray, []byte("somebody else's key"), 0o600); err != nil {
t.Fatal(err)
}
old := time.Now().Add(-time.Hour)
if err := os.Chtimes(stray, old, old); err != nil {
t.Fatal(err)
}
pipe := filepath.Join(dir, "agent.sock")
if err := syscall.Mkfifo(pipe, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(pipe, old, old); err != nil {
t.Fatal(err)
}
opts := options{state: filepath.Join(root, "state.json")}
raw := []byte(`{"declaration":1,"resources":[` +
`{"id":"ssh.dir","type":"directory","path":"` + dir + `","owner":"` + me.Username + `","unknown":"move-out"},` +
`{"id":"ssh.config","type":"file","path":"` + dir + `/config","content":"Host *\n","owner":"` + me.Username + `"}]}`)
var said []string
report := applyAndKeep(context.Background(), opts, raw, nil, nil, func(s string) { said = append(said, s) })
if report.Refused != "" || len(report.Failed) > 0 {
t.Fatalf("refused or failed: %q %v", report.Refused, report.Failed)
}
if _, err := os.Stat(filepath.Join(dir, "config")); err != nil {
t.Errorf("the declared file is not there: %v", err)
}
if _, err := os.Lstat(stray); err == nil {
t.Error("the file nobody declared stayed in the directory")
}
lists, _ := filepath.Glob(filepath.Join(home, ".local", "state", "mesh-moved", "ssh", "*.json"))
if len(lists) != 1 {
t.Fatalf("one list expected: %v", lists)
}
rawList, err := os.ReadFile(lists[0])
if err != nil {
t.Fatal(err)
}
var entries []struct{ From, To string }
if err := json.Unmarshal(rawList, &entries); err != nil || len(entries) != 1 {
t.Fatalf("the list: %s %v", rawList, err)
}
if got, err := os.ReadFile(entries[0].To); err != nil || string(got) != "somebody else's key" {
t.Errorf("the moved file is not whole in the kept folder: %q %v", got, err)
}
if len(report.MovedOut) != 1 || report.MovedOut[0].From != stray || report.MovedOut[0].To != entries[0].To {
t.Errorf("the report does not carry the move: %+v", report.MovedOut)
}
if len(report.Left) != 1 || report.Left[0].Path != pipe {
t.Errorf("the report does not list the pipe left in place: %+v", report.Left)
}
if !strings.Contains(strings.Join(said, "\n"), "moved "+stray) {
t.Errorf("the run does not say it moved the file: %v", said)
}
}
cmd/mesh-host/movesecret.go
-131
View File
@@ -1,131 +0,0 @@
package main
import (
"context"
"crypto/rand"
"crypto/subtle"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"os"
"time"
"golang.org/x/crypto/nacl/box"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// The move of a value sealed to this machine into the vault (design 57's V3, issue 676; internal/link
// movesecret.go). The controller cannot open what it holds; this machine can open what was sealed to it, and so
// hands it to the vault, sealed to a key the vault made for the call. The value is in this process for that one
// call and written nowhere.
// vaultAsk asks one of the vault's verbs in this engine's own name and answers its result.
type vaultAsk func(verb string, args any) (json.RawMessage, error)
// moveSecretFrom is the engine's move: this machine's sealing key, read for each ask, and the vault asked over
// the link open now.
func moveSecretFrom(statePath string, queue *link.Queue) link.MoveSecretFunc {
return func(m link.MoveSecret) (link.MovedSecret, error) {
key, err := identity.LoadSealingKey(identity.SealingKeyPath(statePath))
if err != nil {
return link.MovedSecret{}, err
}
ask := func(verb string, args any) (json.RawMessage, error) {
return queue.AskVault(context.Background(), verb, args)
}
moved, err := moveIntoTheVault(ask, key, m)
if err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: a value was not moved into the vault: %v\n", err)
}
return moved, err
}
}
// moveIntoTheVault opens the row's value with this machine's sealing key, hands it to the vault's move-in in its two
// steps, and answers what the vault kept once its read-back — sealed to a key made here for this one call — opens
// to the same value.
func moveIntoTheVault(ask vaultAsk, key identity.SealingKey, m link.MoveSecret) (link.MovedSecret, error) {
if m.SealedTo != "" && m.SealedTo != key.Public {
return link.MovedSecret{}, errors.New("the controller's store says this value was sealed to a sealing key this " +
"machine no longer has, so it cannot be opened here")
}
value, err := key.Unseal(m.Sealed)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the value does not open with this machine's sealing key: %w", err)
}
defer wipe(value)
// This machine's sealing key, stated in its own name: the key the vault seals the core's credentials to for the
// controller to carry here (novox/hq ADR 0349 §16, issue 763), never a key the controller names.
first := map[string]any{"origin": m.Origin, "generation": m.Generation, "sealing_key": key.Public}
if !m.MadeAt.IsZero() {
first["made_at"] = m.MadeAt.UTC().Format(time.RFC3339Nano)
}
if m.Own() {
first["node"], first["module"], first["name"] = m.Node, m.Module, m.Name
} else {
first["provider"], first["provision"] = m.Provider, m.Provision
first["consumer"], first["consumer_module"] = m.Consumer, m.ConsumerModule
if m.Local != "" {
first["local"] = m.Local
}
}
raw, err := ask("move-in", first)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault did not open the move: %w", err)
}
var opened struct {
Call string `json:"call"`
SealTo string `json:"seal_to"`
Secret string `json:"secret"`
}
if err := json.Unmarshal(raw, &opened); err != nil || opened.Call == "" || opened.SealTo == "" {
return link.MovedSecret{}, errors.New("the vault answered no call and no key to seal to")
}
sealed, err := identity.Seal(opened.SealTo, value)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault's key for the call is not a sealing key: %w", err)
}
backPub, backPriv, err := box.GenerateKey(rand.Reader)
if err != nil {
return link.MovedSecret{}, err
}
defer wipe(backPriv[:])
raw, err = ask("move-in", map[string]any{"call": opened.Call, "sealed": sealed,
"read_back_to": base64.StdEncoding.EncodeToString(backPub[:])})
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault did not take %s: %w", opened.Secret, err)
}
var kept struct {
Name string `json:"name"`
Generation int `json:"generation"`
Unchanged bool `json:"unchanged"`
ReadBack string `json:"read_back"`
}
if err := json.Unmarshal(raw, &kept); err != nil || kept.Name == "" || kept.ReadBack == "" {
return link.MovedSecret{}, fmt.Errorf("the vault answered %s with no read-back", opened.Secret)
}
blob, err := base64.StdEncoding.DecodeString(kept.ReadBack)
if err != nil {
return link.MovedSecret{}, fmt.Errorf("the vault's read-back of %s is not sealed", kept.Name)
}
back, ok := box.OpenAnonymous(nil, blob, backPub, backPriv)
if !ok {
return link.MovedSecret{}, fmt.Errorf("the vault's read-back of %s does not open with the key made for it", kept.Name)
}
defer wipe(back)
if subtle.ConstantTimeCompare(back, value) != 1 {
return link.MovedSecret{}, fmt.Errorf("the vault keeps %s, but it read back another value than the one handed "+
"in: the move of it did not hold", kept.Name)
}
return link.MovedSecret{Secret: kept.Name, Generation: kept.Generation, Unchanged: kept.Unchanged}, nil
}
func wipe(b []byte) {
for i := range b {
b[i] = 0
}
}
cmd/mesh-host/movesecret_test.go
-134
View File
@@ -1,134 +0,0 @@
package main
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"testing"
"golang.org/x/crypto/nacl/box"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// standInVault answers move-in's two steps as the vault does: a key made for the call, the value opened with it,
// and the value kept read back sealed to the key the second step sent. lie makes its read-back another value.
type standInVault struct {
t *testing.T
pub *[32]byte
priv *[32]byte
opened map[string]any
kept string
lie string
refuse string
carried []string
}
func (v *standInVault) ask(verb string, args any) (json.RawMessage, error) {
raw, _ := json.Marshal(args)
v.carried = append(v.carried, string(raw))
if verb != "move-in" {
v.t.Fatalf("asked %s", verb)
}
if v.refuse != "" {
return nil, errors.New(v.refuse)
}
var a map[string]any
_ = json.Unmarshal(raw, &a)
if a["call"] == nil {
v.opened = a
pub, priv, _ := box.GenerateKey(rand.Reader)
v.pub, v.priv = pub, priv
return json.Marshal(map[string]string{"call": "c1", "seal_to": base64.StdEncoding.EncodeToString(pub[:]),
"secret": "own/anchor/postgres/superuser-password"})
}
blob, _ := base64.StdEncoding.DecodeString(a["sealed"].(string))
value, ok := box.OpenAnonymous(nil, blob, v.pub, v.priv)
if !ok {
v.t.Fatal("what the engine sealed does not open with the call's key")
}
v.kept = string(value)
back := v.kept
if v.lie != "" {
back = v.lie
}
key, _ := base64.StdEncoding.DecodeString(a["read_back_to"].(string))
var to [32]byte
copy(to[:], key)
boxed, _ := box.SealAnonymous(nil, []byte(back), &to, rand.Reader)
return json.Marshal(map[string]any{"name": "own/anchor/postgres/superuser-password", "generation": 1,
"read_back": base64.StdEncoding.EncodeToString(boxed)})
}
const theValue = "the-superusers-password"
func aMoveOf(t *testing.T, key identity.SealingKey) link.MoveSecret {
t.Helper()
sealed, err := identity.Seal(key.Public, []byte(theValue))
if err != nil {
t.Fatal(err)
}
return link.MoveSecret{Node: "anchor", By: "the move", MoveSecretAsk: link.MoveSecretAsk{Module: "postgres",
Name: "superuser-password", Origin: "made", Generation: "g-1", Sealed: sealed, SealedTo: key.Public}}
}
// The engine opens what was sealed to its machine, hands it to the vault sealed to the call's key — never in the
// clear — and answers what the vault kept only once the read-back opens to the same value.
func TestTheEngineMovesAValueSealedToItAndChecksTheReadBack(t *testing.T) {
key, err := identity.GenerateSealingKey()
if err != nil {
t.Fatal(err)
}
vault := &standInVault{t: t}
moved, err := moveIntoTheVault(vault.ask, key, aMoveOf(t, key))
if err != nil {
t.Fatal(err)
}
if moved.Secret != "own/anchor/postgres/superuser-password" || moved.Generation != 1 || vault.kept != theValue {
t.Fatalf("moved %+v, the vault kept %q", moved, vault.kept)
}
if vault.opened["module"] != "postgres" || vault.opened["name"] != "superuser-password" || vault.opened["node"] != "anchor" ||
vault.opened["origin"] != "made" || vault.opened["generation"] != "g-1" ||
vault.opened["sealing_key"] != key.Public {
t.Errorf("the first step named %v", vault.opened)
}
if strings.Contains(strings.Join(vault.carried, " "), theValue) {
t.Fatal("the value crossed the bus in the clear")
}
}
// A read-back that is not the value handed in is a failed move, said; so is a value this machine cannot open, and
// the vault's refusal.
func TestTheEngineSaysAMoveThatDidNotHold(t *testing.T) {
key, _ := identity.GenerateSealingKey()
vault := &standInVault{t: t, lie: "something-else"}
if _, err := moveIntoTheVault(vault.ask, key, aMoveOf(t, key)); err == nil ||
!strings.Contains(err.Error(), "read back") {
t.Fatalf("a read-back of another value: %v", err)
}
other, _ := identity.GenerateSealingKey()
if _, err := moveIntoTheVault((&standInVault{t: t}).ask, key, aMoveOf(t, other)); err == nil ||
!strings.Contains(err.Error(), "sealing key") {
t.Fatalf("a value sealed to another key: %v", err)
}
refusing := &standInVault{t: t, refuse: "the move never replaces a value the vault holds"}
if _, err := moveIntoTheVault(refusing.ask, key, aMoveOf(t, key)); err == nil ||
!strings.Contains(err.Error(), "never replaces") {
t.Fatalf("the vault's refusal: %v", err)
}
// A pair credential names both its ends.
pair := aMoveOf(t, key)
pair.Module, pair.Name = "", ""
pair.Provider, pair.Provision, pair.Consumer, pair.ConsumerModule = "anchor", "database", "shanks", "gitea"
ends := &standInVault{t: t}
if _, err := moveIntoTheVault(ends.ask, key, pair); err != nil {
t.Fatal(err)
}
if ends.opened["provider"] != "anchor" || ends.opened["consumer"] != "shanks" || ends.opened["consumer_module"] != "gitea" ||
ends.opened["node"] != nil {
t.Errorf("the pair's first step named %v", ends.opened)
}
}
cmd/mesh-host/network_test.go
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/network"
)
cmd/mesh-host/nodekey.go
-62
View File
@@ -1,62 +0,0 @@
package main
import (
"context"
"fmt"
"time"
"github.com/novox/mesh-host/internal/identity"
)
// This machine's sealing key, stated to the vault in the engine's own name (novox/hq ADR 0372, issue 786). The vault
// records it from the machine's own account alone and, when it is a first or a changed key, says the machine's core
// secrets changed so the controller asks for them again sealed to it. Without this a machine that never moved a
// secret in (adopted after feature 57) has no key at the vault. Nothing but the public half crosses.
const (
// nodeKeyRetryFrom and nodeKeyRetryUpTo bound the wait between tries while the vault does not answer: it starts
// after the bus and may not serve the verb yet.
nodeKeyRetryFrom = 5 * time.Second
nodeKeyRetryUpTo = 5 * time.Minute
// nodeKeyLook is how often the key file is read to see whether it changed.
nodeKeyLook = 30 * time.Second
)
// keepNodeKeyStated states the sealing key load answers to the vault at start, retrying with a growing wait from
// retryFrom to retryUpTo while it does not answer, and states it again whenever the key changes. It says one line
// when a key is stated and one when a try fails for another reason than the last, and returns when ctx ends.
func keepNodeKeyStated(ctx context.Context, load func() (identity.SealingKey, error), ask vaultAsk, say func(string),
retryFrom, retryUpTo time.Duration) {
stated, lastFailure := "", ""
wait := retryFrom
fail := func(why string) {
if why != lastFailure {
say("mesh-host: the sealing key was not stated to the vault, trying again: " + why)
lastFailure = why
}
wait = min(wait*2, retryUpTo)
}
look := min(nodeKeyLook, retryFrom)
for {
delay := look
key, err := load()
switch {
case err != nil:
fail(err.Error())
delay = wait
case key.Public != stated:
if _, err := ask("node-key", map[string]string{"key": key.Public}); err != nil {
fail(err.Error())
delay = wait
} else {
stated, lastFailure, wait = key.Public, "", retryFrom
say(fmt.Sprintf("mesh-host: this machine's sealing key %s is stated to the vault", key.Public))
}
}
select {
case <-ctx.Done():
return
case <-time.After(delay):
}
}
}
cmd/mesh-host/nodekey_test.go
-112
View File
@@ -1,112 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"sync"
"testing"
"time"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
)
// Issue 786 (novox/hq ADR 0372): the engine states its own sealing key to the vault's node-key at start, retries
// while the vault does not answer, and states it again when the key changes; the same key is not stated twice.
type keyVault struct {
mu sync.Mutex
stated []string
fail int
verbs []string
}
func (v *keyVault) ask(verb string, args any) (json.RawMessage, error) {
v.mu.Lock()
defer v.mu.Unlock()
v.verbs = append(v.verbs, verb)
if v.fail > 0 {
v.fail--
return nil, errors.New("the vault does not answer")
}
raw, _ := json.Marshal(args)
var a struct {
Key string `json:"key"`
}
_ = json.Unmarshal(raw, &a)
v.stated = append(v.stated, a.Key)
return json.RawMessage(`{"recorded":true}`), nil
}
func (v *keyVault) said() []string {
v.mu.Lock()
defer v.mu.Unlock()
return append([]string(nil), v.stated...)
}
func waitFor(t *testing.T, what string, ok func() bool) {
t.Helper()
for i := 0; i < 400; i++ {
if ok() {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("timed out: %s", what)
}
func TestTheEngineStatesItsSealingKeyToTheVaultAtStartRetriesAndRestatesAChange(t *testing.T) {
first, _ := identity.GenerateSealingKey()
second, _ := identity.GenerateSealingKey()
var mu sync.Mutex
current := first
load := func() (identity.SealingKey, error) {
mu.Lock()
defer mu.Unlock()
return current, nil
}
vault := &keyVault{fail: 2}
var lines []string
var lmu sync.Mutex
say := func(l string) { lmu.Lock(); lines = append(lines, l); lmu.Unlock() }
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go keepNodeKeyStated(ctx, load, vault.ask, say, time.Millisecond, 5*time.Millisecond)
waitFor(t, "the first key stated after two refusals", func() bool { return len(vault.said()) == 1 })
if vault.said()[0] != first.Public {
t.Errorf("stated %q, want the engine's own key %q", vault.said()[0], first.Public)
}
vault.mu.Lock()
verbs := append([]string(nil), vault.verbs...)
vault.mu.Unlock()
for _, v := range verbs {
if v != "node-key" {
t.Errorf("asked %s", v)
}
}
// The same key again is not stated again.
time.Sleep(60 * time.Millisecond)
if n := len(vault.said()); n != 1 {
t.Errorf("the same key was stated %d times", n)
}
mu.Lock()
current = second
mu.Unlock()
waitFor(t, "the changed key stated", func() bool { return len(vault.said()) == 2 })
if vault.said()[1] != second.Public {
t.Errorf("stated %q after the change, want %q", vault.said()[1], second.Public)
}
lmu.Lock()
defer lmu.Unlock()
if len(lines) == 0 {
t.Error("the engine said nothing")
}
}
// The statement a host makes names this contract (issue 786), one above the vault's box.
func TestTheNodeKeyContractIsThirteen(t *testing.T) {
if link.NodeKeyContract != 13 {
t.Errorf("NodeKeyContract %d", link.NodeKeyContract)
}
}
cmd/mesh-host/probe.go
+1 -1
View File
@@ -6,8 +6,8 @@ import (
"os/exec"
"sort"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
cmd/mesh-host/probe_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/system"
)
cmd/mesh-host/replay233_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
cmd/mesh-host/units_test.go
+5 -8
View File
@@ -5,8 +5,8 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/units"
@@ -87,13 +87,10 @@ func TestTheStatementSaysAnAccountJudgedForRoot(t *testing.T) {
Groups: []string{"openrazer"}}, State: accounts.Healthy, Since: at},
}}
h := withAccounts(healthAsReported(liveness.Statement{At: at}, nil), as)
// At RootContract or later: an engine that states its sealing key to the vault says NodeKeyContract (ADR 0372, issue 786), one above
// VaultBoxContract (it opens the vault's box for the core's credentials, ADR 0349 §16, issue 755), one above HandedContract (design 57 P7), one above ProvisionerContract (novox/hq ADR
// 0349 §4), one above ADR 0349 §5's OwnershipContract.
if h.Contract < link.RootContract || link.RootContract != 3 || h.Contract != link.NodeKeyContract ||
link.NodeKeyContract != link.VaultBoxContract+1 || link.VaultBoxContract != link.HandedContract+1 ||
link.HandedContract != link.ProvisionerContract+1 || link.ProvisionerContract != link.OwnershipContract+1 || link.OwnershipContract != link.BusyContract+1 ||
link.BusyContract != link.OwnGroupContract+1 {
// At RootContract or later: an engine that also reads an identity's runner says OwnGroupContract (novox/hq ADR
// 0334), one above ADR 0328's LibraryContract.
if h.Contract < link.RootContract || link.RootContract != 3 || h.Contract != link.OwnGroupContract ||
link.OwnGroupContract != link.LibraryContract+1 {
t.Fatalf("contract %d", h.Contract)
}
got := map[string]string{}
cmd/mesh-host/vaultbox_nats_test.go
-154
View File
@@ -1,154 +0,0 @@
package main
// Issue 755's proof on a real bus (novox/hq ADR 0349 §16): a core credential the vault sealed to this machine crosses
// the bus inside the declaration, which the controller carries and cannot open, and this engine opens it with its
// sealing key and writes the value. A listener on every subject keeps every byte that crossed; the value is never
// among them. Run against the throwaway bus merge-check.sh starts (MESH_TEST_NATS).
import (
"context"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
const coreCredential = "the-provisioner-clients-bus-password-2025"
func TestNatsTheEngineFillsACoreCredentialFromTheVaultsBoxWithNoValueInTheClear(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
// This machine's sealing key, where the engine reads it.
state := filepath.Join(t.TempDir(), "state.json")
key, err := identity.GenerateSealingKey()
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(identity.SealingKeyPath(state), []byte(key.Private), 0o600); err != nil {
t.Fatal(err)
}
// Every byte that crosses.
tap, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer tap.Close()
var mu sync.Mutex
var crossed []string
if _, err := tap.Subscribe(">", func(m *nats.Msg) {
mu.Lock()
crossed = append(crossed, m.Subject+" "+string(m.Data))
mu.Unlock()
}); err != nil {
t.Fatal(err)
}
_ = tap.Flush()
// The engine hears its declaration on its own subject.
const node = "vaultbox"
engine, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer engine.Close()
heard, err := engine.SubscribeSync(link.DeclareSubject(node))
if err != nil {
t.Fatal(err)
}
_ = engine.Flush()
// The vault's answer to the controller: the value in the ADR 0206 box, sealed to this machine's key. The
// controller carries it in the declaration as it is.
pub, err := key.PublicPEM()
if err != nil {
t.Fatal(err)
}
box, err := seal.Seal(coreCredential, pub)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "broker")
body, _ := json.Marshal(map[string]any{"declaration": 1, "resources": []map[string]any{
{"id": "need-provisioner-broker", "type": "file", "path": path, "vault-sealed": box}}})
controller, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer controller.Close()
if err := controller.Publish(link.DeclareSubject(node), body); err != nil {
t.Fatal(err)
}
_ = controller.Flush()
msg, err := heard.NextMsg(5 * time.Second)
if err != nil {
t.Fatalf("the declaration did not reach the engine: %v", err)
}
d, err := declaration.Parse(msg.Data)
if err != nil {
t.Fatal(err)
}
sys, err := system.For("arch")
if err != nil {
t.Fatal(err)
}
if _, _, err := apply.Apply(context.Background(), sys, d, store.State{}, store.OriginCarried,
func(context.Context, string, ...string) (string, error) { return "", nil }, nil, sealOpener(state)); err != nil {
t.Fatal(err)
}
on, err := os.ReadFile(path)
if err != nil || string(on) != coreCredential {
t.Fatalf("the engine wrote %q, %v", on, err)
}
_ = tap.Flush()
time.Sleep(100 * time.Millisecond)
mu.Lock()
all := strings.Join(crossed, "\n")
mu.Unlock()
if !strings.Contains(all, link.DeclareSubject(node)) || !strings.Contains(all, "vault-sealed") {
t.Fatalf("the listener heard nothing of the declaration:\n%s", all)
}
for _, form := range []string{coreCredential, base64.StdEncoding.EncodeToString([]byte(coreCredential))} {
if strings.Contains(all, form) {
t.Fatal("the core credential crossed the bus in the clear")
}
}
}
// The engine opens a value the controller sealed (the NaCl box) as before, and the vault's box, with one key.
func TestTheEnginesOpenerOpensBothBoxes(t *testing.T) {
state := filepath.Join(t.TempDir(), "state.json")
key, _ := identity.GenerateSealingKey()
if err := os.WriteFile(identity.SealingKeyPath(state), []byte(key.Private), 0o600); err != nil {
t.Fatal(err)
}
open := sealOpener(state)
nacl, _ := identity.Seal(key.Public, []byte("old"))
if got, err := open(nacl); err != nil || string(got) != "old" {
t.Errorf("the controller's box opened to %q, %v", got, err)
}
pub, _ := key.PublicPEM()
box, _ := seal.Seal("new", pub)
raw, _ := json.Marshal(box)
if got, err := open(string(raw)); err != nil || string(got) != "new" {
t.Errorf("the vault's box opened to %q, %v", got, err)
}
}
examples/bundle_test.go
+2 -2
View File
@@ -12,7 +12,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
func bundle(t *testing.T) *declaration.Declaration {
@@ -49,7 +49,7 @@ func TestTheBundleCarriesTheFoundationAndTheControlPlaneAndNothingElse(t *testin
}
sort.Strings(images)
want := []string{"mesh-controller", "nats", "pgvector"}
want := []string{"mesh-controller", "nats", "postgres"}
if strings.Join(images, ",") != strings.Join(want, ",") {
t.Fatalf("the bundle carries %v; expected exactly %v.\n\n"+
"Adding one is a change to what every first node becomes, and to ADR 0006's "+
examples/foundation-first-node.lock
+19 -34
View File
@@ -28,12 +28,11 @@
// image, and this file is applied on a machine with no mesh to ask about anything (ADR 0006). The
// registry prefix is the lab's; the digests are upstream's.
//
// **The store is raised where the `postgres` module keeps it** (novox/hq ADR 0338): the container
// `postgres`, on the module's pgvector image, its data in the directory /var/lib/mesh-store owned by
// the image's postgres user, its superuser password read from the module's own-secret file — so the
// store step adopts it as it runs. Pinned upstream by the module's own digest, not through the lab's
// registry prefix, because the two must be the same reference (mesh-host internal/bootstrap holds
// them to each other against the module's manifest).
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
// creates is owned by root, and the database runs as somebody else inside the container — so it
// could not write, and the container crash-looped. A named volume lets the image set up its own
// ownership, and outlives the container, which is what you want for the thing holding the mesh's
// state.
{
"declaration": 1,
@@ -84,28 +83,17 @@
"boot": "enabled",
"restart-on": ["base-filter"]
},
// The store's data is a directory owned by the image's postgres user (999), made before the
// container starts: a directory the runtime made would be root's, and the database could not
// write it. Measured on the module's image 2026-10-13: it initialises a new store on a directory
// made so (novox/hq ADR 0338 §5).
{
"id": "store-data",
"type": "directory",
"path": "/var/lib/mesh-store",
"mode": "0700",
"owner": "999:70"
},
{
"id": "store",
"type": "container",
"name": "postgres",
"image": "pgvector/pgvector@sha256:cf134a767f474095eeba57e0117be8e568e011a63f33fbf252f14c9b760f8e6f",
"name": "mesh-store",
"image": "192.0.2.250:5000/postgres@sha256:7abf537131b66ed5af448d90653abf1679b0c7e9a1f07efdd4c3108a401b259a",
"env": {
"POSTGRES_PASSWORD": "bootstrap",
"PGDATA": "/var/lib/postgresql/data/pgdata"
},
"ports": ["5432:5432"],
"volumes": ["/var/lib/mesh-store:/var/lib/postgresql/data"]
"volumes": ["mesh-store-data:/var/lib/postgresql/data"]
},
// Over TCP, not the socket. While the store initialises it runs a temporary server on the
// socket ONLY, then stops it and starts the real one — so a socket check passes, the action
@@ -114,21 +102,21 @@
{
"id": "store-ready",
"type": "action",
"in": "postgres",
"in": "mesh-store",
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -h 127.0.0.1 -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -h 127.0.0.1 -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
"verify": ["pg_isready", "-h", "127.0.0.1", "-U", "postgres"]
},
{
"id": "inventory-database",
"type": "action",
"in": "postgres",
"in": "mesh-store",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE inventory'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw inventory"]
},
{
"id": "identity-database",
"type": "action",
"in": "postgres",
"in": "mesh-store",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE identity'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw identity"]
},
@@ -138,20 +126,20 @@
{
"id": "licences-database",
"type": "action",
"in": "postgres",
"in": "mesh-store",
"command": ["sh", "-c", "psql -U postgres -c 'CREATE DATABASE licences'"],
"verify": ["sh", "-c", "psql -U postgres -lqt | cut -d'|' -f1 | grep -qw licences"]
},
{
"id": "context-schemas",
"type": "action",
"command": ["docker", "run", "--rm", "--network", "container:postgres",
"command": ["docker", "run", "--rm", "--network", "container:mesh-store",
"-e", "MESH_STORE_INVENTORY=postgres://postgres:bootstrap@127.0.0.1:5432/inventory?sslmode=disable",
"-e", "MESH_STORE_IDENTITY=postgres://postgres:bootstrap@127.0.0.1:5432/identity?sslmode=disable",
"-e", "MESH_STORE_LICENCES=postgres://postgres:bootstrap@127.0.0.1:5432/licences?sslmode=disable",
"192.0.2.250:5000/mesh-controller@sha256:c67db38439ff0aee242b467486765467bb95801f52175fc5727cc4e437338ace",
"migrate"],
"verify": ["sh", "-c", "docker exec postgres psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec postgres psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec postgres psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
"verify": ["sh", "-c", "docker exec mesh-store psql -U postgres -d inventory -tAc \"select to_regclass('public.node')\" | grep -qx node && docker exec mesh-store psql -U postgres -d identity -tAc \"select to_regclass('public.signing_key')\" | grep -qx signing_key && docker exec mesh-store psql -U postgres -d licences -tAc \"select to_regclass('public.licence')\" | grep -qx licence"]
},
// **The bus is raised where the `nats` module keeps it** (novox/hq ADR 0333 §1): the container
// `mesh-broker-nats`, its configuration and user list in /var/lib/nats-module/conf, its streams in
@@ -221,20 +209,17 @@
"path": "/var/lib/nats-module/conf/accounts.conf",
"mode": "0600",
"create-once": true,
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_assignments.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_declared-config.>\", \"$KV.mesh-controller_lease.>\", \"$KV.mesh-controller_predefined.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.plan-moved\", \"mesh.seat.mesh-controller.event.assignment-changed\", \"mesh.seat.mesh-controller.event.declared-config-changed\", \"mesh.seat.mesh-controller.event.predefined-changed\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.operator-channel.tool.tell\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.ACK.DEAD_LETTER_NOTICES.controller.>\", \"$JS.API.>\", \"$KV.mesh-controller_asked.>\", \"$KV.mesh-controller_calls.>\", \"$KV.mesh-controller_hand-acts.>\", \"$KV.mesh-controller_conditions.>\", \"$KV.mesh-controller_condition-history.>\", \"$KV.mesh-controller_lease.>\", \"$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>\", \"$KV.SEAT_NODE_BUILD_AGENT_cancelled.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.node-build-agent.accept.>\", \"mesh.seat.mesh-build-machine.tool.>\", \"mesh.seat.node-build-agent.tool.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\", \"mesh.seat.mesh-controller.event.rolled-back\", \"mesh.seat.mesh-controller.event.checked\", \"mesh.seat.mesh-controller.event.plan-moved\", \"mesh.seat.mesh-controller.event.condition-raised\", \"mesh.seat.mesh-controller.event.condition-changed\", \"mesh.seat.mesh-controller.event.condition-cleared\", \"mesh.seat.mesh-controller.event.doctor-heartbeat\", \"mesh.seat.mesh-controller.event.secret-replaced\", \"mesh.seat.mesh-controller.event.healer-acted\", \"$SRV.INFO\", \"mesh.seat.node-intrusion-prevention.tool.banned.*\", \"mesh.seat.node-backup.tool.backed-up.*\", \"mesh.seat.node-backup.tool.now.*\", \"mesh.seat.mesh-delivery.tool.stalled\", \"mesh.seat.mesh-delivery.tool.close\", \"mesh.seat.node-launcher.tool.secret.*\", \"mesh.seat.mesh-delivery.tool.release\", \"mesh.seat.mesh-delivery.tool.stop\", \"mesh.seat.mesh-delivery.tool.times\", \"mesh.seat.mesh-delivery.tool.groups\", \"mesh.seat.node-service-manager.tool.restart.*\", \"mesh.seat.mesh-controller.tool.plans\", \"mesh.events.dead.>\", \"mesh.again.>\"] }\n subscribe: { allow: [\"$JS.API.>\", \"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>\", \"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.*.event.provisioner.failing\", \"mesh.mod.*.event.provisioner.recovered\", \"mesh.mod.*.event.provisioner.retirement\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.gitea.event.pull.updated\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.node-build-agent.event.built\", \"mesh.seat.operator-channel.event.decided.mesh-controller\", \"mesh.seat.mesh-controller.tool.>\", \"$SRV.PING\", \"$SRV.INFO\", \"$SRV.PING.mesh-controller\", \"$SRV.PING.mesh-controller.>\", \"$SRV.INFO.mesh-controller\", \"$SRV.INFO.mesh-controller.>\", \"$SRV.STATS\", \"$SRV.STATS.mesh-controller\", \"$SRV.STATS.mesh-controller.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
},
// The upstream server, until the broker step adopts it: the `nats` module's own image, built by the
// mesh, takes its place there once, on these same directories (ADR 0333 §3), as the bus's planned
// step (ADR 0236). The release is the one the module's image is built on (its NATS_BASE, 2.11.17),
// pinned upstream by that same digest: the adoption changes no server release, so it can be undone
// by putting the old image back and is said reversible. The published port is the
// mesh, takes its place there once, on these same directories (ADR 0333 §3). The published port is the
// node's bus port, 5671 unless given, which every enrolment token names; the installer gives the
// module the setting that maps its declared 4222 to it (ADR 0333 §2).
{
"id": "broker",
"type": "container",
"name": "mesh-broker-nats",
"image": "nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00",
"image": "192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927",
"ports": ["5671:4222", "127.0.0.1:8222:8222"],
"volumes": ["/var/lib/mesh-broker-nats:/data", "/var/lib/nats-module/conf:/etc/nats:ro", "/var/lib/mesh-broker-tls:/tls:ro"],
"args": ["-c", "/etc/nats/nats.conf", "-js"]
@@ -242,8 +227,8 @@
{
"id": "broker-ready",
"type": "action",
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh nats@sha256:e4bf19f15fd3218814a4e3c9e0064e1334bd8aa20d5984b9f1a0afd084f8cc00 -c 'nc -z 127.0.0.1 5671'"]
"command": ["sh", "-c", "for i in $(seq 1 60); do docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671' >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
"verify": ["sh", "-c", "docker run --rm --network host --entrypoint sh 192.0.2.250:5000/nats@sha256:b83efabe3e7def1e0a4a31ec6e078999bb17c80363f881df35edc70fcb6bb927 -c 'nc -z 127.0.0.1 5671'"]
},
{
"id": "control-plane",
go.mod
+1 -1
View File
@@ -3,7 +3,7 @@ module github.com/novox/mesh-host
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021153908-725590e06e75
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c
github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0
golang.org/x/sys v0.48.0
go.sum
+6 -2
View File
@@ -1,5 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021153908-725590e06e75 h1:AcYfyJpYaUufhI93VG76+b001msXY4XX7kuRjnf33HY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021153908-725590e06e75/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970 h1:9tFDQsgmI+4X7/BpZGXIr+HemPKE7YddYGqWV0lINAI=
git.novox.be/novox/mesh-sdk/go v0.1.11-0.20261009143344-f047d0a4a970/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d h1:BR8uwv4BqyLaBwITuLylrd9DRUnu0D5DjJw0RuQSXbk=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010190644-622b6b4f556d/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
internal/accounts/accounts.go
+2 -2
View File
@@ -45,7 +45,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// The states, in the words liveness says them (the controller reads them alike).
@@ -119,7 +119,7 @@ func secretsOf(d *declaration.Declaration, account string) []string {
var out []string
for _, r := range d.Resources {
f, ok := r.(*declaration.File)
if !ok || f.Path == "" || (!f.Secret() && len(f.Secrets) == 0) || f.Owner == account {
if !ok || f.Path == "" || (f.Sealed == "" && len(f.Secrets) == 0) || f.Owner == account {
continue
}
out = append(out, f.Path)
internal/accounts/accounts_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// The account judge (novox/hq ADR 0252, "how it is checked"): an account in a group its running session
internal/accounts/root_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// An account declared never to become root without a person (novox/hq ADR 0266, "how it is checked"): each
internal/apply/agent_home_guard.go
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"fmt"
"path/filepath"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// AgentHomeRefusedError is an archive or a written-into file the engine will not place below an agent's home.
internal/apply/agent_home_guard_test.go
+1 -1
View File
@@ -6,7 +6,7 @@ package apply
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
func TestNothingIsUnpackedOrWrittenIntoBelowAnAgentsHome(t *testing.T) {
internal/apply/apply.go
+95 -244
View File
@@ -15,7 +15,6 @@ import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
@@ -27,7 +26,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
@@ -93,11 +92,6 @@ type Outcome struct {
// reads is, for a container, the digest of each file it was created reading, by path — so
// the next apply can say which one changed (novox/hq 04-ISSUES/103).
reads map[string]string
// busySince is, for a container left running because it said busy, when the first apply that would
// have replaced it ran (novox/hq ADR 0335): kept in its record, so the bound holds across passes and
// across a restart of the node-engine. busy is what the report says of it.
busySince *time.Time
busy *BusyWait
}
// Report is what an apply did, in the order it did it.
@@ -114,24 +108,6 @@ type Report struct {
// 224): a machine whose server is stopped at 03:31 because its collector is running is
// working, not broken, and the report is where that difference is said.
Windows []Window `json:"windows,omitempty"`
// Busy is every container this apply left running because it said busy, or replaced after it had
// waited (novox/hq ADR 0335): a machine whose walk waits for it says why, and how it ended.
Busy []BusyWait `json:"busy,omitempty"`
// MovedOut is every path a move-out directory moved into its kept folder (issue 1024, novox/hq ADR 0422).
MovedOut []MovedOut `json:"moved-out,omitempty"`
// Left is every socket, pipe or device a move-out directory holds and did not move (ADR 0422 §4).
Left []LeftInPlace `json:"left,omitempty"`
}
// Waiting says whether a container of this apply waits to be replaced because it is busy: what makes the
// node-engine ask again sooner than its reconcile (novox/hq ADR 0335 §2).
func (r Report) Waiting() bool {
for _, b := range r.Busy {
if b.State == BusyWaiting {
return true
}
}
return false
}
// Changed reports whether anything about the machine actually moved. An apply that changed
@@ -142,10 +118,7 @@ func (r Report) Changed() bool {
// whose account's manager is not running, which nothing moved either (novox/hq ADR 0177);
// held still is a container a maintenance window stopped, which the apply did not touch
// (novox/hq issue 224).
// waits busy is a container left running because it said busy (novox/hq ADR 0335), which nothing
// moved either.
if o.Action != "unchanged" && o.Action != "held" && o.Action != "waiting" && o.Action != heldStill &&
o.Action != waitsBusy {
if o.Action != "unchanged" && o.Action != "held" && o.Action != "waiting" && o.Action != heldStill {
return true
}
}
@@ -179,23 +152,6 @@ type Error struct {
// (novox/hq ADR 0136) or a directory the engine refused (issue 495). Named, because "everything
// was attempted" is then untrue, and a report that says so after skipping a module lies.
Modules []string
// Failures is every resource that failed, the first among them, each with its error (issue 1132, novox/hq ADR
// 0429 §4): Others counts the rest and names none, and a module's state on the bus names what failed in it.
Failures []Failure
}
// Failure is one resource that failed in an apply, and why.
type Failure struct {
Resource string
Err error
}
// ModuleResult is how one module's part of an apply ended (issue 1132).
type ModuleResult struct {
Failed bool
// Detail names each failing resource with its error, or why the module was not attempted.
Detail string
}
func (e *Error) Error() string {
@@ -610,8 +566,6 @@ func ApplyMindingWindows(
// is a different thing — one is "this machine could not do it", the other is "this was never
// a declaration", and they are fixed in different places.
var failures []*Error
// Which resources applied without error in this run, for the move-out pass after the loop (issue 1024).
appliedOK := map[string]bool{}
// Modules whose own step did not complete. What follows *within such a module* is not attempted;
// the rest of the machine is (novox/hq ADR 0136).
gated := map[string]bool{}
@@ -764,7 +718,6 @@ func ApplyMindingWindows(
first := failures[0]
first.Done = report
first.Others = len(failures) - 1
first.Failures = failuresOf(failures)
first.Modules = gatedModules()
log(" kept " + guardPrefix + "*: the guard is not up, so what it replaces was left in force")
return report, known, first
@@ -776,7 +729,7 @@ func ApplyMindingWindows(
// **A module whose step did not complete is skipped from there on** (novox/hq ADR 0136).
// Reported rather than passed over in silence: "not attempted" and "nothing to do" are
// different answers, and only one of them is somebody's to fix.
if module, ours := d.ModuleOfID(resource.Identity()); ours && gated[module] {
if module, ours := moduleOf(resource.Identity()); ours && gated[module] {
report.Outcomes = append(report.Outcomes, Outcome{
ID: resource.Identity(), Type: string(resource.Kind()), Target: resource.Target(),
Action: "skipped", Detail: gatedWhy[module],
@@ -806,16 +759,6 @@ func ApplyMindingWindows(
}
}
// **A filled file is the provisioner client's to write** (novox/hq ADR 0349 §5, design 57 P7): one writer per
// file. The declaration says where it goes and how it is written, and only the write verb (WriteFilled)
// writes it, on the client's word. This apply — the controller's send and the five-minute re-apply alike —
// neither writes it nor records it, and nothing that reads it is moved for it; so only what the verb placed
// is recorded, and only that is removed as an orphan when the file leaves the declaration.
if f, isFile := resource.(*declaration.File); isFile && d.Fills(f.Path) {
report.Outcomes = append(report.Outcomes, leftToTheClient(f, &known, handedFiles, &report, log))
continue
}
// The private network takes over the tunnel it found, ahead of the service that replaces
// it (novox/hq ADR 0105): its configuration kept, its unit stopped and disabled, never
// flushed. A failure here fails the service too — the mesh's interface is not started on a
@@ -938,7 +881,7 @@ func ApplyMindingWindows(
in.refused[filepath.Clean(dir.Path)] = err
}
if dir, ok := resource.(*declaration.Directory); ok && in.refused[filepath.Clean(dir.Path)] != nil {
if module, ours := d.ModuleOfID(resource.Identity()); ours {
if module, ours := moduleOf(resource.Identity()); ours {
gated[module] = true
gatedWhy[module] = fmt.Sprintf("its directory %s was refused (novox/hq issue 495), so "+
"nothing more of this module was attempted", filepath.Clean(dir.Path))
@@ -987,7 +930,7 @@ func ApplyMindingWindows(
//
// An action still gates the machine: the bootstrap is a row of them, each making the
// next possible, and they belong to no module.
if module, ours := d.ModuleOfID(resource.Identity()); ours {
if module, ours := moduleOf(resource.Identity()); ours {
gated[module] = true
gatedWhy[module] = "a step this module declares did not complete"
log(fmt.Sprintf(" gated %s.*: a step it declares did not complete, so the rest "+
@@ -997,7 +940,6 @@ func ApplyMindingWindows(
failed.Stopped = true
failed.Done = report
failed.Others = len(failures) - 1
failed.Failures = failuresOf(failures)
failed.Modules = gatedModules()
return report, known, failed
}
@@ -1035,8 +977,33 @@ func ApplyMindingWindows(
kept, keptMode, keptOwner = from.Kept, from.KeptMode, from.KeptOwner
}
// Only now. The record follows the fact, never leads it.
recordApplied(&known, origin, resource, outcome, kept, keptMode, keptOwner)
appliedOK[resource.Identity()] = true
known.Record(store.Applied{
Origin: origin,
ID: resource.Identity(), Type: string(resource.Kind()),
Target: outcome.Target, AppliedAt: time.Now().UTC(),
Wrote: outcome.wrote,
Into: outcome.into,
Kept: kept,
KeptMode: keptMode,
KeptOwner: keptOwner,
Reads: outcome.reads,
Stateless: outcome.stateless,
Scope: outcome.scope,
User: outcome.user,
Found: outcome.found,
DeclaredUnit: outcome.declared,
Shell: outcome.shell,
Linger: outcome.linger,
Groups: outcome.groups,
Unpacked: outcome.unpacked,
AsFound: outcome.asFound,
Declared: outcome.asFoundDeclared,
FoundAt: outcome.asFoundAt,
Holds: holds(resource),
})
if outcome.found != nil {
known.DropFound(resource.Identity())
}
// Its module has been taken, and what was held for it is now the mesh's. A file written
// into, or a service whose lifecycle is the machine's, replaced nothing that was found, so
// its outcome says what the apply did, not that a cutover happened; a hold from when it was
@@ -1079,13 +1046,10 @@ func ApplyMindingWindows(
Action: "forgotten", Detail: detail})
log(fmt.Sprintf(" forgotten %s (%s): %s", from.ID, from.Target, detail))
}
if outcome.busy != nil {
report.Busy = append(report.Busy, *outcome.busy)
}
if outcome.Action == heldStill || outcome.Action == waitsBusy {
if outcome.Action == heldStill {
// Not changed: nothing about it moved, so nothing that restarts on it restarts.
log(fmt.Sprintf(" %s %s (%s): %s", outcome.Action, outcome.ID, outcome.Target, outcome.Detail))
} else if moved(outcome) {
} else if outcome.Action != "unchanged" {
changed[resource.Identity()] = true
// With the detail, when there is one: "updated app" says a container was replaced;
// which file made that happen is what somebody reading the log at the time needs
@@ -1108,15 +1072,6 @@ func ApplyMindingWindows(
failures = append(failures, identityFailures...)
// A directory or archive that says move-out keeps only what it knows (issue 1024, novox/hq ADR 0422).
movedOut, leftIn, moveFailures := moveOutPass(d, known, appliedOK, log)
report.MovedOut = append(report.MovedOut, movedOut...)
report.Left = append(report.Left, leftIn...)
for _, f := range moveFailures {
f.Done = report
failures = append(failures, f)
}
// A replacement that did not apply — failed, skipped behind its module's step, held — leaves
// what it replaces running and recorded, said, for the next apply to hand over.
unhanded := make([]string, 0, len(handover))
@@ -1181,54 +1136,12 @@ func ApplyMindingWindows(
first := failures[0]
first.Done = report
first.Others = len(failures) - 1
first.Failures = failuresOf(failures)
first.Modules = gatedModules()
return report, known, first
}
return report, known, nil
}
// recordApplied records what applying a resource did, once it is done: the record follows the fact, never leads it.
// kept, keptMode and keptOwner are where the original of what it replaced was kept, and how it was found. What an
// earlier apply found its unit as goes with it.
func recordApplied(known *store.State, origin string, resource declaration.Resource, outcome Outcome,
kept, keptMode, keptOwner string) {
known.Record(store.Applied{
Origin: origin,
ID: resource.Identity(), Type: string(resource.Kind()),
Target: outcome.Target, AppliedAt: time.Now().UTC(),
Wrote: outcome.wrote,
Into: outcome.into,
Kept: kept,
KeptMode: keptMode,
KeptOwner: keptOwner,
Reads: outcome.reads,
BusySince: outcome.busySince,
Stateless: outcome.stateless,
Scope: outcome.scope,
User: outcome.user,
Found: outcome.found,
DeclaredUnit: outcome.declared,
Shell: outcome.shell,
Linger: outcome.linger,
Groups: outcome.groups,
Unpacked: outcome.unpacked,
AsFound: outcome.asFound,
Declared: outcome.asFoundDeclared,
FoundAt: outcome.asFoundAt,
Holds: holds(resource),
})
if outcome.found != nil {
known.DropFound(resource.Identity())
}
}
// moved reports whether applying a resource moved it, so what restarts or reloads on it does: anything but
// unchanged, and not a container held still or waiting for its busy one to say idle.
func moved(outcome Outcome) bool {
return outcome.Action != "unchanged" && outcome.Action != heldStill && outcome.Action != waitsBusy
}
// afterWhatTheyRead is the resources in declared order, except that **a service comes after every
// resource it names under restart-on or reload-on** (novox/hq issue 260).
//
@@ -1297,9 +1210,7 @@ func afterWhatTheyRead(resources []declaration.Resource) []declaration.Resource
const guardPrefix = declaration.AdoptionPrefix + "guard"
// Unseal opens a value the mesh sealed to this node. Nil when the node has no sealing key, which
// makes every sealed file an error rather than a silently skipped one. A value the vault sealed (a core
// credential, novox/hq ADR 0349 §16) is given as its box's JSON, which begins with `{`; one the controller sealed
// is base64, which never does (identity.SealingKey.Open).
// makes every sealed file an error rather than a silently skipped one.
type Unseal func(sealed string) ([]byte, error)
func applyOne(ctx context.Context, sys system.System, r declaration.Resource, run Runner,
@@ -1613,12 +1524,6 @@ func applyFile(r *declaration.File, previous store.Applied, unseal Unseal, keepF
return applyInto(r, previous)
}
out := begin(r)
// A file given by its digest is content to be had, not content: written as it stands it would be an empty file
// (novox/hq ADR 0486 §1, issue 1695). ResolveFileReferences has put its content in before any apply.
if r.IsReference() {
return out, fmt.Errorf("%s is given by digest %s and was not resolved to its content before the apply; "+
"it is not written empty", r.Path, r.Digest)
}
// What actually goes on disk. For a sealed file the mesh never had this, and neither did
// whatever carried the declaration here.
@@ -1645,16 +1550,7 @@ func applyFile(r *declaration.File, previous store.Applied, unseal Unseal, keepF
return out, fmt.Errorf(
"%s is sealed to this node and this node has no sealing key", r.Path)
}
sealed := r.Sealed
if r.VaultSealed != nil {
// The vault's box, handed to the opener as its JSON (Unseal).
raw, err := json.Marshal(r.VaultSealed)
if err != nil {
return out, err
}
sealed = string(raw)
}
opened, err := unseal(sealed)
opened, err := unseal(r.Sealed)
if err != nil {
return out, fmt.Errorf("cannot open %s: %w", r.Path, err)
}
@@ -1695,7 +1591,7 @@ func applyFile(r *declaration.File, previous store.Applied, unseal Unseal, keepF
// A seed that is already there is left exactly as it is — whatever has grown in it since is
// not the mesh's to put back (novox/hq issue 035). What this host records is what it once
// wrote, so a later declaration that changes the seed is not mistaken for drift either.
if r.Seeded() && existed {
if r.CreateOnce && existed {
out.wrote = previous.Wrote
if out.wrote == "" {
out.wrote = digestOf(string(existing))
@@ -2605,11 +2501,6 @@ func containerSpecReading(r *declaration.Container, declares, reads map[string]s
if args := identityArgs(r); len(args) > 0 {
b.WriteString("identity " + strings.Join(args, " ") + "\n")
}
// And the supplementary groups it joins (novox/hq ADR 0480 clause 7): fixed when the container is created, so
// a changed set is a container recreated. Only when there are some, so no other container has its digest moved.
if args := groupArgs(r); len(args) > 0 {
b.WriteString("groups " + strings.Join(args, " ") + "\n")
}
// **What this container reads is part of what it is.**
//
// A container takes its environment and its mounted files once, at start, and never looks
@@ -2751,12 +2642,6 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
// (novox/hq 04-ISSUES/009). Recreating is how a container gets restart-on, which a service
// already has.
reasons := restartedBy(r.RestartOn, changed)
// **A replacement owed since a busy wait began is still owed** (novox/hq ADR 0335): a restart-on trigger
// counts as changed only in the pass it changed, so a container left running then would otherwise read
// unchanged at the next pass, never be replaced, and drop from the report as if it had been.
if owed := owedSinceBusy(r, previous); owed != "" {
reasons = append(reasons, owed)
}
// What this container was created reading, as last recorded. By its declared id first, and by
// its NAME when that id has no record: the bundle's `store` becomes the postgres module's
@@ -2807,8 +2692,6 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
// stop, so a container this apply found stopped by a window is one whose window it can see.
if win, held := in.windows.holding(r.Name, time.Now()); held {
out.reads = wasReading
// A replacement owed since a busy wait stays owed through the window (novox/hq ADR 0335).
out.busySince = previous.BusySince
out.Action = heldStill
out.Detail = fmt.Sprintf("held still by the maintenance window of %s since %s; left as it is, "+
"and converged by the first apply after the window closes (novox/hq issue 224)",
@@ -2824,29 +2707,59 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
out.Action = "unchanged"
return out, nil
case existed:
// **A container that says it is busy is asked before it is replaced** (novox/hq ADR 0335 §2): left
// running while it says busy, within the bound, and replaced on the first pass it says idle. Only a
// running one: a stopped container holds no work in hand.
if r.Busy != nil && before.Running {
if waits := askBeforeReplacing(ctx, r, run, cri, previous, &out); waits {
out.reads = wasReading
return out, nil
}
}
// **The replacement's image is on the machine before the running container is removed**
// (novox/hq issue 1742). `docker run` pulls on its own, but only after the old container is
// gone: an image the registry does not hold then left the service down and every later apply
// failing. Pulled (or found present) first, an unobtainable image fails this resource loudly,
// naming the image, and the old container keeps running.
if err := ensureImage(ctx, cri, r.Image, run); err != nil {
return out, fmt.Errorf("not replacing container %s, which keeps running: %w", r.Name, err)
}
if _, err := run(ctx, cri, "rm", "-f", r.Name); err != nil {
return out, fmt.Errorf("replacing container %s: %w", r.Name, err)
}
}
args := detachedRunArgs(r, want)
args := []string{"run", "--detach", "--name", r.Name, "--restart", "unless-stopped"}
for _, file := range r.EnvFile {
args = append(args, "--env-file", file)
}
if r.Network != "" {
args = append(args, "--network", r.Network)
}
for _, c := range r.Capabilities {
args = append(args, "--cap-add", c)
}
if r.Logging != "" {
// The journal keeps the container's name on every line (CONTAINER_NAME), which is what a
// jail matches on (novox/hq ADR 0179); `docker logs` keeps working against the journal.
args = append(args, "--log-driver", r.Logging)
}
for _, d := range r.Dns {
args = append(args, "--dns", d)
}
if r.IP != "" {
args = append(args, "--ip", r.IP)
}
args = append(args,
"--label", specLabel+"="+want, "--label", idLabel+"="+r.ID)
for _, k := range sortedKeys(r.Env) {
args = append(args, "--env", k+"="+r.Env[k])
}
for _, p := range r.Ports {
args = append(args, "--publish", p)
}
for _, v := range r.Volumes {
args = append(args, "--volume", v)
}
args = append(args, identityArgs(r)...)
// **The one check the container carries is the one the module declared** (ADR 0240 rule 3): an
// exec command, or the image's own adopted by name, with the declared timing. Nothing else on the
// machine sets a container's check.
if r.Health.RunByRuntime() {
args = append(args, runtimeCheckArgs(r.Health)...)
}
for _, h := range r.Hosts {
// Written into the container's own hosts file by the runtime. Per container rather than
// by editing the machine's resolver configuration: that file belongs to something else on
// most machines, and a host that edited it would be fighting whatever owns it on every
// boot — the fault this host exists to avoid, in the place it would be hardest to see.
args = append(args, "--add-host", h)
}
args = append(args, r.Image)
args = append(args, r.Args...)
if _, err := run(ctx, cri, args...); err != nil {
return out, fmt.Errorf("starting container %s: %w", r.Name, err)
@@ -2894,67 +2807,10 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
default:
out.Detail = "replaced; a container's configuration is fixed when it is created"
}
if out.busy != nil {
out.Detail += "; " + out.busy.Words()
}
}
return out, nil
}
// detachedRunArgs is the `docker run` that starts a service container and leaves it running.
func detachedRunArgs(r *declaration.Container, want string) []string {
args := []string{"run", "--detach", "--name", r.Name, "--restart", "unless-stopped"}
for _, file := range r.EnvFile {
args = append(args, "--env-file", file)
}
if r.Network != "" {
args = append(args, "--network", r.Network)
}
for _, c := range r.Capabilities {
args = append(args, "--cap-add", c)
}
if r.Logging != "" {
// The journal keeps the container's name on every line (CONTAINER_NAME), which is what a
// jail matches on (novox/hq ADR 0179); `docker logs` keeps working against the journal.
args = append(args, "--log-driver", r.Logging)
}
for _, d := range r.Dns {
args = append(args, "--dns", d)
}
if r.IP != "" {
args = append(args, "--ip", r.IP)
}
args = append(args,
"--label", specLabel+"="+want, "--label", idLabel+"="+r.ID)
for _, k := range sortedKeys(r.Env) {
args = append(args, "--env", k+"="+r.Env[k])
}
for _, p := range r.Ports {
args = append(args, "--publish", p)
}
for _, v := range r.Volumes {
args = append(args, "--volume", v)
}
args = append(args, identityArgs(r)...)
args = append(args, groupArgs(r)...)
// **The one check the container carries is the one the module declared** (ADR 0240 rule 3): an
// exec command, or the image's own adopted by name, with the declared timing. Nothing else on the
// machine sets a container's check.
if r.Health.RunByRuntime() {
args = append(args, runtimeCheckArgs(r.Health)...)
}
for _, h := range r.Hosts {
// Written into the container's own hosts file by the runtime. Per container rather than
// by editing the machine's resolver configuration: that file belongs to something else on
// most machines, and a host that edited it would be fighting whatever owns it on every
// boot — the fault this host exists to avoid, in the place it would be hardest to see.
args = append(args, "--add-host", h)
}
args = append(args, r.Image)
args = append(args, r.Args...)
return args
}
// runtimeCheckArgs is a declared check as the runtime takes it (novox/hq ADR 0240 rule 3, to-be 48 §3):
// its timing, and for an exec check its command, run by the container's shell. The runtime's own retries
// are the declared failing looks, and its start period the grace — so its retries and start period do the
@@ -3045,7 +2901,6 @@ func foregroundRunArgs(r *declaration.Container, want string) []string {
args = append(args, "--volume", v)
}
args = append(args, identityArgs(r)...)
args = append(args, groupArgs(r)...)
for _, h := range r.Hosts {
args = append(args, "--add-host", h)
}
@@ -3054,16 +2909,6 @@ func foregroundRunArgs(r *declaration.Container, want string) []string {
return args
}
// groupArgs is the runtime's arguments that add a container's supplementary groups, one per number
// (novox/hq ADR 0480 clause 7, issue 1637). None for a container that declares none.
func groupArgs(r *declaration.Container) []string {
var args []string
for _, g := range r.Groups {
args = append(args, "--group-add", strconv.FormatInt(g, 10))
}
return args
}
// specWithoutIdentity is the spec r had before it was told who to run as.
func specWithoutIdentity(r *declaration.Container, declares, reads map[string]string) string {
plain := *r
@@ -3753,10 +3598,16 @@ func unitKey(scope, user, unit string) string {
// False for what the mesh declares in its own right: the foundation's resources carry no dot at all,
// and the adoption's are named for the mesh rather than for a module. Both belong to no module, and
// their gate is therefore the machine's.
//
// This is the guess from an id alone, for a caller with no declaration in hand; a caller with one asks
// Declaration.ModuleOfID, which reads the module the controller said (issue 1132).
func moduleOf(identity string) (string, bool) { return declaration.ModuleGuess(identity) }
func moduleOf(identity string) (string, bool) {
if strings.HasPrefix(identity, declaration.AdoptionPrefix) {
return "", false
}
at := strings.LastIndex(identity, ".")
if at <= 0 {
return "", false
}
return identity[:at], true
}
// handOver removes what a process replaces, once the process is running and still is a moment later
// (novox/hq issue 213, ADR 0184). A replacement that is not up keeps what it replaces in place
internal/apply/apply_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/archive.go
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/block.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/busy.go
-153
View File
@@ -1,153 +0,0 @@
package apply
import (
"context"
"fmt"
"strings"
"time"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that says it is busy is replaced once it is idle, within a bound (novox/hq ADR 0335).
//
// **Asked, never waited on.** The apply asks the container once, and while it says busy leaves it running
// as it is and goes on with everything else of the declaration: the machine's lock is held for one ask,
// never for the wait. The wait is carried between passes in the container's record (store.Applied's
// BusySince), so the next pass — the node-engine asks again sooner than its reconcile while anything waits
// — asks again, and so does a node-engine started again in between: the bound runs from the first apply
// that would have replaced the container, whoever applied it.
// waitsBusy is the action of a container left running because it said busy (novox/hq ADR 0335 §2). Its own
// word rather than "unchanged": its declaration moved, and the machine is not yet what was declared.
const waitsBusy = "waits-busy"
// What a report says of a busy container (novox/hq ADR 0335 §2, §3).
const (
// BusyWaiting is a container left running because it said busy, waiting to be replaced.
BusyWaiting = "waiting"
// BusyReplacedIdle is a container replaced on the first pass it said idle, after it had waited.
BusyReplacedIdle = "replaced-when-idle"
// BusyReplacedAtBound is a container still busy at the bound, replaced anyway.
BusyReplacedAtBound = "replaced-at-bound"
)
// BusyWait is one container a busy command held, as a report says it.
type BusyWait struct {
// ID is the resource's id (`<module>.<resource>`), Container its runtime name.
ID string `json:"id"`
Container string `json:"container"`
// State is BusyWaiting, BusyReplacedIdle or BusyReplacedAtBound.
State string `json:"state"`
// Since is when the first apply that would have replaced it ran; Until when the bound replaces it.
Since time.Time `json:"since"`
Until time.Time `json:"until"`
// Said is the last output its busy command gave.
Said string `json:"said,omitempty"`
}
// Words is the wait in a sentence, for an outcome's detail and the console.
func (b BusyWait) Words() string {
since := b.Since.UTC().Format(time.RFC3339)
said := ""
if b.Said != "" {
said = "; its busy command last said: " + b.Said
}
switch b.State {
case BusyReplacedAtBound:
return fmt.Sprintf("replaced while still busy: it said busy since %s and the bound of %s ran out "+
"(novox/hq ADR 0335 §3)%s", since, declaration.BusyBound, said)
case BusyReplacedIdle:
return fmt.Sprintf("replaced once it said idle, after waiting since %s (novox/hq ADR 0335 §2)%s", since, said)
}
return fmt.Sprintf("busy: left running and replaced on the first pass it says idle, or at %s at the latest; "+
"waiting since %s (novox/hq ADR 0335 §2)%s", b.Until.UTC().Format(time.RFC3339), since, said)
}
// owedSinceBusy is the reason a container still owes the replacement it was left running for, or empty: its
// record keeps when its busy wait began, under the same name.
func owedSinceBusy(r *declaration.Container, previous store.Applied) string {
if previous.BusySince == nil || previous.Target != r.Name {
return ""
}
return "the replacement owed since its busy wait began at " + previous.BusySince.UTC().Format(time.RFC3339)
}
// busyClock is the clock the bound is read on. A test moves it.
var busyClock = time.Now
// busySaidMost is how much of a busy command's output a report carries: its last lines, enough to say what
// it held.
const busySaidMost = 400
// askBeforeReplacing asks a running container that declares busy whether it may be replaced now, and says
// in out what came of it. True when it waits: out is then the whole outcome of this pass. False when the
// caller replaces it — idle, or past the bound — with out carrying the wait it ended, if there was one.
func askBeforeReplacing(ctx context.Context, r *declaration.Container, run Runner, cri string,
previous store.Applied, out *Outcome) bool {
now := busyClock().UTC()
busy, said := askBusy(ctx, r, run, cri)
var since time.Time
waited := previous.BusySince != nil && previous.Target == r.Name
if waited {
since = previous.BusySince.UTC()
// A wait kept from a clock that has since gone back reads as starting now, never as later: the bound
// holds from no later than this pass.
if since.After(now) {
since = now
}
}
wait := BusyWait{ID: r.ID, Container: r.Name, Since: since, Said: said}
switch {
case !busy && !waited:
// Idle at the first ask: replaced at once, as a container that declares nothing is.
return false
case !busy:
wait.State, wait.Until = BusyReplacedIdle, since.Add(declaration.BusyBound)
out.busy = &wait
return false
}
if !waited {
since = now
wait.Since = now
}
wait.Until = since.Add(declaration.BusyBound)
if !now.Before(wait.Until) {
wait.State = BusyReplacedAtBound
out.busy = &wait
return false
}
wait.State = BusyWaiting
out.Action, out.Detail = waitsBusy, wait.Words()
out.busySince, out.busy = &since, &wait
return true
}
// askBusy runs a container's busy command inside it, by its shell, as an exec health check is run: busy is
// an exit of 0 and nothing else. An ask that could not be made, or did not answer within its timeout, is
// not busy: the bound is what keeps a container whose command never answers from being held.
func askBusy(ctx context.Context, r *declaration.Container, run Runner, cri string) (bool, string) {
ask, cancel := context.WithTimeout(ctx, r.Busy.Within())
defer cancel()
output, err := run(ask, cri, "exec", r.Name, "/bin/sh", "-c", r.Busy.Command)
said := strings.TrimSpace(output)
if err != nil && said == "" {
said = err.Error()
}
return err == nil, lastOf(said, busySaidMost)
}
// lastOf is the end of a text, at most n bytes, on one line.
func lastOf(s string, n int) string {
s = strings.Join(strings.Fields(s), " ")
if len(s) <= n {
return s
}
from := len(s) - n
for from < len(s) && !utf8.RuneStart(s[from]) {
from++
}
return "…" + s[from:]
}
internal/apply/busy_test.go
-433
View File
@@ -1,433 +0,0 @@
package apply
import (
"context"
"errors"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that says it is busy is replaced once it is idle, within a bound (novox/hq ADR 0335). The
// machine of these tests runs two containers, a build agent that declares `busy` and a web server that
// does not; every call that replaces a container is counted.
type aBusyMachine struct {
mu sync.Mutex
// containers is each container's name and the spec it was created with; running whether it runs.
containers map[string]string
busy bool // what the agent's busy command answers
said string // what it prints
asked int
calls []string
}
func newBusyMachine() *aBusyMachine {
return &aBusyMachine{containers: map[string]string{}}
}
func (m *aBusyMachine) run(_ context.Context, _ string, args ...string) (string, error) {
m.mu.Lock()
defer m.mu.Unlock()
switch args[0] {
case "info", "image":
return "27.0\n", nil
case "container": // inspect
name := args[len(args)-1]
spec, there := m.containers[name]
if !there {
return "", errors.New("no such container")
}
return "true\t" + spec, nil
case "exec":
m.asked++
m.calls = append(m.calls, "ask "+args[1])
if strings.Join(args[2:], " ") != "/bin/sh -c mesh-builder busy" {
return "", errors.New("asked something else: " + strings.Join(args[2:], " "))
}
if m.busy {
return m.said + "\n", nil
}
return m.said + "\n", errors.New("exited 1")
case "rm":
name := args[len(args)-1]
m.calls = append(m.calls, "rm "+name)
delete(m.containers, name)
case "run":
name := args[slices.Index(args, "--name")+1]
m.calls = append(m.calls, "create "+name)
for i, a := range args {
if a == "--label" && strings.HasPrefix(args[i+1], specLabel+"=") {
m.containers[name] = strings.TrimPrefix(args[i+1], specLabel+"=")
}
}
}
return "", nil
}
func (m *aBusyMachine) mark() int {
m.mu.Lock()
defer m.mu.Unlock()
return len(m.calls)
}
// since is every call since mark that replaced a container, or asked one.
func (m *aBusyMachine) since(mark int) []string {
m.mu.Lock()
defer m.mu.Unlock()
return append([]string(nil), m.calls[mark:]...)
}
func (m *aBusyMachine) answer(busy bool, said string) {
m.mu.Lock()
defer m.mu.Unlock()
m.busy, m.said = busy, said
}
// twoContainers is the declaration at a version: v moves the agent's image tag through its env, w the web's.
func twoContainers(t *testing.T, agent, web string) *declaration.Declaration {
t.Helper()
return parse(t, `{"declaration":1,"resources":[
{"id":"build-agent.server","type":"container","name":"mesh-build-agent","image":"`+pinned+`",
"env":{"V":"`+agent+`"},"busy":{"command":"mesh-builder busy","timeout":"5s"}},
{"id":"web.server","type":"container","name":"web","image":"`+pinned+`","env":{"V":"`+web+`"}}
]}`)
}
// atClock moves the bound's clock for one test.
func atClock(t *testing.T, at *time.Time) {
t.Helper()
was := busyClock
busyClock = func() time.Time { return *at }
t.Cleanup(func() { busyClock = was })
}
func busyOf(r Report, id string) (BusyWait, bool) {
for _, b := range r.Busy {
if b.ID == id {
return b, true
}
}
return BusyWait{}, false
}
func applyBusy(t *testing.T, m *aBusyMachine, d *declaration.Declaration, known store.State) (Report, store.State) {
t.Helper()
report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, m.run, nil, nil)
if err != nil {
t.Fatalf("the apply failed: %v", err)
}
return report, state
}
// The decision's first check: an apply that would replace a container whose busy command exits 0 leaves it
// running, applies the rest, and replaces it on the first pass the command exits non-zero.
func TestABusyContainerIsLeftRunningAndReplacedOnTheFirstPassItSaysIdle(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
// A merge moves both: the agent holds a build, the web server holds nothing.
m.answer(true, "building novox/mesh-controller (check of #280) since 00:58")
mark := m.mark()
report, state := applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm web", "create web"}) {
t.Fatalf("a busy agent was not left running while the rest applied: %v", got)
}
o := outcomeOf(report, "build-agent.server")
if o.Action != waitsBusy {
t.Fatalf("a busy container was reported %q, want %q: %+v", o.Action, waitsBusy, o)
}
for _, says := range []string{"busy", "first pass it says idle", "check of #280", "ADR 0335"} {
if !strings.Contains(o.Detail, says) {
t.Errorf("the outcome does not say %q: %s", says, o.Detail)
}
}
if (Report{Outcomes: []Outcome{o}}).Changed() {
t.Errorf("a container left running reads as a change: %+v", o)
}
b, said := busyOf(report, "build-agent.server")
if !said || b.State != BusyWaiting || !b.Since.Equal(now) || !b.Until.Equal(now.Add(15*time.Minute)) ||
b.Container != "mesh-build-agent" || !strings.Contains(b.Said, "check of #280") {
t.Fatalf("the report does not say the agent waits, since when and until when: %+v", report.Busy)
}
if !report.Waiting() {
t.Error("a report with a container waiting does not say so")
}
if rec, _ := state.Find("build-agent.server"); rec.BusySince == nil || !rec.BusySince.Equal(now) {
t.Fatalf("the wait is not kept in the container's record: %+v", rec)
}
// Asked again a minute later, still busy: still left running, the wait counted from the first apply.
now = now.Add(time.Minute)
mark = m.mark()
report, state = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent"}) {
t.Fatalf("a second pass while busy did more than ask: %v", got)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyWaiting || !b.Since.Equal(now.Add(-time.Minute)) {
t.Fatalf("the second pass does not keep the wait from the first apply: %+v", report.Busy)
}
// The build ends: the next pass replaces it, and says it waited.
now = now.Add(time.Minute)
m.answer(false, "idle")
mark = m.mark()
report, state = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("the agent was not replaced on the first pass it said idle: %v", got)
}
if o := outcomeOf(report, "build-agent.server"); o.Action != "updated" || !strings.Contains(o.Detail, "once it said idle") {
t.Errorf("the replacement does not say it waited: %+v", o)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyReplacedIdle {
t.Errorf("the report does not say the agent was replaced when idle: %+v", report.Busy)
}
if report.Waiting() {
t.Error("a report with nothing waiting says something waits")
}
if rec, _ := state.Find("build-agent.server"); rec.BusySince != nil {
t.Errorf("the wait is still kept after the replacement: %+v", rec)
}
// And the pass after that has nothing to say about it.
report, _ = applyBusy(t, m, twoContainers(t, "2", "2"), state)
if len(report.Busy) != 0 {
t.Errorf("a container replaced is still said: %+v", report.Busy)
}
}
// An idle container is replaced at once, as one that declares nothing is: asked once, nothing waits.
func TestAnIdleContainerIsReplacedAtOnce(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(false, "")
mark := m.mark()
report, _ := applyBusy(t, m, twoContainers(t, "2", "1"), state)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("an idle agent was not replaced at once: %v", got)
}
if len(report.Busy) != 0 {
t.Errorf("an idle container replaced at once is said as a wait: %+v", report.Busy)
}
// And a container whose declaration did not move is never asked.
mark = m.mark()
applyBusy(t, m, twoContainers(t, "2", "1"), state)
if got := m.since(mark); slices.Contains(got, "ask mesh-build-agent") {
t.Errorf("a container nothing replaces was asked: %v", got)
}
}
// Past the bound it is replaced anyway, the report carrying the command's last output — and the bound holds
// across a restart of the node-engine: the wait is read back from the state on disk.
func TestABusyContainerIsReplacedAtTheBoundEvenAfterARestart(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building novox/mesh-host (a check that never ends)")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
// The node-engine stops and starts again: what it knows is what it saved.
path := filepath.Join(t.TempDir(), "state.json")
if err := store.Save(path, state); err != nil {
t.Fatal(err)
}
reloaded, err := store.Load(path)
if err != nil {
t.Fatal(err)
}
now = now.Add(14 * time.Minute)
mark := m.mark()
report, reloaded := applyBusy(t, m, twoContainers(t, "2", "1"), reloaded)
if got := m.since(mark); slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("a container busy within the bound was replaced after a restart: %v", got)
}
if b, _ := busyOf(report, "build-agent.server"); b.State != BusyWaiting || !b.Since.Equal(now.Add(-14*time.Minute)) {
t.Fatalf("a restart started the wait again: %+v", report.Busy)
}
now = now.Add(time.Minute)
mark = m.mark()
report, _ = applyBusy(t, m, twoContainers(t, "2", "1"), reloaded)
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("a container still busy at the bound was not replaced: %v", got)
}
b, _ := busyOf(report, "build-agent.server")
if b.State != BusyReplacedAtBound || !strings.Contains(b.Said, "never ends") {
t.Fatalf("the report does not say it replaced a busy container, with its last output: %+v", report.Busy)
}
if o := outcomeOf(report, "build-agent.server"); !strings.Contains(o.Detail, "still busy") ||
!strings.Contains(o.Detail, "never ends") {
t.Errorf("the outcome does not say a busy container was replaced, with what it said: %s", o.Detail)
}
}
// While one container waits, a send that changes another module on the same machine is applied at once: the
// apply returns after one ask, and holds nothing for the wait.
func TestASendToAnotherModuleIsAppliedWhileOneWaits(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
// The next send moves only the web server.
mark := m.mark()
done := make(chan struct{})
var report Report
go func() {
defer close(done)
report, _ = applyBusy(t, m, twoContainers(t, "2", "2"), state)
}()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Fatal("an apply while a container waits did not return")
}
got := m.since(mark)
if !slices.Contains(got, "create web") || slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("the other module's send was not applied at once beside the wait: %v", got)
}
if o := outcomeOf(report, "web.server"); o.Action != "updated" {
t.Errorf("the web server was reported %q: %+v", o.Action, o)
}
}
// A container's busy is held to its shape where it arrives.
func TestBusyIsReadAndHeldToItsShape(t *testing.T) {
d := twoContainers(t, "1", "1")
c := d.Resources[0].(*declaration.Container)
if c.Busy == nil || c.Busy.Command != "mesh-builder busy" || c.Busy.Within() != 5*time.Second {
t.Fatalf("busy was not read: %+v", c.Busy)
}
for name, raw := range map[string]string{
"no command": `"busy":{"command":" "}`,
"a long timeout": `"busy":{"command":"x","timeout":"5m"}`,
"not a duration": `"busy":{"command":"x","timeout":"soon"}`,
"an unknown key": `"busy":{"command":"x","every":"1m"}`,
"on a step": `"run-once":true,"busy":{"command":"x"}`,
} {
_, err := declaration.Parse([]byte(`{"declaration":1,"resources":[{"id":"a.b","type":"container",` +
`"name":"b","image":"` + pinned + `",` + raw + `}]}`))
if err == nil {
t.Errorf("%s: a busy of that shape was accepted", name)
}
}
}
// A replacement owed to a restart-on trigger is still owed after the wait: a trigger that is no file or
// directory (a service, an identity's or a library's change) is in no container's spec and counts as changed only
// in the pass it changed, so the container must not read unchanged at the next pass and drop from the report as if
// replaced (novox/hq ADR 0335).
func TestAReplacementOwedToARestartTriggerIsMadeOnceIdle(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
d := parse(t, `{"declaration":1,"resources":[
{"id":"build-agent.server","type":"container","name":"mesh-build-agent","image":"`+pinned+`",
"restart-on":["identity:build-agent"],"busy":{"command":"mesh-builder busy"}}
]}`)
r := d.Resources[0].(*declaration.Container)
in := inputs{declares: map[string]string{}}
m := newBusyMachine()
m.containers["mesh-build-agent"] = containerSpec(r, in)
ctx := context.Background()
// The trigger moved this pass, and the agent holds a build: left running.
m.answer(true, "building")
out, err := applyContainer(ctx, r, m.run, map[string]bool{"identity:build-agent": true}, in, store.Applied{})
if err != nil || out.Action != waitsBusy || out.busySince == nil {
t.Fatalf("a busy container was not left running for its trigger: %+v %v", out, err)
}
record := store.Applied{ID: r.ID, Target: r.Name, BusySince: out.busySince}
// The next pass: nothing triggers now, and the replacement is still owed.
now = now.Add(time.Minute)
m.answer(false, "idle")
mark := m.mark()
out, err = applyContainer(ctx, r, m.run, map[string]bool{}, in, record)
if err != nil {
t.Fatal(err)
}
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("the replacement owed to the trigger was dropped: %v (%+v)", got, out)
}
if out.busy == nil || out.busy.State != BusyReplacedIdle || out.busySince != nil {
t.Fatalf("the replacement does not say it was made once idle: %+v", out)
}
}
// A wait kept from a clock that later went back is never longer than the bound from now.
func TestAWaitFromAClockThatWentBackIsClampedToNow(t *testing.T) {
now := time.Date(2026, 10, 13, 1, 0, 0, 0, time.UTC)
atClock(t, &now)
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "building")
_, state = applyBusy(t, m, twoContainers(t, "2", "1"), state)
now = now.Add(-time.Hour) // the clock went back an hour
report, _ := applyBusy(t, m, twoContainers(t, "2", "1"), state)
b, _ := busyOf(report, "build-agent.server")
if b.State != BusyWaiting || b.Since.After(now) || !b.Until.Equal(now.Add(15*time.Minute)) {
t.Fatalf("a wait from a later clock is not clamped to now: %+v (now %s)", b, now)
}
}
// An ask that fails — no shell in the image, a timeout, a paused container — is not busy: replaced at once.
func TestAnAskThatFailsIsIdle(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
failing := func(ctx context.Context, name string, args ...string) (string, error) {
if args[0] == "exec" {
m.mu.Lock()
m.calls = append(m.calls, "ask "+args[1])
m.mu.Unlock()
return "", errors.New("OCI runtime exec failed: exec: \"/bin/sh\": no such file or directory")
}
return m.run(ctx, name, args...)
}
mark := m.mark()
report, _, err := Apply(context.Background(), archHost(t), twoContainers(t, "2", "1"), state,
store.OriginDeclared, failing, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := m.since(mark); !slices.Equal(got, []string{"ask mesh-build-agent", "rm mesh-build-agent", "create mesh-build-agent"}) {
t.Fatalf("an ask that failed held the container: %v", got)
}
if len(report.Busy) != 0 {
t.Errorf("an ask that failed is said as a wait: %+v", report.Busy)
}
}
// A stopped container holds no work: it is replaced without being asked.
func TestAStoppedContainerIsNotAsked(t *testing.T) {
m := newBusyMachine()
_, state := applyBusy(t, m, twoContainers(t, "1", "1"), store.State{})
m.answer(true, "would say busy")
stopped := func(ctx context.Context, name string, args ...string) (string, error) {
if args[0] == "container" && args[len(args)-1] == "mesh-build-agent" {
out, err := m.run(ctx, name, args...)
return strings.Replace(out, "true\t", "false\t", 1), err
}
return m.run(ctx, name, args...)
}
mark := m.mark()
if _, _, err := Apply(context.Background(), archHost(t), twoContainers(t, "2", "1"), state,
store.OriginDeclared, stopped, nil, nil); err != nil && !strings.Contains(err.Error(), "not running") {
t.Fatal(err)
}
got := m.since(mark)
if slices.Contains(got, "ask mesh-build-agent") || !slices.Contains(got, "rm mesh-build-agent") {
t.Fatalf("a stopped container was asked, or not replaced: %v", got)
}
}
internal/apply/container_groups_test.go
-54
View File
@@ -1,54 +0,0 @@
package apply
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// Issue 1637 (novox/hq ADR 0480 clause 7): a container's supplementary groups reach the runtime as --group-add.
func groupsContainer(groups ...int64) *declaration.Container {
return &declaration.Container{ID: "n8n.app", Name: "n8n", Image: pinned, Groups: groups}
}
// 1637/AC1: both run commands carry one --group-add per group, and none without groups.
func TestAContainerIsAddedItsSupplementaryGroups(t *testing.T) {
with := groupsContainer(2000, 2001)
for name, args := range map[string][]string{
"detached": detachedRunArgs(with, "x"),
"foreground": foregroundRunArgs(with, "x"),
} {
got := " " + strings.Join(args, " ") + " "
if !strings.Contains(got, " --group-add 2000 ") || !strings.Contains(got, " --group-add 2001 ") {
t.Errorf("1637/AC1: the %s run does not add both groups: %s", name, got)
}
}
without := groupsContainer()
for name, args := range map[string][]string{
"detached": detachedRunArgs(without, "x"),
"foreground": foregroundRunArgs(without, "x"),
} {
if strings.Contains(strings.Join(args, " "), "--group-add") {
t.Errorf("1637/AC1: the %s run adds a group none was declared", name)
}
}
}
// 1637/AC2: a changed set of groups recreates the container; no groups leaves the spec as it was.
func TestAChangeOfGroupsRecreatesTheContainer(t *testing.T) {
none, a, b := groupsContainer(), groupsContainer(2000), groupsContainer(2000, 2001)
if containerSpec(none, inputs{}) == containerSpec(a, inputs{}) || containerSpec(a, inputs{}) == containerSpec(b, inputs{}) {
t.Error("1637/AC2: a change of groups does not change the container's spec")
}
same := groupsContainer(2000)
if containerSpec(a, inputs{}) != containerSpec(same, inputs{}) {
t.Error("1637/AC2: the same groups changed the spec")
}
plain := *none
plain.Groups = nil
if containerSpec(none, inputs{}) != containerSpec(&plain, inputs{}) {
t.Error("1637/AC2: a container with no groups has its spec moved")
}
}
internal/apply/declared_home_test.go
-158
View File
@@ -1,158 +0,0 @@
package apply
import (
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// A module that declares its own system account and a .ssh in that account's home is placed (novox/hq issue 1542):
// the account is not in the user database yet when the declaration is judged, and its home is under /var/lib.
func declaredHomeFixture(t *testing.T) (home string, user *declaration.User) {
t.Helper()
dir := t.TempDir()
passwd := filepath.Join(dir, "passwd")
if err := os.WriteFile(passwd, []byte("root:x:0:0::/root:/bin/sh\n"), 0o644); err != nil {
t.Fatal(err)
}
before := passwdFile
passwdFile = passwd
t.Cleanup(func() { passwdFile = before })
home = filepath.Join(dir, "lib", "mesh-build-go")
return home, &declaration.User{ID: "mesh-build-go.account", Type: declaration.TypeUser, Name: "mesh-build-go",
Shell: "/usr/bin/nologin", Home: home}
}
// 1542/AC1: the .ssh of a declared account, owned by it, is refused by nothing.
func TestIssue1542ADeclaredAccountsSSHIsPlaced(t *testing.T) {
home, user := declaredHomeFixture(t)
ssh := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
refused := refusedPlaces([]declaration.Resource{user, ssh})
if len(refused) != 0 {
t.Fatalf("a declared account's own .ssh was refused: %v", refused)
}
if err := (inputs{refused: refused}).refused[filepath.Clean(ssh.Path)]; err != nil {
t.Fatalf("the apply-time check refuses it: %v", err)
}
if line := JudgeDirectories(&declaration.Declaration{Resources: []declaration.Resource{user, ssh}}); strings.Contains(line, "refused already by the placement guard: ") {
t.Fatalf("the read-only judgement refuses it: %s", line)
}
}
// 1542/AC2: another account's directory below the declared home is refused as below any home, and a .ssh below
// no home of the machine or the declaration is refused with today's words.
func TestIssue1542OnlyTheDeclaredAccountsOwnIsPlaced(t *testing.T) {
home, user := declaredHomeFixture(t)
other := &declaration.Directory{ID: "notes.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "intruder"}
refused := refusedPlaces([]declaration.Resource{user, other})
err := refused[filepath.Clean(other.Path)]
if err == nil || !strings.Contains(err.Error(), "below mesh-build-go's home and declared intruder's") {
t.Fatalf("another account's directory below a declared home: %v", err)
}
stray := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory,
Path: filepath.Join(filepath.Dir(home), "elsewhere", ".ssh"), Mode: "0700", Owner: "mesh-build-go"}
refused = refusedPlaces([]declaration.Resource{user, stray})
err = refused[filepath.Clean(stray.Path)]
if err == nil || !strings.Contains(err.Error(), "a .ssh directory is placed only below its account's home, as that account's") {
t.Fatalf("a .ssh below no home: %v", err)
}
// Not a .ssh: a root-owned directory below the declared home is placed exactly as on main.
data := &declaration.Directory{ID: "mesh-build-go.data", Type: declaration.TypeDirectory, Path: home + "/data",
Mode: "0755", Owner: "root"}
if refused := refusedPlaces([]declaration.Resource{user, data}); len(refused) != 0 {
t.Fatalf("a root-owned directory below a declared home, not a .ssh, was refused: %v", refused)
}
// And without the account declared, the same .ssh is refused as it was.
own := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
if refusedPlaces([]declaration.Resource{own})[filepath.Clean(own.Path)] == nil {
t.Fatal("a .ssh below an account nobody makes was placed")
}
}
// 1542/AC3: a declared account is no home to anyone but the directory placement guard.
func TestIssue1542DeclaredAccountsAreNotHomesToTheOtherGuards(t *testing.T) {
home, user := declaredHomeFixture(t)
_ = refusedPlaces([]declaration.Resource{user})
if _, ok := accountsOfHomes()[home]; ok {
t.Fatal("a declared account became a home of the machine")
}
if homeAbove(home+"/.ssh") != "" {
t.Fatal("a declared account became a home for the links and the move-out")
}
}
// A declared account is added as a home and never takes one over (reviewer, issue 1542): the person's own .ssh
// stays refused to every other account, whatever a declaration says about a user.
func TestIssue1542ADeclaredAccountTakesNoExistingHome(t *testing.T) {
dir := t.TempDir()
passwd := filepath.Join(dir, "passwd")
body := "root:x:0:0::/root:/bin/sh\njochen:x:1000:1000::/home/jochen:/bin/sh\n" +
"svc:x:998:998::/var/lib/svc:/usr/bin/nologin\n"
if err := os.WriteFile(passwd, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
before := passwdFile
passwdFile = passwd
t.Cleanup(func() { passwdFile = before })
newHome := filepath.Join(dir, "lib", "mesh-build-go")
ssh := func(path, owner string) *declaration.Directory {
return &declaration.Directory{ID: "x.ssh", Type: declaration.TypeDirectory, Path: path, Mode: "0700", Owner: owner}
}
user := func(name, home string) *declaration.User {
return &declaration.User{ID: "x.user", Type: declaration.TypeUser, Name: name, Home: home}
}
// A declared user over a person's home, whoever it is named for.
for _, name := range []string{"evil", "postgres", "root"} {
d := ssh("/home/jochen/.ssh", name)
if refusedPlaces([]declaration.Resource{user(name, "/home/jochen"), d})[d.Path] == nil {
t.Errorf("%s, declared over jochen's home, was given jochen's .ssh", name)
}
}
// A declared home below a person's home.
d := ssh("/home/jochen/sub/.ssh", "evil")
if refusedPlaces([]declaration.Resource{user("evil", "/home/jochen/sub"), d})[d.Path] == nil {
t.Error("a home declared below jochen's home was counted")
}
// A name the machine has already, declared with another home: that is not its home.
d = ssh(newHome+"/e/.ssh", "svc")
if refusedPlaces([]declaration.Resource{user("svc", newHome+"/e"), d})[d.Path] == nil {
t.Error("an existing account's declared home was counted")
}
// A number is no name: chown -1 leaves the owner as it was.
for _, owner := range []string{"-1", "-1:0", "0", "1001"} {
d := ssh(newHome+"/.ssh", owner)
if refusedPlaces([]declaration.Resource{user("mesh-build-go", newHome), d})[d.Path] == nil {
t.Errorf("owner %q was placed below a declared home", owner)
}
}
// And the account that is new, by name, still is.
d = ssh(newHome+"/.ssh", "mesh-build-go")
if got := refusedPlaces([]declaration.Resource{user("mesh-build-go", newHome), d}); len(got) != 0 {
t.Errorf("a new account's own .ssh was refused: %v", got)
}
}
// The account the mesh made, as declared, is judged the same at the next apply (issue 1542): once it is in the
// user database with exactly the declared home, its .ssh is still placed.
func TestIssue1542TheAccountTheMeshMadeIsStillPlacedAtTheNextApply(t *testing.T) {
home, user := declaredHomeFixture(t)
ssh := &declaration.Directory{ID: "mesh-build-go.ssh-dir", Type: declaration.TypeDirectory, Path: home + "/.ssh",
Mode: "0700", Owner: "mesh-build-go"}
resources := []declaration.Resource{user, ssh}
if got := refusedPlaces(resources); len(got) != 0 {
t.Fatalf("first apply, the account not yet made: %v", got)
}
body := "root:x:0:0::/root:/bin/sh\nmesh-build-go:x:998:998::" + home + ":/usr/bin/nologin\n"
if err := os.WriteFile(passwdFile, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
if got := refusedPlaces(resources); len(got) != 0 {
t.Fatalf("second apply, the account made as declared: %v", got)
}
}
internal/apply/filled.go
-234
View File
@@ -1,234 +0,0 @@
package apply
import (
"context"
"fmt"
"slices"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// WriteFilled writes what the provisioner's client filled into one file the declaration names as filled (novox/hq
// ADR 0349 §4), and makes what reads the file take the change, exactly as an apply that changed it would.
//
// **The declared file says everything but the content.** Its path, mode, owner and who owns it at runtime are the
// declaration's; the client's request carries the content alone. So the file is applied as the declared one with
// that content in place of the declaration's: an owned file whole through applyFile, a seed only when absent, a
// managed file's keys through applyInto or its block through applyBlock — the same guards on where nothing is
// placed, the same Keep of an original found there, the same record. What the declaration seals or fills with
// `${secret:}` the client has already filled, so the content is written as it came; a file that carried a secret
// keeps the mode a secret is written with when the declaration gives none.
//
// **Then what reads it, in the apply's order** (afterWhatTheyRead): a service naming it under restart-on is
// restarted and under reload-on reloaded, a process or container naming it under restart-on is restarted, and a
// running container that reads it at creation — an env-file, a file mounted into it — is recreated, because what
// it read is recorded by digest and this write changed the digest. Each of those is applied and recorded as an
// apply does, and what moves goes on moving what names it. Nothing else of the declaration is touched.
//
// The caller holds the apply's lock: a write and an apply never write one file at once. log is said the path, the
// action and the digest, never the content.
//
// **It does not refresh the runtimes' data roots.** A filled write is one file, not an apply; the placement guard
// judges it with the roots the last apply read (runtimeDataRoots), falling back to the runtimes' configuration
// files when no apply has read them yet — the runtimes themselves are asked once, at an apply's start, not once
// per file (novox/hq issue 1138).
func WriteFilled(
ctx context.Context,
sys system.System,
d *declaration.Declaration,
known store.State,
path string,
content []byte,
run Runner,
log func(string),
keep Keep,
windows *Windows,
) (report Report, _ store.State, _ error) {
if log == nil {
log = func(string) {}
}
if !d.Fills(path) {
return report, known, &Error{Resource: path, Err: fmt.Errorf("this node's declaration does not name %s "+
"as a file the provisioner fills, so nothing was written", path)}
}
var declared []*declaration.File
for _, r := range d.Resources {
if f, ok := r.(*declaration.File); ok && f.Path == path {
declared = append(declared, f)
}
}
if len(declared) != 1 {
// The parser refuses a filled path no file declares; two at one path would need the request to say whose
// keys or block it carries, and it carries the content only.
return report, known, &Error{Resource: path, Err: fmt.Errorf("%d resources declare %s, and a filled file "+
"is written for exactly one, so nothing was written", len(declared), path)}
}
file := filledAs(declared[0], content)
if held, isHeld := known.HeldAt(file.ID); isHeld {
return report, known, &Error{Resource: file.ID, Err: fmt.Errorf("%s is held as found for a module not "+
"yet taken (%s), so nothing was written", path, held.Why)}
}
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: groupsWanted(d.Resources), handed: Handed{}}
in.refused = refusedPlaces(d.Resources)
in.agentHomes = rootNeverHomes(d)
in.dirsBefore = map[string]bool{}
// A container that reads the file and mounts a library whose disk or mount is not there is held, as a full apply
// holds it: not created over the missing disk (novox/hq ADR 0328 §9).
in.held = heldLibraries(ctx, d.Libraries)
for _, r := range d.Resources {
in.declares[r.Identity()] = declaredDigest(r)
}
changed := map[string]bool{}
// The file, as the apply loop applies it.
was, _ := known.Find(file.ID)
var keepFound Keep
if (was.ID == "" || was.Target != file.Path) && !known.Recorded(string(declaration.TypeFile), file.Path) {
keepFound = keep
}
outcome, err := applyOne(ctx, sys, file, run, changed, in, was, nil, keepFound)
if err != nil {
log(fmt.Sprintf(" failed %s (%s): %v", file.ID, file.Path, err))
return report, known, &Error{Resource: file.ID, Err: err, Done: report}
}
recordKeeping(&known, file, outcome, was)
report.Outcomes = append(report.Outcomes, outcome)
if !moved(outcome) {
return report, known, nil
}
changed[file.ID] = true
line := fmt.Sprintf(" %s %s (%s), filled: %s", outcome.Action, file.ID, file.Path, outcome.wrote)
if outcome.Detail != "" {
line += ": " + outcome.Detail
}
log(line)
// What reads it, in the apply's order, each moving what names it in turn.
var failures []*Error
for _, r := range afterWhatTheyRead(d.Resources) {
if r.Identity() == file.ID || !readsChange(r, path, changed) {
continue
}
if _, isHeld := known.HeldAt(r.Identity()); isHeld {
log(fmt.Sprintf(" held %s (%s): held as found for a module not yet taken, so it was not restarted "+
"for %s", r.Identity(), r.Target(), file.ID))
continue
}
was, _ := known.Find(r.Identity())
o, err := applyOne(ctx, sys, r, run, changed, in, was, nil, nil)
if err != nil {
if o.found != nil {
known.KeepFound(r.Identity(), store.OriginDeclared, *o.found)
}
failures = append(failures, &Error{Resource: r.Identity(), Err: err})
log(fmt.Sprintf(" failed %s (%s): %v", r.Identity(), o.Target, err))
continue
}
if o.Action == "waiting" {
if o.found != nil {
known.KeepFound(r.Identity(), store.OriginDeclared, *o.found)
}
report.Outcomes = append(report.Outcomes, o)
log(fmt.Sprintf(" waiting %s (%s): %s", o.ID, o.Target, o.Detail))
continue
}
recordKeeping(&known, r, o, was)
report.Outcomes = append(report.Outcomes, o)
if o.busy != nil {
report.Busy = append(report.Busy, *o.busy)
}
if moved(o) {
changed[r.Identity()] = true
line := fmt.Sprintf(" %s %s (%s)", o.Action, o.ID, o.Target)
if o.Detail != "" {
line += ": " + o.Detail
}
log(line)
} else if o.Action != "unchanged" {
log(fmt.Sprintf(" %s %s (%s): %s", o.Action, o.ID, o.Target, o.Detail))
}
}
if len(failures) > 0 {
first := failures[0]
first.Done = report
first.Others = len(failures) - 1
return report, known, first
}
return report, known, nil
}
// filledAs is the declared file with the filled content in place of the declaration's.
func filledAs(declared *declaration.File, content []byte) *declaration.File {
f := *declared
f.Content, f.Sealed, f.VaultSealed, f.Bytes, f.Secrets = string(content), "", nil, "", nil
if f.Mode == "" && (declared.Secret() || len(declared.Secrets) > 0) {
// Written as the declaration would have written it: a file carrying a credential is not world-readable.
f.Mode = "0600"
}
return &f
}
// readsChange reports whether applying r is what makes it take a change of what changed: a service's restart-on or
// reload-on, a process's or container's restart-on, or a running container that reads path at creation.
func readsChange(r declaration.Resource, path string, changed map[string]bool) bool {
switch res := r.(type) {
case *declaration.Service:
return len(restartedBy(res.RestartOn, changed)) > 0 || len(restartedBy(res.ReloadOn, changed)) > 0
case *declaration.Process:
return len(restartedBy(res.RestartOn, changed)) > 0
case *declaration.Container:
if len(restartedBy(res.RestartOn, changed)) > 0 {
return true
}
if res.RunOnce || res.Schedule != "" {
return false
}
if slices.Contains(res.EnvFile, path) {
return true
}
for _, v := range res.Volumes {
if mountSource(v) == path {
return true
}
}
}
return false
}
// recordKeeping records r as applied, carrying where the original it replaced was kept from its record when this
// write kept none — for as long as the resource is recorded at the same target, as the apply loop carries it.
func recordKeeping(known *store.State, r declaration.Resource, o Outcome, was store.Applied) {
kept, keptMode, keptOwner := o.kept, o.keptMode, o.keptOwner
if kept == "" && was.Target == o.Target {
kept, keptMode, keptOwner = was.Kept, was.KeptMode, was.KeptOwner
}
recordApplied(known, store.OriginDeclared, r, o, kept, keptMode, keptOwner)
}
// leftToTheClient is a filled file as an apply leaves it (novox/hq ADR 0349 §5, design 57 P7): untouched, and said
// so. A file the mesh wrote at the same path under another id, going in this apply, is handed to it as an apply hands
// a whole file (novox/hq ADR 0223): its record — what the mesh wrote there, and the original it kept — is the filled
// file's now, so the verb judges what is there against it, and the file is neither removed nor written.
func leftToTheClient(f *declaration.File, known *store.State, handedFiles map[string]store.Applied, report *Report,
log func(string)) Outcome {
if from, handed := handedFiles[f.ID]; handed {
if _, recorded := known.Find(f.ID); !recorded {
rec := from
rec.ID = f.ID
known.Record(rec)
}
delete(handedFiles, f.ID)
known.Forget(from.ID)
detail := "no longer declared; " + f.ID + " declares the file now, filled by the provisioner's client, so " +
"it was handed to it, not removed"
report.Outcomes = append(report.Outcomes, Outcome{ID: from.ID, Type: from.Type, Target: from.Target,
Action: "forgotten", Detail: detail})
log(fmt.Sprintf(" forgotten %s (%s): %s", from.ID, from.Target, detail))
}
return Outcome{ID: f.ID, Type: string(declaration.TypeFile), Target: f.Path, Action: "unchanged",
Detail: "filled by the provisioner's client, which alone has it written"}
}
internal/apply/filled_apply_test.go
-218
View File
@@ -1,218 +0,0 @@
package apply
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §5, one writer per file (design 57 P7, issue 673): a file the declaration names as
// filled is written only through the write verb (WriteFilled), on the provisioner client's word. An apply — the
// controller's send and the five-minute re-apply alike — never writes it, never records it, and removes it as an
// orphan only when the verb placed it.
// An apply leaves a filled file alone, whatever who owns it at runtime says: absent stays absent and is not
// recorded, and what the verb wrote stays as it wrote it through every re-apply.
func TestAnApplyNeverWritesAFilledFile(t *testing.T) {
for _, c := range []struct{ name, fields, content string }{
{"owned", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n"},
{"seed", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n"},
{"managed block", `"into":"block","content":"","owner-at-runtime":"managed"`, "10.42.0.1 ace\n"},
} {
t.Run(c.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
d := filledDecl(t, path, c.fields, "")
report, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared,
noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(path); err == nil {
t.Fatalf("an apply wrote a filled file the client has not filled: %q", readText(t, path))
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("an apply recorded a filled file it did not write")
}
if o := outcomeOf(report, "m.conf"); o.Action != "unchanged" || !strings.Contains(o.Detail, "provisioner") {
t.Errorf("an apply did not say it left the file to the provisioner's client: %+v", o)
}
_, state, err = WriteFilled(context.Background(), archHost(t), d, state, path, []byte(c.content),
noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
placed, _ := state.Find("m.conf")
want, wantMode := readText(t, path), modeAt(t, path)
// The re-apply, twice over: nothing written, the verb's record kept as it wrote it.
for i := 0; i < 2; i++ {
var said []string
report, state, err = Apply(context.Background(), archHost(t), d, state, store.OriginDeclared,
noServices, func(line string) { said = append(said, line) }, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != want || modeAt(t, path) != wantMode {
t.Fatalf("re-apply %d wrote the filled file again: %q, want %q", i+1, got, want)
}
if o := outcomeOf(report, "m.conf"); o.Action != "unchanged" {
t.Errorf("re-apply %d: %+v", i+1, o)
}
if rec, _ := state.Find("m.conf"); rec.Wrote != placed.Wrote || rec.Target != path {
t.Errorf("re-apply %d changed the verb's record: %+v, was %+v", i+1, rec, placed)
}
if len(said) != 0 {
t.Errorf("re-apply %d said something of an untouched file: %q", i+1, said)
}
}
})
}
}
// What reads a filled file is moved by the verb's write, once, never again by an apply: a service under
// restart-on is not restarted and a container reading it is not recreated by the re-apply.
func TestAnApplyMovesNothingForAFilledFile(t *testing.T) {
env := filepath.Join(t.TempDir(), "forge.env")
d := filledDecl(t, env, `"content":"","mode":"0600","owner-at-runtime":"owned"`,
`{"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}`)
m := &machine{containers: map[string]*fakeContainer{}}
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, env, []byte("PORT=5432\n"),
m.run, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
m.asked = nil
if _, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, m.run, nil,
nil); err != nil {
t.Fatal(err)
}
if m.removed("forge") || m.did("docker run") {
t.Errorf("the re-apply recreated the container reading the filled file: %v", m.asked)
}
if got := readText(t, env); got != "PORT=5432\n" {
t.Errorf("the re-apply wrote the filled file: %q", got)
}
conf := filepath.Join(t.TempDir(), "daemon.conf")
d = filledDecl(t, conf, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","restart-on":["m.conf"]}`)
var commands []string
_, state, err = WriteFilled(context.Background(), archHost(t), d, store.State{}, conf, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil)
if err != nil {
t.Fatal(err)
}
commands = nil
if _, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared,
recordingServices(&commands), nil, nil); err != nil {
t.Fatal(err)
}
if n := count(commands, "systemctl stop m.service"); n != 0 {
t.Errorf("the re-apply restarted what reads the filled file: %v", commands)
}
}
// A filled file the verb never placed is not the mesh's to remove: one found there, or put there by hand, stays
// when the file leaves the declaration.
func TestAFilledFileTheVerbNeverPlacedIsNotRemoved(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
if err := os.WriteFile(path, []byte("the machine's own\n"), 0o644); err != nil {
t.Fatal(err)
}
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
gone := elsewhere(t)
report, state, err := Apply(context.Background(), archHost(t), gone, state, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != "the machine's own\n" {
t.Errorf("a filled file the verb never placed was touched when it left: %q", got)
}
if o := outcomeOf(report, "m.conf"); o.Action == "removed" {
t.Errorf("a filled file the verb never placed was removed: %+v", o)
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("a record of the filled file was left")
}
}
// What the verb placed is the mesh's: when the file leaves the declaration it is removed as an orphan, as a file
// an apply wrote is.
func TestAFilledFileTheVerbPlacedIsRemovedWhenItLeaves(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
_, state, err = WriteFilled(context.Background(), archHost(t), d, state, path, []byte("port = 8080\n"),
noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
gone := elsewhere(t)
report, state, err := Apply(context.Background(), archHost(t), gone, state, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(path); err == nil {
t.Errorf("a filled file the verb placed stayed when it left the declaration: %q", readText(t, path))
}
if o := outcomeOf(report, "m.conf"); o.Action != "removed" {
t.Errorf("not removed: %+v", o)
}
if _, ok := state.Find("m.conf"); ok {
t.Errorf("its record stayed")
}
}
// A file the mesh wrote the old way and that is now filled keeps its record for the verb to judge against, and is
// not written over by the apply that hands it to the client; a file at the same path recorded under another id
// is handed to the filled one, not removed.
func TestAFileThatMovesToTheClientIsHandedNotRewritten(t *testing.T) {
path := filepath.Join(t.TempDir(), "the-file")
old := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"m.old","type":"file","path":%q,`+
`"content":"port = 1\n"}]}`, path))
_, state, err := Apply(context.Background(), archHost(t), old, store.State{}, store.OriginDeclared, noServices,
nil, nil)
if err != nil {
t.Fatal(err)
}
was, _ := state.Find("m.old")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, state, err = Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := readText(t, path); got != "port = 1\n" {
t.Errorf("the file handed to the client was rewritten or removed: %q", got)
}
if _, ok := state.Find("m.old"); ok {
t.Errorf("the record going was kept beside the filled file's")
}
if rec, ok := state.Find("m.conf"); !ok || rec.Wrote != was.Wrote || rec.Target != path {
t.Errorf("the filled file did not take the record of what the mesh wrote there: %+v", rec)
}
}
// elsewhere is a declaration with nothing at the filled path: one unrelated file.
func elsewhere(t *testing.T) *declaration.Declaration {
t.Helper()
return parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"n.other","type":"file","path":%q,`+
`"content":"x\n"}]}`, filepath.Join(t.TempDir(), "other")))
}
internal/apply/filled_library_test.go
-57
View File
@@ -1,57 +0,0 @@
//go:build linux
package apply
import (
"context"
"fmt"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A container that reads a filled file and mounts a library whose disk is not there is held by the write verb as a
// full apply holds it (novox/hq ADR 0328 §9, issue 668): never created or recreated over the missing disk.
func TestAWriteDoesNotRecreateAContainerHeldForItsLibrary(t *testing.T) {
// No folder is given the group: the library's root is absent, and nothing is made.
gid := 61000
libraries(t, nil, nil)
base := t.TempDir()
root := filepath.Join(base, "gone", "media")
env := filepath.Join(base, "radarr.env")
raw := fmt.Sprintf(`{"declaration":1,"provisioner_account":"mesh-node-provisioner","filled_files":[%q],
"groups":[{"name":"mesh-media","gid":%d}],"libraries":[%s],"resources":[
{"id":"radarr.env","type":"file","path":%q,"content":"","mode":"0600","owner-at-runtime":"owned"},
{"id":"radarr.app","type":"container","name":"radarr","image":%q,"env-file":[%q],"volumes":[%q]}]}`,
env, gid, libJSON("media", root, gid, true, false, "movies"), env, pinned, env,
filepath.Join(root, "movies")+":/movies")
d, err := declaration.Parse([]byte(raw))
if err != nil {
t.Fatal(err)
}
var ran []string
run := func(_ context.Context, name string, args ...string) (string, error) {
ran = append(ran, name+" "+strings.Join(args, " "))
return "", nil
}
var state store.State
for _, content := range []string{"PORT=1\n", "PORT=2\n"} {
var report Report
report, state, err = WriteFilled(context.Background(), archHost(t), d, state, env, []byte(content), run,
nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if o := outcomeOf(report, "radarr.app"); o.Action != "waiting" {
t.Errorf("after writing %q the container is %+v, not held for its library", content, o)
}
}
for _, line := range ran {
if strings.HasPrefix(line, "docker run") || strings.HasPrefix(line, "docker rm") {
t.Errorf("the container was created or removed over a missing disk: %v", ran)
}
}
}
internal/apply/filled_runtime_roots_test.go
-168
View File
@@ -1,168 +0,0 @@
package apply
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// A filled write is one file, not an apply: it asks no container runtime, and judges where nothing is placed
// (issue 339) with the roots the last apply read, falling back to the runtimes' configuration files when no apply
// has read them yet (novox/hq issue 1138). These tests set the package's runtimeRoots and AskRuntimes, so none
// runs in parallel with another that reads or sets them, and each restores what it changed.
// setRuntimeRoots makes runtimeDataRoots answer roots, as if an apply had just read them (or, with nil, as if none
// ever had), restoring what a test before it left there once this one is done.
func setRuntimeRoots(t *testing.T, roots []string) {
t.Helper()
runtimeRootsMu.Lock()
before := runtimeRoots
runtimeRoots = roots
runtimeRootsMu.Unlock()
t.Cleanup(func() {
runtimeRootsMu.Lock()
runtimeRoots = before
runtimeRootsMu.Unlock()
})
}
// askNoRuntime makes AskRuntimes record the name of every runtime it is asked for, answering as one that is not
// running does (nothing, no error): were it consulted, that is what would say so. Restored once the test is done.
func askNoRuntime(t *testing.T, asked *[]string) {
t.Helper()
before := AskRuntimes
AskRuntimes = func(_ context.Context, name string, _ ...string) (string, error) {
*asked = append(*asked, name)
return "", nil
}
t.Cleanup(func() { AskRuntimes = before })
}
// aRuntimeConfig points the runtimes' configuration at a fresh, unmounted root below a temp directory, restoring
// the engine's real configuration once the test is done.
func aRuntimeConfig(t *testing.T) (dir, root string) {
t.Helper()
dir = t.TempDir()
root = filepath.Join(dir, "docker")
daemon := filepath.Join(dir, "daemon.json")
if err := os.WriteFile(daemon, []byte(`{"data-root":"`+root+`"}`), 0o644); err != nil {
t.Fatal(err)
}
before := runtimeConfigs
runtimeConfigs = runtimeFiles{daemonJSON: daemon, storageConf: filepath.Join(dir, "none.conf")}
t.Cleanup(func() { runtimeConfigs = before })
return dir, root
}
// refusedAsPlacement fails the test unless err is a PlacementRefusedError.
func refusedAsPlacement(t *testing.T, err error) {
t.Helper()
var refusal *PlacementRefusedError
if err == nil || !errors.As(err, &refusal) {
t.Fatalf("a filled file below a refused runtime root was not refused: %v", err)
}
}
// 1138/AC1: a filled write asks no runtime, and judges with the roots the last apply read.
func TestAFilledWriteAsksNoRuntimeAndUsesTheLastApplysRoots(t *testing.T) {
root := filepath.Join(t.TempDir(), "docker-root")
setRuntimeRoots(t, []string{root}) // as if an apply had just read it
var asked []string
askNoRuntime(t, &asked)
path := filepath.Join(root, "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
if len(asked) != 0 {
t.Errorf("a filled write asked the container runtimes: %v", asked)
}
}
// 1138/AC2: given a real apply that read a runtime data root, a filled write below it is refused as today, with
// no apply run again in between.
func TestAFilledWriteBelowARootARealApplyReadIsRefused(t *testing.T) {
dir, root := aRuntimeConfig(t)
t.Cleanup(func() {
runtimeRootsMu.Lock()
runtimeRoots = nil
runtimeRootsMu.Unlock()
})
// A full apply of something unrelated reads the runtimes' configuration once, at its start.
elsewhere := filepath.Join(dir, "elsewhere.conf")
if _, _, err := Apply(context.Background(), archHost(t), declare(t,
fmt.Sprintf(`{"id":"m.other","type":"file","path":%q,"content":"x\n"}`, elsewhere)),
store.State{}, store.OriginDeclared, noServices, nil, nil); err != nil {
t.Fatal(err)
}
// A filled write below the runtime's root, with no apply run since, is still refused.
path := filepath.Join(root, "sub", "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
}
// 1138/AC3: with no apply having read the roots yet, a filled write judges by the runtimes' configuration files,
// asking no runtime.
func TestAFilledWriteWithNoApplyYetUsesTheRuntimesConfigFiles(t *testing.T) {
setRuntimeRoots(t, nil) // no apply has read the roots yet
_, root := aRuntimeConfig(t)
var asked []string
askNoRuntime(t, &asked)
path := filepath.Join(root, "sub", "app.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`, "")
_, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("x\n"), noServices,
nil, nil, nil)
refusedAsPlacement(t, err)
if len(asked) != 0 {
t.Errorf("a filled write with no apply yet asked the container runtimes: %v", asked)
}
}
// 1138/AC4: a full apply still asks each runtime once, at its start, whatever the number of files it writes.
func TestAFullApplyAsksEachRuntimeOnceRegardlessOfFileCount(t *testing.T) {
dir := t.TempDir()
var asked []string
before := AskRuntimes
AskRuntimes = func(_ context.Context, name string, _ ...string) (string, error) {
asked = append(asked, name)
return "", errors.New("not running in this test")
}
t.Cleanup(func() {
AskRuntimes = before
runtimeRootsMu.Lock()
runtimeRoots = nil
runtimeRootsMu.Unlock()
})
var resources []string
for i := 0; i < 3; i++ {
resources = append(resources, fmt.Sprintf(`{"id":"m.f%d","type":"file","path":%q,"content":"x\n"}`, i,
filepath.Join(dir, fmt.Sprintf("f%d.conf", i))))
}
if _, _, err := Apply(context.Background(), archHost(t), declare(t, strings.Join(resources, ",")),
store.State{}, store.OriginDeclared, noServices, nil, nil); err != nil {
t.Fatal(err)
}
if got := count(asked, "docker"); got != 1 {
t.Errorf("docker was asked %d times for one apply of 3 files; want 1: %v", got, asked)
}
if got := count(asked, "podman"); got != 1 {
t.Errorf("podman was asked %d times for one apply of 3 files; want 1: %v", got, asked)
}
}
internal/apply/filled_test.go
-271
View File
@@ -1,271 +0,0 @@
package apply
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §4 (issue 668): the provisioner's client has the node-engine write a file it filled,
// and the node-engine writes it as an apply of the declared file with that content would — through applyFile,
// applyInto or applyBlock by who owns it at runtime — and makes what reads it take the change as an apply does.
// filledDecl is a declaration naming path as filled, with the file's fields after id, type and path, and more
// resources after it.
func filledDecl(t *testing.T, path, fields, more string) *declaration.Declaration {
t.Helper()
if more != "" {
more = "," + more
}
return parseTrusted(t, fmt.Sprintf(`{"declaration":1,"provisioner_account":"mesh-node-provisioner",
"filled_files":[%q],"resources":[{"id":"m.conf","type":"file","path":%q,%s}%s]}`, path, path, fields, more))
}
func TestAWriteOfAPathNotFilledIsRefused(t *testing.T) {
dir := t.TempDir()
filled := filepath.Join(dir, "filled.conf")
other := filepath.Join(dir, "other.conf")
d := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"filled_files":[%q],"resources":[
{"id":"m.conf","type":"file","path":%q,"content":"x\n","owner-at-runtime":"owned"},
{"id":"m.other","type":"file","path":%q,"content":"y\n"}
]}`, filled, filled, other))
for _, path := range []string{other, filepath.Join(dir, "nothing.conf")} {
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("evil\n"),
noServices, nil, nil, nil)
if err == nil || !strings.Contains(err.Error(), "does not name") {
t.Errorf("%s: a write of a path the declaration does not fill was not refused: %v", path, err)
}
if _, err := os.Stat(path); err == nil && path != other {
t.Errorf("%s was written", path)
}
if len(state.Resources) != 0 {
t.Errorf("a refused write recorded something: %+v", state.Resources)
}
}
if _, err := os.Stat(other); err == nil {
t.Errorf("%s was written", other)
}
}
// Each ownership is written by the write verb exactly as an apply of the declared file with the same content: the
// same action, the same bytes on disk, the same mode.
func TestAFilledFileIsWrittenAsItsOwnershipSays(t *testing.T) {
for _, c := range []struct {
name, fields, content, before, action string
}{
{"owned, absent", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n", "", "created"},
{"owned, over what is there", `"content":"","mode":"0640","owner-at-runtime":"owned"`, "port = 8080\n",
"port = 1\n", "updated"},
{"seed, absent", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n", "", "created"},
{"seed, present", `"content":"","mode":"0600","owner-at-runtime":"seed"`, "user default on\n",
"user default on\nuser app-one on\n", "kept"},
{"managed json", `"into":"json","content":"{}","owner-at-runtime":"managed"`,
`{"insecure-registries":["10.42.0.1:5000"]}`, machinesOwn, "updated"},
{"managed block", `"into":"block","content":"x\n","owner-at-runtime":"managed"`, "10.42.0.1 ace\n",
workstationHosts, "updated"},
} {
t.Run(c.name, func(t *testing.T) {
at := func() string {
path := filepath.Join(t.TempDir(), "the-file")
if c.before != "" {
if err := os.WriteFile(path, []byte(c.before), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
written := at()
report, state, err := WriteFilled(context.Background(), archHost(t), filledDecl(t, written, c.fields, ""),
store.State{}, written, []byte(c.content), noServices, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
applied := at()
withContent := strings.Replace(c.fields, fmt.Sprintf(`"content":%q`, contentOf(t, c.fields)),
fmt.Sprintf(`"content":%q`, c.content), 1)
// The same file declared the old way: an apply writes a filled file only on the client's word (P7).
plain := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"m.conf","type":"file",`+
`"path":%q,%s}]}`, applied, withContent))
want, _, err := Apply(context.Background(), archHost(t), plain, store.State{}, store.OriginDeclared,
noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := report.Outcomes[0].Action; got != c.action || got != want.Outcomes[0].Action {
t.Errorf("written %s; an apply of the same content %s; want %s", got, want.Outcomes[0].Action, c.action)
}
if got, apply := readText(t, written), readText(t, applied); got != apply {
t.Errorf("written, the file holds\n%q\nan apply of the same content leaves\n%q", got, apply)
}
if a, b := modeAt(t, written), modeAt(t, applied); a != b {
t.Errorf("written mode %o, an apply's %o", a, b)
}
if rec, ok := state.Find("m.conf"); !ok || rec.Target != written || rec.Origin != store.OriginDeclared {
t.Errorf("the write is not recorded as the file's: %+v", rec)
}
})
}
}
func contentOf(t *testing.T, fields string) string {
t.Helper()
var holder struct {
Content string `json:"content"`
}
if err := json.Unmarshal([]byte("{"+fields+"}"), &holder); err != nil {
t.Fatal(err)
}
return holder.Content
}
func modeAt(t *testing.T, path string) os.FileMode {
t.Helper()
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
return info.Mode().Perm()
}
// A file that carried a secret keeps the mode a secret is written with: the client filled the value, and the file
// is no more readable for it.
func TestAFilledFileThatCarriedASecretIsNotWorldReadable(t *testing.T) {
path := filepath.Join(t.TempDir(), "token.env")
d := filledDecl(t, path, `"content":"TOKEN=${secret:token}\n","secrets":{"token":"c2VhbGVk"},"owner-at-runtime":"owned"`, "")
if _, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("TOKEN=abc\n"),
noServices, nil, nil, nil); err != nil {
t.Fatal(err)
}
if m := modeAt(t, path); m != 0o600 {
t.Errorf("a filled file that carried a secret is mode %o", m)
}
}
func TestAWriteUnderReloadOnReloadsItsProgramOnce(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","reload-on":["m.conf"]},
{"id":"m.other","type":"service","unit":"other.service","state":"running"}`)
var commands []string
var said []string
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("level = 1\n"),
recordingServices(&commands), func(line string) { said = append(said, line) }, nil, nil)
if err != nil {
t.Fatal(err)
}
if n := count(commands, "systemctl reload m.service"); n != 1 {
t.Errorf("the program was reloaded %d times; commands were %v", n, commands)
}
joined := strings.Join(commands, "\n")
if strings.Contains(joined, "restart m.service") || strings.Contains(joined, "stop m.service") {
t.Errorf("the program was restarted, not reloaded: %v", commands)
}
if strings.Contains(joined, "other.service") {
t.Errorf("a service that does not read the file was touched: %v", commands)
}
if _, ok := state.Find("m.daemon"); !ok {
t.Errorf("the reloaded service was not recorded")
}
// The log says the path and the digest, never the content.
for _, line := range said {
if strings.Contains(line, "level = 1") {
t.Errorf("the content was said: %q", line)
}
}
if !strings.Contains(strings.Join(said, "\n"), digestOf("level = 1\n")) {
t.Errorf("the digest was not said: %q", said)
}
// The same content again moves nothing, and reloads nothing.
commands = nil
report, _, err := WriteFilled(context.Background(), archHost(t), d, state, path, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if report.Outcomes[0].Action != "unchanged" || count(commands, "reload") != 0 {
t.Errorf("an unchanged write reloaded: %v %+v", commands, report.Outcomes)
}
}
func TestAWriteUnderRestartOnRestartsItsProgram(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.conf")
d := filledDecl(t, path, `"content":"","owner-at-runtime":"owned"`,
`{"id":"m.daemon","type":"service","unit":"m.service","state":"running","restart-on":["m.conf"]}`)
var commands []string
if _, _, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("level = 1\n"),
recordingServices(&commands), nil, nil, nil); err != nil {
t.Fatal(err)
}
// A restart is a stop and a start, once.
stops, starts := count(commands, "systemctl stop m.service"), count(commands, "systemctl start m.service")
if stops != 1 || starts != 1 {
t.Errorf("the program was stopped %d and started %d times; commands were %v", stops, starts, commands)
}
}
// A running container that reads the file at creation is recreated, as an apply recreates it for a changed file.
func TestAWriteOfAFileAContainerReadsRecreatesIt(t *testing.T) {
env := filepath.Join(t.TempDir(), "forge.env")
d := filledDecl(t, env, `"content":"","mode":"0600","owner-at-runtime":"owned"`,
`{"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}`)
m := &machine{containers: map[string]*fakeContainer{}}
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, env, []byte("PORT=5432\n"),
m.run, nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if !m.did("docker run") {
t.Fatalf("the container was not created: %v", m.asked)
}
m.asked = nil
report, _, err := WriteFilled(context.Background(), archHost(t), d, state, env, []byte("PORT=5433\n"), m.run,
nil, nil, nil)
if err != nil {
t.Fatal(err)
}
if !m.removed("forge") || !m.did("docker run") {
t.Fatalf("the container kept what it read before the write: %v", m.asked)
}
if o := outcomeOf(report, "forge.server"); o.Action != "updated" || !strings.Contains(o.Detail, env) {
t.Errorf("the recreate did not say which file changed: %+v", o)
}
}
func count(lines []string, prefix string) int {
n := 0
for _, l := range lines {
if strings.Contains(l, prefix) {
n++
}
}
return n
}
// Two resources declaring one filled path — two modules' blocks in one file — are refused: the request carries the
// content only, so it cannot say whose block it is, and nothing is written.
func TestAFilledPathTwoResourcesDeclareIsRefused(t *testing.T) {
path := filepath.Join(t.TempDir(), "hosts")
if err := os.WriteFile(path, []byte(workstationHosts), 0o644); err != nil {
t.Fatal(err)
}
d := parseTrusted(t, fmt.Sprintf(`{"declaration":1,"filled_files":[%q],"resources":[
{"id":"a.hosts","type":"file","path":%q,"into":"block","content":"10.42.0.1 ace\n","owner-at-runtime":"managed"},
{"id":"b.hosts","type":"file","path":%q,"into":"block","content":"10.42.0.2 g14\n","owner-at-runtime":"managed"}
]}`, path, path, path))
_, state, err := WriteFilled(context.Background(), archHost(t), d, store.State{}, path, []byte("6.6.6.6 evil\n"),
noServices, nil, nil, nil)
if err == nil || !strings.Contains(err.Error(), "2 resources declare") {
t.Fatalf("a path two resources declare was not refused: %v", err)
}
if got := readText(t, path); got != workstationHosts || len(state.Resources) != 0 {
t.Errorf("something was written or recorded: %q %+v", got, state.Resources)
}
}
internal/apply/groups.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/health_check_test.go
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/hold.go
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"syscall"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/hold_test.go
+1 -8
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
@@ -185,13 +185,6 @@ func (m *machine) run(_ context.Context, name string, args ...string) (string, e
return "", errors.New("no such image")
}
return "", nil
case "pull":
// A real runtime leaves the image present after a pull.
if m.images == nil {
m.images = map[string]string{}
}
m.images[args[len(args)-1]] = "2026-01-01T00:00:00Z"
return "", nil
case "network":
if len(args) > 1 && args[1] == "inspect" {
return strings.Join(m.members[args[len(args)-1]], ",") + ",\n", nil
internal/apply/homes_other.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"os"
"path/filepath"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
func chmodUnder(home, path string, mode os.FileMode) error {
internal/apply/identities.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/identities_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/into.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"slices"
"sort"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/into_handover_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/into_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/keep.go
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
internal/apply/left_out_test.go
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/library.go
+7 -13
View File
@@ -11,7 +11,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
@@ -318,17 +318,6 @@ func heldRoots(verdicts map[string]libraryVerdict, libs []declaration.Library) m
return out
}
// heldLibraries is the root of every library whose disk or mount is not there, with its name, judged now and read
// only: what a full apply holds its containers for (ADR 0328 §9), for one that judges without applying libraries.
func heldLibraries(ctx context.Context, libs []declaration.Library) map[string]string {
verdicts := map[string]libraryVerdict{}
for _, l := range libs {
rec, _ := libraryRecord(l.Library)
verdicts[l.Library] = assessLibrary(ctx, l, rec)
}
return heldRoots(verdicts, libs)
}
// mountsHeld is the library a container waits for: one whose root it mounts, or mounts below, is held.
func mountsHeld(c *declaration.Container, held map[string]string) string {
for _, v := range c.Volumes {
@@ -352,7 +341,12 @@ func WaitingOnLibrary(d *declaration.Declaration) map[string]bool {
if d == nil || len(d.Libraries) == 0 {
return out
}
held := heldLibraries(context.Background(), d.Libraries)
verdicts := map[string]libraryVerdict{}
for _, l := range d.Libraries {
rec, _ := libraryRecord(l.Library)
verdicts[l.Library] = assessLibrary(context.Background(), l, rec)
}
held := heldRoots(verdicts, d.Libraries)
for _, r := range d.Resources {
if c, ok := r.(*declaration.Container); ok && mountsHeld(c, held) != "" {
out[c.ID] = true
internal/apply/library_linux.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/library_test.go
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"golang.org/x/sys/unix"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/logging_test.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/maintenance_window_test.go
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/mesh_dirs.go
+2 -3
View File
@@ -30,7 +30,7 @@ import (
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// LinkInPathError is a mesh directory refused because a link in its path is one another account could replace.
@@ -94,14 +94,13 @@ func groupNamed(gid uint32) string {
func JudgeDirectories(d *declaration.Declaration) string {
var lines []string
var placed, refused, guarded, homed int
declared := declaredHomes(d.Resources)
for _, r := range d.Resources {
dir, ok := r.(*declaration.Directory)
if !ok {
continue
}
path := filepath.Clean(dir.Path)
if err := refusePathWith(path, placingDirectory, moduleOfID(dir.ID), dir.Owner, declared); err != nil {
if err := refusePath(path, placingDirectory, moduleOfID(dir.ID), dir.Owner); err != nil {
guarded++
lines = append(lines, path+": refused already by the placement guard: "+err.Error())
continue
internal/apply/mesh_dirs_apply_linux.go
+1 -1
View File
@@ -14,7 +14,7 @@ import (
"golang.org/x/sys/unix"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// meshDirsByDescriptor says this system can open a directory component by component without following a link.
internal/apply/mesh_dirs_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/mesh_names_test.go
+1 -1
View File
@@ -3,7 +3,7 @@ package apply
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// **A container's mesh names are part of what it is** (novox/hq 04-ISSUES/135).
internal/apply/mode.go
+1 -1
View File
@@ -3,7 +3,7 @@ package apply
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/module_results.go
-99
View File
@@ -1,99 +0,0 @@
package apply
import (
"errors"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// failuresOf is every failure of an apply as the error carries it: the first among them, then the rest, so a module
// whose resource failed second is still named (issue 1132, and issue 500, which counted the rest and named none).
func failuresOf(failures []*Error) []Failure {
out := make([]Failure, 0, len(failures))
for _, f := range failures {
out = append(out, Failure{Resource: f.Resource, Err: f.Err})
}
return out
}
// ModuleResults groups an apply's outcome per module the declaration names (novox/hq ADR 0429 §4, issue 1132): the
// module's resources all applied, or it failed and Detail says which resource failed with its error, which resources
// were skipped behind a step that did not complete, or that its resources were not attempted because a step of no
// module stopped the apply. A module is named by what the declaration says of each resource (ModuleOfID), so one
// whose id would guess another is still its own.
//
// A failure that is no resource's (an error that is not an Error, or a resource of no module) belongs to no module and
// changes none: the node says it in its report. The report is the outcome of an apply that did not fail; an apply that
// failed carries its own account in the Error, and that is read in its place.
func ModuleResults(d *declaration.Declaration, report Report, err error) map[string]ModuleResult {
if d == nil {
return nil
}
out := map[string]ModuleResult{}
attempted := map[string]bool{}
var failed *Error
errors.As(err, &failed)
outcomes := report.Outcomes
if failed != nil {
outcomes = append(append([]Outcome(nil), outcomes...), failed.Done.Outcomes...)
}
skipped := map[string]string{}
for _, o := range outcomes {
module, ok := d.ModuleOfID(o.ID)
if !ok {
continue
}
if o.Action == "skipped" {
skipped[module] = o.Detail
continue
}
attempted[o.ID] = true
}
var reasons = map[string][]string{}
for _, r := range d.Resources {
if module, ok := d.ModuleOfID(r.Identity()); ok {
out[module] = ModuleResult{}
if failed != nil && failed.Stopped && !attempted[r.Identity()] {
reasons[module] = append(reasons[module], fmt.Sprintf("%s was not attempted: %s stopped the apply", r.Identity(), failed.Resource))
}
}
}
if failed != nil {
for _, f := range failed.Failures {
if module, ok := d.ModuleOfID(f.Resource); ok {
if _, named := out[module]; named {
reasons[module] = append(reasons[module], fmt.Sprintf("%s failed: %v", f.Resource, f.Err))
}
}
}
if len(failed.Failures) == 0 {
if module, ok := d.ModuleOfID(failed.Resource); ok {
if _, named := out[module]; named {
reasons[module] = append(reasons[module], fmt.Sprintf("%s failed: %v", failed.Resource, failed.Err))
}
}
}
for _, module := range failed.Modules {
if _, named := out[module]; named {
why := skipped[module]
if why == "" {
why = "a step it declares did not complete"
}
reasons[module] = append(reasons[module], "gated: "+why)
}
}
}
for module, why := range skipped {
if _, named := out[module]; named && len(reasons[module]) == 0 {
reasons[module] = append(reasons[module], "gated: "+why)
}
}
for module, why := range reasons {
sort.Strings(why)
out[module] = ModuleResult{Failed: true, Detail: strings.Join(why, "; ")}
}
return out
}
internal/apply/module_results_test.go
-76
View File
@@ -1,76 +0,0 @@
package apply
import (
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
func twoModules(t *testing.T) *declaration.Declaration {
t.Helper()
d, err := declaration.Parse([]byte(`{"declaration":1,"builds":{"a":"b-1","b":"b-2"},"resources":[
{"id":"a.dir","type":"directory","path":"/srv/a","module":"a"},
{"id":"b.dir","type":"directory","path":"/srv/b","module":"b"},
{"id":"b.conf","type":"file","path":"/srv/b/c","content":"x","module":"b"},
{"id":"c.dir","type":"directory","path":"/srv/c","module":"c"}]}`))
if err != nil {
t.Fatal(err)
}
return d
}
// 1132/AC3: every resource of module A applied and one of module B failed: A is a success, B failed with the failing
// resource and its error named; a module gated by another's failure is failed too.
func TestAnApplyIsGroupedPerModule(t *testing.T) {
d := twoModules(t)
done := Report{Outcomes: []Outcome{
{ID: "a.dir", Action: "created"},
{ID: "c.dir", Action: "skipped", Detail: "a step this module declares did not complete"},
}}
err := &Error{Resource: "b.dir", Err: errors.New("no space left"), Done: done, Others: 1,
Failures: []Failure{{Resource: "b.dir", Err: errors.New("no space left")}, {Resource: "b.conf", Err: errors.New("denied")}},
Modules: []string{"c"}}
got := ModuleResults(d, Report{}, err)
if len(got) != 3 {
t.Fatalf("modules %v, want a, b and c", got)
}
if got["a"].Failed || got["a"].Detail != "" {
t.Errorf("a: %+v, want a clean success", got["a"])
}
b := got["b"]
if !b.Failed || !strings.Contains(b.Detail, "b.dir") || !strings.Contains(b.Detail, "no space left") ||
!strings.Contains(b.Detail, "b.conf") || !strings.Contains(b.Detail, "denied") {
t.Errorf("b: %+v, want both failing resources and their errors", b)
}
c := got["c"]
if !c.Failed || !strings.Contains(c.Detail, "did not complete") {
t.Errorf("c: %+v, want it failed as gated, with why", c)
}
}
// 1132/AC3: an apply that did not fail is a success for every module the declaration names.
func TestACleanApplyIsASuccessForEveryModule(t *testing.T) {
got := ModuleResults(twoModules(t), Report{}, nil)
for _, m := range []string{"a", "b", "c"} {
if r, ok := got[m]; !ok || r.Failed {
t.Errorf("%s: %+v (%v), want a success", m, r, ok)
}
}
}
// 1132/AC3: a step that belongs to no module stops the apply there; a module with a resource not attempted is not a
// success.
func TestAModuleNotAttemptedIsNotASuccess(t *testing.T) {
d := twoModules(t)
err := &Error{Resource: "boot", Err: errors.New("boom"), Stopped: true, Failures: []Failure{{Resource: "boot", Err: errors.New("boom")}},
Done: Report{Outcomes: []Outcome{{ID: "a.dir", Action: "created"}}}}
got := ModuleResults(d, Report{}, err)
if got["a"].Failed {
t.Errorf("a was done: %+v", got["a"])
}
if !got["b"].Failed || !strings.Contains(got["b"].Detail, "boot") {
t.Errorf("b was not attempted: %+v", got["b"])
}
}
internal/apply/move_out.go
-371
View File
@@ -1,371 +0,0 @@
package apply
// A directory or archive that says unknown "move-out" keeps only what it knows (issue 1024, novox/hq ADR 0422).
//
// Everything inside it that nothing declared, no archive unpacked and `knows` does not name is moved, whole, into
// `<home>/.local/state/mesh-moved/<module>/<UTC time>/`, with a list beside it that names each move before it is
// made (ADR 0417 §7). Nothing is deleted: the owner of the path reacts to a file it did not write by keeping it
// where its writer can find it. The pass runs after the resources of an apply, so a reconcile (every five minutes)
// runs it too.
import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// movedOutWait is how old a path is before it is moved: a younger one may be mid-write by whatever made it, and
// waits for the next look (ADR 0417 §4).
const movedOutWait = time.Minute
// MovedOut is one path a move-out directory moved.
type MovedOut struct {
Resource string `json:"resource"`
From string `json:"from"`
To string `json:"to"`
}
// LeftInPlace is a socket, pipe or device a move-out directory did not move (ADR 0422 §4).
type LeftInPlace struct {
Resource string `json:"resource"`
Path string `json:"path"`
}
// renamePath and deviceOf are the machine's calls, variables so a test can fail a move or name another file system.
// renamePath renames from to to without following a link in either parent below home (move_out_linux.go).
var renamePath = renameUnder
type movedEntry struct {
From string `json:"from"`
To string `json:"to"`
}
// moveOutOwner is the user and group the kept folders and the list are given: the declared owner, or the home's
// account.
func moveOutOwner(home, owner string) (uid, gid int, err error) {
if owner != "" {
return idsOf(owner)
}
acc, ok := accountsOfHomes()[home]
if !ok {
return 0, 0, fmt.Errorf("%s is not the home of an account", home)
}
if u, g, err := idsOf(acc.Name); err == nil {
return u, g, nil
}
return acc.UID, acc.UID, nil
}
// knownUnder is what a directory knows of its own contents: the declared targets, the files an archive unpacked,
// `knows` entries, and every folder above one of them.
type knownUnder struct {
exact map[string]bool
prefixes []string
// own are nested directories and archives that say what they do with unknown paths themselves.
own map[string]bool
}
func (k *knownUnder) add(root, path string) {
path = filepath.Clean(path)
for path != root && strings.HasPrefix(path, root+string(os.PathSeparator)) {
k.exact[path] = true
path = filepath.Dir(path)
}
}
func (k *knownUnder) knows(path string) bool {
if k.exact[path] {
return true
}
for _, p := range k.prefixes {
if path == p || strings.HasPrefix(path, p+string(os.PathSeparator)) {
return true
}
}
return false
}
func knownOf(d *declaration.Declaration, known store.State, id, root string, knows []string) *knownUnder {
k := &knownUnder{exact: map[string]bool{}, own: map[string]bool{}}
for _, r := range d.Resources {
target := filepath.Clean(r.Target())
if target == "" || target == "." {
continue
}
if r.Identity() != id && strings.HasPrefix(target, root+string(os.PathSeparator)) {
k.add(root, target)
switch o := r.(type) {
case *declaration.Directory:
if o.Unknown != "" {
k.own[target] = true
}
case *declaration.Archive:
if o.Unknown != "" {
k.own[target] = true
}
}
}
if a, ok := r.(*declaration.Archive); ok {
if rec, found := known.Find(a.ID); found && rec.Unpacked != nil {
base := filepath.Clean(a.Path)
for _, f := range append(append([]string{}, rec.Unpacked.Files...), rec.Unpacked.Dirs...) {
p := filepath.Join(base, filepath.FromSlash(f))
if p == root || strings.HasPrefix(p, root+string(os.PathSeparator)) {
k.add(root, p)
}
}
}
}
}
for _, e := range knows {
p := filepath.Join(root, filepath.FromSlash(strings.TrimSuffix(e, "/")))
if strings.HasSuffix(e, "/") {
k.prefixes = append(k.prefixes, p)
} else {
k.exact[p] = true
}
k.add(root, p)
}
return k
}
// moveOut moves what the directory at dir does not know, for the resource id. It answers the paths moved.
func moveOut(d *declaration.Declaration, known store.State, id, module, dir, owner string, knows []string,
log func(string)) ([]MovedOut, []LeftInPlace, error) {
dir = filepath.Clean(dir)
home := homeAbove(dir)
if home == "" {
return nil, nil, fmt.Errorf("%s: move-out is only for a directory under an account's home, and %s is under none "+
"(ADR 0422 §5); nothing was moved", id, dir)
}
k := knownOf(d, known, id, dir, knows)
skip := filepath.Join(keptRoot(home), module)
var candidates []string
var left []string
err := filepath.WalkDir(dir, func(path string, e fs.DirEntry, err error) error {
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil
}
return err
}
if path == dir {
return nil
}
if path == skip || path == keptRoot(home) {
return fs.SkipDir
}
if k.own[path] {
if e.IsDir() {
return fs.SkipDir
}
return nil
}
if k.knows(path) {
if e.IsDir() && !k.exact[path] {
return fs.SkipDir
}
return nil
}
// Unknown. Moved whole, so never descended into.
info, err := e.Info()
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil
}
return err
}
switch {
case info.Mode()&(os.ModeSocket|os.ModeNamedPipe|os.ModeDevice|os.ModeCharDevice) != 0:
left = append(left, path)
case time.Since(info.ModTime()) < movedOutWait:
default:
candidates = append(candidates, path)
}
if e.IsDir() {
return fs.SkipDir
}
return nil
})
if err != nil {
return nil, nil, fmt.Errorf("%s: cannot look through %s: %w", id, dir, err)
}
for _, p := range left {
log(fmt.Sprintf(" left %s in %s: a socket, pipe or device is not moved (ADR 0422)", p, dir))
}
var stayed []LeftInPlace
for _, p := range left {
stayed = append(stayed, LeftInPlace{Resource: id, Path: p})
}
if len(candidates) == 0 {
return nil, stayed, nil
}
sort.Strings(candidates)
uid, gid, err := moveOutOwner(home, owner)
if err != nil {
return nil, nil, fmt.Errorf("%s: %w", id, err)
}
// The same file system first, so a move is a rename and nothing is copied (ADR 0417 §5).
ancestor := skip
for {
if _, err := os.Lstat(ancestor); err == nil {
break
}
ancestor = filepath.Dir(ancestor)
}
devDir, err1 := deviceOf(dir)
devKept, err2 := deviceOf(ancestor)
if err1 != nil || err2 != nil {
return nil, nil, fmt.Errorf("%s: cannot read the file system of %s or %s: %v %v", id, dir, ancestor, err1, err2)
}
if devDir != devKept {
return nil, nil, fmt.Errorf("%s: %s and %s are on different file systems (devices %d and %d), so a move would "+
"copy; nothing was moved (ADR 0417 §5)", id, dir, ancestor, devDir, devKept)
}
// The kept folder and the list, named by the second; a clash in the same second gets -2, -3.
stamp := time.Now().UTC().Format("20060102T150405Z")
kept, listPath := "", ""
for n := 1; ; n++ {
name := stamp
if n > 1 {
name = fmt.Sprintf("%s-%d", stamp, n)
}
kept, listPath = filepath.Join(skip, name), filepath.Join(skip, name+".json")
if !pathExists(kept) && !pathExists(listPath) {
break
}
}
var made []string
cleanup := func() {
_ = os.Remove(listPath)
// In the reverse of the order they were made, so a folder goes after what is in it.
for i := len(made) - 1; i >= 0; i-- {
_ = os.Remove(made[i])
}
}
chown := func(paths []string) error {
for _, p := range paths {
if err := chownIfNeeded(home, p, uid, gid); err != nil {
return err
}
}
return nil
}
madeNow, err := mkdirAllUnder(home, kept, 0o700)
made = append(made, reversed(madeNow)...)
if err == nil {
err = chown(madeNow)
}
if err != nil {
cleanup()
return nil, nil, fmt.Errorf("%s: cannot make the kept folder %s: %w", id, kept, err)
}
var list []movedEntry
var moved []MovedOut
var failures []string
for _, from := range candidates {
rel, _ := filepath.Rel(dir, from)
to := filepath.Join(kept, rel)
// Named before it is moved: a move that stops half way leaves its entry (ADR 0417 §7).
list = append(list, movedEntry{From: from, To: to})
raw, _ := json.MarshalIndent(list, "", " ")
if err := writeUnder(home, listPath, append(raw, '\n'), 0o600); err != nil {
failures = append(failures, fmt.Sprintf("cannot write the list %s: %v", listPath, err))
break
}
if err := chownIfNeeded(home, listPath, uid, gid); err != nil {
failures = append(failures, err.Error())
break
}
parents, err := mkdirAllUnder(home, filepath.Dir(to), 0o700)
made = append(made, reversed(parents)...)
if err == nil {
err = chown(parents)
}
if err == nil {
err = renamePath(home, from, to)
}
if err != nil {
failures = append(failures, fmt.Sprintf("cannot move %s to %s: %v", from, to, err))
continue
}
log(fmt.Sprintf(" moved %s to %s (ADR 0422)", from, to))
moved = append(moved, MovedOut{Resource: id, From: from, To: to})
}
if len(moved) == 0 {
cleanup()
}
if len(failures) > 0 {
return moved, stayed, fmt.Errorf("%s: %s", id, strings.Join(failures, "; "))
}
return moved, stayed, nil
}
// reversed is paths shallowest first: mkdirAllUnder answers deepest first.
func reversed(in []string) []string {
out := make([]string, len(in))
for i, p := range in {
out[len(in)-1-i] = p
}
return out
}
func keptRoot(home string) string { return filepath.Join(home, ".local", "state", "mesh-moved") }
// chownIfNeeded gives a path to uid and gid through the home-safe call, unless it is theirs already.
func chownIfNeeded(home, path string, uid, gid int) error {
if u, g, ok := ownerIDs(path); ok && u == uid && g == gid {
return nil
}
return chownUnder(home, path, uid, gid)
}
// moveOutPass runs move-out for every directory and archive of d that says so and applied in this run.
func moveOutPass(d *declaration.Declaration, known store.State, applied map[string]bool, log func(string)) (
moved []MovedOut, left []LeftInPlace, failures []*Error) {
for _, r := range d.Resources {
var unknown, owner string
var knows []string
switch o := r.(type) {
case *declaration.Directory:
unknown, owner, knows = o.Unknown, o.Owner, o.Knows
case *declaration.Archive:
unknown, owner, knows = o.Unknown, o.Owner, o.Knows
default:
continue
}
if unknown != declaration.UnknownMoveOut || !applied[r.Identity()] {
continue
}
module, _ := d.ModuleOfID(r.Identity())
if module == "" {
module = r.Identity()
}
m, l, err := moveOut(d, known, r.Identity(), module, r.Target(), owner, knows, log)
moved = append(moved, m...)
left = append(left, l...)
if err != nil {
log(fmt.Sprintf(" failed %s (%s): %v", r.Identity(), r.Target(), err))
failures = append(failures, &Error{Resource: r.Identity(), Err: err})
}
}
return moved, left, failures
}
func pathExists(path string) bool {
_, err := os.Lstat(path)
return err == nil
}
internal/apply/move_out_linux.go
-61
View File
@@ -1,61 +0,0 @@
//go:build linux
package apply
import (
"fmt"
"path/filepath"
"syscall"
"golang.org/x/sys/unix"
)
// deviceOf is the device a path lives on; a variable so a test can name another file system. Below a home it is
// read through a descriptor opened without following a link (issue 1024).
var deviceOf = func(path string) (uint64, error) {
var st unix.Stat_t
if home := homeAbove(path); home != "" {
fd, err := openDirUnder(home, path)
if err != nil {
return 0, err
}
defer unix.Close(fd)
if err := unix.Fstat(fd, &st); err != nil {
return 0, err
}
return uint64(st.Dev), nil
}
if err := unix.Stat(path, &st); err != nil {
return 0, err
}
return uint64(st.Dev), nil
}
// renameUnder moves from to to, both below home, through descriptors of their parents opened without following a
// link, and never over something that is there: an account that owns the folders cannot point either at another
// place between the look and the move (issue 1024, novox/hq ADR 0266).
func renameUnder(home, from, to string) error {
fromDir, err := openDirUnder(home, filepath.Dir(from))
if err != nil {
return err
}
defer unix.Close(fromDir)
toDir, err := openDirUnder(home, filepath.Dir(to))
if err != nil {
return err
}
defer unix.Close(toDir)
if err := unix.Renameat2(fromDir, filepath.Base(from), toDir, filepath.Base(to), unix.RENAME_NOREPLACE); err != nil {
return fmt.Errorf("rename %s to %s: %w", from, to, err)
}
return nil
}
// ownerIDs is the user and group of a path, without following a link.
func ownerIDs(path string) (uid, gid int, ok bool) {
var st syscall.Stat_t
if err := syscall.Lstat(path, &st); err != nil {
return 0, 0, false
}
return int(st.Uid), int(st.Gid), true
}
internal/apply/move_out_other.go
-19
View File
@@ -1,19 +0,0 @@
//go:build !linux
package apply
import "os"
var deviceOf = func(string) (uint64, error) { return 0, nil }
func ownerIDs(string) (int, int, bool) { return 0, 0, false }
func renameUnder(home, from, to string) error {
if err := refuseLinksUnderHome(from); err != nil {
return err
}
if err := refuseLinksUnderHome(to); err != nil {
return err
}
return os.Rename(from, to)
}
internal/apply/move_out_test.go
-355
View File
@@ -1,355 +0,0 @@
package apply
// Issue 1024, novox/hq ADR 0422: a directory that says unknown "move-out" keeps only what it knows, and moves every
// other path into a kept folder under the account's home, listed before it is moved.
import (
"context"
"encoding/json"
"errors"
"os"
osuser "os/user"
"path/filepath"
"strings"
"syscall"
"testing"
"time"
"github.com/novox/mesh-host/internal/store"
"golang.org/x/sys/unix"
)
// me is the account running the test, which owns the home the test names.
func me() string {
u, err := osuser.Current()
if err != nil {
panic(err)
}
return u.Username
}
// aMoveOutHome is a home the test names, of the account running the test, so no chown needs privilege.
func aMoveOutHome(t *testing.T) string {
t.Helper()
home := filepath.Join(t.TempDir(), "home", "agent")
if err := os.MkdirAll(home, 0o755); err != nil {
t.Fatal(err)
}
wasHomes, wasAccounts := homes, accountsOfHomes
homes = func() []string { return []string{home} }
accountsOfHomes = func() map[string]homeAccount {
return map[string]homeAccount{home: {Name: me(), UID: os.Getuid()}}
}
t.Cleanup(func() { homes, accountsOfHomes = wasHomes, wasAccounts })
return home
}
func put(t *testing.T, path, content string, age time.Duration) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
old := time.Now().Add(-age)
if err := os.Chtimes(path, old, old); err != nil {
t.Fatal(err)
}
}
func keptLists(t *testing.T, home string) []string {
t.Helper()
out, _ := filepath.Glob(filepath.Join(keptRoot(home), "*", "*.json"))
return out
}
func readList(t *testing.T, path string) []movedEntry {
t.Helper()
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var list []movedEntry
if err := json.Unmarshal(raw, &list); err != nil {
t.Fatal(err)
}
return list
}
func TestAMoveOutDirectoryMovesOnlyWhatIsNotKnown(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, ".ssh")
outside := filepath.Join(t.TempDir(), "target")
put(t, outside, "elsewhere", time.Hour)
put(t, filepath.Join(dir, "config"), "declared", time.Hour)
put(t, filepath.Join(dir, "known_hosts"), "knows", time.Hour)
put(t, filepath.Join(dir, "x", "inside"), "in x", time.Hour)
put(t, filepath.Join(dir, "stray"), "stray content", time.Hour)
put(t, filepath.Join(dir, "oldfolder", "deep", "f"), "deep", time.Hour)
put(t, filepath.Join(dir, "young"), "just made", 5*time.Second)
if err := os.Symlink(outside, filepath.Join(dir, "alink")); err != nil {
t.Fatal(err)
}
old := time.Now().Add(-time.Hour)
_ = os.Chtimes(filepath.Join(dir, "oldfolder"), old, old)
_ = os.Chtimes(filepath.Join(dir, "x"), old, old)
// A named pipe stands for the socket: a socket's path is too long under a test directory to bind.
if err := syscall.Mkfifo(filepath.Join(dir, "agent.sock"), 0o644); err != nil {
t.Fatal(err)
}
_ = os.Chtimes(filepath.Join(dir, "agent.sock"), old, old)
// The directory itself and its declared file are old enough too.
// The symbolic link is older than a minute as well.
ts := []unix.Timespec{unix.NsecToTimespec(old.UnixNano()), unix.NsecToTimespec(old.UnixNano())}
if err := unix.UtimesNanoAt(unix.AT_FDCWD, filepath.Join(dir, "alink"), ts, unix.AT_SYMLINK_NOFOLLOW); err != nil {
t.Fatal(err)
}
var logged []string
report, _, err := Apply(context.Background(), archHost(t), declare(t,
`{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out","knows":["known_hosts","x/"]},`+
`{"id":"mod.config","type":"file","path":"`+dir+`/config","content":"declared"}`),
store.State{}, store.OriginDeclared, noServices, func(s string) { logged = append(logged, s) }, nil)
if err != nil {
t.Fatal(err)
}
_ = report
for _, kept := range []string{"config", "known_hosts", "x/inside", "young", "agent.sock"} {
if !exists(filepath.Join(dir, kept)) {
t.Errorf("%s should stay", kept)
}
}
for _, gone := range []string{"stray", "oldfolder", "alink"} {
if exists(filepath.Join(dir, gone)) {
t.Errorf("%s should have moved", gone)
}
}
lists := keptLists(t, home)
if len(lists) != 1 {
t.Fatalf("one list expected, got %v", lists)
}
if !strings.Contains(lists[0], filepath.Join("mesh-moved", "mod")) {
t.Errorf("the module's name is not in the kept path: %s", lists[0])
}
list := readList(t, lists[0])
if len(list) != 3 {
t.Fatalf("the list names three paths: %+v", list)
}
kept := strings.TrimSuffix(lists[0], ".json")
for _, e := range list {
if !strings.HasPrefix(e.To, kept+string(os.PathSeparator)) || !exists(e.To) {
t.Errorf("not at %s: %+v", kept, e)
}
}
if b, _ := os.ReadFile(filepath.Join(kept, "stray")); string(b) != "stray content" {
t.Errorf("content changed: %q", b)
}
if b, _ := os.ReadFile(filepath.Join(kept, "oldfolder", "deep", "f")); string(b) != "deep" {
t.Errorf("the folder was not moved whole: %q", b)
}
if info, err := os.Lstat(filepath.Join(kept, "alink")); err != nil || info.Mode()&os.ModeSymlink == 0 {
t.Errorf("the link is not still a link: %v", err)
}
if !strings.Contains(strings.Join(logged, "\n"), "(ADR 0422)") {
t.Errorf("the moves were not logged: %v", logged)
}
if len(report.Left) != 1 || filepath.Base(report.Left[0].Path) != "agent.sock" {
t.Errorf("the report does not list the pipe it left: %+v", report.Left)
}
if len(report.MovedOut) != 3 {
t.Errorf("the report does not count the moves: %+v", report.MovedOut)
}
}
func TestAMoveOutListIsWrittenBeforeTheMove(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "d")
put(t, filepath.Join(dir, "a"), "a", time.Hour)
put(t, filepath.Join(dir, "b"), "b", time.Hour)
was := renamePath
renamePath = func(h, from, to string) error {
if filepath.Base(from) == "b" {
return errors.New("injected")
}
return was(h, from, to)
}
t.Cleanup(func() { renamePath = was })
_, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out"}`, nil, store.State{})
if err == nil {
t.Fatal("a failed move must fail the resource")
}
if !exists(filepath.Join(dir, "b")) || exists(filepath.Join(dir, "a")) {
t.Error("b stays, a moved")
}
lists := keptLists(t, home)
if len(lists) != 1 {
t.Fatalf("lists: %v", lists)
}
names := map[string]bool{}
for _, e := range readList(t, lists[0]) {
names[filepath.Base(e.From)] = true
}
if !names["a"] || !names["b"] {
t.Errorf("the entry for the failed move was not listed first: %v", names)
}
}
func TestAMoveOutThatMovedNothingMakesNothing(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "d")
put(t, filepath.Join(dir, "known"), "k", time.Hour)
put(t, filepath.Join(dir, "young"), "y", time.Second)
if _, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out","knows":["known"]}`, nil, store.State{}); err != nil {
t.Fatal(err)
}
if exists(keptRoot(home)) {
t.Error("a kept folder was made with nothing to move")
}
// And when every move fails, what was made goes.
put(t, filepath.Join(dir, "stray"), "s", time.Hour)
was := renamePath
renamePath = func(string, string, string) error { return errors.New("injected") }
t.Cleanup(func() { renamePath = was })
if _, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out","knows":["known"]}`, nil, store.State{}); err == nil {
t.Fatal("expected a failure")
}
if len(keptLists(t, home)) != 0 {
t.Error("a list stayed though nothing moved")
}
if matches, _ := filepath.Glob(filepath.Join(keptRoot(home), "*", "*")); len(matches) != 0 {
t.Errorf("a kept folder stayed though nothing moved: %v", matches)
}
}
func TestAMoveOutOfANestedStrayThatFailsLeavesNoKeptFolder(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "d")
put(t, filepath.Join(dir, "x", "f"), "declared", time.Hour)
put(t, filepath.Join(dir, "x", "stray"), "s", time.Hour)
was := renamePath
renamePath = func(string, string, string) error { return errors.New("injected") }
t.Cleanup(func() { renamePath = was })
_, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out"},`+
`{"id":"mod.f","type":"file","path":"`+dir+`/x/f","content":"declared","owner":"`+me()+`"}`, nil, store.State{})
if err == nil {
t.Fatal("expected a failure")
}
if exists(keptRoot(home)) {
matches, _ := filepath.Glob(filepath.Join(keptRoot(home), "*", "*"))
t.Errorf("something was left behind though nothing moved: %v", matches)
}
}
// A link swapped into a parent folder, on either side, between the look and the move, is not followed.
func TestAMoveOutDoesNotFollowALinkSwappedIntoAParent(t *testing.T) {
for _, side := range []string{"from", "to"} {
t.Run(side, func(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "d")
outside := filepath.Join(t.TempDir(), "etc")
put(t, filepath.Join(outside, "stray"), "root's", time.Hour)
put(t, filepath.Join(dir, "x", "f"), "declared", time.Hour)
put(t, filepath.Join(dir, "x", "stray"), "s", time.Hour)
was := renamePath
renamePath = func(h, from, to string) error {
victim := filepath.Dir(from)
if side == "to" {
victim = filepath.Dir(to)
_ = os.Remove(filepath.Join(victim, "nothing"))
}
if err := os.RemoveAll(victim); err != nil {
t.Fatal(err)
}
if side == "from" {
put(t, filepath.Join(outside, "other"), "o", time.Hour)
}
if err := os.Symlink(outside, victim); err != nil {
t.Fatal(err)
}
return was(h, from, to)
}
t.Cleanup(func() { renamePath = was })
_, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out"},`+
`{"id":"mod.f","type":"file","path":"`+dir+`/x/f","content":"declared","owner":"`+me()+`"}`, nil, store.State{})
if err == nil {
t.Fatal("a move through a swapped link must fail")
}
if b, e := os.ReadFile(filepath.Join(outside, "stray")); e != nil || string(b) != "root's" {
t.Errorf("the file outside the home was touched: %q %v", b, e)
}
entries, _ := os.ReadDir(outside)
if side == "to" && len(entries) != 1 || side == "from" && len(entries) != 2 {
t.Errorf("something was moved into or out of the folder outside the home: %v", entries)
}
})
}
}
func TestOtherUnknownValuesAndNoValueMoveNothing(t *testing.T) {
home := aMoveOutHome(t)
for _, v := range []string{`,"unknown":"program"`, `,"unknown":"module"`, ``} {
dir := filepath.Join(home, "d")
put(t, filepath.Join(dir, "stray"), "s", time.Hour)
if _, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`"`+v+`}`, nil, store.State{}); err != nil {
t.Fatal(err)
}
if !exists(filepath.Join(dir, "stray")) || exists(keptRoot(home)) {
t.Errorf("%q moved something", v)
}
}
}
func TestAMoveOutArchiveKeepsWhatItUnpackedAndMovesAStray(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "theme")
put(t, filepath.Join(dir, "stray"), "s", time.Hour)
res, _ := archiveDecl(t, "mod.theme", dir, map[string]string{"one.txt": "1", "sub/two.txt": "2"}, "")
res = strings.Replace(res, `"path":`, `"owner":"`+me()+`","unknown":"move-out","path":`, 1)
if _, _, err := Apply(context.Background(), archHost(t), declare(t, res), store.State{},
store.OriginDeclared, noServices, nil, nil); err != nil {
t.Fatal(err)
}
for _, f := range []string{"one.txt", "sub/two.txt"} {
if !exists(filepath.Join(dir, f)) {
t.Errorf("%s was moved", f)
}
}
if exists(filepath.Join(dir, "stray")) || len(keptLists(t, home)) != 1 {
t.Error("the stray file did not move")
}
}
func TestMoveOutNeedsAHome(t *testing.T) {
aMoveOutHome(t)
dir := filepath.Join(t.TempDir(), "elsewhere")
put(t, filepath.Join(dir, "stray"), "s", time.Hour)
_, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out"}`, nil, store.State{})
if err == nil || !strings.Contains(err.Error(), "home") {
t.Fatalf("expected a refusal naming the home: %v", err)
}
if !exists(filepath.Join(dir, "stray")) {
t.Error("something moved")
}
}
func TestMoveOutRefusesAnotherFileSystem(t *testing.T) {
home := aMoveOutHome(t)
dir := filepath.Join(home, "d")
put(t, filepath.Join(dir, "stray"), "s", time.Hour)
was := deviceOf
deviceOf = func(p string) (uint64, error) {
if p == dir {
return 1, nil
}
return 2, nil
}
t.Cleanup(func() { deviceOf = was })
_, _, err := applyOnce(t, `{"id":"mod.dir","type":"directory","path":"`+dir+`","owner":"`+me()+`","unknown":"move-out"}`, nil, store.State{})
if err == nil || !strings.Contains(err.Error(), "file system") {
t.Fatalf("expected a refusal naming the file systems: %v", err)
}
if !exists(filepath.Join(dir, "stray")) || exists(keptRoot(home)) {
t.Error("something moved or was made")
}
}
internal/apply/opening.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/opening_test.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/ownership_test.go
-129
View File
@@ -1,129 +0,0 @@
package apply
import (
"context"
"fmt"
"os"
"path/filepath"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// Defends novox/hq ADR 0349 §5 (issue 667): a file that says who owns it at runtime is written as the
// form it names always was — owned as a whole file whose drift is written back, seed as create-once,
// managed as only its keys or its marked block. Each is applied beside its old form, through the same
// drift, and the two must end the same: the same actions, the same bytes on disk.
// ownershipCase is one ownership word and the old form it maps onto.
type ownershipCase struct {
name string
// said and old are the resource's fields after id, type and path: with the word, and in the old form.
said, old string
// before is what is at the path before the first apply; "" leaves the path absent.
before string
// drift is what the machine writes there between the two applies.
drift string
// first and second are the actions the old form is known to report.
first, second string
}
func TestEachOwnershipBehavesAsItsOldForm(t *testing.T) {
for _, c := range []ownershipCase{
{
name: "owned is a whole file whose drift is written back",
said: `"content":"correct\n","mode":"0644","owner-at-runtime":"owned"`,
old: `"content":"correct\n","mode":"0644"`,
drift: "someone edited this\n",
first: "created", second: "corrected",
},
{
name: "seed is written once and then the program's",
said: `"content":"user default on\n","mode":"0600","owner-at-runtime":"seed"`,
old: `"content":"user default on\n","mode":"0600","create-once":true`,
drift: "user default on\nuser app-one on\n",
first: "created", second: "kept",
},
{
name: "managed json sets only its keys",
said: `"into":"json","content":"{\"insecure-registries\":[\"10.42.0.1:5000\"]}","owner-at-runtime":"managed"`,
old: `"into":"json","content":"{\"insecure-registries\":[\"10.42.0.1:5000\"]}"`,
before: machinesOwn,
drift: `{"data-root":"/srv/other","log-opts":{"max-size":"10m"},"insecure-registries":["192.0.2.7:5000"]}`,
first: "updated", second: "corrected",
},
{
name: "managed block holds only its region",
said: `"into":"block","content":"10.42.0.1 ace\n","owner-at-runtime":"managed"`,
old: `"into":"block","content":"10.42.0.1 ace\n"`,
before: workstationHosts,
drift: workstationHosts + "\n" + marked("f", "6.6.6.6 evil\n") + "192.0.2.9 the-machines-own\n",
first: "updated", second: "corrected",
},
} {
t.Run(c.name, func(t *testing.T) {
saidActions, saidBytes := applyThroughDrift(t, c, c.said)
oldActions, oldBytes := applyThroughDrift(t, c, c.old)
if saidActions != oldActions {
t.Errorf("with the word the actions were %s; the old form's were %s", saidActions, oldActions)
}
if want := c.first + " " + c.second; oldActions != want {
t.Errorf("the old form's actions were %s; known to be %s", oldActions, want)
}
if saidBytes != oldBytes {
t.Errorf("with the word the file ends\n%q\nthe old form's ends\n%q", saidBytes, oldBytes)
}
})
}
}
// applyThroughDrift applies one file, lets the machine write the case's drift into it, applies again,
// and says the two actions and what the file holds at the end.
func applyThroughDrift(t *testing.T, c ownershipCase, fields string) (string, string) {
t.Helper()
path := filepath.Join(t.TempDir(), "the-file")
if c.before != "" {
if err := os.WriteFile(path, []byte(c.before), 0o644); err != nil {
t.Fatal(err)
}
}
d := parse(t, fmt.Sprintf(`{"declaration":1,"resources":[{"id":"f","type":"file","path":%q,%s}]}`, path, fields))
first, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(c.drift), 0o644); err != nil {
t.Fatal(err)
}
second, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
return first.Outcomes[0].Action + " " + second.Outcomes[0].Action, readText(t, path)
}
// A seed said by the word alone, with no create-once beside it, is still kept once present: the word is
// enough, and the file the program grew is not written back.
func TestASeedByTheWordAloneKeepsWhatGrew(t *testing.T) {
path := filepath.Join(t.TempDir(), "acl.conf")
d := parse(t, fmt.Sprintf(`{"declaration":1,"resources":[
{"id":"acl","type":"file","path":%q,"content":"user default on\n","owner-at-runtime":"seed"}
]}`, path))
_, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("grown\n"), 0o600); err != nil {
t.Fatal(err)
}
report, _, err := Apply(context.Background(), archHost(t), d, state, store.OriginDeclared, noServices, nil, nil)
if err != nil {
t.Fatal(err)
}
if got := report.Outcomes[0].Action; got != "kept" {
t.Errorf("a seed was %q", got)
}
if got := readText(t, path); got != "grown\n" {
t.Errorf("what grew in the seed was written over: %q", got)
}
}
internal/apply/placement_guard.go
+4 -99
View File
@@ -56,7 +56,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// protectedRoots are paths no directory, access or mount source may be, nor hold.
@@ -384,9 +384,6 @@ func ownedByAccount(owner string, a homeAccount) bool {
if owner == a.Name {
return true
}
if a.UID < 0 {
return false // an account the declaration makes has no uid yet: by name alone, never by a number
}
user, _, _ := strings.Cut(owner, ":")
if uid, err := strconv.Atoi(user); err == nil {
return uid == a.UID
@@ -416,91 +413,12 @@ const (
// refusePath says why a path is not placed or mounted; nil when it may be. module is the resource's module,
// owner a directory's declared owner.
func refusePath(path string, kind placing, module, owner string) error {
return refusePathWith(path, kind, module, owner, nil)
}
// declaredHomes is the home of each account the declaration itself makes (a user resource with a home), by the
// account's name; the account is not in the user database yet when the declaration is judged, and the engine makes
// it where it is declared, before the directory that is owned by it is reached (novox/hq issue 1542). Its uid is
// not known, so a directory is owned by it by name alone. Only for the directory placement guard; no other
// reader of accountsOfHomes sees these.
func declaredHomes(resources []declaration.Resource) map[string]homeAccount {
var accounts []passwdEntry
var out map[string]homeAccount
for _, r := range resources {
u, ok := r.(*declaration.User)
if !ok || u.Name == "" {
continue
}
home := filepath.Clean(u.Home)
if u.Home == "" || !filepath.IsAbs(home) || home == "/" {
continue
}
if accounts == nil {
accounts = passwdEntries()
}
// Only ever adds a home. An account the machine has keeps the home it has (applyUser never moves it), so
// a declared home of one is not its home, unless it is the very home it has: the account an earlier apply
// made, as declared, which must be judged the same at every later apply. And a home at or below another
// account's, or at or above a protected root, is not a new one.
taken := false
for _, e := range accounts {
if e.name == u.Name && e.home != home {
taken = true
}
if e.name != u.Name && e.home != "" && atOrBelow(home, e.home) {
taken = true
}
}
for _, root := range protectedRoots {
taken = taken || home == root || below(root, home)
}
if taken {
continue
}
if out == nil {
out = map[string]homeAccount{}
}
out[home] = homeAccount{Name: u.Name, UID: -1}
}
return out
}
// passwdEntry is an account of the user database and its home, whatever its uid.
type passwdEntry struct{ name, home string }
// passwdEntries is every account of the user database with its home (empty and root homes as "").
func passwdEntries() []passwdEntry {
out := []passwdEntry{}
f, err := os.Open(passwdFile)
if err != nil {
return out
}
defer f.Close()
sc := bufio.NewScanner(f)
for sc.Scan() {
fields := strings.Split(sc.Text(), ":")
if len(fields) < 6 {
continue
}
home := filepath.Clean(fields[5])
if home == "." || home == "/" || home == "/nonexistent" {
home = ""
}
out = append(out, passwdEntry{fields[0], home})
}
return out
}
// refusePathWith is refusePath with the homes of the accounts the declaration makes (declaredHomes) counted
// among the machine's.
func refusePathWith(path string, kind placing, module, owner string, declared map[string]homeAccount) error {
clean := filepath.Clean(path)
if !filepath.IsAbs(clean) {
return nil // the declaration refuses a relative path already; a named volume is not a path
}
for _, p := range []string{clean, resolved(clean)} {
if err := refuseOne(p, kind, module, owner, declared); err != nil {
if err := refuseOne(p, kind, module, owner); err != nil {
if p != clean {
err.Why = fmt.Sprintf("through a link, it is %s, and %s", p, err.Why)
err.Path = clean
@@ -511,7 +429,7 @@ func refusePathWith(path string, kind placing, module, owner string, declared ma
return nil
}
func refuseOne(path string, kind placing, module, owner string, declared map[string]homeAccount) *PlacementRefusedError {
func refuseOne(path string, kind placing, module, owner string) *PlacementRefusedError {
for _, root := range protectedRoots {
if path == root || below(root, path) {
return &PlacementRefusedError{Path: path, Why: root + " is one of the machine's own directories, " +
@@ -564,18 +482,6 @@ func refuseOne(path string, kind placing, module, owner string, declared map[str
return nil
}
homes := accountsOfHomes()
if ssh && len(declared) > 0 {
merged := map[string]homeAccount{}
for home, a := range homes {
merged[home] = a
}
for home, a := range declared {
if _, held := merged[home]; !held {
merged[home] = a
}
}
homes = merged
}
var deepest string
for home := range homes {
if below(path, home) && len(home) > len(deepest) {
@@ -612,12 +518,11 @@ func moduleOfID(id string) string {
// a file into it, because nothing judged the file's path.
func refusedPlaces(resources []declaration.Resource) map[string]error {
out := map[string]error{}
declared := declaredHomes(resources)
for _, r := range resources {
var err error
switch res := r.(type) {
case *declaration.Directory:
err = refusePathWith(res.Path, placingDirectory, moduleOfID(res.ID), res.Owner, declared)
err = refusePath(res.Path, placingDirectory, moduleOfID(res.ID), res.Owner)
// And outside every home, the walk's judgement before anything is applied (novox/hq issue 358): a
// link another account could make in its path refuses it, and with it what is below it, whatever
// order they are declared in. The apply walks again at the moment of use. A directory handed over
internal/apply/plan.go
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/plan_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/process.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/witness"
)
internal/apply/process_test.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/reads_test.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/references.go
-95
View File
@@ -1,95 +0,0 @@
package apply
import (
"context"
"fmt"
"os"
"path/filepath"
"unicode/utf8"
"encoding/base64"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// FetchObject reads the object of the declaration-files bucket named by name, no larger than limit bytes (novox/hq ADR
// 0486 §1, §2). Nil is a node-engine with no link to the bus: a file by reference is then only the file already there.
type FetchObject func(ctx context.Context, name string, limit int64) ([]byte, error)
// ResolveFileReferences puts the content of every file given by digest and size into the declaration, so what is
// applied afterwards is the declaration as it was before references existed (novox/hq ADR 0486 §1, issue 1695).
//
// **Nothing of the declaration is applied before every reference is resolved**: a missing object, one whose size or
// sha256 differs from what the signed declaration names, or a fetch that fails, refuses the declaration whole. The
// content is checked with File.Verify before it is kept, and is only ever written by the apply that follows.
//
// A file already on the machine whose content verifies is used as it is and fetches nothing: the reconcile of an
// unchanged declaration, and a node that is asleep and has no bus, apply as they always did. One object is fetched
// once however many files name its digest.
func ResolveFileReferences(ctx context.Context, d *declaration.Declaration, fetch FetchObject) error {
fetched := map[string][]byte{}
for i, r := range d.Resources {
f, ok := r.(*declaration.File)
if !ok || !f.IsReference() {
continue
}
body, err := referencedContent(ctx, f, fetch, fetched)
if err != nil {
return err
}
inline := *f
inline.Digest, inline.Size = "", 0
if utf8.Valid(body) {
inline.Content = string(body)
} else {
inline.Bytes = base64.StdEncoding.EncodeToString(body)
}
d.Resources[i] = &inline
}
return nil
}
func referencedContent(ctx context.Context, f *declaration.File, fetch FetchObject,
fetched map[string][]byte) ([]byte, error) {
if body, ok := fetched[f.Digest]; ok {
return body, nil
}
if body, ok := alreadyThere(f); ok {
fetched[f.Digest] = body
return body, nil
}
name, err := declaration.ObjectName(f.Digest)
if err != nil {
return nil, fmt.Errorf("file %s (%s): %w", f.ID, f.Path, err)
}
if fetch == nil {
return nil, fmt.Errorf("file %s (%s) is given by digest %s and is not on this machine, and this node-engine "+
"has no link to the bus to fetch it from", f.ID, f.Path, f.Digest)
}
body, err := fetch(ctx, name, f.Size)
if err != nil {
return nil, fmt.Errorf("file %s (%s): fetching the object %s of this node's own object bucket: %w",
f.ID, f.Path, name, err)
}
if err := f.Verify(body); err != nil {
return nil, fmt.Errorf("%s (%s): %w; nothing was applied", f.ID, f.Path, err)
}
fetched[f.Digest] = body
return body, nil
}
// alreadyThere is the content of the file on the machine when it is what the reference names.
func alreadyThere(f *declaration.File) ([]byte, bool) {
if !filepath.IsAbs(f.Path) {
return nil, false
}
info, err := os.Lstat(f.Path)
if err != nil || !info.Mode().IsRegular() || info.Size() != f.Size {
return nil, false
}
body, err := os.ReadFile(f.Path)
if err != nil || f.Verify(body) != nil {
return nil, false
}
return body, true
}
internal/apply/references_test.go
-132
View File
@@ -1,132 +0,0 @@
package apply
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
func referenceTo(t *testing.T, path string, body []byte) *declaration.File {
t.Helper()
return &declaration.File{ID: "m.big", Type: declaration.TypeFile, Path: path,
Digest: declaration.DigestOf(body), Size: int64(len(body))}
}
func objects(m map[string][]byte, asked *[]string) FetchObject {
return func(_ context.Context, name string, limit int64) ([]byte, error) {
*asked = append(*asked, name)
b, ok := m[name]
if !ok {
return nil, errors.New("object not found")
}
return b, nil
}
}
func objectOf(body []byte) string {
name, _ := declaration.ObjectName(declaration.DigestOf(body))
return name
}
// 1695/AC1: both forms apply: a file by reference becomes the inline file, an inline one is left as it is.
func TestAFileByReferenceBecomesTheInlineFile(t *testing.T) {
body := []byte(strings.Repeat("a line\n", 20000))
inline := &declaration.File{ID: "m.small", Type: declaration.TypeFile, Path: "/etc/small", Content: "x"}
d := &declaration.Declaration{Resources: []declaration.Resource{inline, referenceTo(t, "/nowhere/big", body)}}
var asked []string
if err := ResolveFileReferences(context.Background(), d, objects(map[string][]byte{objectOf(body): body}, &asked)); err != nil {
t.Fatal(err)
}
got := d.Resources[1].(*declaration.File)
if got.IsReference() || got.Content != string(body) || got.Path != "/nowhere/big" || d.Resources[0] != declaration.Resource(inline) {
t.Errorf("1695/AC1: resolved to %+v", got)
}
if len(asked) != 1 || asked[0] != objectOf(body) {
t.Errorf("1695/AC1: asked %v", asked)
}
}
// 1695/AC2: an object that differs by one byte, or in size, fails the whole resolve and gives no content.
func TestAnObjectThatDiffersFailsTheApply(t *testing.T) {
body := []byte(strings.Repeat("x", 70000))
bad := append([]byte(nil), body...)
bad[3] ^= 1
for name, served := range map[string][]byte{"one byte": bad, "shorter": body[1:]} {
d := &declaration.Declaration{Resources: []declaration.Resource{referenceTo(t, "/nowhere/big", body)}}
var asked []string
err := ResolveFileReferences(context.Background(), d, objects(map[string][]byte{objectOf(body): served}, &asked))
if err == nil || !strings.Contains(err.Error(), "nothing was applied") {
t.Errorf("1695/AC2: %s: %v", name, err)
}
if !d.Resources[0].(*declaration.File).IsReference() {
t.Errorf("1695/AC2: %s: the reference was replaced by unverified content", name)
}
}
}
// 1695/AC3: a missing object, or a node with no bus, fails the apply naming the file.
func TestAMissingObjectFailsTheApply(t *testing.T) {
body := []byte(strings.Repeat("y", 70000))
d := &declaration.Declaration{Resources: []declaration.Resource{referenceTo(t, "/nowhere/big", body)}}
var asked []string
if err := ResolveFileReferences(context.Background(), d, objects(nil, &asked)); err == nil ||
!strings.Contains(err.Error(), "m.big") || !strings.Contains(err.Error(), "object bucket") {
t.Errorf("1695/AC3: missing object: %v", err)
}
if err := ResolveFileReferences(context.Background(), d, nil); err == nil || !strings.Contains(err.Error(), "no link to the bus") {
t.Errorf("1695/AC3: no bus: %v", err)
}
}
// 1695/AC4: a file already on the machine that verifies is used and nothing is fetched; one that does not is fetched.
func TestAVerifiedFileOnTheMachineFetchesNothing(t *testing.T) {
body := []byte(strings.Repeat("z", 70000))
path := filepath.Join(t.TempDir(), "big")
if err := os.WriteFile(path, body, 0o644); err != nil {
t.Fatal(err)
}
var asked []string
d := &declaration.Declaration{Resources: []declaration.Resource{referenceTo(t, path, body)}}
if err := ResolveFileReferences(context.Background(), d, objects(nil, &asked)); err != nil || len(asked) != 0 {
t.Errorf("1695/AC4: err %v, asked %v", err, asked)
}
if err := os.WriteFile(path, body[:100], 0o644); err != nil {
t.Fatal(err)
}
d = &declaration.Declaration{Resources: []declaration.Resource{referenceTo(t, path, body)}}
if err := ResolveFileReferences(context.Background(), d, objects(map[string][]byte{objectOf(body): body}, &asked)); err != nil || len(asked) != 1 {
t.Errorf("1695/AC4: a differing file: err %v, asked %v", err, asked)
}
}
// 1695/AC5: binary content comes back as bytes, and two files naming one digest fetch it once.
func TestBinaryContentAndOneFetchPerDigest(t *testing.T) {
body := append([]byte{0xff, 0xfe, 0x00}, []byte(strings.Repeat("b", 70000))...)
d := &declaration.Declaration{Resources: []declaration.Resource{referenceTo(t, "/n/a", body), referenceTo(t, "/n/b", body)}}
var asked []string
if err := ResolveFileReferences(context.Background(), d, objects(map[string][]byte{objectOf(body): body}, &asked)); err != nil {
t.Fatal(err)
}
a, _ := d.Resources[0].(*declaration.File).Body()
if d.Resources[0].(*declaration.File).Bytes == "" || string(a) != string(body) || len(asked) != 1 {
t.Errorf("1695/AC5: bytes %q asked %v", d.Resources[0].(*declaration.File).Bytes[:4], asked)
}
}
// 1695/AC9: an unresolved reference reaching the apply is refused, never written as an empty file.
func TestAnUnresolvedReferenceIsNeverWrittenEmpty(t *testing.T) {
path := filepath.Join(t.TempDir(), "big")
f := referenceTo(t, path, []byte(strings.Repeat("q", 70000)))
if _, err := applyFile(f, store.Applied{}, nil, nil); err == nil || !strings.Contains(err.Error(), "not written empty") {
t.Errorf("1695/AC9: %v", err)
}
if _, err := os.Stat(path); err == nil {
t.Error("1695/AC9: a file was written")
}
}
internal/apply/replace_image_test.go
-93
View File
@@ -1,93 +0,0 @@
package apply
import (
"context"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Issue 1742: the engine removed a running container and only then found the replacement image
// unobtainable, leaving the service down. The replacement image comes first.
func TestARunningContainerIsKeptWhenItsReplacementImageCannotBeObtained(t *testing.T) {
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
]}`)
var removed, created bool
run := func(ctx context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
switch args[0] {
case "info":
return "27.0\n", nil
case "container":
return "true\tsome-older-spec", nil
case "image":
return "", errors.New("no such image")
case "pull":
return "", errors.New("manifest unknown")
case "rm":
removed = true
return "", nil
case "run":
created = true
return "deadbeef\n", nil
}
return "", nil
}
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if removed || created {
t.Fatalf("the running container was touched although its image is unobtainable (removed=%v created=%v)", removed, created)
}
if err == nil {
t.Fatal("an unobtainable image failed nothing")
}
if !strings.Contains(err.Error(), pinned) {
t.Errorf("the failure does not name the image %s: %v", pinned, err)
}
}
func TestAReplacementImageIsPresentBeforeTheRunningContainerIsRemoved(t *testing.T) {
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
]}`)
want := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
var order []string
created := false
run := func(ctx context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
switch args[0] {
case "info":
return "27.0\n", nil
case "container":
if created {
return "true\t" + want, nil
}
return "true\tsome-older-spec", nil
case "image":
order = append(order, "inspect")
if len(order) == 1 {
return "", errors.New("no such image")
}
return "", nil
case "pull":
order = append(order, "pull")
case "rm":
order = append(order, "rm")
case "run":
order = append(order, "run")
created = true
return "deadbeef\n", nil
}
return "", nil
}
if _, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil); err != nil {
t.Fatalf("apply failed: %v", err)
}
got := strings.Join(order, ",")
if !strings.HasPrefix(got, "inspect,pull,inspect,rm,run") {
t.Errorf("order of runtime calls = %s, want the image pulled before rm", got)
}
}
internal/apply/replay358_test.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/root_moved.go
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// rootPaths are the files and directories through which an account comes to root, or by which it is told who it
internal/apply/runonce_test.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/schedule.go
+1 -1
View File
@@ -26,7 +26,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
)
// Clock is the source of "now", injected so scheduled steps can be tested without waiting on the
internal/apply/schedule_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/sealed_test.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/step_failed_test.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/store_away_test.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/strays.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/takeover.go
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
"github.com/novox/mesh-host/internal/tunnel"
internal/apply/takeover_test.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
internal/apply/test_hooks.go
-10
View File
@@ -1,10 +0,0 @@
package apply
// UseAccountsFileForTest makes the accounts and homes this package knows come from the user database at path, until
// the returned function is called. For the node-engine's own integration tests, which cannot name a home any other
// way; nothing in the node-engine calls it (issue 1024).
func UseAccountsFileForTest(path string) (restore func()) {
was := passwdFile
passwdFile = path
return func() { passwdFile = was }
}
internal/apply/user.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strconv"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)

Some files were not shown because too many files have changed in this diff Show More