novox/mesh-sdk
18
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Remove the controller from the delivery contract, and its command line client and hand composer

Browse Source 
The controller ended (ADR 0590): the grants issued are the grants part's alone, so its grant value
is Grant and GrantSource goes; a node trusts the vault's key alone, follows the grants part's
user list alone and composes its own or none; the controller's composed and builds bucket names
go. nox calls seat verbs (ADR 0590 clause 7), so the cli package that spoke to the node-engine's
socket goes, and mesh-hand-compose, which wrote the controller's compositions. A have an engine
from before wrote, composes controller, still reads. Issue 1126 B4a, B6.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
This commit is contained in:
jschoubben
2026-11-02 11:36:26 +01:00
parent ba1e90fb2d
commit dcb2e6e966
22 changed files with 119 additions and 1031 deletions
Download Patch File Download Diff File Expand all files Collapse all files
go/cli/cli.go
-201
View File
@@ -1,201 +0,0 @@
// Package cli is the local protocol between `mesh-cli` and the node-engine on its machine (novox/hq ADR 0272).
//
// `mesh-cli <command line>` asks the controller to run one of its command lines. It holds no credential and
// reads no file of the mesh: it connects to the node-engine's socket, writes one Request, and reads one Answer.
// The node-engine reads the connecting process's account from the kernel and asks the controller over its own
// link; that request is the node-engine's and the controller's, and lives in neither this package nor any SDK
// (ADR 0039 as ADR 0188 reads it: the transport lives in the runtime).
//
// One request and one answer per connection, each one line of JSON. Standard library only.
package cli
import (
"bufio"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"time"
)
// Socket is where the node-engine listens for mesh-cli: a file under its runtime directory, with no network
// address (ADR 0272 §3).
const Socket = "/run/nox-mesh-host/cli.sock"
// MaxStdin bounds the standard input mesh-cli carries for a line: a secret, a key, a settings file — kilobytes. More
// is refused by mesh-cli before anything is asked, and by the node-engine and the controller again.
const MaxStdin = 64 << 10
// MaxRequest bounds what the node-engine reads of a request: a command line of a few hundred bytes, settings given
// inline of a few kilobytes, and standard input of at most MaxStdin, which JSON writes as base64 (4 bytes for every 3).
const MaxRequest = 64<<10 + MaxStdin*4/3 + 4
// MaxAnswer bounds what mesh-cli reads of an answer. The controller cuts what it runs to fit one bus message, and
// says so; this is the local bound above that.
const MaxAnswer = 16 << 20
// AskWithin is how long mesh-cli waits for its answer: the controller's own bound on a call (five minutes),
// and a minute for the link on either side.
const AskWithin = 6 * time.Minute
// Request is what mesh-cli asks: the controller's command line, as words, and what mesh-cli's own standard input
// held when it was not a terminal.
//
// **Stdin is for the controller's terminal alone** (novox/hq ADR 0259 §10, ADR 0272): a secret given with
// `secret accept … --from -`, so it never stands on a command line. The controller gives it to a line that runs as
// its terminal and refuses any other line that carries it; nobody keeps it — not the node-engine, not the
// controller's record of calls, not its journal.
type Request struct {
Line []string `json:"line"`
Stdin []byte `json:"stdin,omitempty"`
}
// Answer is what comes back: what the command printed, how it exited, and whether it ran as the controller's
// terminal and why. Refused is set when nothing ran at all, and says why; Exit is then 1.
type Answer struct {
Stdout []byte `json:"stdout,omitempty"`
Stderr []byte `json:"stderr,omitempty"`
Exit int `json:"exit"`
Terminal bool `json:"terminal"`
// Why says whether the line ran as the terminal, and why or why not, in the controller's words.
Why string `json:"why,omitempty"`
// Refused says why nothing ran: the account, the node, the controller unreachable.
Refused string `json:"refused,omitempty"`
// Cut says the output was longer than one bus message and was cut there.
Cut bool `json:"cut,omitempty"`
// Unknown says the line was asked and its outcome is not known: no answer came in time, and it may have
// run. Never said as a refusal, which means nothing ran.
Unknown string `json:"unknown,omitempty"`
}
// Refusal is an answer saying that nothing ran, and why.
func Refusal(why string) Answer { return Answer{Exit: 1, Refused: why} }
// NotKnown is an answer saying the line was asked and what came of it is not known: it may have run. call is
// the controller's call the line runs as, when it is known, which `calls` answers.
func NotKnown(call, why string) Answer {
where := "the controller's `calls` lists what came of each line"
if call != "" {
where = "see the call " + call + " (`calls " + call + "`)"
}
return Answer{Exit: 1, Unknown: why + "; the line may have run — " + where}
}
// ErrTooLong is a request or an answer over its bound.
var ErrTooLong = errors.New("longer than the protocol allows")
// WriteRequest writes one request, as one line.
func WriteRequest(w io.Writer, r Request) error { return writeLine(w, r) }
// ReadRequest reads one request, refusing one over MaxRequest or one that names no command.
func ReadRequest(r io.Reader) (Request, error) {
var req Request
if err := readLine(r, MaxRequest, &req); err != nil {
return Request{}, err
}
if len(req.Line) == 0 {
return Request{}, errors.New("the request names no command")
}
if len(req.Stdin) > MaxStdin {
return Request{}, fmt.Errorf("%w: standard input of more than %d bytes", ErrTooLong, MaxStdin)
}
return req, nil
}
// WriteAnswer writes one answer, as one line.
func WriteAnswer(w io.Writer, a Answer) error { return writeLine(w, a) }
// ReadAnswer reads one answer, refusing one over MaxAnswer.
func ReadAnswer(r io.Reader) (Answer, error) {
var a Answer
err := readLine(r, MaxAnswer, &a)
return a, err
}
// Ask is mesh-cli's side: connect to the socket, ask, and wait for the answer within AskWithin or ctx.
func Ask(ctx context.Context, socket string, line []string) (Answer, error) {
return AskWith(ctx, socket, Request{Line: line})
}
// AskWith asks a whole request: a line, and the standard input it carries, at most MaxStdin.
func AskWith(ctx context.Context, socket string, req Request) (Answer, error) {
line := req.Line
if len(line) == 0 {
return Answer{}, errors.New("no command given")
}
if len(req.Stdin) > MaxStdin {
return Answer{}, fmt.Errorf("standard input is more than %d bytes, which mesh-cli does not carry; nothing was asked", MaxStdin)
}
ctx, cancel := context.WithTimeout(ctx, AskWithin)
defer cancel()
var d net.Dialer
conn, err := d.DialContext(ctx, "unix", socket)
if err != nil {
return Answer{}, fmt.Errorf("the node-engine on this machine does not answer at %s: %w", socket, err)
}
defer conn.Close()
if deadline, ok := ctx.Deadline(); ok {
_ = conn.SetDeadline(deadline)
}
if err := WriteRequest(conn, req); err != nil {
// The engine may have answered before it read the request — a refusal (busy, an unknown account) written
// and the connection closed — and the write then fails on the close. That answer is already waiting, and
// it says why where the write error says only that the pipe broke (novox/hq issue 444, found under issue
// 434). So read it, within the deadline the connection already has, and only when none came say the write.
// Only a refusal is taken: the request was never wholly written, so nothing can have run, and any other
// answer here is not one a correct engine gives — passing it on as success would hide the failure.
a, rerr := ReadAnswer(conn)
if rerr != nil {
return Answer{}, fmt.Errorf("asking the node-engine: %w; and no answer came (%v)", err, rerr)
}
if a.Refused == "" {
return Answer{}, fmt.Errorf("asking the node-engine: %w; and the answer that came is not a refusal, "+
"though the request was never wholly written", err)
}
return a, nil
}
a, err := ReadAnswer(conn)
if err != nil {
return Answer{}, fmt.Errorf("reading the node-engine's answer: %w", err)
}
return a, nil
}
func writeLine(w io.Writer, v any) error {
body, err := json.Marshal(v)
if err != nil {
return err
}
_, err = w.Write(append(body, '\n'))
return err
}
// readLine reads one line of JSON of at most limit bytes. A line over the limit is refused rather than read
// whole: the reader is the side that pays for it.
func readLine(r io.Reader, limit int, into any) error {
br := bufio.NewReaderSize(io.LimitReader(r, int64(limit)+1), 64<<10)
var body []byte
for {
chunk, err := br.ReadSlice('\n')
body = append(body, chunk...)
if len(body) > limit {
return fmt.Errorf("%w: more than %d bytes", ErrTooLong, limit)
}
if err == nil {
break
}
if errors.Is(err, bufio.ErrBufferFull) {
continue
}
if errors.Is(err, io.EOF) && len(body) > 0 {
break
}
return err
}
if err := json.Unmarshal(body, into); err != nil {
return fmt.Errorf("not one: %w", err)
}
return nil
}
go/cli/cli_test.go
-141
View File
@@ -1,141 +0,0 @@
package cli
import (
"bytes"
"context"
"encoding/json"
"errors"
"net"
"path/filepath"
"strings"
"testing"
)
// A request and an answer survive a round trip, bytes and all.
func TestARequestAndAnAnswerSurviveTheirRoundTrip(t *testing.T) {
var b bytes.Buffer
line := []string{"settings", "set", "zsh", `{"execute":"withhold"}`, "--node", "a"}
if err := WriteRequest(&b, Request{Line: line}); err != nil {
t.Fatal(err)
}
got, err := ReadRequest(&b)
if err != nil {
t.Fatal(err)
}
if strings.Join(got.Line, "\x00") != strings.Join(line, "\x00") {
t.Fatalf("the line came back as %q", got.Line)
}
b.Reset()
sent := Answer{Stdout: []byte("ok\n\xff"), Stderr: []byte("said"), Exit: 3, Terminal: true, Why: "the terminal", Cut: true,
Unknown: "no answer; the line may have run — see the call call-1"}
if err := WriteAnswer(&b, sent); err != nil {
t.Fatal(err)
}
a, err := ReadAnswer(&b)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(a.Stdout, sent.Stdout) || string(a.Stderr) != "said" || a.Exit != 3 || !a.Terminal || a.Why != sent.Why || !a.Cut ||
a.Unknown != sent.Unknown {
t.Fatalf("the answer came back as %+v", a)
}
}
// A request over the bound is refused, not read whole; one naming no command is refused.
func TestARequestOverTheBoundOrNamingNothingIsRefused(t *testing.T) {
long := Request{Line: []string{strings.Repeat("x", MaxRequest)}}
big := Request{Line: []string{"secret"}, Stdin: bytes.Repeat([]byte("s"), MaxStdin+1)}
var bb bytes.Buffer
if err := WriteRequest(&bb, big); err != nil {
t.Fatal(err)
}
if _, err := ReadRequest(&bb); !errors.Is(err, ErrTooLong) {
t.Fatalf("standard input over its bound was read: %v", err)
}
if _, err := AskWith(context.Background(), "/nonexistent.sock", big); err == nil || !strings.Contains(err.Error(), "nothing was asked") {
t.Fatalf("mesh-cli would ask with standard input over its bound: %v", err)
}
var b bytes.Buffer
if err := WriteRequest(&b, long); err != nil {
t.Fatal(err)
}
if _, err := ReadRequest(&b); !errors.Is(err, ErrTooLong) {
t.Fatalf("a request over the bound was read: %v", err)
}
if _, err := ReadRequest(strings.NewReader(`{"line":[]}` + "\n")); err == nil {
t.Fatal("a request naming no command was taken")
}
if _, err := ReadRequest(strings.NewReader("not json\n")); err == nil {
t.Fatal("a request that is not one was taken")
}
}
// Ask speaks the protocol over a socket, as mesh-cli does to the node-engine.
func TestAskSpeaksTheProtocolOverASocket(t *testing.T) {
socket := filepath.Join(t.TempDir(), "cli.sock")
l, err := net.Listen("unix", socket)
if err != nil {
t.Skipf("no unix socket here: %v", err)
}
defer l.Close()
go func() {
conn, err := l.Accept()
if err != nil {
return
}
defer conn.Close()
req, err := ReadRequest(conn)
if err != nil {
_ = WriteAnswer(conn, Refusal(err.Error()))
return
}
_ = WriteAnswer(conn, Answer{Stdout: []byte(strings.Join(req.Line, " "))})
}()
a, err := Ask(context.Background(), socket, []string{"status"})
if err != nil {
t.Fatal(err)
}
if string(a.Stdout) != "status" || a.Exit != 0 || a.Refused != "" {
t.Fatalf("asked status and heard %+v", a)
}
if _, err := Ask(context.Background(), filepath.Join(t.TempDir(), "none.sock"), []string{"status"}); err == nil ||
!strings.Contains(err.Error(), "does not answer") {
t.Fatalf("a missing socket was not said: %v", err)
}
}
// An outcome not known is never a refusal: it says the line may have run, and exits non-zero.
func TestAnUnknownOutcomeIsNotARefusal(t *testing.T) {
a := NotKnown("call-7", "no answer within 6m0s")
if a.Refused != "" || a.Exit == 0 || !strings.Contains(a.Unknown, "may have run") || !strings.Contains(a.Unknown, "call-7") {
t.Fatalf("an unknown outcome reads as %+v", a)
}
if b := NotKnown("", "no answer"); strings.Contains(b.Unknown, "call ") {
t.Fatalf("an unknown outcome with no call names one: %+v", b)
}
}
// Standard input up to its bound travels whole, beside the line: a secret given with --from -.
func TestStandardInputTravelsWithItsLine(t *testing.T) {
given := append(bytes.Repeat([]byte{0, 0xff, '\n', 'x'}, MaxStdin/4-1), "end"...)
var b bytes.Buffer
if err := WriteRequest(&b, Request{Line: []string{"secret", "accept", "anchor", "telegram", "telegram-token", "--from", "-"},
Stdin: given}); err != nil {
t.Fatal(err)
}
if b.Len() > MaxRequest {
t.Fatalf("a request with standard input at its bound is %d bytes, over MaxRequest %d", b.Len(), MaxRequest)
}
got, err := ReadRequest(&b)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(got.Stdin, given) || got.Line[1] != "accept" {
t.Fatalf("standard input did not travel whole: %d bytes, want %d", len(got.Stdin), len(given))
}
none, _ := json.Marshal(Request{Line: []string{"status"}})
if strings.Contains(string(none), "stdin") {
t.Fatalf("a request without standard input says it has one: %s", none)
}
}
go/cli/refusal_test.go
-123
View File
@@ -1,123 +0,0 @@
package cli
import (
"context"
"errors"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// anEngineThatClosesFirst listens like the node-engine before mesh-host#80: it writes its answer, if any, and
// closes without reading the request (novox/hq issue 434). It answers one connection.
func anEngineThatClosesFirst(t *testing.T, answer *Answer) string {
t.Helper()
socket := filepath.Join(t.TempDir(), "cli.sock")
l, err := net.Listen("unix", socket)
if err != nil {
t.Skipf("no unix socket here: %v", err)
}
t.Cleanup(func() { l.Close() })
go func() {
conn, err := l.Accept()
if err != nil {
return
}
if answer != nil {
_ = WriteAnswer(conn, *answer)
}
conn.Close()
}()
return socket
}
// aLineTheSocketCannotHold is a request larger than a unix socket buffers, so its write cannot finish before
// the engine reads it — and an engine that closes first makes it fail every time, not only when it loses a race.
func aLineTheSocketCannotHold() []string { return []string{"status", strings.Repeat("x", 4<<20)} }
// An engine that refuses and closes before reading the request is heard: its refusal reaches the caller, not
// the broken pipe its close left the client's write (novox/hq issue 444).
func TestARefusalWrittenBeforeTheRequestIsReadReachesTheCaller(t *testing.T) {
busy := Refusal("the node-engine is busy: too many lines at once")
socket := anEngineThatClosesFirst(t, &busy)
a, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
if err != nil {
t.Fatalf("the engine said %q and the caller heard: %v", busy.Refused, err)
}
if a.Refused != busy.Refused || a.Exit != 1 {
t.Fatalf("the engine said %q and the caller heard %+v", busy.Refused, a)
}
}
// When the write fails and no answer is waiting, the error says both: what failed, and that no answer came.
func TestAFailedWriteWithNoAnswerSaysBoth(t *testing.T) {
socket := anEngineThatClosesFirst(t, nil)
start := time.Now()
_, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
if err == nil {
t.Fatal("a request the engine never read, and never answered, came back without an error")
}
if !strings.Contains(err.Error(), "asking the node-engine") || !strings.Contains(err.Error(), "no answer came") {
t.Fatalf("the error does not say what failed and that no answer came: %v", err)
}
if waited := time.Since(start); waited > 30*time.Second {
t.Fatalf("waited %s on a connection the engine had closed", waited)
}
}
// After a failed write only a refusal is taken: nothing can have run, so an answer saying it ran and exited 0
// is an error, never a success (novox/hq issue 444).
func TestAnAnswerOtherThanARefusalAfterAFailedWriteIsAnError(t *testing.T) {
ran := Answer{Exit: 0, Stdout: []byte("ok")}
socket := anEngineThatClosesFirst(t, &ran)
a, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
if err == nil {
t.Fatalf("a request never wholly written came back as success: %+v", a)
}
if !strings.Contains(err.Error(), "not a refusal") {
t.Fatalf("the error does not say the answer was not a refusal: %v", err)
}
}
// Reading after a failed write never waits longer than the ask already allowed: a peer that never reads and
// never answers is given up on at the context's deadline, not later (novox/hq issue 444).
func TestAFailedWriteWaitsNoLongerThanTheDeadline(t *testing.T) {
socket := filepath.Join(t.TempDir(), "cli.sock")
l, err := net.Listen("unix", socket)
if err != nil {
t.Skipf("no unix socket here: %v", err)
}
t.Cleanup(func() { l.Close() })
held := make(chan net.Conn, 1)
go func() {
conn, err := l.Accept()
if err != nil {
return
}
held <- conn // neither read nor answered, nor closed, until the test ends
}()
t.Cleanup(func() {
select {
case c := <-held:
c.Close()
default:
}
})
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
start := time.Now()
_, err = Ask(ctx, socket, aLineTheSocketCannotHold())
waited := time.Since(start)
if err == nil {
t.Fatal("a peer that never read nor answered gave no error")
}
if !errors.Is(err, os.ErrDeadlineExceeded) {
t.Fatalf("the error is not the deadline: %v", err)
}
if waited > time.Second {
t.Fatalf("waited %s with a deadline of 50ms", waited)
}
}
go/cmd/mesh-hand-compose/main.go
-210
View File
@@ -1,210 +0,0 @@
// mesh-hand-compose composes one module for one node by hand, from the inputs the controller keeps, and signs it with the
// operator's key, for the hand path of novox/hq ADR 0539 clause 10: a fingerprint the controller never composed (a
// hand build), while the controller cannot compose its own fix. It composes with mesh-sdk's composition, the one the
// controller writes with (compose.Compositions), and prints the composition, its key and its digest; only with -write
// does it write them, with the operator's credential: the composition into the node's bucket composed-<node>, each
// file over 64 KiB into the node's object bucket, and the digest into mesh-controller_composed. The want is the
// operator's next act, naming that digest; each act is recorded in the hand-act log.
//
// mesh-hand-compose -input <inputs.json> -node <node> -module <module> -fp <src2:...> -key <operator key>
// [-write -nats <url> -creds <operator credential>]
//
// The inputs are a compose.Input as JSON. The key is the operator's ed25519 private key, PKCS#8 in PEM; the
// node-engine trusts it beside the controller's (ADR 0539 clause 3).
package main
import (
"context"
"crypto/ed25519"
"crypto/x509"
"encoding/json"
"encoding/pem"
"errors"
"flag"
"fmt"
"io"
"os"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/kvlimit"
)
func main() {
if err := run(os.Args[1:], os.Stdout); err != nil {
fmt.Fprintln(os.Stderr, "mesh-hand-compose:", err)
os.Exit(1)
}
}
// printed is what mesh-hand-compose prints: where the composition goes, its digest, the composition, and whether it was
// written.
type printed struct {
Bucket string `json:"bucket"`
Key string `json:"key"`
Digest string `json:"digest"`
Composition json.RawMessage `json:"composition"`
Objects []string `json:"objects,omitempty"`
Written bool `json:"written"`
Next string `json:"next"`
}
func run(args []string, stdout io.Writer) error {
fs := flag.NewFlagSet("mesh-hand-compose", flag.ContinueOnError)
input := fs.String("input", "", "the controller's kept inputs, a compose.Input as JSON")
node := fs.String("node", "", "the node to compose for")
module := fs.String("module", "", "the module to compose")
fp := fs.String("fp", "", "the module's fingerprint, src2: and 64 hex digits")
keyFile := fs.String("key", "", "the operator's ed25519 private key, PKCS#8 in PEM")
write := fs.Bool("write", false, "write the composition, its files and its digest, with the operator's credential")
url := fs.String("nats", nats.DefaultURL, "with -write: the bus")
creds := fs.String("creds", "", "with -write: the operator's credential file")
if err := fs.Parse(args); err != nil {
return err
}
switch {
case *input == "" || *node == "" || *module == "" || *fp == "":
return errors.New("-input, -node, -module and -fp are each required")
case *keyFile == "":
return errors.New("-key is required: a composition made by hand is signed by the operator's key, and the node-engine applies no other")
case *write && *creds == "":
return errors.New("-write needs -creds: the operator's credential is the only hand writer of a composition")
}
key, err := readKey(*keyFile)
if err != nil {
return err
}
key2, err := delivery.ComposedKey(*node, *module, *fp)
if err != nil {
return err
}
bucket, err := delivery.ComposedBucketFor(*node)
if err != nil {
return err
}
raw, err := os.ReadFile(*input)
if err != nil {
return err
}
var in compose.Input
if err := json.Unmarshal(raw, &in); err != nil {
return fmt.Errorf("the inputs in %s are not a compose.Input: %w", *input, err)
}
all, _, err := compose.Compositions(in, *node, map[string]string{*module: *fp}, compose.Reading)
if err != nil {
return err
}
c, ok := all[key2]
if !ok {
return fmt.Errorf("%s does not run %s: its composition holds none of it", *node, *module)
}
c, objects, err := c.ByDigest()
if err != nil {
return err
}
if c, err = delivery.SignComposed(c, key); err != nil {
return err
}
encoded, err := delivery.EncodeComposed(c)
if err != nil {
return err
}
digest, err := delivery.ComposedDigestOf(c)
if err != nil {
return err
}
out := printed{Bucket: bucket, Key: key2, Digest: digest, Composition: encoded,
Next: fmt.Sprintf("write want.%s.%s naming fp %s and composed %s with the operator's credential, and record each act in the hand-act log", *node, *module, *fp, digest)}
for d := range objects {
out.Objects = append(out.Objects, d)
}
if *write {
if err := writeIt(*url, *creds, *node, bucket, key2, encoded, digest, objects); err != nil {
return err
}
out.Written = true
}
enc := json.NewEncoder(stdout)
enc.SetIndent("", " ")
return enc.Encode(out)
}
func readKey(path string) (ed25519.PrivateKey, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
block, _ := pem.Decode(raw)
if block == nil {
return nil, fmt.Errorf("%s holds no PEM block", path)
}
parsed, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
key, ok := parsed.(ed25519.PrivateKey)
if !ok {
return nil, fmt.Errorf("%s is not an ed25519 key", path)
}
return key, nil
}
// writeIt writes the files by digest, then the composition, then its digest: a reader that finds the digest finds the
// composition and its files already there.
func writeIt(url, creds, node, bucket, key string, encoded []byte, digest string, objects map[string][]byte) error {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
nc, err := nats.Connect(url, nats.UserCredentials(creds), nats.Name("mesh-hand-compose"))
if err != nil {
return err
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
return err
}
if len(objects) > 0 {
filesBucket, err := declaration.FileBucketFor(node)
if err != nil {
return err
}
store, err := js.ObjectStore(ctx, filesBucket)
if err != nil {
return fmt.Errorf("the node's object bucket %s: %w", filesBucket, err)
}
for d, body := range objects {
name, err := declaration.ObjectName(d)
if err != nil {
return err
}
if _, err := store.PutBytes(ctx, name, body); err != nil {
return fmt.Errorf("the file %s: %w", d, err)
}
}
}
composed, err := js.KeyValue(ctx, bucket)
if err != nil {
return fmt.Errorf("the bucket %s: %w", bucket, err)
}
if _, err := kvlimit.Put(ctx, composed, key, encoded); err != nil {
return fmt.Errorf("%s in %s: %w", key, bucket, err)
}
value, err := delivery.EncodeComposedDigest(delivery.ComposedDigest{Digest: digest, Judgement: delivery.JudgedCannot,
Words: "composed by hand with mesh-hand-compose (ADR 0539 clause 10): the operator writes the want"})
if err != nil {
return err
}
digests, err := js.KeyValue(ctx, delivery.ControllerComposedBucket)
if err != nil {
return fmt.Errorf("the bucket %s: %w", delivery.ControllerComposedBucket, err)
}
if _, err := kvlimit.Put(ctx, digests, key, value); err != nil {
return fmt.Errorf("%s in %s: %w", key, delivery.ControllerComposedBucket, err)
}
return nil
}
go/cmd/mesh-hand-compose/main_test.go
-145
View File
@@ -1,145 +0,0 @@
package main
import (
"bytes"
"crypto/ed25519"
"crypto/rand"
"crypto/x509"
"encoding/base64"
"encoding/json"
"encoding/pem"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// aMesh is the Input of mesh-controller's aMesh, as mesh-sdk's compose tests make it (compose/fixture_test.go): two
// machines on the private network, the module the mesh provides for itself assigned to both. The manifest is the one
// the controller provides (overlay.Manifest), read back as the catalogue reads it.
func aMesh(t *testing.T) compose.Input {
t.Helper()
var m catalogue.Manifest
b, _ := json.Marshal(overlay.Manifest())
if err := json.Unmarshal(b, &m); err != nil {
t.Fatal(err)
}
in := compose.Input{Catalogue: map[string]catalogue.Manifest{overlay.Name: m}}
for i, name := range []string{"anchor", "laptop"} {
pub, _, _ := ed25519.GenerateKey(rand.Reader)
in.Nodes = append(in.Nodes, compose.Node{Name: name,
Place: compose.Place{Key: base64.StdEncoding.EncodeToString(pub), Endpoint: name + ".example:51820", Site: "here", Hub: name == "anchor", Address: []string{"10.77.0.1", "10.77.0.2"}[i]},
Assigned: []string{overlay.Name}, Capabilities: map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true}})
}
return in
}
func writeKey(t *testing.T, dir string) ed25519.PublicKey {
t.Helper()
pub, priv, _ := ed25519.GenerateKey(rand.Reader)
der, err := x509.MarshalPKCS8PrivateKey(priv)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "operator.key"), pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der}), 0o600); err != nil {
t.Fatal(err)
}
return pub
}
// 1943/AC5: mesh-hand-compose, given the kept inputs, a node, a module and its fingerprint, prints that module's
// composition for that node signed by the operator's key and its digest: its resources are the controller's for that
// module on that node (compose.Compose), and it verifies against the operator's key. Without --write it writes nothing.
func TestMeshComposeGivesTheControllersResourcesSignedByTheOperator(t *testing.T) {
dir := t.TempDir()
in := aMesh(t)
raw, err := json.Marshal(in)
if err != nil {
t.Fatal(err)
}
input := filepath.Join(dir, "input.json")
if err := os.WriteFile(input, raw, 0o600); err != nil {
t.Fatal(err)
}
operator := writeKey(t, dir)
fp := delivery.SourcePrefix2 + strings.Repeat("a", 64)
var out bytes.Buffer
if err := run([]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp,
"-key", filepath.Join(dir, "operator.key")}, &out); err != nil {
t.Fatal(err)
}
var printed struct {
Key string `json:"key"`
Bucket string `json:"bucket"`
Digest string `json:"digest"`
Composition json.RawMessage `json:"composition"`
Written bool `json:"written"`
}
if err := json.Unmarshal(out.Bytes(), &printed); err != nil {
t.Fatalf("%s: %v", out.String(), err)
}
c, err := delivery.DecodeComposed(printed.Composition)
if err != nil {
t.Fatal(err)
}
if digest, err := delivery.VerifyComposed(c, []ed25519.PublicKey{operator}); err != nil || digest != printed.Digest {
t.Errorf("the composition verifies %q, %v against the operator's key; printed %s", digest, err, printed.Digest)
}
wantKey, _ := delivery.ComposedKey("laptop", overlay.Name, fp)
if printed.Key != wantKey || printed.Bucket != "composed-laptop" || printed.Written {
t.Errorf("printed key %s in %s, written %v", printed.Key, printed.Bucket, printed.Written)
}
d, _, err := compose.Compose(in, "laptop")
if err != nil {
t.Fatal(err)
}
var want, got []string
for _, r := range d.Resources {
want = append(want, r["id"].(string))
}
for _, r := range c.Resources {
var head struct{ ID string }
_ = json.Unmarshal(r, &head)
got = append(got, head.ID)
}
sort.Strings(want)
sort.Strings(got)
if len(got) == 0 || strings.Join(got, " ") != strings.Join(want, " ") {
t.Errorf("mesh-hand-compose gives %v, the controller's composition %v", got, want)
}
}
// 1943/AC5: mesh-hand-compose refuses what it cannot compose, naming why: no key, a fingerprint not of the new form, a
// module the node does not run, a node the mesh does not have.
func TestMeshComposeRefuses(t *testing.T) {
dir := t.TempDir()
raw, _ := json.Marshal(aMesh(t))
input := filepath.Join(dir, "input.json")
_ = os.WriteFile(input, raw, 0o600)
writeKey(t, dir)
key := filepath.Join(dir, "operator.key")
fp := delivery.SourcePrefix2 + strings.Repeat("a", 64)
for _, c := range []struct {
args []string
why string
}{
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp}, "-key"},
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", "src2:1", "-key", key}, "fingerprint"},
{[]string{"-input", input, "-node", "laptop", "-module", "mesh-mail", "-fp", fp, "-key", key}, "does not run mesh-mail"},
{[]string{"-input", input, "-node", "nowhere", "-module", overlay.Name, "-fp", fp, "-key", key}, "nowhere"},
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp, "-key", key, "-write"}, "-creds"},
} {
var out bytes.Buffer
if err := run(c.args, &out); err == nil || !strings.Contains(err.Error(), c.why) {
t.Errorf("mesh-hand-compose %v = %v, want a refusal saying %q", c.args, err, c.why)
}
}
}
go/compose/composed.go
+1 -1
View File
@@ -18,7 +18,7 @@ import (
// the controller composes it, split into each module's composition at the fingerprint fps gives it and the node's own
// (`<node>.node`), by key (delivery.ComposedKey). Each resource says its module, whatever the node reported it reads.
// A module fps gives no fingerprint for is not composed and is named in skipped, so a caller composing one module
// (mesh-hand-compose) gets that one; but for a module the node runs inside its own composition (Node.InOwn, and the
// gets that one; but for a module the node runs inside its own composition (Node.InOwn, and the
// private network's always), which fps names at no fingerprint of the new form: its resources go into the node's own,
// their module taken off and their ids still naming it, and its filled paths with them, as the controller's composed
// writer folds them (issue 1126). Each module's Data says whether its manifest declares data. The compositions are not
go/delivery/assigner.go
+1 -1
View File
@@ -140,7 +140,7 @@ const (
RuleStep = 4 // at most step other nodes are ahead (step 0: no limit, ADR 0528 §3)
RuleProviders = 5 // every provider the module requires runs healthy on that node, or at its canary
RuleGrant = 6 // the module's credential for F exists (grants.<module>.<fp>)
RuleComposed = 7 // the module's composition for that node at F exists (mesh-controller_composed, ADR 0539 §6)
RuleComposed = 7 // the module's composition for that node at F exists (ADR 0539 §6)
// RuleProvisions: every provision the module requires on that node is provided (novox/hq ADR 0570 §4): a want's
// words name the key and its state that hold it, as "the database of keycloak on novox is asked of postgres".
RuleProvisions = 8
go/delivery/assigner_test.go
+15 -34
View File
@@ -268,58 +268,39 @@ func TestCompositionVerdictRefused(t *testing.T) {
// 1394/B1b: a grant key round-trips; a want or held key is refused, naming it; a grant round-trips and one naming an
// empty account is refused.
func TestControllerGrant(t *testing.T) {
func TestTheGrantsIssued(t *testing.T) {
fp := fp2of('c')
key, err := ControllerGrantKey("mesh-delivery-assigner", fp)
key, err := PartGrantKey("mesh-delivery-assigner", fp)
if err != nil || key != "mesh-delivery-assigner."+strings.Repeat("c", 64) {
t.Fatalf("ControllerGrantKey = %q, %v", key, err)
t.Fatalf("PartGrantKey = %q, %v", key, err)
}
module, back, err := ParseControllerGrantKey(key)
module, back, err := ParsePartGrantKey(key)
if err != nil || module != "mesh-delivery-assigner" || back != fp {
t.Errorf("ParseControllerGrantKey = %q, %q, %v", module, back, err)
t.Errorf("ParsePartGrantKey = %q, %q, %v", module, back, err)
}
held, _ := HeldKey("mesh-mail", fp)
for _, bad := range []string{"want.ace.mesh-mail", held, "ace.mesh-mail", ""} {
if _, _, err := ParseControllerGrantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseControllerGrantKey(%q) = %v, want a refusal naming it", bad, err)
if _, _, err := ParsePartGrantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParsePartGrantKey(%q) = %v, want a refusal naming it", bad, err)
}
}
for _, g := range []ControllerGrant{{}, {Accounts: []string{"UABC", "UDEF"}}} {
data, err := EncodeControllerGrant(g)
for _, g := range []Grant{{}, {Accounts: []string{"UABC", "UDEF"}}} {
data, err := EncodeGrant(g)
if err != nil {
t.Fatal(err)
}
got, err := DecodeControllerGrant(data)
got, err := DecodeGrant(data)
g.V = Version
if err != nil || !SameControllerGrant(got, g) {
if err != nil || !SameGrant(got, g) {
t.Errorf("grant %+v came back %+v, %v", g, got, err)
}
}
if _, err := EncodeControllerGrant(ControllerGrant{Accounts: []string{""}}); err == nil {
if _, err := EncodeGrant(Grant{Accounts: []string{""}}); err == nil {
t.Errorf("a grant naming an empty account was accepted")
}
// The grants part's keys are the controller's: the same key round-trips; the source is the part's bucket only for a
// handed-away lease (issue 774).
if pk, err := PartGrantKey("mesh-delivery-assigner", fp); err != nil || pk != key {
t.Errorf("PartGrantKey = %q, %v, want the controller's %q", pk, err, key)
}
if m, f, err := ParsePartGrantKey(key); err != nil || m != "mesh-delivery-assigner" || f != fp {
t.Errorf("ParsePartGrantKey = %q, %q, %v", m, f, err)
}
g := Grant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct {
away bool
bucket, read string
}{
{false, "mesh-controller_grants", "mesh-controller.grants"},
{true, "mesh-grants_grants", "mesh-grants.grants"},
} {
if b, r := GrantSource(c.away); b != c.bucket || r != c.read {
t.Errorf("GrantSource(%v) = %q, %q, want %q, %q", c.away, b, r, c.bucket, c.read)
}
}
g := ControllerGrant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct {
g ControllerGrant
g Grant
node, module string
names bool
}{
@@ -327,7 +308,7 @@ func TestControllerGrant(t *testing.T) {
{g, "shanks", "mesh-mail", true},
{g, "g14", "mesh-build-go", false},
{g, "ace", "mesh-build", false},
{ControllerGrant{}, "novox", "nats", true},
{Grant{}, "novox", "nats", true},
} {
if got := c.g.NamesNode(c.node, c.module); got != c.names {
t.Errorf("%v names %s.%s: %v, want %v", c.g.Accounts, c.node, c.module, got, c.names)
go/delivery/builder.go
+2 -2
View File
@@ -10,8 +10,8 @@ import (
// The builder's keys (novox/hq ADR 0429 clauses 5 and 16, ADR 0437 clause 5, ADR 0405 clauses 2 and 7, ADR 0453
// clause 4): a build is `<module>.<fingerprint key part>` in the builder's bucket, one key per fingerprint, since equal
// fingerprints are equal content and the same source is never built twice. They are written only by the builder part
// from R4c and read by the checker, the assigner and the view, and from R4e by the controller. They are not the
// controller's build records (buildrecord.go: BuildsBucket, BuildKey), which stay as they are.
// from R4c and read by the checker, the assigner and the view. They are not the ended controller's build records
// (buildrecord.go: BuildKey).
//
// **The rules its writer keeps.** A builder claims a fingerprint that has no build key by creating it queued or
// building, a create that fails when the key exists, and moves it by compare-and-set; a builder reclaims only a build
go/delivery/builder_test.go
+1 -4
View File
@@ -166,10 +166,7 @@ func TestBuilderLayoutDoesNotCollideWithTheControllers(t *testing.T) {
if BuilderBuildsBucket != "mesh-delivery-builder_builds" || BuilderBuildsHistory != 5 || BuilderModule != "mesh-delivery-builder" {
t.Errorf("bucket %s history %d module %s", BuilderBuildsBucket, BuilderBuildsHistory, BuilderModule)
}
if BuilderBuildsBucket == BuildsBucket || BuilderModule == "mesh-controller" {
t.Error("the builder's bucket is the controller's")
}
if BuildsBucket != "mesh-controller_builds" || BuildKey("m", "7") != "m.build.7" || NewestKey("m") != "m.newest" {
if BuildKey("m", "7") != "m.build.7" || NewestKey("m") != "m.newest" {
t.Error("the controller's layout changed")
}
// the two key forms are never read as each other
go/delivery/buildrecord.go
+2 -7
View File
@@ -2,8 +2,8 @@ package delivery
import "time"
// BuildRecord is the layout mesh-controller internal/chain/records.go Build writes (line 91), the fields a reader outside
// the controller reads (novox/hq ADR 0437 §4 §5, issue 1223). The controller's record carries more; a decode ignores
// BuildRecord is the layout the ended controller's chain wrote (mesh-controller internal/chain/records.go Build, line 91),
// the fields a reader outside it read (novox/hq ADR 0437 §4 §5, issue 1223). The controller's record carries more; a decode ignores
// what is not here. Path is the module's directory in its repository, as the controller writes it (Build.Path, records.go:100,
// set from the build request trimmed of slashes at internal/inventory/chain_requests.go:19); empty for a module whose
// manifest is the repository's root. SourceInputs has no json keys of its own: it is written under its Go field names, as the controller
@@ -32,11 +32,6 @@ type Artifact struct {
Reference string `json:"reference"`
}
// BuildsBucket is the bucket the builds are kept in: mesh-controller internal/broker/controller_buckets.go:101,
// BucketName(ControllerSeat, ChainBuilds), with ControllerSeat "mesh-controller" (internal/broker/streams.go:359) and
// ChainBuilds "builds" (controller_buckets.go:113).
const BuildsBucket = "mesh-controller_builds"
// BuildKey is a build's record under its module, a build that knew none under the request's key: copied from
// mesh-controller internal/chain/keys.go:35-40 (BuildKey, with RequestKey at line 42).
func BuildKey(module, id string) string {
go/delivery/buildrecord_test.go
-3
View File
@@ -114,9 +114,6 @@ func TestBuildRecordWithoutInputs(t *testing.T) { // 1223/AC2
func TestBuildKeys(t *testing.T) { // 1223/AC3
// Literal values of mesh-controller internal/broker/controller_buckets.go:101 (BucketName(ControllerSeat, ChainBuilds)
// is "mesh-controller_builds") and internal/chain/keys.go:35-44.
if BuildsBucket != "mesh-controller_builds" {
t.Errorf("BuildsBucket = %q", BuildsBucket)
}
if got := BuildKey("build-agent", "build-1"); got != "build-agent.build.build-1" {
t.Errorf("BuildKey = %q", got)
}
go/delivery/composed.go
+8 -20
View File
@@ -7,29 +7,17 @@ import (
"strings"
)
// The controller's compositions (novox/hq ADR 0539 clauses 1 to 4). A **composition** is one module's resources
// composed for one node at one fingerprint, written by the controller, which keeps the inputs. It lives in a bucket of
// each node, `composed-<node>`, which only that node's node-engine reads (a composition can carry a sealed value), under
// `<node>.<module>.<fingerprint hex>`, and the node's own resources (adoption, firewall openings, accounts, the
// resolver, identities and groups) under `<node>.node`. Beside it, under the same key in `mesh-controller_composed`,
// which the assigner reads, the controller writes the composition's digest and its judgement of the move from the
// composition the node runs now (ComposedDigest). The controller writes both by compare-and-set while it holds the
// lease; the operator's credential is a second writer on the hand path alone (clause 10, mesh-hand-compose).
//
// A composition is derived state (clause 4): its key and its digest are deleted by the controller when its fingerprint
// is no longer among the module's five newest and no want names it, and each bucket keeps five revisions of a key.
// The compositions (novox/hq ADR 0539 clauses 1 to 4). A **composition** is one module's resources composed for one node
// at one fingerprint: a node composes its own (ADR 0547 §2), and the ended controller wrote them into a bucket of each
// node, `composed-<node>`, which nothing writes since it ended (ADR 0590), under `<node>.<module>.<fingerprint hex>`, and
// the node's own resources (adoption, firewall openings, accounts, the resolver, identities and groups) under
// `<node>.node`. A composition's digest and its judgement of the move from the composition the node runs now is
// ComposedDigest.
const (
// ComposedBucketPrefix starts the name of a node's bucket of compositions, ComposedBucketFor(node).
ComposedBucketPrefix = "composed-"
// ComposedHistory is the revisions a node's composition bucket keeps per key (ADR 0539 clause 4).
ComposedHistory = 5
// ControllerModule is the module of the controller, the writer of the compositions.
ControllerModule = "mesh-controller"
// ControllerComposedBucket is the controller's bucket of each composition's digest and judgement, `<owner
// module>_<name>`, read by the assigner (ADR 0539 clauses 2 and 6).
ControllerComposedBucket = ControllerModule + "_composed"
// ControllerComposedHistory is the revisions that bucket keeps per key.
ControllerComposedHistory = 5
// ComposedKeep is how many of a module's newest fingerprints keep their compositions though no want names them
// (ADR 0539 clause 4).
ComposedKeep = 5
@@ -142,8 +130,8 @@ func checkDigest(digest, of string) error {
//
// Policy is the module's upgrade policy as the controller holds it (ADR 0236, ADR 0539 clause 14), with PolicyFrom
// where it comes from and PolicyWhy why: the assigner holds a move of a module of policy record from it, the operator's
// own choice (`upgrade` with policy) included. Rewritten when the policy changes. Empty on a composition made by hand
// (mesh-hand-compose), whose want the operator writes.
// own choice (`upgrade` with policy) included. Rewritten when the policy changes. Empty on a composition made by hand,
// whose want the operator wrote.
type ComposedDigest struct {
V int `json:"v"`
Digest string `json:"digest"`
go/delivery/composed_signature.go
+3 -4
View File
@@ -6,9 +6,8 @@ import (
"fmt"
)
// A composition's signature (novox/hq ADR 0539 clause 3): the controller signs each composition it writes with its
// key, and the operator signs one made by hand with mesh-hand-compose with theirs. The node-engine trusts exactly these two
// signers and applies nothing of a composition whose signature fails or whose signer is neither. The signature is an
// A composition's signature (novox/hq ADR 0539 clause 3): the signer of a composition signs it with its key (the ended
// controller signed those it wrote, the operator one made by hand). The node-engine trusts the signers it is given and applies nothing of a composition whose signature fails or whose signer is neither. The signature is an
// ed25519 signature, as a declaration's is, of the composition's digest (ComposedDigestOf) after a context line, so a
// signature made for anything else never verifies here.
const composedSigningContext = "novox composition v1\n"
@@ -84,7 +83,7 @@ func VerifyComposed(c Composed, trusted []ed25519.PublicKey) (string, error) {
}
}
if key == nil {
return "", fmt.Errorf("delivery: the composition of %q on %s is signed by %s, which is not a trusted signer (the controller's key or the operator's)",
return "", fmt.Errorf("delivery: the composition of %q on %s is signed by %s, which is not a trusted signer (the vault's key or the operator's)",
c.Module, c.Node, c.Signature.Signer)
}
digest, err := ComposedDigestOf(c)
go/delivery/composed_signature_test.go
+3 -3
View File
@@ -108,7 +108,7 @@ func TestACompositionIsSignedByAKeptKey(t *testing.T) {
// signers a node trusts and names which; an unknown key and another kind's context are refused.
vaultPub, vault := aSigningKey(t)
controllerPub, controller := aSigningKey(t)
trusted := []Signer{{Name: SignerController, Key: controllerPub}, {Name: SignerVault, Key: vaultPub}}
trusted := []Signer{{Name: "operator", Key: controllerPub}, {Name: SignerVault, Key: vaultPub}}
enrol, _ := SignContext(KindEnrolment)
certCtx, _ := SignContext(KindNodeCertificate)
msg := []byte("the reply")
@@ -120,7 +120,7 @@ func TestACompositionIsSignedByAKeptKey(t *testing.T) {
want string
}{
{"the vault's key verifies", vault, enrol, enrol, SignerVault},
{"the controller's key verifies", controller, enrol, enrol, SignerController},
{"the controller's key verifies", controller, enrol, enrol, "operator"},
{"an unknown key is refused", other, enrol, enrol, ""},
{"the wrong context is refused", vault, certCtx, enrol, ""},
} {
@@ -148,7 +148,7 @@ func TestACompositionIsSignedByAKeptKey(t *testing.T) {
}
}
if got := Trusts(trusted, nil); len(got) != 2 || !TrustsKey(got, SignerVault, SignerFingerprint(vaultPub)) ||
!TrustsKey(got, SignerController, SignerFingerprint(controllerPub)) {
!TrustsKey(got, "operator", SignerFingerprint(controllerPub)) {
t.Errorf("trusts: %v", got)
}
// A certificate the vault's authority signs for a node's name and serving key checks against a bundle holding that
go/delivery/delivery.go
+14 -15
View File
@@ -367,8 +367,7 @@ type Have struct {
// holds now. The bus reloads the file itself when its digest changes (the module's entrypoint), so Digest is the list
// the bus loads. Wanted differs from Digest only while a write failed, and Error says why.
type UserListWritten struct {
// Source is the key the list was taken from: the controller's composition of the bus module for this node
// (`<node>.<module>.<fp>`), or the grants part's `user-list` once it took over.
// Source is the key the list was taken from: the grants part's `user-list` (GrantsUserListSource).
Source string `json:"source"`
Revision uint64 `json:"revision"`
// Wanted is the digest of the list the source holds, `sha256:<hex>`.
@@ -378,8 +377,8 @@ type UserListWritten struct {
Users int `json:"users"`
At time.Time `json:"at"`
Error string `json:"error,omitempty"`
// Follows is which list the node follows now (FollowController or FollowGrants, novox/hq ADR 0547 clause 8), empty
// from a node-engine before the switch.
// Follows is which list the node follows now (FollowGrants, novox/hq ADR 0547 clause 8), empty from a node-engine
// before the switch.
Follows string `json:"follows,omitempty"`
// Key is the bus node's sealing key, its public half as SPKI PEM: the key the grants part seals its list to
// (clause 7). Public, as the vault holds it.
@@ -417,19 +416,19 @@ type HaveNode struct {
// its shadow composer whenever an answer changes, removed when the candidate key goes. The node applies nothing of a
// candidate.
Candidates map[string]CandidateAnswer `json:"candidates,omitempty"`
// Composes is whose compositions the node applies: its own (ComposesOwn, placement places it) or the
// controller's (ComposesController); ComposesWhy says why in words. Written by the node agent at each start, so a
// node returning to the controller's compositions when its placement key is deleted says so (issue 1126). Empty
// from an engine older than the field.
// Composes is whose compositions the node applies: its own (ComposesOwn, placement places it), or none, empty, with
// ComposesWhy saying why in words. Written by the node agent at each start (issue 1126). An engine from before the
// controller ended may still say `controller` (composedByTheEndedController), which is read and never written.
Composes string `json:"composes,omitempty"`
ComposesWhy string `json:"composes-why,omitempty"`
}
// Whose compositions a node applies (HaveNode.Composes).
const (
ComposesOwn = "own"
ComposesController = "controller"
)
// ComposesOwn is a node applying its own compositions (HaveNode.Composes).
const ComposesOwn = "own"
// composedByTheEndedController is what an engine from before the controller ended (ADR 0590) wrote in Composes while
// it applied the controller's compositions: still read, so its have is not refused, and written by no engine now.
const composedByTheEndedController = "controller"
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
// writes it.
@@ -526,9 +525,9 @@ func DecodeHave(data []byte) (Have, error) {
func (n HaveNode) check() error {
switch n.Composes {
case "", ComposesOwn, ComposesController:
case "", ComposesOwn, composedByTheEndedController:
default:
return fmt.Errorf("delivery: a node composes %q, neither %s nor %s", n.Composes, ComposesOwn, ComposesController)
return fmt.Errorf("delivery: a node composes %q, not %s", n.Composes, ComposesOwn)
}
for group, a := range n.Candidates {
if group == "" {
go/delivery/delivery_test.go
+6 -14
View File
@@ -242,29 +242,21 @@ func TestRoundTrip(t *testing.T) {
t.Fatal(err)
}
back, err := DecodeGrantsUserList(raw)
if err != nil || back.Digest != digest || back.LeftOut == nil || back.Follow != FollowGrants || back.LeaseHanded {
if err != nil || back.Digest != digest || back.LeftOut == nil || back.Follow != FollowGrants {
t.Fatalf("decoded %+v, %v", back, err)
}
// ADR 0582 clause 3 (issue 774, 774/S3): a key written before the flag existed reads not handed; one written with
// it reads handed.
// Issue 1126 B4a (ADR 0590): a key written with the lease's flag still reads; the controller's list is no source.
if old, err := DecodeGrantsUserList([]byte(`{"v":1,"follow":"grants","digest":"","object":"","sealed_to":"",` +
`"node":"novox","revision":3,"users":40,"left_out":[],"at":"2026-10-30T10:00:00Z"}`)); err != nil || old.LeaseHanded {
t.Errorf("a key from before the lease write decodes %+v, %v", old, err)
`"node":"novox","revision":3,"users":40,"left_out":[],"at":"2026-10-30T10:00:00Z","lease-handed":true}`)); err != nil || old.Revision != 3 {
t.Errorf("a key with the lease's flag decodes %+v, %v", old, err)
}
g.LeaseHanded = true
if raw, err := EncodeGrantsUserList(g); err != nil || !bytes.Contains(raw, []byte(`"lease-handed":true`)) {
t.Errorf("the handed flag is written as %s, %v", raw, err)
} else if back, err := DecodeGrantsUserList(raw); err != nil || !back.LeaseHanded {
t.Errorf("the handed flag reads back %+v, %v", back, err)
}
g.LeaseHanded = false
for _, bad := range []GrantsUserList{{Follow: "both", Digest: digest, Object: g.Object},
for _, bad := range []GrantsUserList{{Follow: "both", Digest: digest, Object: g.Object}, {Follow: "controller"},
{Follow: FollowGrants, Digest: digest, Object: "other"}, {Follow: FollowGrants, Digest: "md5:x", Object: "md5:x"}} {
if _, err := EncodeGrantsUserList(bad); err == nil {
t.Errorf("%+v was written", bad)
}
}
if _, err := EncodeGrantsUserList(GrantsUserList{Follow: FollowController}); err != nil {
if _, err := EncodeGrantsUserList(GrantsUserList{Follow: FollowGrants}); err != nil {
t.Errorf("a switch with no list yet refused: %v", err)
}
object, err := SealUserList(list, node.PublicKey)
go/delivery/grants.go
+30 -63
View File
@@ -7,41 +7,44 @@ import (
"strings"
)
// The controller's grants (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6): when the controller has issued a module's
// bus credential for a fingerprint, it writes `<module>.<fingerprint hex>` in its bucket grants. The assigner reads
// it: a node's want moves to a fingerprint only once its grant exists. Only the controller writes the bucket.
//
// The names say Controller, since the catalogue's GrantKey (catalogue/streams.go) is a stream's grant pattern, unrelated.
// **The grants issued** (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, ADR 0547 §4, §5, ADR 0590): when the grants
// part has issued a module's bus credentials for a fingerprint, it writes `<module>.<fingerprint hex>` in its state
// `grants` (PartGrantsBucket). The assigner reads it: a node's want moves to a fingerprint only once its grant exists.
// Only the grants part writes the bucket.
const (
// ControllerGrantsBucket is the controller's bucket of issued credentials by fingerprint.
ControllerGrantsBucket = "mesh-controller_grants"
// ControllerGrantsHistory is the revisions the bucket keeps per key.
ControllerGrantsHistory = 5
// PartGrantsState is the grants part's state of issued credentials by fingerprint, as its manifest names it.
PartGrantsState = "grants"
// PartGrantsBucket is that state's bucket, `<owner module>_<name>`.
PartGrantsBucket = "mesh-grants_" + PartGrantsState
// PartGrantsRead is that state as a reader names it in `reads`.
PartGrantsRead = "mesh-grants." + PartGrantsState
// PartGrantsHistory is the revisions the bucket keeps per key.
PartGrantsHistory = 5
)
// ControllerGrantKey is the key, in ControllerGrantsBucket, of a module's credential issued for a fingerprint:
// `<module>.<hex>`, the hex being the fingerprint's key part (FingerprintKeyPart).
func ControllerGrantKey(module, fingerprint string) (string, error) {
// PartGrantKey is the key, in PartGrantsBucket, of a module's credential issued for a fingerprint: `<module>.<hex>`,
// the hex being the fingerprint's key part (FingerprintKeyPart).
func PartGrantKey(module, fingerprint string) (string, error) {
return BuilderBuildKey(module, fingerprint)
}
// ParseControllerGrantKey is the inverse of ControllerGrantKey; it refuses a want or held key, naming it.
func ParseControllerGrantKey(key string) (module, fingerprint string, err error) {
// ParsePartGrantKey is the inverse of PartGrantKey; it refuses a want or held key, naming it.
func ParsePartGrantKey(key string) (module, fingerprint string, err error) {
if module, fingerprint, err = ParseBuilderBuildKey(key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a grant key, <module>.<fingerprint hex>: %w", key, err)
}
return module, fingerprint, nil
}
// ControllerGrant is the value of a grant key: the bus accounts (their public keys) the module's processes were issued
// Grant is the value of a grant key: the bus accounts (their public keys) the module's processes were issued
// for that fingerprint, none for a module whose grants the tool runner's account carries, so a reissue for another
// account is a new revision. The entry's own time says when it was issued.
type ControllerGrant struct {
type Grant struct {
V int `json:"v"`
Accounts []string `json:"accounts,omitempty"`
}
func (g ControllerGrant) check() error {
func (g Grant) check() error {
for _, a := range g.Accounts {
if strings.TrimSpace(a) == "" || strings.ContainsAny(a, " \t\n") {
return fmt.Errorf("delivery: a grant names an empty or spaced account %q", a)
@@ -50,8 +53,8 @@ func (g ControllerGrant) check() error {
return nil
}
// EncodeControllerGrant is g as JSON at Version; it refuses an empty or spaced account.
func EncodeControllerGrant(g ControllerGrant) ([]byte, error) {
// EncodeGrant is g as JSON at Version; it refuses an empty or spaced account.
func EncodeGrant(g Grant) ([]byte, error) {
if err := g.check(); err != nil {
return nil, err
}
@@ -59,26 +62,26 @@ func EncodeControllerGrant(g ControllerGrant) ([]byte, error) {
return json.Marshal(g)
}
// DecodeControllerGrant reads a grant; it refuses an unknown version (ErrUnknownVersion) and an empty or spaced account.
func DecodeControllerGrant(data []byte) (ControllerGrant, error) {
var g ControllerGrant
// DecodeGrant reads a grant; it refuses an unknown version (ErrUnknownVersion) and an empty or spaced account.
func DecodeGrant(data []byte) (Grant, error) {
var g Grant
if err := decode(data, &g); err != nil {
return ControllerGrant{}, err
return Grant{}, err
}
if err := g.check(); err != nil {
return ControllerGrant{}, err
return Grant{}, err
}
return g, nil
}
// SameControllerGrant says whether two grants are the same value.
func SameControllerGrant(a, b ControllerGrant) bool { return reflect.DeepEqual(a, b) }
// SameGrant says whether two grants are the same value.
func SameGrant(a, b Grant) bool { return reflect.DeepEqual(a, b) }
// NamesNode says whether the grant names module's accounts on node: its own `<node>.<module>` or a process's
// `<node>.<module>.<process>`. A grant that names no account (a module that does not speak on the bus) names every
// node. The assigner wants a node only once the grant names its account (issue 1945), and a put waits for it (ADR
// 0546 clause 4).
func (g ControllerGrant) NamesNode(node, module string) bool {
func (g Grant) NamesNode(node, module string) bool {
if len(g.Accounts) == 0 {
return true
}
@@ -90,39 +93,3 @@ func (g ControllerGrant) NamesNode(node, module string) bool {
}
return false
}
// **The grants part's grants** (novox/hq ADR 0547 §4, §5, ADR 0561 §2, §6; issue 774). The grants part writes the same
// keys, with the same value, into its own state `grants` (the bucket PartGrantsBucket), beside the controller's: first in
// shadow, which nobody reads, so a tool can compare the two before anything switches. The assigner reads the part's keys
// only once the controller's lease is handed away (GrantSource), and the controller's until then.
const (
// PartGrantsState is the grants part's state of issued credentials by fingerprint, as its manifest names it.
PartGrantsState = "grants"
// PartGrantsBucket is that state's bucket, `<owner module>_<name>`.
PartGrantsBucket = "mesh-grants_" + PartGrantsState
// PartGrantsRead is that state as a reader names it in `reads`.
PartGrantsRead = "mesh-grants." + PartGrantsState
// PartGrantsHistory is the revisions the bucket keeps per key.
PartGrantsHistory = ControllerGrantsHistory
)
// PartGrantKey is the key, in PartGrantsBucket, of a module's credential issued for a fingerprint: the layout of the
// controller's key (ControllerGrantKey), so a key of one is the key of the other and a compare is by key.
func PartGrantKey(module, fingerprint string) (string, error) {
return ControllerGrantKey(module, fingerprint)
}
// ParsePartGrantKey is the inverse of PartGrantKey.
func ParsePartGrantKey(key string) (module, fingerprint string, err error) {
return ParseControllerGrantKey(key)
}
// GrantSource names the bucket the assigner reads grants from, and the `reads` name that grants it: the grants part's
// once the controller's lease is handed away (buscompose.LeaseHandedAway, which refuses every other value of the lease,
// an unreadable one included), the controller's in every other case.
func GrantSource(leaseHandedAway bool) (bucket, read string) {
if leaseHandedAway {
return PartGrantsBucket, PartGrantsRead
}
return ControllerGrantsBucket, "mesh-controller.grants"
}
go/delivery/placement_candidate.go
+6 -8
View File
@@ -127,10 +127,9 @@ type CandidateAnswer struct {
Held []ModuleRefused `json:"held,omitempty"`
// Refused is why the node could not compose with the candidate at all; Refusals is then empty.
Refused string `json:"refused,omitempty"`
// Returns is, for a proposal that takes the node's placement key away, what the node does then: it returns to the
// controller's compositions (ReturnsToController), as a node with no placement key composes none of its own. Same, Differs, Added and Removed then compare the controller's compositions at
// the node's wants with what it runs, and the verdict is theirs; a want the controller holds no composition for is
// refused (Refused, without Returns).
// Returns is, for a proposal that takes the node's placement key away, what the node does then (ReturnsToNothing): a
// node with no placement key composes nothing (ADR 0590), so Removed names every module it runs and the verdict is
// theirs.
Returns string `json:"returns,omitempty"`
// Note is what the node says of how it answered, in words, beside the verdict: a pass that ran over a minute, inputs
// it waited for (issue 1126: the shadow shows its work). It changes no verdict.
@@ -141,8 +140,8 @@ type CandidateAnswer struct {
ValuesDiffer int `json:"values_differ,omitempty"`
}
// ReturnsToController is what a node answers a proposal that takes its placement key away.
const ReturnsToController = "the proposal takes this node's placement away: it returns to the controller's compositions"
// ReturnsToNothing is what a node answers a proposal that takes its placement key away.
const ReturnsToNothing = "the proposal takes this node's placement away: it composes nothing then"
// ModuleDiffers is one module the node runs and the candidate still places whose composition is not the one it runs:
// both digests (Composed with the candidate, Running what runs), the first resource that differs by its id, or empty
@@ -227,8 +226,7 @@ func (a CandidateAnswer) check() error {
return fmt.Errorf("delivery: a candidate the node could not compose names modules")
}
if a.Returns != "" && (a.Refused != "" || len(a.Refusals) > 0) {
return fmt.Errorf("delivery: a candidate the node returns to the controller's compositions on is compared, never refused: " +
"a composition the controller holds none of is refused without returning")
return fmt.Errorf("delivery: a candidate that takes the node's placement away is compared, never refused")
}
for _, d := range a.Differs {
if d.Module == "" || d.Why == "" {
go/delivery/placement_test.go
+10 -6
View File
@@ -388,8 +388,8 @@ func TestPlacement(t *testing.T) {
"a value": {Proposal: id, Revision: 1, Verdict: CandidateRefused, Same: []string{"node"}, Refusals: []ModuleRefused{{Module: "db", Why: "stands in for db/password"}}},
// A head that deletes the node's placement file: the node returns to the controller's compositions (issue 1126,
// the seed's revert, mesh-catalog #526).
"returns": {Proposal: id, Revision: 1, Verdict: CandidateSame, Returns: ReturnsToController, Same: []string{"app", "node"}},
"returns, the controller's differs": {Proposal: id, Revision: 1, Verdict: CandidateDiffers, Returns: ReturnsToController, Same: []string{"node"},
"returns": {Proposal: id, Revision: 1, Verdict: CandidateSame, Returns: ReturnsToNothing, Same: []string{"app", "node"}},
"returns, differs": {Proposal: id, Revision: 1, Verdict: CandidateDiffers, Returns: ReturnsToNothing, Same: []string{"node"},
Differs: []ModuleDiffers{{Module: "app", Resource: "file:/etc/app", Why: "the resource differs"}}, Removed: []string{"old"}},
}
n := HaveNode{Composed: "sha256:" + strings.Repeat("a", 64), Sealed: []string{"box"}, Candidates: answers,
@@ -407,11 +407,15 @@ func TestPlacement(t *testing.T) {
t.Error("a node's have without its answers counts as the same")
}
if SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Candidates: answers,
Composes: ComposesController, ComposesWhy: n.ComposesWhy}) {
ComposesWhy: n.ComposesWhy}) {
t.Error("a node's have composing otherwise counts as the same")
}
if _, err := EncodeHaveNode(HaveNode{Composes: "both"}); err == nil {
t.Error("a node composing neither its own nor the controller's was taken")
t.Error("a node composing neither its own nor none was taken")
}
// Issue 1126 B4a: a have an engine from before the controller ended wrote still reads.
if _, err := DecodeHaveNode([]byte(`{"v":1,"composes":"controller"}`)); err != nil {
t.Errorf("an older engine's have is refused: %v", err)
}
// CompareComposed: the bus's user list is left out on both sides; anything else that differs is named.
file := func(id, content string) json.RawMessage {
@@ -605,8 +609,8 @@ func TestPlacement(t *testing.T) {
"same with an added": {Proposal: "p-x1", Revision: 1, Verdict: CandidateSame, Added: []ModuleAdded{{Module: "a"}}},
"differs refusing": {Proposal: "p-x1", Revision: 1, Verdict: CandidateDiffers, Removed: []string{"a"}, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"refused and modules": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Refused: "x", Same: []string{"a"}},
"returns refusing": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToController, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"returns refused": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToController, Refused: "x"},
"returns refusing": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToNothing, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"returns refused": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToNothing, Refused: "x"},
} {
if _, err := EncodeHaveNode(HaveNode{Candidates: map[string]CandidateAnswer{"g": a}}); err == nil {
t.Errorf("%s: taken", name)
go/delivery/signers.go
+7 -9
View File
@@ -24,9 +24,8 @@ import (
// serving key from its record), and says `signed` naming the caller, the kind, the node and the digest, never the key.
//
// A node trusts the vault's key through the bus grants alone (the lead's design limit, 2026-10-28): only the vault may
// write `signers`, the same trust the wants and the placement rest on. Until the controller's lease is handed away the
// node trusts the controller's key beside it; from then on the vault's alone (LeaseHandedAway). What a node trusts now
// it states in its record, NodeRecord.Trusts, so the gate before the lease write reads it from the bus.
// write `signers`, the same trust the wants and the placement rest on. A node trusts the vault's key alone since the
// controller ended (ADR 0590). What a node trusts now it states in its record, NodeRecord.Trusts.
const (
// VaultModule is the module holding the seat mesh-vault, whose state `signers` is.
@@ -54,12 +53,11 @@ const (
// KindAuthority is the vault's own authority certificate, which the vault signs for itself.
KindAuthority = "authority"
// SignerController and SignerVault name the two signers a node may trust.
SignerController = "controller"
SignerVault = "vault"
// SignerVault names the signer a node trusts.
SignerVault = "vault"
// CertificateLife is how long a certificate the vault signs for a node lasts: the controller's ten years
// (identity.forever), renewed by asking again, never by a clock running out unseen.
// CertificateLife is how long a certificate the vault signs for a node lasts: ten years, renewed by asking again,
// never by a clock running out unseen.
CertificateLife = 10 * 365 * 24 * time.Hour
)
@@ -166,7 +164,7 @@ func DecodeSignerAuthority(raw []byte) (SignerAuthority, error) {
return a, nil
}
// Signer is one key a node trusts, by who holds it (SignerController, SignerVault, or the operator's).
// Signer is one key a node trusts, by who holds it (SignerVault, or the operator's).
type Signer struct {
Name string
Key ed25519.PublicKey
go/delivery/userlist.go
+10 -17
View File
@@ -21,11 +21,10 @@ import (
// list's digest (UserListObjectName).
// - It writes the key GrantsUserListKey of its state `user-list` (bucket GrantsUserListBucket): the GrantsUserList
// below, which names the object, the list's digest, its revision, its user count and the users left out, and which
// source the bus node follows (Follow: FollowGrants from take-over, FollowController in shadow and after hand-back).
// - The node-engine of the bus node watches that key alone. While it says FollowGrants it opens the object, checks
// the digest, writes the bus's user-list file (mode 0600) only when the file's digest differs, and says what it
// loaded in its have of the bus module (UserListWritten, Source GrantsUserListSource); otherwise it follows the
// controller's compositions as before (clause 8). Never both.
// source the bus node follows (Follow: FollowGrants, the one source since the controller ended, ADR 0590).
// - The node-engine of the bus node watches that key alone. It opens the object, checks the digest, writes the bus's
// user-list file (mode 0600) only when the file's digest differs, and says what it loaded in its have of the bus
// module (UserListWritten, Source GrantsUserListSource).
// The names of the grants part's user list on the bus.
const (
@@ -41,16 +40,13 @@ const (
GrantsUserListSource = GrantsUserListBucket + "." + GrantsUserListKey
)
// Which list the bus node follows.
const (
FollowController = "controller"
FollowGrants = "grants"
)
// FollowGrants is the list the bus node follows: the grants part's, the one source (ADR 0590).
const FollowGrants = "grants"
// GrantsUserList is the value of the grants part's key `user-list.sealed`.
type GrantsUserList struct {
V int `json:"v"`
// Follow is which list the bus node follows: FollowGrants from take-over, FollowController otherwise.
// Follow is which list the bus node follows: FollowGrants, the one value it takes.
Follow string `json:"follow"`
// Digest is the list's digest, `sha256:<hex>` of the file's bytes as the bus loads them.
Digest string `json:"digest"`
@@ -65,10 +61,6 @@ type GrantsUserList struct {
// LeftOut are the users the list leaves out for want of a hash in the vault.
LeftOut []string `json:"left_out"`
At time.Time `json:"at"`
// LeaseHanded says the controller's lease is handed to the grants part (novox/hq ADR 0582 clause 3, ADR 0571 clause
// 8): from then on the bus node follows this list alone and never falls back to the controller's. A key written
// before it reads false.
LeaseHanded bool `json:"lease-handed,omitempty"`
}
// EncodeGrantsUserList is g as JSON at Version; it refuses a value the bus node could not follow.
@@ -96,8 +88,9 @@ func DecodeGrantsUserList(data []byte) (GrantsUserList, error) {
}
func (g GrantsUserList) check() error {
if g.Follow != FollowGrants && g.Follow != FollowController {
return fmt.Errorf("delivery: the user list's follow %q is not %s or %s", g.Follow, FollowGrants, FollowController)
if g.Follow != FollowGrants {
return fmt.Errorf("delivery: the user list's follow %q is not %s, the one list since the controller ended (ADR 0590)",
g.Follow, FollowGrants)
}
if g.Digest == "" {
return nil // a switch with no list written yet: the bus node keeps what it follows