diff --git a/modules/mesh-vault/README.md b/modules/mesh-vault/README.md new file mode 100644 index 00000000..b34b94c0 --- /dev/null +++ b/modules/mesh-vault/README.md @@ -0,0 +1,164 @@ +# mesh-vault + +Holds the `mesh-vault` seat: the mesh's shared secrets, each kept sealed, handed out only on request and +reply, sealed to whoever asked (novox/hq ADR 0085, ADR 0113, ADR 0349 §7 and §11). Go, one binary +(`cmd/mesh-vault`), launched by its node's runtime and speaking MCP to it through the Go SDK. + +## Where the secrets are kept, and why there + +In the vault's own store: one file per secret in `${dir:store}` on the vault's own disk, mode 0600 in a +0700 directory. Not a database, as most Go modules keep their data, because: + +- the vault is installed before any other module built on the runtime base (ADR 0113 §14), and a database + login is itself a shared secret the vault makes — the vault cannot need one to start; +- every value in the store is ciphertext anyway, so a database would add nothing a file does not; +- no secret is in a bucket, sealed or not (ADR 0201 §13), so the module declares no `state`. + +Each record holds the value twice, never in the clear: + +- **at rest**, sealed to the vault's own key (the sealed box of ADR 0206, `mesh-sdk/go/seal`). The key is + an X25519 pair the vault makes once in `${dir:state}/vault-key.json`; its private half never leaves it. + The key is made only for an empty store: with the key file missing and secrets in the store, the vault + does not start and says so, rather than make a key that opens none of them. +- **for the operator**, sealed to the operator key (ADR 0113 §22), so nothing exists only inside the + vault. A secret is refused, loudly, while the vault has no operator key. + +Every file is written whole, synced, renamed into place and its directory synced. + +### What the encryption at rest protects against, and what not + +The vault's private key lives in plaintext in `${dir:state}`, on the same disk as the store, readable by the +vault's account alone. So: + +- **It protects** against a copy of the store directory alone — a `store` backup, a file sent somewhere, + a listing of `${dir:store}` by a tool that reads files — holding anything usable, and against a value + being found by searching the disk for it. +- **It does not protect** against anyone who can read the vault's state directory too: root on the + vault's node, the vault's own account, or a backup that holds both `state` and `store` (both are + class `valuable`, so the node's backup holds both). Against those, at-rest encryption adds nothing; the + protection is the node's own security. Keeping the key elsewhere (another directory class, a key the + operator unlocks) is for V3's review, before real secrets move in. + +## Where the operator key comes from + +From the export the mesh writes to the vault's root directory (`${dir:root}/export.json`, the manifest's +`keeps`): its `operator-key` is the public half `operator key set` recorded in the controller. The vault +checks it every minute; a new key makes the vault seal every operator copy again to it — it can, because +it opens its own copy. Each record names the key its copy is sealed to, so a failure partway is finished on +the next minute's pass, and nothing is left sealed to an earlier key for good. + +The vault takes an operator key only as SPKI PEM (`SetOperatorKey`). A PEM private key is refused, and so +is a raw 32-byte key, because a raw private key looks the same. The one raw key it reads is the export's, +which the controller writes from what it recorded; the vault wraps it as SPKI before taking it. It keeps +the key raw in base64, the form of the operator's NaCl box. When the secrets move to the vault (design 57's +V3), the operator key moves with them. + +### How the operator opens a copy + +`mesh-vault.secret_vault_export` answers the vault's copies in the shape `secret export` writes. Saved to a +file, one is opened with the operator's private key by + + nox secret recover vault mesh-vault --key --from-export --out + +(`--out` is needed: a name holds `/`). `TestTheOperatorRecoversAVaultSecretFromItsExport` reads the file +as that command does and opens it as it does. `secret_export` still answers the controller's export, as +before; until V3 the controller's secrets are there. + +## The boxes and the key formats — settled + +Two boxes on the mesh today, both over X25519: + +| | format | keys | who uses it | +|---|---|---|---| +| NaCl anonymous box | XSalsa20-Poly1305, `box.SealAnonymous` | raw 32 bytes, base64 | the controller sealing into declarations; the node-engine's `sealing.key` (mesh-host `internal/identity/sealing.go`); the operator key and `secret recover --from-export` | +| the sealed box (ADR 0206) | X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}` | PEM: SPKI public, PKCS #8 private | the licence manager and `claude-code`; `mesh-sdk/go/seal` | + +What the vault uses: + +- **Its answers to `get` are the ADR 0206 box, for every asker, a node included.** The asker sends its + public key as SPKI PEM. A node's provisioner client (V4) needs no second key: its node's sealing key is + X25519 already, so it sends that key's public half wrapped as SPKI and opens the answer with the private + half wrapped as PKCS #8 (`TestANodesSealingKeyOpensTheAnswer` does exactly this). A raw 32-byte key is + refused: a raw private key looks the same, and the vault takes no private key. +- **Its operator copies are the NaCl anonymous box**, to the operator key as it is made today, so the + operator's recovery opens them with the key and the command it already has. +- **The at-rest copy is the ADR 0206 box**, to the vault's own key. + +The NaCl box stays on the node-engine only for what the controller seals into declarations, until the last +module has moved (ADR 0349 §17). For the core's own credentials that travel in a declaration sealed by the +vault (ADR 0349 §16), V4 adds opening the ADR 0206 box with the same sealing key to the node-engine. + +## The seat's verb + +`mesh-vault.get` — `name`, `public_key` (SPKI PEM). It answers `{name, generation, changed-at, sealed}`, +`sealed` being the secret sealed to that key, and only to an asker allowed for that secret, the asker being +the bus user the runtime proved — never an argument. + +Every answer and every refusal is recorded in the vault's own store (`handed.jsonl`: who, which secret, +which generation, when, and why a refusal), synced before the answer is given — as ADR 0277 §25 records +asks; a secret whose hand-out cannot be recorded is not handed out. A refusal is recorded only for a caller +of a bus user's shape and a well-formed secret name, so an arbitrary argument fills nothing. The record is +bounded: rolled at 1 MiB into `handed.jsonl.1` … `.3`, the oldest dropped, each roll said loudly in the +journal and counted in `vault_status`. `secret_handed` reads it. + +`vault_status` says what needs a person: operator copies still sealed to an earlier key, an operator key in +the export the vault refused, and how often the record was rolled. **It is not yet a condition**: a health +check is declared on a long-running resource (ADR 0240), and the vault is a bundle the tool runner loads, so +nothing asks it on its own; until it runs as a resource of its own (V3's review) these reach a person +through the journal and this tool only. + +**Who may ask, today: nobody.** The rule (ADR 0349 §11) is a node's provisioner client under a bus account +of its own, for what its node's declaration names, and a module under its own account (issue 569). Neither +exists yet, so the running vault refuses every asker; the tests set a stand-in list. The real rule goes +behind the `Askers` interface in `askers.go` (design 57's V4 and P3). + +## Events + +- `secret-changed` — `name`, `by`, `at`, `generation`: which secret, who changed it, when. Never a value + or a sealed copy. +- `provisioned`, `rotated` — the custody ledger's, by fingerprint, as before. +- `provisioner.retirement` (ADR 0230 §35) — `waiting`, `settled`, `approved`, `rejected`, `retired`, + `reenabled`, `deleted`, naming holders, never a value. + +## The tools + +`secret_export`, `secret_holders`, `secret_verify` answer as the TypeScript vault's did, and its custody +ledger (`${dir:ledger}`, the same JSON files) is kept by a pass over the contributions file every minute. +`secret_handed` and `secret_vault_export` read the vault's own store. None returns a value. + +**The ledger follows ADR 0230.** A contributions file that does not say who is asked for (unreadable, not +JSON, no `given` list or a null one, or for another requirement) is not "nobody": the pass records nothing, +retires nobody and starts its count again. A holder no longer asked for is retired — a mark with when and +why in its own ledger file, its history kept — only after the same set has gone unasked in five passes and +ten minutes; more than three at once, or more than half of those held, waits for a person +(`provisioner_retire_approve`, `provisioner_retire_reject`); asked for again it is re-enabled; it is deleted +only by `provisioner_delete`, a person's act with a why. `provisioner_retirement` shows it all. The vault is +mark-only: it holds no access to switch off. + +The words are the controller's: the `provisioner.retirement` event is mesh-controller's `link.Retirement` +(`provider-node`, `consumers` as objects) and the `provisioner_retirement` answer its `link.RetirementState` +(`held`, `bound`, `retires_by: mark-only`), so `retire approve`, `retire reject` and `cleanup list` read the +vault and a set waiting for a person raises the controller's urgent condition. The acts take what the SDK's +harness takes: approve and reject repeat the set in `consumers` and refuse any other set; delete repeats the +name in `confirm`; each records the `by` and `via` the controller sends. `testdata/retirement-*.json` are +those types marshalled by the controller (feat/57 at dc36536), and the tests hold the vault's words to them. + +What the TypeScript vault did and this one does not: it consumed its own events only to print an audit line, +which the pass now prints itself. Its retirement ran on the SDK's harness, which this one follows: mark-only, +and a holder's ledger file removed only through a person's delete, as here. The difference is where the mark +lives: the harness kept it in its process, so a restart forgot it (ADR 0230 §13); here it is in the ledger +file. + +## What comes later + +Taking values in (`secret accept`, `secret-ask`, `issued`, `rotate`) is V2; the controller's secrets move +in with V3; nodes ask with V4; pair credentials are minted here with V5. Until V3 the vault's store is +empty and the controller still serves every secret. + +## Tests + + go test ./... + +`vault_test.go` is ADR 0349's vault test list as far as this package goes: no bucket holds a secret; no +private key is accepted; every secret has an operator-sealed copy; `get` answers only an allowed asker, +sealed to its key; `secret-changed` carries no value. diff --git a/modules/mesh-vault/client.ts b/modules/mesh-vault/client.ts deleted file mode 100644 index c322db20..00000000 --- a/modules/mesh-vault/client.ts +++ /dev/null @@ -1,172 +0,0 @@ -// mesh-vault's ledger — vault's own code, living in the module (novox/hq ADR 0039). The provisioner and -// the tools both import it, and nothing outside vault does. -// -// **The vault holds no value.** A `secret` is an ordinary pair credential: the controller mints it, -// seals it to the consumer's node and to this one, and the host unseals this node's copy into the -// file the contribution names (ADR 0048). That file is already on this machine, readable by nothing -// but the vault's runtime, and it is the only copy the vault ever sees. Writing a second copy — -// plain, or sealed to a key the vault keeps — would put back exactly the single place that can open -// everything, which is what sealing to the machine was built to remove (ADR 0085's open question is -// how to recover WITHOUT that; the answer is not "keep one anyway"). -// -// So what the vault keeps is what makes a secret *owned* rather than merely delivered: who holds -// one, since when, its fingerprint, and every time it changed. Enough to say "this holder's value is -// the one the mesh last delivered" and "it has been rotated twice, last on Tuesday" — and never -// enough to say what it is. The fingerprint is the only thing a tool may take or return, which is -// the rule the source mesh's secret tools were built on: the secret is never an argument. - -import { createHash } from "node:crypto"; -import { mkdirSync, readdirSync, readFileSync, renameSync, unlinkSync, writeFileSync } from "node:fs"; -import { join } from "node:path"; - -/** One holder of a secret this vault provides — everything the vault knows, and no value. */ -export interface Held { - /** The login the mesh derived for the consumer — `-`, so it names the holder. */ - readonly as: string; - /** The consumer's node. */ - readonly consumer: string; - /** sha256 of the value the mesh last delivered, `sha256:`. Compared, never inverted. */ - readonly fingerprint: string; - /** Length of the value, so a holder can tell a truncated file from a wrong one. */ - readonly length: number; - /** When this holder was first granted a secret. */ - readonly since: string; - /** When the value last changed — equal to `since` until the first rotation. */ - readonly changed: string; - /** How many times the value has changed since `since`. */ - readonly rotations: number; - /** Every earlier fingerprint, oldest first: the audit trail a rotation leaves. */ - readonly history: readonly { readonly fingerprint: string; readonly until: string }[]; -} - -/** What recording a delivery found: a new holder, a changed value, or nothing new. */ -export type Outcome = "granted" | "rotated" | "unchanged"; - -/** sha256 of a value, as `sha256:`. The one thing about a secret that may be spoken. */ -export function fingerprint(value: string): string { - return "sha256:" + createHash("sha256").update(value, "utf8").digest("hex"); -} - -export class Ledger { - private readonly dir: string; - - constructor(dir: string) { - this.dir = dir; - mkdirSync(dir, { recursive: true, mode: 0o700 }); - } - - /** Build from the module's resolved environment: $MESH_VAULT_LEDGER is where holders are kept. */ - static fromEnv(env: NodeJS.ProcessEnv = process.env): Ledger { - const dir = env.MESH_VAULT_LEDGER; - if (!dir) { - throw new Error("MESH_VAULT_LEDGER is not set — the vault has nowhere to keep its ledger"); - } - return new Ledger(dir); - } - - /** - * Record that the mesh delivered `value` for `as`. Idempotent: the same value again changes - * nothing, a different value is a rotation and is remembered as one. The value is fingerprinted - * here and goes no further. - */ - record(as: string, consumer: string, value: string, now = new Date()): { held: Held; outcome: Outcome } { - const fp = fingerprint(value); - const at = now.toISOString(); - const before = this.get(as); - if (!before) { - const held: Held = { - as, consumer, fingerprint: fp, length: value.length, - since: at, changed: at, rotations: 0, history: [], - }; - this.write(held); - return { held, outcome: "granted" }; - } - if (before.fingerprint === fp && before.length === value.length) { - return { held: before, outcome: "unchanged" }; - } - const held: Held = { - ...before, consumer, fingerprint: fp, length: value.length, changed: at, - rotations: before.rotations + 1, - history: [...before.history, { fingerprint: before.fingerprint, until: at }], - }; - this.write(held); - return { held, outcome: "rotated" }; - } - - /** Forget a holder the mesh withdrew. Returns whether there was one to forget. */ - withdraw(as: string): boolean { - try { - unlinkSync(this.pathOf(as)); - return true; - } catch { - return false; - } - } - - get(as: string): Held | undefined { - try { - return JSON.parse(readFileSync(this.pathOf(as), "utf8")) as Held; - } catch { - return undefined; - } - } - - /** Every holder, by login. */ - list(): Held[] { - let names: string[]; - try { - names = readdirSync(this.dir); - } catch { - return []; - } - return names - .filter((n) => n.endsWith(".json")) - .map((n) => this.get(n.slice(0, -".json".length))) - .filter((h): h is Held => h !== undefined) - .sort((a, b) => a.as.localeCompare(b.as)); - } - - private pathOf(as: string): string { - if (!/^[a-z0-9][a-z0-9_.-]*$/.test(as)) { - throw new Error(`a login is a name, not a path: ${JSON.stringify(as)}`); - } - return join(this.dir, `${as}.json`); - } - - /** Written whole and renamed into place, so a reader never sees half a record. */ - private write(held: Held): void { - const final = this.pathOf(held.as); - const tmp = `${final}.${process.pid}.tmp`; - writeFileSync(tmp, JSON.stringify(held, null, 2) + "\n", { mode: 0o600 }); - renameSync(tmp, final); - } -} - -/** One entry of the mesh's contributions file, as the vault reads it for its tools. */ -export interface Contribution { - readonly from?: string; - readonly node?: string; - readonly as: string; - readonly secret: string; -} - -/** The consumers the mesh currently asks this vault to serve — the `receives` file, read plainly. */ -export function contributions(receives: string): Contribution[] { - let doc: { given?: Contribution[] }; - try { - doc = JSON.parse(readFileSync(receives, "utf8")) as { given?: Contribution[] }; - } catch { - return []; - } - return (doc.given ?? []).filter((g) => g.as && g.secret); -} - -/** Fingerprint of the value the host currently holds for one contribution, or why it could not. */ -export function deliveredFingerprint(c: Contribution): { fingerprint: string; length: number } | { error: string } { - try { - const value = readFileSync(c.secret, "utf8").replace(/\n$/, ""); - return { fingerprint: fingerprint(value), length: value.length }; - } catch (err) { - return { error: `the delivered secret is not readable: ${err}` }; - } -} diff --git a/modules/mesh-vault/cmd/mesh-vault/askers.go b/modules/mesh-vault/cmd/mesh-vault/askers.go new file mode 100644 index 00000000..44c367c7 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/askers.go @@ -0,0 +1,37 @@ +package main + +// Who may ask `get` for which secret: the seam the real rule fills (novox/hq ADR 0349 §11). +// +// The rule is: a node's provisioner client, running as a service under a bus account of its own, for a +// secret that node's declaration names; or a module, for a secret its manifest declares, only once its +// code runs under a bus account of its own (issue 569). Neither exists yet — the client and its account are +// design 57's P3, the vault's answer to a node V4 — so the vault that runs allows nobody (NoAsker), and the +// tests set a stand-in list (AllowList). V4 and P3 put the real rule behind this interface. + +// Askers decides whether caller (the bus user the runtime proved) may have the secret name, and if not, +// why not. +type Askers interface { + May(caller, name string) (bool, string) +} + +// NoAsker allows nobody: the vault's rule until who may ask is decided by a declaration and an account. +type NoAsker struct{} + +// May refuses every caller. +func (NoAsker) May(string, string) (bool, string) { + return false, "the vault hands a secret to no asker yet: a node's provisioner client under its own account, " + + "for what its declaration names, is the first it will answer (novox/hq ADR 0349 §11, issue 647)" +} + +// AllowList is a stand-in rule: each caller, and the names it may have. +type AllowList map[string][]string + +// May allows a caller only the names listed for it. +func (l AllowList) May(caller, name string) (bool, string) { + for _, n := range l[caller] { + if n == name { + return true, "" + } + } + return false, "not on the list of who may ask for it" +} diff --git a/modules/mesh-vault/cmd/mesh-vault/keys.go b/modules/mesh-vault/cmd/mesh-vault/keys.go new file mode 100644 index 00000000..033d324d --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/keys.go @@ -0,0 +1,168 @@ +package main + +// The keys the vault meets, and the one it holds (novox/hq ADR 0113 §2, ADR 0206). +// +// - Its own key: an X25519 pair made once in its state directory, the private half never leaving it. Every +// secret is kept sealed to it at rest. +// - An asker's key: the public half only, as SPKI PEM — the form of the ADR 0206 box (mesh-sdk go/seal). +// A raw 32-byte key is refused, because a raw private key looks the same and the vault takes none. +// - The operator key: the public half, taken as SPKI PEM only and kept raw in base64, the form `operator key +// make` writes and the NaCl box the operator opens uses. +// +// No private key is ever accepted: not as a key to seal to, not as a secret to hold. + +import ( + "crypto/ecdh" + "crypto/rand" + "crypto/x509" + "encoding/base64" + "encoding/json" + "encoding/pem" + "errors" + "fmt" + "os" + "path/filepath" + "regexp" + "strings" + + "git.novox.be/novox/mesh-sdk/go/seal" + "golang.org/x/crypto/nacl/box" +) + +// OwnKeyFile is where the vault's own key lives in its state directory. +const OwnKeyFile = "vault-key.json" + +// OwnKey is the vault's own key pair, made once in dir. Written whole and synced, then linked into place and +// the directory synced, so a second process reads the first's key and never half of it. +// +// **A key is made only for an empty store.** When the key file is missing and the store holds secrets, the +// key they are sealed to is lost or misplaced, and a new key would open none of them: the vault refuses to +// start, loudly, rather than seal new secrets to a key beside old ones nothing opens. The operator copies +// are what re-raise it (ADR 0113 §21). +func OwnKey(dir string, store *Store) (seal.KeyPair, error) { + if dir == "" { + return seal.KeyPair{}, errors.New("the vault has no state directory: MESH_VAULT_STATE is not set") + } + if err := os.MkdirAll(dir, 0o700); err != nil { + return seal.KeyPair{}, err + } + file := filepath.Join(dir, OwnKeyFile) + if _, err := os.Stat(file); errors.Is(err, os.ErrNotExist) { + if store != nil { + empty, err := store.Empty() + if err != nil { + return seal.KeyPair{}, fmt.Errorf("the vault's key is missing and its store cannot be read: %w", err) + } + if !empty { + return seal.KeyPair{}, fmt.Errorf("THE VAULT'S KEY IS MISSING (%s) WHILE ITS STORE HOLDS SECRETS SEALED TO IT: "+ + "the vault does not start and makes no new key, which would open none of them — put the key back, "+ + "or re-raise the vault from the operator copies (novox/hq ADR 0113 §21)", file) + } + } + k, err := seal.GenerateKeyPair() + if err != nil { + return seal.KeyPair{}, err + } + raw, _ := json.Marshal(k) + tmp := filepath.Join(dir, fmt.Sprintf(".vault-key.%d.json", os.Getpid())) + if err := writeSynced(tmp, raw); err != nil { + return seal.KeyPair{}, err + } + _ = os.Link(tmp, file) // fails when another made it first, which is right + _ = os.Remove(tmp) + if err := syncDir(dir); err != nil { + return seal.KeyPair{}, err + } + } + raw, err := os.ReadFile(file) + if err != nil { + return seal.KeyPair{}, err + } + var k seal.KeyPair + if err := json.Unmarshal(raw, &k); err != nil || k.PublicKey == "" || k.PrivateKey == "" { + return seal.KeyPair{}, fmt.Errorf("%s is not the vault's key", file) + } + return k, nil +} + +var privateKeyPEM = regexp.MustCompile(`-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----`) + +// privateKeyIn says whether text carries a private key. +func privateKeyIn(text string) bool { + return privateKeyPEM.MatchString(text) || strings.Contains(text, "PRIVATE KEY") +} + +// askerKey is an asker's public key, checked and as PEM: SPKI, X25519, and never a private key. +func askerKey(key string) (string, error) { + if privateKeyIn(key) { + return "", errors.New("that is a private key, and a private key is never sent to the vault (ADR 0113 §2): send the public half") + } + block, rest := pem.Decode([]byte(strings.TrimSpace(key))) + if block == nil || block.Type != "PUBLIC KEY" || strings.TrimSpace(string(rest)) != "" { + return "", errors.New("the key to seal to is one X25519 public key as SPKI PEM (`-----BEGIN PUBLIC KEY-----`), the form of the sealed box (ADR 0206)") + } + k, err := x509.ParsePKIXPublicKey(block.Bytes) + if err != nil { + return "", fmt.Errorf("the key to seal to is not a public key: %w", err) + } + pub, ok := k.(*ecdh.PublicKey) + if !ok || pub.Curve() != ecdh.X25519() { + return "", errors.New("the key to seal to is not an X25519 key") + } + return string(pem.EncodeToMemory(block)), nil +} + +// checkOperatorKey takes the operator's public key as SPKI PEM — never a private key, and never a raw key, +// which cannot be told from a private one — and gives it in the raw base64 form the operator's NaCl box and +// `secret recover` use. +func checkOperatorKey(key string) (string, error) { + if privateKeyIn(key) { + return "", errors.New("that is a private key: the operator key's private half never enters the mesh (ADR 0113 §2)") + } + block, rest := pem.Decode([]byte(strings.TrimSpace(key))) + if block == nil || block.Type != "PUBLIC KEY" || strings.TrimSpace(string(rest)) != "" { + return "", errors.New("the operator key is one X25519 public key as SPKI PEM (`-----BEGIN PUBLIC KEY-----`); " + + "a raw key is refused, because a raw private key looks the same") + } + k, err := x509.ParsePKIXPublicKey(block.Bytes) + if err != nil { + return "", fmt.Errorf("the operator key is not a public key: %w", err) + } + pub, ok := k.(*ecdh.PublicKey) + if !ok || pub.Curve() != ecdh.X25519() { + return "", errors.New("the operator key is not an X25519 key") + } + return base64.StdEncoding.EncodeToString(pub.Bytes()), nil +} + +// publicPEM wraps a raw X25519 public key, base64, as SPKI PEM. +func publicPEM(raw string) (string, error) { + b, err := base64.StdEncoding.DecodeString(strings.TrimSpace(raw)) + if err != nil || len(b) != 32 { + return "", errors.New("not a raw X25519 key in base64") + } + pub, err := ecdh.X25519().NewPublicKey(b) + if err != nil { + return "", err + } + der, err := x509.MarshalPKIXPublicKey(pub) + if err != nil { + return "", err + } + return string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: der})), nil +} + +// operatorSeal seals value to the operator key in the box the operator opens today. +func operatorSeal(key string, value []byte) (string, error) { + raw, err := base64.StdEncoding.DecodeString(key) + if err != nil || len(raw) != 32 { + return "", errors.New("the operator key is not usable") + } + var pub [32]byte + copy(pub[:], raw) + sealed, err := box.SealAnonymous(nil, value, &pub, rand.Reader) + if err != nil { + return "", err + } + return base64.StdEncoding.EncodeToString(sealed), nil +} diff --git a/modules/mesh-vault/cmd/mesh-vault/ledger.go b/modules/mesh-vault/cmd/mesh-vault/ledger.go new file mode 100644 index 00000000..17465276 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/ledger.go @@ -0,0 +1,251 @@ +package main + +// The custody ledger (novox/hq ADR 0085, design 24), carried over from the TypeScript vault: for each pair +// credential the controller delivers to this vault as the `secret` provider, who holds it, since when, its +// fingerprint and every rotation — and never the value. The controller still mints and seals those until +// the secrets move to the vault (design 57's V3, V4); the pass records what it delivered, so +// `secret_holders` and `secret_verify` answer as before. Its files are the same JSON, read as they are. +// +// **A holder the mesh stops asking for is retired, never deleted by a pass** (novox/hq ADR 0230). The vault +// is a mark-only provider: it has no access to switch off, so retiring writes the mark — when and why — into +// the holder's own ledger file, which keeps its rotation history. A holder is retired only once the same set +// has gone unasked in five consecutive passes that read the contributions file and for ten minutes; a set +// over the bound (more than three, or more than half of those held) waits for a person; asked for again, a +// retired holder is re-enabled at once; a holder is deleted only by `provisioner_delete`, a person's act +// with a why. Because the mark is in the ledger file, a restart forgets nothing. + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "regexp" + "sort" + "strings" + "time" +) + +// isoMillis is JavaScript's toISOString, the form the ledger's times were first written in. +const isoMillis = "2006-01-02T15:04:05.000Z" + +// Held is one holder of a secret this vault provides: everything the vault knows of it, and no value. +type Held struct { + As string `json:"as"` + Consumer string `json:"consumer"` + Fingerprint string `json:"fingerprint"` + Length int `json:"length"` + Since string `json:"since"` + Changed string `json:"changed"` + Rotations int `json:"rotations"` + History []Earlier `json:"history"` + // Retired is the mark a retirement writes; nil while the holder is active. + Retired *RetiredMark `json:"retired,omitempty"` +} + +// RetiredMark is when and why a holder was retired. +type RetiredMark struct { + At string `json:"at"` + Why string `json:"why"` +} + +// Earlier is a fingerprint a rotation replaced, and when. +type Earlier struct { + Fingerprint string `json:"fingerprint"` + Until string `json:"until"` +} + +// Outcome is what recording a delivery found. +type Outcome string + +const ( + Granted Outcome = "granted" + Rotated Outcome = "rotated" + Unchanged Outcome = "unchanged" +) + +// Fingerprint is sha256 of a value, as `sha256:`: the one thing about a secret that may be said. +func Fingerprint(value string) string { + sum := sha256.Sum256([]byte(value)) + return "sha256:" + hex.EncodeToString(sum[:]) +} + +// Ledger is the directory of holders. +type Ledger struct{ dir string } + +// NewLedger opens (and makes) the ledger at dir. +func NewLedger(dir string) (*Ledger, error) { + if dir == "" { + return nil, errors.New("MESH_VAULT_LEDGER is not set: the vault has nowhere to keep its ledger") + } + if err := os.MkdirAll(dir, 0o700); err != nil { + return nil, err + } + return &Ledger{dir: dir}, nil +} + +var login = regexp.MustCompile(`^[a-z0-9][a-z0-9_.-]*$`) + +func (l *Ledger) pathOf(as string) (string, error) { + if !login.MatchString(as) || strings.Contains(as, "..") { + return "", fmt.Errorf("a login is a name, not a path: %q", as) + } + return filepath.Join(l.dir, as+".json"), nil +} + +// Record says the mesh delivered value for as: the same value again changes nothing, another is a rotation. +// The value is fingerprinted here and goes no further. A retirement mark is left as it is. +func (l *Ledger) Record(as, consumer, value string, now time.Time) (Held, Outcome, error) { + path, err := l.pathOf(as) + if err != nil { + return Held{}, "", err + } + fp, at := Fingerprint(value), now.UTC().Format(isoMillis) + before, ok := l.Get(as) + if !ok { + h := Held{As: as, Consumer: consumer, Fingerprint: fp, Length: len(value), Since: at, Changed: at, History: []Earlier{}} + return h, Granted, l.write(path, h) + } + if before.Fingerprint == fp && before.Length == len(value) { + return before, Unchanged, nil + } + h := before + h.Consumer, h.Fingerprint, h.Length, h.Changed = consumer, fp, len(value), at + h.Rotations++ + h.History = append(append([]Earlier{}, before.History...), Earlier{Fingerprint: before.Fingerprint, Until: at}) + return h, Rotated, l.write(path, h) +} + +// Mark writes or clears a holder's retirement mark. +func (l *Ledger) Mark(as string, mark *RetiredMark) error { + path, err := l.pathOf(as) + if err != nil { + return err + } + h, ok := l.Get(as) + if !ok { + return fmt.Errorf("the ledger holds nothing for %s", as) + } + h.Retired = mark + return l.write(path, h) +} + +// Delete forgets a holder, saying whether there was one. Only a person's `provisioner_delete` reaches it. +func (l *Ledger) Delete(as string) bool { + path, err := l.pathOf(as) + if err != nil { + return false + } + if os.Remove(path) != nil { + return false + } + _ = syncDir(l.dir) + return true +} + +// Get is one holder. +func (l *Ledger) Get(as string) (Held, bool) { + path, err := l.pathOf(as) + if err != nil { + return Held{}, false + } + raw, err := os.ReadFile(path) + if err != nil { + return Held{}, false + } + var h Held + if json.Unmarshal(raw, &h) != nil { + return Held{}, false + } + return h, true +} + +// List is every holder, by login. +func (l *Ledger) List() []Held { + entries, err := os.ReadDir(l.dir) + if err != nil { + return nil + } + var out []Held + for _, e := range entries { + if strings.HasSuffix(e.Name(), ".json") { + if h, ok := l.Get(strings.TrimSuffix(e.Name(), ".json")); ok { + out = append(out, h) + } + } + } + sort.Slice(out, func(i, j int) bool { return out[i].As < out[j].As }) + return out +} + +func (l *Ledger) write(path string, h Held) error { + raw, err := json.MarshalIndent(h, "", " ") + if err != nil { + return err + } + return writeWhole(path, append(raw, '\n')) +} + +// Contribution is one entry of the contributions file the mesh delivers to the vault. +type Contribution struct { + From string `json:"from,omitempty"` + Node string `json:"node,omitempty"` + As string `json:"as"` + Secret string `json:"secret"` +} + +// Contributions is what the mesh asks this vault to serve: the entries with a login and a secret file. +// +// **A file that does not say who is asked for is not "nobody".** One that is unreadable, not JSON, without a +// `given` list (missing or null), or written for another requirement than `secret` is refused, so a pass +// over it retires nobody. An empty `given` list is an answer: nobody is asked for. +func Contributions(receives string) ([]Contribution, error) { + raw, err := os.ReadFile(receives) + if err != nil { + return nil, err + } + var doc struct { + Requirement *string `json:"requirement"` + Given json.RawMessage `json:"given"` + } + if err := json.Unmarshal(raw, &doc); err != nil { + return nil, fmt.Errorf("the contributions file is not JSON: %w", err) + } + if doc.Requirement != nil && *doc.Requirement != "secret" { + return nil, fmt.Errorf("the contributions file is for %q, not for secret", *doc.Requirement) + } + if len(doc.Given) == 0 || string(doc.Given) == "null" { + return nil, errors.New("the contributions file has no `given` list, so it says nothing of who is asked for") + } + var given []Contribution + if err := json.Unmarshal(doc.Given, &given); err != nil { + return nil, fmt.Errorf("the contributions file's `given` is not a list of contributions: %w", err) + } + var out []Contribution + for _, g := range given { + if g.As != "" && g.Secret != "" { + out = append(out, g) + } + } + return out, nil +} + +// delivered is the value the mesh delivered for c, without the newline a file may end with. +func delivered(c Contribution) (string, error) { + raw, err := os.ReadFile(c.Secret) + if err != nil { + return "", fmt.Errorf("the delivered secret is not readable: %v", err) + } + return strings.TrimSuffix(string(raw), "\n"), nil +} + +// DeliveredFingerprint is the fingerprint and length of what the mesh currently delivers for c. +func DeliveredFingerprint(c Contribution) (string, int, error) { + v, err := delivered(c) + if err != nil { + return "", 0, err + } + return Fingerprint(v), len(v), nil +} diff --git a/modules/mesh-vault/cmd/mesh-vault/ledger_test.go b/modules/mesh-vault/cmd/mesh-vault/ledger_test.go new file mode 100644 index 00000000..74aaa70e --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/ledger_test.go @@ -0,0 +1,129 @@ +package main + +// The custody ledger the TypeScript vault kept (its client.ts and test/ledger.test.ts), carried over whole: +// who holds a secret the controller delivered for this vault, since when, and its fingerprint — never the +// value. Its files are the same JSON, so the ledger on the vault's disk is read as it is. + +import ( + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestAFirstDeliveryIsAGrantTheSameAgainNothingANewValueARotation(t *testing.T) { + l, _ := NewLedger(t.TempDir()) + t0 := time.Date(2026, 9, 20, 10, 0, 0, 0, time.UTC) + t1 := time.Date(2026, 9, 21, 10, 0, 0, 0, time.UTC) + + held, outcome, err := l.Record("anchor-redis", "anchor", "first-value", t0) + if err != nil || outcome != Granted || held.Rotations != 0 || held.Since != t0.Format(isoMillis) || + held.Fingerprint != Fingerprint("first-value") { + t.Fatalf("%+v %s %v", held, outcome, err) + } + if _, outcome, _ := l.Record("anchor-redis", "anchor", "first-value", t1); outcome != Unchanged { + t.Errorf("the same value again is %s", outcome) + } + held, outcome, _ = l.Record("anchor-redis", "anchor", "second-value", t1) + if outcome != Rotated || held.Rotations != 1 || held.Since != t0.Format(isoMillis) || held.Changed != t1.Format(isoMillis) { + t.Errorf("%+v %s", held, outcome) + } + if len(held.History) != 1 || held.History[0].Fingerprint != Fingerprint("first-value") || held.History[0].Until != t1.Format(isoMillis) { + t.Errorf("history %+v", held.History) + } +} + +func TestTheLedgerHoldsFingerprintsNeverTheValue(t *testing.T) { + dir := t.TempDir() + l, _ := NewLedger(dir) + _, _, _ = l.Record("anchor-redis", "anchor", "the-actual-password", time.Now()) + _, _, _ = l.Record("anchor-redis", "anchor", "the-rotated-password", time.Now()) + entries, _ := os.ReadDir(dir) + for _, e := range entries { + raw, _ := os.ReadFile(filepath.Join(dir, e.Name())) + if strings.Contains(string(raw), "the-actual-password") || strings.Contains(string(raw), "the-rotated-password") { + t.Errorf("%s holds a value", e.Name()) + } + info, _ := e.Info() + if info.Mode().Perm() != 0o600 { + t.Errorf("%s is %v", e.Name(), info.Mode()) + } + } +} + +func TestDeletingForgetsAHolderAndListingIsByLogin(t *testing.T) { + l, _ := NewLedger(t.TempDir()) + _, _, _ = l.Record("b-app", "b", "x", time.Now()) + _, _, _ = l.Record("a-app", "a", "y", time.Now()) + if got := logins(l.List()); got != "a-app,b-app" { + t.Errorf("listed %s", got) + } + if !l.Delete("a-app") || l.Delete("a-app") { + t.Error("deleting found the wrong thing") + } + if got := logins(l.List()); got != "b-app" { + t.Errorf("listed %s", got) + } +} + +func TestALoginIsANameNotAPath(t *testing.T) { + l, _ := NewLedger(t.TempDir()) + if _, _, err := l.Record("../etc/passwd", "n", "v", time.Now()); err == nil { + t.Error("a path was taken for a login") + } +} + +// A ledger the TypeScript vault wrote is read as it is. +func TestALedgerTheTypeScriptVaultWroteIsRead(t *testing.T) { + dir := t.TempDir() + written := `{ + "as": "anchor-redis", + "consumer": "anchor", + "fingerprint": "sha256:abc", + "length": 40, + "since": "2026-09-20T10:00:00.000Z", + "changed": "2026-09-21T10:00:00.000Z", + "rotations": 1, + "history": [{"fingerprint": "sha256:old", "until": "2026-09-21T10:00:00.000Z"}] +} +` + _ = os.WriteFile(filepath.Join(dir, "anchor-redis.json"), []byte(written), 0o600) + l, _ := NewLedger(dir) + h, ok := l.Get("anchor-redis") + if !ok || h.Rotations != 1 || h.Length != 40 || h.History[0].Fingerprint != "sha256:old" { + t.Fatalf("%+v %v", h, ok) + } +} + +func TestWhatTheMeshDeliversIsFingerprintedNeverReturned(t *testing.T) { + dir := t.TempDir() + secret := filepath.Join(dir, "anchor.redis.secret") + _ = os.WriteFile(secret, []byte("minted-value\n"), 0o600) + receives := filepath.Join(dir, "mesh.json") + doc, _ := json.Marshal(map[string]any{"requirement": "secret", "given": []map[string]string{ + {"from": "redis", "node": "anchor", "as": "anchor-redis", "secret": secret}, + {"from": "offer-only", "node": "anchor"}, + }}) + _ = os.WriteFile(receives, doc, 0o600) + asked, err := Contributions(receives) + if err != nil || len(asked) != 1 || asked[0].As != "anchor-redis" { + t.Fatalf("%+v %v", asked, err) + } + fp, length, err := DeliveredFingerprint(asked[0]) + if err != nil || fp != Fingerprint("minted-value") || length != len("minted-value") { + t.Errorf("%s %d %v", fp, length, err) + } + if _, _, err := DeliveredFingerprint(Contribution{As: "x", Secret: filepath.Join(dir, "missing")}); err == nil || !strings.Contains(err.Error(), "not readable") { + t.Errorf("a missing file said %v", err) + } +} + +func logins(hs []Held) string { + var out []string + for _, h := range hs { + out = append(out, h.As) + } + return strings.Join(out, ",") +} diff --git a/modules/mesh-vault/cmd/mesh-vault/main.go b/modules/mesh-vault/cmd/mesh-vault/main.go new file mode 100644 index 00000000..25f5cd4d --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/main.go @@ -0,0 +1,94 @@ +// mesh-vault (novox/hq ADR 0085, ADR 0113, ADR 0349 §7 and §11): one binary, launched by its node's runtime +// and speaking MCP to it over stdio through the Go SDK. It holds the `mesh-vault` seat: it keeps every +// secret in its own store, sealed at rest and with a copy sealed to the operator key; it answers the seat's +// verb `get` with a secret sealed to the asker's key; it raises `secret-changed` without a value. Beside +// that it keeps the custody ledger of the pair credentials the controller still delivers to it. +// +// stdout is the MCP channel; everything this module says, it says on stderr. +package main + +import ( + "fmt" + "os" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +func say(format string, args ...any) { + fmt.Fprintf(os.Stderr, "[mesh-vault] "+format+"\n", args...) +} + +func emit(event string, body map[string]any) error { return stdio.Emit(event, body) } + +// build makes the vault from the module's words: directories, never values. +func build() *Env { + env := &Env{Receives: os.Getenv("MESH_RECEIVES"), Root: os.Getenv("MESH_VAULT_ROOT")} + if l, err := NewLedger(os.Getenv("MESH_VAULT_LEDGER")); err == nil { + env.Ledger = l + } else { + say("%v", err) + } + if env.Ledger != nil && env.Receives != "" { + env.Custody = &Custody{Ledger: env.Ledger, Receives: env.Receives, Resource: "secret", Node: os.Getenv("MESH_NODE"), + Emit: emit, Now: time.Now, Log: say} + } + store, err := OpenStore(os.Getenv("MESH_VAULT_STORE")) + if err != nil { + env.Broken = fmt.Errorf("the vault could not open its store: %w", err) + say("%v", env.Broken) + return env + } + own, err := OwnKey(os.Getenv("MESH_VAULT_STATE"), store) + if err != nil { + env.Broken = fmt.Errorf("the vault could not open its key: %w", err) + say("%v", env.Broken) + return env + } + // Askers is left nil: the vault that runs answers nobody until who may ask arrives (askers.go). + env.Vault = &Vault{Store: store, Own: own, Emit: emit, Now: time.Now, Log: say} + return env +} + +// adoptOperatorKey takes the operator key from the export the mesh writes to the vault's root (what +// `operator key set` recorded; the controller keeps only a public half there), and seals again any +// operator copy not yet sealed to the current key. Run every minute, so a failure partway is finished. +func adoptOperatorKey(env *Env) { + if env.Vault == nil { + return + } + if raw := exportedOperatorKey(env.Root); env.Root != "" && raw != "" { + key, err := publicPEM(raw) + if err == nil { + _, err = env.Vault.SetOperatorKey(key, "the mesh's export") + } + env.refuseKey("") + if err != nil { + env.refuseKey(err.Error()) + say("the operator key in the mesh's export is not taken, or not every copy is sealed to it yet: %v", err) + } + return + } + if _, err := env.Vault.Reseal(); err != nil { + say("%v", err) + } +} + +func main() { + env := build() + go func() { + tick := time.NewTicker(time.Minute) + defer tick.Stop() + for { + adoptOperatorKey(env) + if env.Custody != nil { + env.Custody.Pass() + } + <-tick.C + } + }() + if err := stdio.Serve("", tools(env)); err != nil { + say("%v", err) + os.Exit(1) + } +} diff --git a/modules/mesh-vault/cmd/mesh-vault/retirement.go b/modules/mesh-vault/cmd/mesh-vault/retirement.go new file mode 100644 index 00000000..c614a9d1 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/retirement.go @@ -0,0 +1,466 @@ +package main + +// Retirement of a holder the mesh stops asking for (novox/hq ADR 0230), in the words the controller reads: +// the `provisioner.retirement` event as mesh-controller's link.Retirement, the `provisioner_retirement` +// answer as link.RetirementState, and the acts as the SDK's harness takes them (mesh-sdk +// src/provisioner/index.ts) — the set repeated in `consumers`, a `why`, and the person's `by` and the +// `via` the controller's verbs say. testdata/retirement-*.json are those types marshalled by the controller +// at the commit its README names, and the tests hold this file's words to them. +// +// The vault is mark-only: it holds no access to switch off, so a retired holder keeps what it has until a +// person deletes it, and the mark — when and why — is in the holder's own ledger file, so a restart forgets +// nothing. + +import ( + "encoding/json" + "errors" + "fmt" + "sort" + "strings" + "sync" + "time" +) + +// EventRetirement is the one event a provider says about retirement (ADR 0230 §35). +const EventRetirement = "provisioner.retirement" + +// The rules of ADR 0230 §15 and §18. +const ( + StablePasses = 5 + StableFor = 10 * time.Minute + RetireAtOnce = 3 + RetireFraction = 0.5 + SayAgainEvery = time.Hour +) + +// RetiredConsumer is one holder in a retirement word or answer (link.RetiredConsumer). +type RetiredConsumer struct { + Consumer string `json:"consumer"` + Node string `json:"node,omitempty"` + RetiredAt string `json:"retired_at,omitempty"` + Why string `json:"why,omitempty"` + SizeBytes *int64 `json:"size_bytes,omitempty"` + Kind string `json:"kind,omitempty"` + Access string `json:"access,omitempty"` +} + +// RetirementWord is the event's body (link.Retirement). +type RetirementWord struct { + Provider string `json:"provider"` + ProviderNode string `json:"provider-node"` + Change string `json:"change"` + Consumers []RetiredConsumer `json:"consumers"` + Held int `json:"held"` + Bound string `json:"bound,omitempty"` + Why string `json:"why,omitempty"` + By string `json:"by,omitempty"` + Via string `json:"via,omitempty"` + At time.Time `json:"at"` + Since *time.Time `json:"since,omitempty"` +} + +// RetirementWaiting, RetirementRejected and RetirementState are the answer to provisioner_retirement +// (link.RetirementWaiting, link.RetirementRejected, link.RetirementState). +type RetirementWaiting struct { + Consumers []RetiredConsumer `json:"consumers"` + Since string `json:"since"` + Held int `json:"held"` +} + +type RetirementRejected struct { + Consumers []RetiredConsumer `json:"consumers"` + By string `json:"by"` + Why string `json:"why"` + At string `json:"at"` +} + +type RetirementState struct { + Resource string `json:"resource"` + Node string `json:"node"` + Held []string `json:"held"` + StablePasses int `json:"stable_passes"` + StableForSeconds int `json:"stable_for_seconds,omitempty"` + RetiresBy string `json:"retires_by,omitempty"` + Bound string `json:"bound"` + Waiting *RetirementWaiting `json:"waiting"` + Rejected *RetirementRejected `json:"rejected"` + Retired []RetiredConsumer `json:"retired"` +} + +// Act is a person's act on retirement: why, who (the person the controller proved) and through what. +type Act struct { + Why, By, Via string +} + +// Custody is the pass over the contributions file that keeps the ledger, and what it remembers between +// passes about the holders the mesh no longer asks for. +type Custody struct { + Ledger *Ledger + Receives string + // Resource is the provision the vault provides; Node the machine it runs on. + Resource string + Node string + Emit func(event string, body map[string]any) error + Now func() time.Time + Log func(format string, args ...any) + + mu sync.Mutex + key string + count int + firstSeen time.Time + asked map[string]bool + read bool + waiting *heldSet + rejected *heldSet +} + +type heldSet struct { + set []string + since time.Time + saidAt time.Time + held int + by, why string +} + +func (c *Custody) resource() string { + if c.Resource == "" { + return "secret" + } + return c.Resource +} + +// Pass records every delivered value, re-enables a retired holder asked for again, and counts the holders +// no longer asked for toward their retirement. +func (c *Custody) Pass() { + c.mu.Lock() + defer c.mu.Unlock() + asked, err := Contributions(c.Receives) + if err != nil { + c.Log("THE CONTRIBUTIONS FILE IS NOT USABLE (%s): %v — nothing is recorded or retired on this pass, and "+ + "the count toward retiring anyone starts again", c.Receives, err) + c.key, c.count = "", 0 + return + } + now := c.Now() + keep := map[string]bool{} + for _, a := range asked { + keep[a.As] = true + value, err := delivered(a) + if err != nil { + c.Log("%s: %v", a.As, err) + continue + } + held, outcome, err := c.Ledger.Record(a.As, a.Node, value, now) + if err != nil { + c.Log("%s: %v", a.As, err) + continue + } + if held.Retired != nil { + if err := c.Ledger.Mark(a.As, nil); err != nil { + c.Log("%s: asked for again, and its retirement mark could not be cleared: %v", a.As, err) + } else { + c.Log("%s: asked for again — re-enabled, its retirement mark cleared", a.As) + c.tell(RetirementWord{Change: "reenabled", Consumers: []RetiredConsumer{{Consumer: a.As, Node: a.Node}}}, now) + } + } + if outcome == Unchanged { + continue + } + event := "provisioned" + if outcome == Rotated { + event = "rotated" + } + c.Log("%s: %s (%s…, rotations %d)", held.As, outcome, held.Fingerprint[:19], held.Rotations) + c.say(event, map[string]any{"consumer": held.Consumer, "as": held.As, "fingerprint": held.Fingerprint, "rotations": held.Rotations}) + } + c.asked, c.read = keep, true + c.countUnasked(now) +} + +func (c *Custody) active() []Held { + var out []Held + for _, h := range c.Ledger.List() { + if h.Retired == nil { + out = append(out, h) + } + } + return out +} + +func (c *Custody) countUnasked(now time.Time) { + active := c.active() + var unasked []string + for _, h := range active { + if !c.asked[h.As] { + unasked = append(unasked, h.As) + } + } + key := strings.Join(unasked, ",") + if c.waiting != nil && strings.Join(c.waiting.set, ",") != key { + c.settle(c.waiting, "the set the mesh no longer asks for changed", now) + c.waiting = nil + } + if c.rejected != nil && strings.Join(c.rejected.set, ",") != key { + c.settle(c.rejected, "the set the mesh no longer asks for changed", now) + c.rejected = nil + } + switch { + case len(unasked) == 0: + c.key, c.count = "", 0 + return + case key != c.key: + c.key, c.count, c.firstSeen = key, 1, now + c.Log("no longer asked for: %s; retired once the same set holds for %d passes and %s (novox/hq ADR 0230)", + key, StablePasses, StableFor) + default: + c.count++ + } + if c.rejected != nil || c.count < StablePasses || now.Sub(c.firstSeen) < StableFor { + return + } + if overTheBound(len(unasked), len(active)) { + c.wait(unasked, len(active), now) + return + } + c.retire(unasked, len(active), fmt.Sprintf("the mesh stopped asking for it: the same answer in %d passes over %s, from %s", + c.count, now.Sub(c.firstSeen).Round(time.Second), c.firstSeen.UTC().Format(time.RFC3339)), now) +} + +func overTheBound(n, held int) bool { + return n > RetireAtOnce || (held > 1 && float64(n) > RetireFraction*float64(held)) +} + +func bound(held int) string { + return fmt.Sprintf("more than %d, or more than %d%% of the %d held", RetireAtOnce, int(RetireFraction*100), held) +} + +func (c *Custody) wait(set []string, held int, now time.Time) { + if c.waiting == nil { + c.waiting = &heldSet{set: set, since: now, held: held, saidAt: now} + c.Log("RETIREMENT WAITS FOR A PERSON: the mesh no longer asks for %d of the %d holder(s) (%s), %s. "+ + "Nothing is retired until `retire approve` or `retire reject` (novox/hq ADR 0230)", + len(set), held, strings.Join(set, ", "), bound(held)) + } else if now.Sub(c.waiting.saidAt) < SayAgainEvery { + return + } else { + c.waiting.saidAt = now + } + since := c.waiting.since.UTC() + c.tell(RetirementWord{Change: "waiting", Consumers: c.named(set), Held: c.waiting.held, Since: &since}, now) +} + +func (c *Custody) settle(s *heldSet, why string, now time.Time) { + c.Log("the set waiting on retirement (%s) is settled, nothing retired: %s", strings.Join(s.set, ", "), why) + c.tell(RetirementWord{Change: "settled", Consumers: c.named(s.set), Why: why}, now) +} + +// retire marks each holder in set retired: the ledger keeps it, and its history, until a person deletes it. +func (c *Custody) retire(set []string, held int, why string, now time.Time) []string { + at := now.UTC().Format(isoMillis) + var done []string + for _, as := range set { + if err := c.Ledger.Mark(as, &RetiredMark{At: at, Why: why}); err != nil { + c.Log("%s: could not be marked retired, asked again once its set is stable again: %v", as, err) + continue + } + done = append(done, as) + } + c.key, c.count = "", 0 + if len(done) == 0 { + return done + } + c.Log("%s: RETIRED, MARK ONLY — the vault holds no access to switch off, so each keeps what it holds until a "+ + "person deletes it with `cleanup delete` (novox/hq ADR 0230): %s", strings.Join(done, ", "), why) + c.tell(RetirementWord{Change: "retired", Consumers: c.retiredOnes(done), Held: held, Why: why}, now) + return done +} + +// named is each holder as the controller's words name it. +func (c *Custody) named(set []string) []RetiredConsumer { + out := make([]RetiredConsumer, 0, len(set)) + for _, as := range set { + h, _ := c.Ledger.Get(as) + out = append(out, RetiredConsumer{Consumer: as, Node: h.Consumer}) + } + return out +} + +func (c *Custody) retiredOnes(set []string) []RetiredConsumer { + out := make([]RetiredConsumer, 0, len(set)) + for _, as := range set { + h, _ := c.Ledger.Get(as) + out = append(out, retiredOne(h)) + } + return out +} + +func retiredOne(h Held) RetiredConsumer { + unknown := int64(-1) + r := RetiredConsumer{Consumer: h.As, Node: h.Consumer, SizeBytes: &unknown, Kind: "consumer", Access: "kept"} + if h.Retired != nil { + r.RetiredAt, r.Why = h.Retired.At, h.Retired.Why + } + return r +} + +func sameSet(a, b []string) bool { + x, y := append([]string{}, a...), append([]string{}, b...) + sort.Strings(x) + sort.Strings(y) + return strings.Join(x, ",") == strings.Join(y, ",") +} + +// Approve retires exactly the set waiting (or rejected) for a person, repeated in consumers; any other set +// is refused (ADR 0230 §20). +func (c *Custody) Approve(consumers []string, act Act) ([]string, error) { + if strings.TrimSpace(act.Why) == "" { + return nil, errors.New("approve: why is required") + } + c.mu.Lock() + defer c.mu.Unlock() + s := c.waiting + if s == nil { + s = c.rejected + } + if s == nil || !sameSet(s.set, consumers) { + return nil, fmt.Errorf("approve: nothing waits with exactly that set (%s); waiting: %s; rejected: %s. Look again", + strings.Join(consumers, ", "), setOf(c.waiting), setOf(c.rejected)) + } + now := c.Now() + held := len(c.active()) + c.Log("retirement of %s approved by %s: %s", strings.Join(s.set, ", "), someone(act.By), act.Why) + c.tell(RetirementWord{Change: "approved", Consumers: c.named(s.set), Held: held, Why: act.Why, By: act.By, Via: act.Via}, now) + set := s.set + c.waiting, c.rejected = nil, nil + return c.retire(set, held, "approved by "+someone(act.By)+": "+act.Why, now), nil +} + +// Reject keeps the waiting set active, repeated in consumers, and asks no more about it while it stays the +// same set. +func (c *Custody) Reject(consumers []string, act Act) ([]string, error) { + if strings.TrimSpace(act.Why) == "" { + return nil, errors.New("reject: why is required") + } + c.mu.Lock() + defer c.mu.Unlock() + if c.waiting == nil || !sameSet(c.waiting.set, consumers) { + return nil, fmt.Errorf("reject: nothing waits with exactly that set (%s); waiting: %s. Look again", + strings.Join(consumers, ", "), setOf(c.waiting)) + } + now := c.Now() + c.rejected = &heldSet{set: c.waiting.set, since: now, by: act.By, why: act.Why} + c.waiting = nil + c.Log("retirement of %s rejected by %s: %s; kept active while the mesh says the same", + strings.Join(c.rejected.set, ", "), someone(act.By), act.Why) + c.tell(RetirementWord{Change: "rejected", Consumers: c.named(c.rejected.set), Why: act.Why, By: act.By, Via: act.Via}, now) + return c.rejected.set, nil +} + +// Delete forgets one retired holder on a person's word, its name repeated in confirm. Refused: before a +// pass has read the contributions file, a holder that is not retired, one the contributions file asks for +// now, and any act without a why or a matching confirm. +func (c *Custody) Delete(consumer, confirm string, act Act) error { + if strings.TrimSpace(act.Why) == "" { + return errors.New("delete: why is required") + } + if confirm != consumer { + return errors.New("delete: confirm must repeat the consumer's name") + } + c.mu.Lock() + defer c.mu.Unlock() + if !c.read { + return errors.New("delete: no pass has read the contributions file yet, so whether the mesh asks for it is not known; try again in a minute") + } + h, ok := c.Ledger.Get(consumer) + if !ok { + return fmt.Errorf("delete: %s is not held here", consumer) + } + if h.Retired == nil { + return fmt.Errorf("delete: %s is active, not retired — only a retired holder is deleted", consumer) + } + asked, err := Contributions(c.Receives) + if err != nil { + return fmt.Errorf("delete: the contributions file cannot say whether the mesh asks for %s: %v", consumer, err) + } + for _, a := range asked { + if a.As == consumer { + return fmt.Errorf("delete: the mesh asks for %s again; the next pass re-enables it", consumer) + } + } + if !c.Ledger.Delete(consumer) { + return fmt.Errorf("delete: %s could not be deleted", consumer) + } + c.Log("%s: DELETED from the ledger, asked by %s: %s", consumer, someone(act.By), act.Why) + unknown := int64(-1) + c.tell(RetirementWord{Change: "deleted", Consumers: []RetiredConsumer{{Consumer: consumer, Node: h.Consumer, SizeBytes: &unknown}}, + Why: act.Why, By: act.By, Via: act.Via}, c.Now()) + return nil +} + +func someone(by string) string { + if by == "" { + return "someone" + } + return by +} + +func setOf(s *heldSet) string { + if s == nil || len(s.set) == 0 { + return "none" + } + return strings.Join(s.set, ", ") +} + +// Retirement is the answer to provisioner_retirement. +func (c *Custody) Retirement() RetirementState { + c.mu.Lock() + defer c.mu.Unlock() + st := RetirementState{Resource: c.resource(), Node: c.Node, Held: []string{}, StablePasses: StablePasses, + StableForSeconds: int(StableFor.Seconds()), RetiresBy: "mark-only", Retired: []RetiredConsumer{}} + for _, h := range c.Ledger.List() { + if h.Retired == nil { + st.Held = append(st.Held, h.As) + } else { + st.Retired = append(st.Retired, retiredOne(h)) + } + } + st.Bound = bound(len(st.Held)) + if c.waiting != nil { + st.Waiting = &RetirementWaiting{Consumers: c.named(c.waiting.set), + Since: c.waiting.since.UTC().Format(time.RFC3339), Held: c.waiting.held} + } + if c.rejected != nil { + st.Rejected = &RetirementRejected{Consumers: c.named(c.rejected.set), By: c.rejected.by, Why: c.rejected.why, + At: c.rejected.since.UTC().Format(time.RFC3339)} + } + return st +} + +// tell says one retirement word, with the provider, its machine, the bound and the time filled in. +func (c *Custody) tell(w RetirementWord, now time.Time) { + w.Provider, w.ProviderNode, w.At = c.resource(), c.Node, now.UTC() + if w.Bound == "" { + w.Bound = bound(len(c.active())) + } + if w.Consumers == nil { + w.Consumers = []RetiredConsumer{} + } + c.say(EventRetirement, wordBody(w)) +} + +// wordBody is a word as the map the SDK's Emit takes, through its JSON, so the bus carries exactly the tags. +func wordBody(w RetirementWord) map[string]any { + raw, _ := json.Marshal(w) + var m map[string]any + _ = json.Unmarshal(raw, &m) + return m +} + +func (c *Custody) say(event string, body map[string]any) { + if c.Emit == nil { + return + } + if err := c.Emit(event, body); err != nil { + c.Log("%s could not be said: %v", event, err) + } +} diff --git a/modules/mesh-vault/cmd/mesh-vault/retirement_test.go b/modules/mesh-vault/cmd/mesh-vault/retirement_test.go new file mode 100644 index 00000000..83c23abf --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/retirement_test.go @@ -0,0 +1,537 @@ +package main + +// ADR 0230 in the vault's custody ledger, and its words held to the controller's: testdata/retirement-*.json +// are mesh-controller's link.Retirement and link.RetirementState marshalled at dc36536 (feat/57, with #317). + +import ( + "bytes" + "encoding/json" + "fmt" + "os" + "path/filepath" + "sort" + "strings" + "testing" + "time" +) + +// custodyAt is a custody pass over a contributions file in dir, on a clock the test moves. +type custodyAt struct { + *Custody + bus *published + receives string + dir string + clock *time.Time +} + +func newCustody(t *testing.T) custodyAt { + dir := t.TempDir() + l, _ := NewLedger(filepath.Join(dir, "ledger")) + bus := &published{} + clock := time.Date(2026, 10, 14, 9, 0, 0, 0, time.UTC) + c := custodyAt{bus: bus, receives: filepath.Join(dir, "mesh.json"), dir: dir, clock: &clock} + c.Custody = &Custody{Ledger: l, Receives: c.receives, Resource: "secret", Node: "novox", Emit: bus.emit, + Now: func() time.Time { return *c.clock }, Log: t.Logf} + return c +} + +// ask writes a contributions file asking for these logins, each with its value. +func (c custodyAt) ask(values map[string]string) { + given := []map[string]string{} + for as, v := range values { + f := filepath.Join(c.dir, as+".secret") + _ = os.WriteFile(f, []byte(v), 0o600) + given = append(given, map[string]string{"node": "anchor", "as": as, "secret": f}) + } + doc, _ := json.Marshal(map[string]any{"requirement": "secret", "given": given}) + _ = os.WriteFile(c.receives, doc, 0o600) +} + +// apps is n holders named a-app, b-app, … +func apps(n int) map[string]string { + out := map[string]string{} + for i := 0; i < n; i++ { + out[fmt.Sprintf("%c-app", 'a'+i)] = fmt.Sprint(i) + } + return out +} + +func keepOnly(all map[string]string, n int) map[string]string { + names := make([]string, 0, len(all)) + for k := range all { + names = append(names, k) + } + sort.Strings(names) + out := map[string]string{} + for _, k := range names[:n] { + out[k] = all[k] + } + return out +} + +// passes runs n passes a minute apart. +func (c custodyAt) passes(n int) { + for i := 0; i < n; i++ { + c.Pass() + *c.clock = c.clock.Add(time.Minute) + } +} + +// stableFor runs StablePasses passes over an hour: past both of ADR 0230's conditions. +func (c custodyAt) stable() { + for i := 0; i < StablePasses; i++ { + c.Pass() + *c.clock = c.clock.Add(15 * time.Minute) + } +} + +func (c custodyAt) changes() []string { + var out []string + for _, e := range c.bus.events { + if e["event"] == EventRetirement { + out = append(out, e["body"].(map[string]any)["change"].(string)) + } else { + out = append(out, e["event"].(string)) + } + } + return out +} + +func (c custodyAt) retired(as string) bool { + h, _ := c.Ledger.Get(as) + return h.Retired != nil +} + +func waitingNames(st RetirementState) []string { + if st.Waiting == nil { + return nil + } + var out []string + for _, w := range st.Waiting.Consumers { + out = append(out, w.Consumer) + } + return out +} + +func act(why string) Act { return Act{Why: why, By: "person.jochen", Via: "mesh-controller"} } + +// ---- the controller's words ---------------------------------------------------------------------------- + +// paths is every key path in a JSON document, arrays read through their first element. +func paths(t *testing.T, raw []byte) map[string]bool { + t.Helper() + var v any + if err := json.Unmarshal(raw, &v); err != nil { + t.Fatal(err) + } + out := map[string]bool{} + var walk func(prefix string, v any) + walk = func(prefix string, v any) { + switch x := v.(type) { + case map[string]any: + for k, e := range x { + out[prefix+k] = true + walk(prefix+k+".", e) + } + case []any: + if len(x) > 0 { + walk(prefix, x[0]) + } + } + } + walk("", v) + return out +} + +// sameWords says every key the vault says is a key the controller's type has, and every key the controller +// needs is said. +func sameWords(t *testing.T, what string, ours []byte, golden string, needed ...string) { + t.Helper() + theirs, err := os.ReadFile(filepath.Join("testdata", golden)) + if err != nil { + t.Fatal(err) + } + want := paths(t, theirs) + got := paths(t, ours) + for p := range got { + if !want[p] { + t.Errorf("%s says %q, which the controller's type has no field for", what, p) + } + } + for _, p := range needed { + if !got[p] { + t.Errorf("%s does not say %q, which the controller reads", what, p) + } + } + // And decoding strictly into the shape of the controller's golden document's own Go type would succeed: + // the same kinds at the same keys. + var g, o any + _ = json.Unmarshal(theirs, &g) + _ = json.Unmarshal(ours, &o) + sameKinds(t, what, "", g, o) +} + +func sameKinds(t *testing.T, what, at string, want, got any) { + switch w := want.(type) { + case map[string]any: + g, ok := got.(map[string]any) + if !ok { + if got != nil { + t.Errorf("%s: %q is %T, the controller's is an object", what, at, got) + } + return + } + for k, v := range g { + if wv, known := w[k]; known { + sameKinds(t, what, at+"."+k, wv, v) + } + } + case []any: + g, ok := got.([]any) + if !ok { + if got != nil { + t.Errorf("%s: %q is %T, the controller's is a list", what, at, got) + } + return + } + if len(w) > 0 { + for _, e := range g { + sameKinds(t, what, at+"[]", w[0], e) + } + } + default: + if got != nil && fmt.Sprintf("%T", want) != fmt.Sprintf("%T", got) { + t.Errorf("%s: %q is %T, the controller's is %T", what, at, got, want) + } + } +} + +// Every retirement word the vault says is one the controller's link.ReadRetirement reads: its machine +// named, its change known, its consumers objects. +func TestTheRetirementWordsAreTheControllers(t *testing.T) { + c := newCustody(t) + all := apps(5) + c.ask(all) + c.passes(1) + c.ask(keepOnly(all, 1)) + c.stable() // waiting: four of five + st := c.Retirement() + if _, err := c.Reject(waitingNames(st), act("kept for now")); err != nil { + t.Fatal(err) + } + if _, err := c.Approve(waitingNames(st), act("they are gone")); err != nil { + t.Fatal(err) + } + c.passes(1) + if err := c.Delete("b-app", "b-app", act("gone for good")); err != nil { + t.Fatal(err) + } + c.ask(all) + c.passes(1) // re-enabled + seen := map[string]bool{} + for i, e := range c.bus.events { + if e["event"] != EventRetirement { + continue + } + raw := []byte(c.bus.raw[i]) + var outer struct { + Body json.RawMessage `json:"body"` + } + _ = json.Unmarshal(raw, &outer) + var w struct { + Change string `json:"change"` + ProviderNode string `json:"provider-node"` + Consumers []struct { + Consumer string `json:"consumer"` + } `json:"consumers"` + } + dec := json.NewDecoder(bytes.NewReader(outer.Body)) + if err := dec.Decode(&w); err != nil { + t.Fatalf("a retirement word the controller cannot read: %v: %s", err, outer.Body) + } + if w.ProviderNode != "novox" || len(w.Consumers) == 0 || w.Consumers[0].Consumer == "" { + t.Errorf("a %s word names no machine or no consumer: %s", w.Change, outer.Body) + } + seen[w.Change] = true + sameWords(t, "the "+w.Change+" word", outer.Body, "retirement-word.json", + "provider", "provider-node", "change", "consumers", "consumers.consumer", "at") + } + for _, change := range []string{"waiting", "rejected", "approved", "retired", "deleted", "reenabled"} { + if !seen[change] { + t.Errorf("no %s word was said", change) + } + } +} + +// The answer to provisioner_retirement is the controller's RetirementState, so `retire approve`, `retire +// reject` and `cleanup list` read the vault. +func TestTheRetirementAnswerIsTheControllers(t *testing.T) { + c := newCustody(t) + all := apps(5) + c.ask(all) + c.passes(1) + c.ask(keepOnly(all, 1)) + c.stable() + raw, _ := json.Marshal(c.Retirement()) + sameWords(t, "the waiting answer", raw, "retirement-state.json", "resource", "node", "held", "stable_passes", + "retires_by", "bound", "waiting", "waiting.consumers", "waiting.consumers.consumer", "waiting.since", "retired") + if _, err := c.Approve(waitingNames(c.Retirement()), act("gone")); err != nil { + t.Fatal(err) + } + raw, _ = json.Marshal(c.Retirement()) + sameWords(t, "the retired answer", raw, "retirement-state.json", "retired.consumer", "retired.retired_at", + "retired.why", "retired.access") + st := c.Retirement() + if st.RetiresBy != "mark-only" || st.Resource != "secret" || st.Node != "novox" || len(st.Retired) != 4 || + st.Retired[0].Access != "kept" { + t.Errorf("the answer is %+v", st) + } +} + +// ---- the rules, one at a time --------------------------------------------------------------------------- + +func TestTheCustodyPassRecordsAndRotates(t *testing.T) { + c := newCustody(t) + c.ask(map[string]string{"anchor-redis": "v1"}) + c.passes(1) + c.ask(map[string]string{"anchor-redis": "v2"}) + c.passes(1) + if got := strings.Join(c.changes(), ","); got != "provisioned,rotated" { + t.Errorf("announced %s", got) + } + for _, raw := range c.bus.raw { + if strings.Contains(raw, `"v1"`) || strings.Contains(raw, `"v2"`) { + t.Errorf("announced a value: %s", raw) + } + } +} + +// A contributions file that does not say who is asked for is no answer: nobody is retired. +func TestAFileThatSaysNothingRetiresNobody(t *testing.T) { + for _, doc := range []string{"{not json", `{"requirement":"secret"}`, `{"requirement":"secret","given":null}`, + `{"requirement":"redis-cache","given":[]}`} { + c := newCustody(t) + c.ask(map[string]string{"anchor-redis": "v1"}) + c.passes(1) + _ = os.WriteFile(c.receives, []byte(doc), 0o600) + c.passes(30) + if h, ok := c.Ledger.Get("anchor-redis"); !ok || h.Retired != nil { + t.Errorf("%s: the holder was retired or lost: %+v %v", doc, h, ok) + } + if _, err := Contributions(c.receives); err == nil { + t.Errorf("%s was read as an answer", doc) + } + } + if _, err := Contributions(filepath.Join(t.TempDir(), "missing.json")); err == nil { + t.Error("a missing file was read as an answer") + } +} + +// Five passes, not four, however much time has passed. +func TestRetiringTakesFivePasses(t *testing.T) { + c := newCustody(t) + c.ask(apps(3)) + c.passes(1) + c.ask(keepOnly(apps(3), 2)) + for i := 0; i < StablePasses-1; i++ { + c.Pass() + *c.clock = c.clock.Add(time.Hour) + } + if c.retired("c-app") { + t.Fatal("retired after four passes") + } + c.Pass() + if !c.retired("c-app") { + t.Fatal("not retired after five passes and hours") + } +} + +// Ten minutes, not less, however many passes. +func TestRetiringTakesTenMinutes(t *testing.T) { + c := newCustody(t) + c.ask(apps(3)) + c.passes(1) + c.ask(keepOnly(apps(3), 2)) + for i := 0; i < 20; i++ { + c.Pass() + *c.clock = c.clock.Add(25 * time.Second) + } + if c.retired("c-app") { + t.Fatal("retired before ten minutes") + } + *c.clock = c.clock.Add(5 * time.Minute) + c.Pass() + if !c.retired("c-app") { + t.Fatal("not retired after twenty passes and ten minutes") + } +} + +// More than three at once waits, though under half: 4 of 10. +func TestMoreThanThreeWaits(t *testing.T) { + c := newCustody(t) + c.ask(apps(10)) + c.passes(1) + c.ask(keepOnly(apps(10), 6)) + c.stable() + if got := waitingNames(c.Retirement()); len(got) != 4 { + t.Fatalf("four of ten did not wait: %v", got) + } + if c.retired("j-app") { + t.Fatal("retired while waiting") + } +} + +// More than half waits, though three or fewer: 2 of 3. +func TestMoreThanHalfWaits(t *testing.T) { + c := newCustody(t) + c.ask(apps(3)) + c.passes(1) + c.ask(keepOnly(apps(3), 1)) + c.stable() + if got := waitingNames(c.Retirement()); len(got) != 2 { + t.Fatalf("two of three did not wait: %v", got) + } +} + +// Within both bounds it retires without a person: 3 of 7. +func TestWithinTheBoundRetires(t *testing.T) { + c := newCustody(t) + c.ask(apps(7)) + c.passes(1) + c.ask(keepOnly(apps(7), 4)) + c.stable() + if c.Retirement().Waiting != nil || !c.retired("g-app") { + t.Fatal("three of seven did not retire") + } +} + +// A holder is retired, never deleted by a pass, keeps its history, and is re-enabled when asked again. +func TestAHolderNoLongerAskedForIsRetiredNotDeleted(t *testing.T) { + c := newCustody(t) + c.ask(apps(3)) + c.passes(1) + c.ask(keepOnly(apps(3), 2)) + c.stable() + h, ok := c.Ledger.Get("c-app") + if !ok || h.Retired == nil || h.Retired.Why == "" || h.Fingerprint != Fingerprint("2") { + t.Fatalf("%+v %v", h, ok) + } + c.ask(apps(3)) + c.Pass() + if c.retired("c-app") { + t.Error("asked for again, the holder stayed retired") + } + if got := strings.Join(c.changes(), ","); got != "provisioned,provisioned,provisioned,retired,reenabled" { + t.Errorf("announced %s", got) + } +} + +// A file that says nothing starts the count again. +func TestTheCountStartsAgain(t *testing.T) { + c := newCustody(t) + c.ask(apps(3)) + c.passes(1) + c.ask(keepOnly(apps(3), 2)) + c.passes(4) + _ = os.WriteFile(c.receives, []byte(`{"given":null}`), 0o600) + c.passes(1) + c.ask(keepOnly(apps(3), 2)) + c.passes(4) + *c.clock = c.clock.Add(time.Hour) + if c.retired("c-app") { + t.Fatal("retired though the count was broken") + } + c.passes(1) + if !c.retired("c-app") { + t.Fatal("not retired after five unbroken passes and an hour") + } +} + +// Approve and reject take exactly the set waiting, and refuse any other (ADR 0230 §20). +func TestApproveAndRejectTakeExactlyTheSetWaiting(t *testing.T) { + for _, decide := range []string{"approve", "reject"} { + c := newCustody(t) + c.ask(apps(5)) + c.passes(1) + c.ask(keepOnly(apps(5), 1)) + c.stable() + set := waitingNames(c.Retirement()) + for _, wrong := range [][]string{set[:3], append(append([]string{}, set...), "a-app"), {}} { + if _, err := c.Approve(wrong, act("x")); err == nil { + t.Errorf("approve took the set %v", wrong) + } + if _, err := c.Reject(wrong, act("x")); err == nil { + t.Errorf("reject took the set %v", wrong) + } + } + if _, err := c.Approve(set, Act{}); err == nil { + t.Error("an approval without a why was taken") + } + reversed := append([]string{}, set...) + sort.Sort(sort.Reverse(sort.StringSlice(reversed))) + if decide == "approve" { + done, err := c.Approve(reversed, act("the four apps are gone")) + if err != nil || len(done) != 4 || !c.retired("b-app") || c.retired("a-app") { + t.Fatalf("%v %v", done, err) + } + } else { + if _, err := c.Reject(reversed, act("they come back")); err != nil { + t.Fatal(err) + } + c.passes(10) + if c.retired("b-app") || c.Retirement().Waiting != nil { + t.Error("a rejected set was retired or asked about again") + } + } + } +} + +// retiredC is a custody with c-app retired, d-app active. +func retiredC(t *testing.T) custodyAt { + c := newCustody(t) + c.ask(apps(4)) + c.passes(1) + c.ask(keepOnly(apps(4), 3)) + c.stable() + if !c.retired("d-app") { + t.Fatal("d-app was not retired") + } + return c +} + +// Each refusal of a delete, one at a time. +func TestEachDeleteRefusal(t *testing.T) { + fresh := newCustody(t) + fresh.ask(apps(1)) + if err := fresh.Delete("a-app", "a-app", act("x")); err == nil || !strings.Contains(err.Error(), "no pass") { + t.Errorf("a delete before any pass read the file: %v", err) + } + c := retiredC(t) + if err := c.Delete("a-app", "a-app", act("x")); err == nil || !strings.Contains(err.Error(), "active") { + t.Errorf("an active holder: %v", err) + } + if err := c.Delete("d-app", "d-app", Act{By: "person.jochen"}); err == nil || !strings.Contains(err.Error(), "why") { + t.Errorf("no why: %v", err) + } + if err := c.Delete("d-app", "c-app", act("x")); err == nil || !strings.Contains(err.Error(), "confirm") { + t.Errorf("a confirm naming another: %v", err) + } + if err := c.Delete("nobody", "nobody", act("x")); err == nil { + t.Error("a holder not held") + } + c.ask(apps(4)) // asked for again, no pass yet + if err := c.Delete("d-app", "d-app", act("x")); err == nil || !strings.Contains(err.Error(), "asks for") { + t.Errorf("a holder the file asks for again: %v", err) + } + c.ask(keepOnly(apps(4), 3)) + if err := c.Delete("d-app", "d-app", act("gone for good")); err != nil { + t.Fatal(err) + } + if _, ok := c.Ledger.Get("d-app"); ok { + t.Error("deleted, but still in the ledger") + } + last := c.bus.events[len(c.bus.events)-1]["body"].(map[string]any) + if last["change"] != "deleted" || last["by"] != "person.jochen" || last["via"] != "mesh-controller" { + t.Errorf("the deletion said %v", last) + } +} diff --git a/modules/mesh-vault/cmd/mesh-vault/store.go b/modules/mesh-vault/cmd/mesh-vault/store.go new file mode 100644 index 00000000..192c60fe --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/store.go @@ -0,0 +1,336 @@ +package main + +// The vault's own store (novox/hq ADR 0349 §7, ADR 0113 §22): one file per secret on the vault's own disk, +// every value in it sealed — once to the vault's own key, the copy the vault opens to answer `get`, and once +// to the operator key, the copy the operator opens when the vault is lost (ADR 0113 §21). No database and no +// bucket: the vault is installed before any other module (ADR 0113 §14), a database login would be a +// secret only the vault makes, and a bucket holds no secret, sealed or not (ADR 0201 §13). +// +// Each file is written whole and renamed into place, so a reader never sees half a record. + +import ( + "encoding/json" + "errors" + "fmt" + "net/url" + "os" + "path/filepath" + "regexp" + "sort" + "strings" + "sync" + "time" + + "git.novox.be/novox/mesh-sdk/go/seal" +) + +// OperatorBox names the box an operator copy is sealed in: the NaCl anonymous box the operator key is made +// for today (`operator key make`) and `secret recover --from-export` opens. +const OperatorBox = "nacl-anonymous-box" + +// Record is one secret as the store keeps it: never its value. +type Record struct { + Name string `json:"name"` + Generation int `json:"generation"` + // AtRest is the value sealed to the vault's own key (the ADR 0206 box). + AtRest seal.Box `json:"at-rest"` + // Operator is the value sealed to the operator key. + Operator OperatorCopy `json:"operator"` + MadeAt time.Time `json:"made-at"` + ChangedAt time.Time `json:"changed-at"` + ChangedBy string `json:"changed-by"` +} + +// OperatorCopy is a value sealed to the operator key named in Key. +type OperatorCopy struct { + Box string `json:"box"` + Key string `json:"key"` + Sealed string `json:"sealed"` +} + +// Store is the vault's directory of secrets. +type Store struct { + dir string + // failWrite, when a test sets it, fails the write of a record: how a failure partway is proven. + failWrite func(Record) error + // HandedMax is the size at which the record of hand-outs is rolled (0: HandedMaxDefault). + HandedMax int64 + + handedMu sync.Mutex + rolled int +} + +// The record of hand-outs is bounded: rolled at HandedMaxDefault into handed.jsonl.1 … .HandedKept, the +// oldest dropped, so a flood of asks costs at most (HandedKept+1) × HandedMaxDefault on the vault's disk. +const ( + HandedMaxDefault = 1 << 20 + HandedKept = 3 +) + +// OpenStore opens (and makes) the store at dir. +func OpenStore(dir string) (*Store, error) { + if dir == "" { + return nil, errors.New("the vault's store has no directory: MESH_VAULT_STORE is not set") + } + if err := os.MkdirAll(filepath.Join(dir, "secrets"), 0o700); err != nil { + return nil, err + } + return &Store{dir: dir}, nil +} + +var secretName = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]*(/[a-z0-9][a-z0-9._-]*)*$`) + +// validName says why name is not a secret's name, or nil. A name is words joined by `/`, never a path. +func validName(name string) error { + if len(name) == 0 || len(name) > 200 || !secretName.MatchString(name) { + return fmt.Errorf("%q is not a secret's name: lower-case words joined by /, at most 200 characters", name) + } + return nil +} + +func (s *Store) pathOf(name string) string { + return filepath.Join(s.dir, "secrets", url.PathEscape(name)+".json") +} + +// Read is the record of one secret, and whether there is one. +func (s *Store) Read(name string) (Record, bool, error) { + if err := validName(name); err != nil { + return Record{}, false, err + } + raw, err := os.ReadFile(s.pathOf(name)) + if errors.Is(err, os.ErrNotExist) { + return Record{}, false, nil + } + if err != nil { + return Record{}, false, err + } + var r Record + if err := json.Unmarshal(raw, &r); err != nil { + return Record{}, false, fmt.Errorf("the record of %s is not readable: %w", name, err) + } + return r, true, nil +} + +// Write keeps one record, replacing any earlier one. +func (s *Store) Write(r Record) error { + if err := validName(r.Name); err != nil { + return err + } + if s.failWrite != nil { + if err := s.failWrite(r); err != nil { + return err + } + } + raw, err := json.MarshalIndent(r, "", " ") + if err != nil { + return err + } + return writeWhole(s.pathOf(r.Name), append(raw, '\n')) +} + +// Empty says whether the store holds no secret at all. +func (s *Store) Empty() (bool, error) { + all, err := s.List() + return len(all) == 0, err +} + +// Handing is one answer to `get`, or one refusal, as the store records it (as ADR 0277 §25 records asks): +// who asked, for what, when, and what it got — never the value. +type Handing struct { + At time.Time `json:"at"` + To string `json:"to"` + Name string `json:"name"` + Generation int `json:"generation,omitempty"` + Outcome string `json:"outcome"` + Why string `json:"why,omitempty"` +} + +// RecordHanding appends one handing to the store's record, synced before it returns. It says whether the +// record was rolled to make room. +func (s *Store) RecordHanding(h Handing) (rolled bool, err error) { + raw, err := json.Marshal(h) + if err != nil { + return false, err + } + s.handedMu.Lock() + defer s.handedMu.Unlock() + path := filepath.Join(s.dir, "handed.jsonl") + max := s.HandedBound() + if info, err := os.Stat(path); err == nil && info.Mode().IsRegular() && info.Size()+int64(len(raw))+1 > max { + if err := s.roll(path); err != nil { + return false, err + } + rolled = true + } + return rolled, appendSynced(path, append(raw, '\n')) +} + +// roll moves handed.jsonl to .1, .1 to .2 and so on, dropping what falls past HandedKept. +func (s *Store) roll(path string) error { + _ = os.Remove(fmt.Sprintf("%s.%d", path, HandedKept)) + for i := HandedKept - 1; i >= 1; i-- { + if err := os.Rename(fmt.Sprintf("%s.%d", path, i), fmt.Sprintf("%s.%d", path, i+1)); err != nil && !errors.Is(err, os.ErrNotExist) { + return err + } + } + if err := os.Rename(path, path+".1"); err != nil { + return err + } + s.rolled++ + return syncDir(s.dir) +} + +// Rolled is how many times the record of hand-outs was rolled since the vault started. +func (s *Store) Rolled() int { + s.handedMu.Lock() + defer s.handedMu.Unlock() + return s.rolled +} + +func appendSynced(path string, raw []byte) error { + f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600) + if err != nil { + return err + } + if _, err := f.Write(raw); err != nil { + f.Close() + return err + } + if err := f.Sync(); err != nil { + f.Close() + return err + } + return f.Close() +} + +// Handed is the latest handings, newest first, at most limit: the current record, then the rolled ones. +func (s *Store) Handed(limit int) ([]Handing, error) { + s.handedMu.Lock() + defer s.handedMu.Unlock() + path := filepath.Join(s.dir, "handed.jsonl") + out := []Handing{} + for i := 0; i <= HandedKept && len(out) < limit; i++ { + file := path + if i > 0 { + file = fmt.Sprintf("%s.%d", path, i) + } + raw, err := os.ReadFile(file) + if errors.Is(err, os.ErrNotExist) { + continue + } + if err != nil { + return nil, err + } + lines := strings.Split(strings.TrimSpace(string(raw)), "\n") + for j := len(lines) - 1; j >= 0 && len(out) < limit; j-- { + var h Handing + if json.Unmarshal([]byte(lines[j]), &h) == nil { + out = append(out, h) + } + } + } + return out, nil +} + +// HandedBound is the size at which the record of hand-outs is rolled. +func (s *Store) HandedBound() int64 { + if s.HandedMax > 0 { + return s.HandedMax + } + return HandedMaxDefault +} + +// List is every record, by name. +func (s *Store) List() ([]Record, error) { + entries, err := os.ReadDir(filepath.Join(s.dir, "secrets")) + if err != nil { + return nil, err + } + var out []Record + for _, e := range entries { + if !strings.HasSuffix(e.Name(), ".json") { + continue + } + name, err := url.PathUnescape(strings.TrimSuffix(e.Name(), ".json")) + if err != nil { + continue + } + r, ok, err := s.Read(name) + if err != nil { + return nil, err + } + if ok { + out = append(out, r) + } + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return out, nil +} + +type operatorFile struct { + Key string `json:"key"` + Since time.Time `json:"since"` + By string `json:"by"` +} + +// OperatorKey is the operator's public key the store seals to, or "" when it has none. +func (s *Store) OperatorKey() (string, error) { + raw, err := os.ReadFile(filepath.Join(s.dir, "operator.json")) + if errors.Is(err, os.ErrNotExist) { + return "", nil + } + if err != nil { + return "", err + } + var f operatorFile + if err := json.Unmarshal(raw, &f); err != nil { + return "", err + } + return f.Key, nil +} + +func (s *Store) setOperatorKey(key, by string, at time.Time) error { + raw, _ := json.MarshalIndent(operatorFile{Key: key, Since: at, By: by}, "", " ") + return writeWhole(filepath.Join(s.dir, "operator.json"), append(raw, '\n')) +} + +// writeWhole writes a file readable by the vault alone, synced, then renames it into place and syncs the +// directory, so a crash leaves the old file or the new one, never neither. +func writeWhole(path string, raw []byte) error { + tmp := fmt.Sprintf("%s.%d.tmp", path, os.Getpid()) + if err := writeSynced(tmp, raw); err != nil { + return err + } + if err := os.Rename(tmp, path); err != nil { + _ = os.Remove(tmp) + return err + } + return syncDir(filepath.Dir(path)) +} + +// writeSynced writes a new file at 0600 and syncs it to disk. +func writeSynced(path string, raw []byte) error { + f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return err + } + if _, err := f.Write(raw); err != nil { + f.Close() + return err + } + if err := f.Sync(); err != nil { + f.Close() + return err + } + return f.Close() +} + +// syncDir syncs a directory, so a rename or a link in it is on disk. +func syncDir(dir string) error { + d, err := os.Open(dir) + if err != nil { + return err + } + defer d.Close() + return d.Sync() +} diff --git a/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-state.json b/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-state.json new file mode 100644 index 00000000..600bf79f --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-state.json @@ -0,0 +1,56 @@ +{ + "resource": "secret", + "node": "novox", + "held": [ + "a" + ], + "held_sizes": { + "a": 1 + }, + "stable_passes": 5, + "stable_for_seconds": 600, + "retires_by": "mark-only", + "bound": "b", + "waiting": { + "consumers": [ + { + "consumer": "c", + "node": "n", + "retired_at": "t", + "why": "w", + "size_bytes": -1, + "kind": "consumer", + "access": "kept" + } + ], + "since": "t", + "held": 1 + }, + "rejected": { + "consumers": [ + { + "consumer": "c", + "node": "n", + "retired_at": "t", + "why": "w", + "size_bytes": -1, + "kind": "consumer", + "access": "kept" + } + ], + "by": "b", + "why": "w", + "at": "t" + }, + "retired": [ + { + "consumer": "c", + "node": "n", + "retired_at": "t", + "why": "w", + "size_bytes": -1, + "kind": "consumer", + "access": "kept" + } + ] +} diff --git a/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-word.json b/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-word.json new file mode 100644 index 00000000..57792616 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/testdata/retirement-word.json @@ -0,0 +1,23 @@ +{ + "provider": "secret", + "provider-node": "novox", + "change": "waiting", + "consumers": [ + { + "consumer": "c", + "node": "n", + "retired_at": "t", + "why": "w", + "size_bytes": -1, + "kind": "consumer", + "access": "kept" + } + ], + "held": 1, + "bound": "b", + "why": "w", + "by": "person.x", + "via": "mesh-controller", + "at": "2026-10-14T09:00:00Z", + "since": "2026-10-14T09:00:00Z" +} diff --git a/modules/mesh-vault/cmd/mesh-vault/tools.go b/modules/mesh-vault/cmd/mesh-vault/tools.go new file mode 100644 index 00000000..1bd56770 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/tools.go @@ -0,0 +1,399 @@ +package main + +// What the vault serves: the seat's verb `get`, and the three tools the TypeScript vault offered — +// secret_export, secret_holders and secret_verify — answering as they did; beside them secret_handed and +// secret_vault_export over the vault's own store, and ADR 0230's four retirement tools over the custody +// ledger, and vault_status. None returns a value: `get` returns one sealed to the asker's key, the others +// fingerprints and ciphertext only. + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + "sync" + + stdio "git.novox.be/novox/mesh-sdk/go" +) + +// Env is what the tools work on. A part the module's words did not give is nil, and a tool that needs it +// says so. +type Env struct { + Vault *Vault + Ledger *Ledger + Receives string + Root string + Custody *Custody + // refused is why the operator key in the mesh's export was not taken, or empty. + mu sync.Mutex + refused string + // Broken is why the vault could not be built, said by `get`. + Broken error +} + +func (e *Env) keyRefused() string { + e.mu.Lock() + defer e.mu.Unlock() + return e.refused +} + +func (e *Env) refuseKey(why string) { + e.mu.Lock() + defer e.mu.Unlock() + e.refused = why +} + +// toolDef is a tool with its caller given apart from its arguments, so a test can be the runtime. +type toolDef struct { + Name string + Description string + Input map[string]any + run func(caller string, a map[string]any) (any, error) +} + +// Run answers a call the runtime made: the caller is the one it proved. +func (t toolDef) Run(a map[string]any) (any, error) { return t.run(stdio.CallerOf(a), a) } + +func (t toolDef) runAs(caller string, a map[string]any) (any, error) { return t.run(caller, a) } + +func str(description string) map[string]any { + return map[string]any{"type": "string", "description": description} +} + +func definitions(env *Env) []toolDef { + return []toolDef{ + {Name: Seat + ".get", + Description: "One secret, sealed to the public key the asker sends (the sealed box of ADR 0206), with its " + + "generation; answered only to an asker the vault allows, as the runtime proved it (ADR 0349 §11). " + + "Never the value in the clear, and never to a private key.", + Input: map[string]any{"name": str("the secret's name"), + "public_key": str("the asker's X25519 public key as SPKI PEM — never a private key")}, + run: func(caller string, a map[string]any) (any, error) { + if env.Vault == nil { + if env.Broken != nil { + return nil, env.Broken + } + return nil, errors.New("the vault is not ready") + } + name, _ := a["name"].(string) + key, _ := a["public_key"].(string) + return env.Vault.Get(caller, strings.TrimSpace(name), key) + }}, + {Name: "secret_export", + Description: "The mesh's root secrets as this vault keeps them: every secret a module holds for itself, " + + "sealed to the operator's key (novox/hq ADR 0085, amended). Ciphertext — nothing here can open a line " + + "of it; the operator, holding the private key off the mesh, recovers one with " + + "`nox secret recover --from-export` on the control-node. Also lists what is NOT recoverable: secrets " + + "made before the mesh had an operator key.", + Input: map[string]any{"sealed": map[string]any{"type": "boolean", + "description": "include the sealed blobs (default true); false lists holders and the key only"}}, + run: func(_ string, a map[string]any) (any, error) { + return exportAnswer(env.Root, a["sealed"] != false), nil + }}, + {Name: "secret_holders", + Description: "Who holds a secret from this vault: each consumer's login, node and module, when it was " + + "granted, how many times it has been rotated and when, and the fingerprint of the current value. " + + "Fingerprints only — the value is never returned.", + run: func(string, map[string]any) (any, error) { + if env.Ledger == nil { + return map[string]any{"holders": []any{}, "count": 0, "error": "the vault has no ledger (MESH_VAULT_LEDGER is not set)"}, nil + } + var asked []Contribution + if env.Receives != "" { + asked, _ = Contributions(env.Receives) + } + holders := []map[string]any{} + for _, h := range env.Ledger.List() { + row := map[string]any{} + raw, _ := json.Marshal(h) + _ = json.Unmarshal(raw, &row) + row["module"], row["asked"] = nil, false + for _, c := range asked { + if c.As == h.As { + row["module"], row["asked"] = c.From, true + } + } + holders = append(holders, row) + } + return map[string]any{"holders": holders, "count": len(holders)}, nil + }}, + {Name: "secret_verify", + Description: "Check one holder's secret without seeing it: the fingerprint the vault recorded against the " + + "fingerprint of the value the mesh currently delivers here, and optionally against a fingerprint " + + "computed on the holder's own machine (sha256 of the file, as `sha256:`). Two ends agreeing " + + "proves they agree, not that either works — the login itself is the test.", + Input: map[string]any{"as": str("the holder's login, e.g. anchor-redis"), + "fingerprint": str("optional: sha256: of the value as the holder reads it, computed there — never the value")}, + run: func(_ string, a map[string]any) (any, error) { + as, _ := a["as"].(string) + given, _ := a["fingerprint"].(string) + return verifyAnswer(env, as, given), nil + }}, + {Name: "secret_handed", + Description: "Who was handed which secret by `get`, and when, and who was refused, newest first, as the " + + "vault's own store records it. Never a value.", + Input: map[string]any{"limit": map[string]any{"type": "number", "description": "how many (default 50)"}}, + run: func(_ string, a map[string]any) (any, error) { + if env.Vault == nil { + return nil, errors.New("the vault is not ready") + } + return env.Vault.Store.Handed(limitOf(a)) + }}, + {Name: "secret_vault_export", + Description: "Every secret the vault itself holds, as its copy sealed to the operator key, in the shape of " + + "`secret export`: saved to a file, `nox secret recover vault mesh-vault --key " + + "--from-export --out ` opens one with the operator's private key. Ciphertext only.", + run: func(string, map[string]any) (any, error) { + if env.Vault == nil { + return nil, errors.New("the vault is not ready") + } + return vaultExport(env.Vault) + }}, + {Name: "provisioner_retirement", + Description: "This provider's retirement (novox/hq ADR 0230): the holders it keeps active, a retirement waiting " + + "for a person, one a person rejected, and every holder retired with when and why. Mark-only: a retired " + + "holder keeps what it has until a person deletes it.", + run: func(string, map[string]any) (any, error) { + if env.Custody == nil { + return nil, errors.New("the vault has no custody ledger") + } + return env.Custody.Retirement(), nil + }}, + {Name: "provisioner_retire_approve", + Description: "Retire exactly the set waiting for a person (or rejected earlier). Repeat the set in consumers; why is required.", + Input: actInput(true, false), + run: func(caller string, a map[string]any) (any, error) { + if env.Custody == nil { + return nil, errors.New("the vault has no custody ledger") + } + done, err := env.Custody.Approve(stringsOf(a["consumers"]), actOf(caller, a)) + if err != nil { + return nil, err + } + return map[string]any{"retired": done}, nil + }}, + {Name: "provisioner_retire_reject", + Description: "Keep the set waiting for a person active, though the mesh no longer asks for it. Repeat the set in consumers; why is required.", + Input: actInput(true, false), + run: func(caller string, a map[string]any) (any, error) { + if env.Custody == nil { + return nil, errors.New("the vault has no custody ledger") + } + kept, err := env.Custody.Reject(stringsOf(a["consumers"]), actOf(caller, a)) + if err != nil { + return nil, err + } + return map[string]any{"kept": kept}, nil + }}, + {Name: "provisioner_delete", + Description: "Delete one RETIRED holder from the custody ledger, its rotation history with it — never an active one. " + + "Repeat its name in confirm; why is required.", + Input: actInput(false, true), + run: func(caller string, a map[string]any) (any, error) { + if env.Custody == nil { + return nil, errors.New("the vault has no custody ledger") + } + consumer, _ := a["consumer"].(string) + confirm, _ := a["confirm"].(string) + if err := env.Custody.Delete(consumer, confirm, actOf(caller, a)); err != nil { + return nil, err + } + return map[string]any{"deleted": consumer, "freed_bytes": -1}, nil + }}, + {Name: "vault_status", + Description: "What needs a person's eye in the vault: operator copies still sealed to an earlier key, the operator " + + "key the mesh's export holds if the vault refused it, and how often the record of hand-outs was rolled.", + run: func(string, map[string]any) (any, error) { + if env.Vault == nil { + return map[string]any{"ready": false, "error": fmt.Sprint(env.Broken)}, nil + } + left, err := env.Vault.StillSealedToAnother() + if err != nil { + return nil, err + } + key, _ := env.Vault.Store.OperatorKey() + return map[string]any{"ready": true, "operator_key": key, "copies_on_an_earlier_key": left, + "operator_key_refused": env.keyRefused(), "handed_rolled": env.Vault.Store.Rolled(), + "ok": left == 0 && env.keyRefused() == "" && key != ""}, nil + }}, + } +} + +// actInput is the arguments of an act on retirement, as the SDK's harness takes them. +func actInput(set, one bool) map[string]any { + text := map[string]any{"type": "string"} + in := map[string]any{"why": text, "by": text, "via": text} + if set { + in["consumers"] = map[string]any{"type": "array", "items": text} + } + if one { + in["consumer"], in["confirm"] = text, text + } + return in +} + +// actOf is an act's why, by and via. `by` is the person the controller names; with none, the caller the +// runtime proved. +func actOf(caller string, a map[string]any) Act { + act := Act{} + act.Why, _ = a["why"].(string) + act.By, _ = a["by"].(string) + act.Via, _ = a["via"].(string) + if act.By == "" { + act.By = caller + } + return act +} + +func stringsOf(v any) []string { + list, _ := v.([]any) + out := []string{} + for _, x := range list { + if s, ok := x.(string); ok { + out = append(out, s) + } + } + return out +} + +func limitOf(a map[string]any) int { + if v, ok := a["limit"].(float64); ok && v > 0 { + return int(v) + } + return 50 +} + +// keyFingerprint is a key's fingerprint as the controller's secrets.Fingerprint says it: sha256 of the key, +// its first eight bytes in hex. +func keyFingerprint(key string) string { + if key == "" { + return "" + } + sum := sha256.Sum256([]byte(strings.TrimSpace(key))) + return "sha256:" + hex.EncodeToString(sum[:8]) +} + +// vaultExport is the vault's operator copies in the shape `secret export` writes and `secret recover +// --from-export` reads: each as node "vault", module "mesh-vault" and the secret's name. +func vaultExport(v *Vault) (map[string]any, error) { + key, err := v.Store.OperatorKey() + if err != nil { + return nil, err + } + all, err := v.Store.List() + if err != nil { + return nil, err + } + kept, earlier := []map[string]any{}, []map[string]any{} + for _, rec := range all { + k := map[string]any{"node": "vault", "module": Seat, "name": rec.Name, "kind": "own", "origin": "made", + "sealed": rec.Operator.Sealed, "key": rec.Operator.Key, "made-at": rec.MadeAt} + if rec.Operator.Key == key { + kept = append(kept, k) + } else { + earlier = append(earlier, k) + } + } + return map[string]any{"export": 1, "operator-key": key, "fingerprint": keyFingerprint(key), "kept": kept, + "sealed-to-earlier-key": earlier}, nil +} + +func exportAnswer(root string, withBlobs bool) map[string]any { + if root == "" { + return map[string]any{"available": false, "error": "this vault keeps no root secrets (MESH_VAULT_ROOT is not set)"} + } + raw, err := os.ReadFile(filepath.Join(root, "export.json")) + if err != nil { + return map[string]any{"available": false, "error": "the mesh has not written an export here yet: " + err.Error()} + } + var doc struct { + Export int `json:"export"` + OperatorKey string `json:"operator-key"` + Fingerprint string `json:"fingerprint"` + Kept []map[string]any `json:"kept"` + Unrecoverable []map[string]any `json:"unrecoverable"` + } + if err := json.Unmarshal(raw, &doc); err != nil { + return map[string]any{"available": false, "error": "the export is not readable: " + err.Error()} + } + kept := []map[string]any{} + for _, k := range doc.Kept { + if !withBlobs { + k = map[string]any{"node": k["node"], "module": k["module"], "name": k["name"], "origin": k["origin"], "made-at": k["made-at"]} + } + kept = append(kept, k) + } + if doc.Unrecoverable == nil { + doc.Unrecoverable = []map[string]any{} + } + return map[string]any{"available": true, "export": doc.Export, "operator-key": doc.OperatorKey, + "fingerprint": doc.Fingerprint, "count": len(kept), "kept": kept, "unrecoverable": doc.Unrecoverable} +} + +// exportedOperatorKey is the operator key in the export the mesh writes to the vault's root, or "". +func exportedOperatorKey(root string) string { + raw, err := os.ReadFile(filepath.Join(root, "export.json")) + if err != nil { + return "" + } + var doc struct { + OperatorKey string `json:"operator-key"` + } + _ = json.Unmarshal(raw, &doc) + return doc.OperatorKey +} + +func verifyAnswer(env *Env, as, given string) map[string]any { + if env.Ledger == nil { + return map[string]any{"as": as, "known": false, "error": "the vault has no ledger (MESH_VAULT_LEDGER is not set)"} + } + recorded, ok := env.Ledger.Get(as) + if !ok { + return map[string]any{"as": as, "known": false, "error": "this vault holds nothing for " + as} + } + var delivered, deliveredErr any + var deliveredMatches any + var asked *Contribution + if env.Receives != "" { + all, _ := Contributions(env.Receives) + for i := range all { + if all[i].As == as { + asked = &all[i] + } + } + } + if asked == nil { + deliveredErr = "the mesh does not currently ask this vault to serve that login" + } else if fp, _, err := DeliveredFingerprint(*asked); err != nil { + deliveredErr = err.Error() + } else { + delivered, deliveredMatches = fp, fp == recorded.Fingerprint + } + var givenAny, givenMatches, earlier any + if given != "" { + givenAny, givenMatches = given, given == recorded.Fingerprint + was := false + for _, h := range recorded.History { + was = was || h.Fingerprint == given + } + earlier = was + } + return map[string]any{"as": recorded.As, "known": true, "recorded": recorded.Fingerprint, + "rotations": recorded.Rotations, "changed": recorded.Changed, + "delivered": delivered, "deliveredError": deliveredErr, "deliveredMatchesRecorded": deliveredMatches, + "given": givenAny, "givenMatchesRecorded": givenMatches, "givenIsAnEarlierValue": earlier, + "ok": deliveredMatches != false && givenMatches != false} +} + +// tools are the definitions as the SDK serves them. +func tools(env *Env) []stdio.Tool { + var out []stdio.Tool + for _, d := range definitions(env) { + out = append(out, stdio.Tool{Name: d.Name, Description: d.Description, Input: d.Input, Run: d.Run}) + } + return out +} diff --git a/modules/mesh-vault/cmd/mesh-vault/tools_test.go b/modules/mesh-vault/cmd/mesh-vault/tools_test.go new file mode 100644 index 00000000..dee060cb --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/tools_test.go @@ -0,0 +1,152 @@ +package main + +// The tools the TypeScript vault offered, answering as they did: secret_export, secret_holders and +// secret_verify — none returns a value. And the seat's verb `get`, reached the way the runtime calls it. + +import ( + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "git.novox.be/novox/mesh-sdk/go/seal" +) + +func toolNamed(t *testing.T, tools []toolDef, name string) toolDef { + t.Helper() + for _, tl := range tools { + if tl.Name == name { + return tl + } + } + t.Fatalf("no tool %s", name) + return toolDef{} +} + +func TestTheManifestNamesWhatTheBundleServes(t *testing.T) { + raw, _ := os.ReadFile("../../module.json") + var m struct { + Tools []string `json:"tools"` + Claims []struct { + Name string `json:"name"` + Serves []string `json:"serves"` + } `json:"claims"` + Emits []string `json:"emits"` + } + if err := json.Unmarshal(raw, &m); err != nil { + t.Fatal(err) + } + served := map[string]bool{} + for _, tl := range definitions(&Env{}) { + served[tl.Name] = true + } + for _, name := range m.Tools { + if !served[name] { + t.Errorf("the manifest names the tool %s, which the bundle does not serve", name) + } + } + for _, c := range m.Claims { + for _, verb := range c.Serves { + if !served[c.Name+"."+verb] { + t.Errorf("the claim of %s serves %s, which the bundle does not", c.Name, verb) + } + } + } + for name := range served { + if !strings.HasPrefix(name, Seat+".") && !contains(m.Tools, name) { + t.Errorf("the bundle serves %s, which the manifest does not name", name) + } + } + if !contains(m.Emits, ChangedEvent) { + t.Errorf("the manifest does not say the vault emits %s", ChangedEvent) + } +} + +func TestSecretExportAnswersTheMeshsExport(t *testing.T) { + root := t.TempDir() + env := &Env{Root: root} + export := toolNamed(t, definitions(env), "secret_export") + got, _ := export.Run(map[string]any{}) + if got.(map[string]any)["available"] != false { + t.Errorf("no export said %v", got) + } + doc := `{"export":1,"operator-key":"b3A=","fingerprint":"sha256:f","kept":[{"node":"g14","module":"dunst","name":"token","kind":"own","origin":"made","sealed":"c2VhbGVk","key":"b3A=","made-at":"2026-10-01T00:00:00Z"}],"unrecoverable":[{"node":"ace","module":"x","name":"y"}]}` + _ = os.WriteFile(filepath.Join(root, "export.json"), []byte(doc), 0o600) + got, _ = export.Run(map[string]any{}) + a := got.(map[string]any) + if a["available"] != true || a["count"] != 1 || a["operator-key"] != "b3A=" { + t.Errorf("the export said %v", a) + } + got, _ = export.Run(map[string]any{"sealed": false}) + kept := got.(map[string]any)["kept"].([]map[string]any) + if _, ok := kept[0]["sealed"]; ok { + t.Error("sealed=false still returned the blobs") + } +} + +func TestSecretHoldersAndVerifySayFingerprintsOnly(t *testing.T) { + dir := t.TempDir() + l, _ := NewLedger(filepath.Join(dir, "ledger")) + secret := filepath.Join(dir, "s") + _ = os.WriteFile(secret, []byte("the-value"), 0o600) + receives := filepath.Join(dir, "mesh.json") + doc, _ := json.Marshal(map[string]any{"given": []map[string]string{{"from": "redis", "node": "anchor", "as": "anchor-redis", "secret": secret}}}) + _ = os.WriteFile(receives, doc, 0o600) + _, _, _ = l.Record("anchor-redis", "anchor", "the-value", time.Now()) + env := &Env{Ledger: l, Receives: receives} + + holders, _ := toolNamed(t, definitions(env), "secret_holders").Run(map[string]any{}) + verify, _ := toolNamed(t, definitions(env), "secret_verify").Run(map[string]any{"as": "anchor-redis", "fingerprint": Fingerprint("the-value")}) + unknown, _ := toolNamed(t, definitions(env), "secret_verify").Run(map[string]any{"as": "nobody"}) + for _, answer := range []any{holders, verify, unknown} { + raw, _ := json.Marshal(answer) + if strings.Contains(string(raw), "the-value") { + t.Errorf("a tool returned the value: %s", raw) + } + } + v := verify.(map[string]any) + if v["ok"] != true || v["deliveredMatchesRecorded"] != true || v["givenMatchesRecorded"] != true { + t.Errorf("verify said %v", v) + } + h := holders.(map[string]any) + if h["count"] != 1 { + t.Errorf("holders said %v", h) + } + if unknown.(map[string]any)["known"] != false { + t.Errorf("an unknown login said %v", unknown) + } +} + +// The seat's verb, called as the runtime calls it: the caller is the one the runtime proved, never an +// argument. +func TestTheGetVerbTakesItsCallerFromTheRuntime(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + get := toolNamed(t, definitions(&Env{Vault: v.Vault}), Seat+".get") + // No runtime proved a caller here, so the call is unproven whatever the arguments say. + if _, err := get.Run(map[string]any{"name": "pair/postgres/gitea", "public_key": asker.PublicKey, "caller": "g14.node-provisioner"}); err == nil { + t.Error("get answered a caller named in its arguments") + } + answer, err := get.runAs("g14.node-provisioner", map[string]any{"name": "pair/postgres/gitea", "public_key": asker.PublicKey}) + if err != nil { + t.Fatal(err) + } + a := answer.(Answer) + if got, err := seal.Open(a.Sealed, asker.PrivateKey); err != nil || got != "x" { + t.Errorf("%q %v", got, err) + } +} + +func contains(list []string, s string) bool { + for _, x := range list { + if x == s { + return true + } + } + return false +} diff --git a/modules/mesh-vault/cmd/mesh-vault/vault.go b/modules/mesh-vault/cmd/mesh-vault/vault.go new file mode 100644 index 00000000..2ba4ec53 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/vault.go @@ -0,0 +1,301 @@ +package main + +// The vault's rules (novox/hq ADR 0349 §7 and §11; ADR 0113 §1, §2, §22): it holds every secret sealed at +// rest, with a copy sealed to the operator key; it hands one out only on request and reply, sealed to the +// asker's key, and only to an asker it allows; it says a secret changed — who, when, never a value. + +import ( + "crypto/rand" + "encoding/base64" + "errors" + "fmt" + "regexp" + "sync" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" + "git.novox.be/novox/mesh-sdk/go/seal" +) + +// Seat is the role this module holds. +const Seat = "mesh-vault" + +// ChangedEvent is what the vault raises when a secret it holds changes: who, when, which, its generation. +const ChangedEvent = "secret-changed" + +// Vault holds the secrets. +type Vault struct { + Store *Store + // Own is the vault's own key; the private half never leaves its state directory. + Own seal.KeyPair + // Askers decides who may ask `get` for what. Nil allows nobody. + Askers Askers + Emit func(event string, body map[string]any) error + Now func() time.Time + Log func(format string, args ...any) + + mu sync.Mutex +} + +// Changed is what one change to a secret was. Never its value. +type Changed struct { + Name string `json:"name"` + Generation int `json:"generation"` + At time.Time `json:"at"` + By string `json:"by"` + Unchanged bool `json:"unchanged,omitempty"` +} + +// Answer is `get`'s reply: the secret sealed to the asker's key, and which generation it is. +type Answer struct { + Name string `json:"name"` + Generation int `json:"generation"` + ChangedAt time.Time `json:"changed-at"` + Sealed seal.Box `json:"sealed"` +} + +func (v *Vault) log(format string, args ...any) { + if v.Log != nil { + v.Log(format, args...) + } +} + +// Put keeps value as the secret name: sealed to the vault's key and to the operator key, a new generation +// when it differs from the one held, and `secret-changed` said. The same value again changes nothing. +// Refused: a private key (it stays where it was made), and any value while the vault has no operator key — +// a secret only the vault could open is what ADR 0113 §22 rules out. +func (v *Vault) Put(name, value, by string) (Changed, error) { + if err := validName(name); err != nil { + return Changed{}, err + } + if value == "" { + return Changed{}, fmt.Errorf("%s: a secret is not empty", name) + } + if privateKeyIn(value) { + return Changed{}, fmt.Errorf("%s: that is a private key, and a private key stays where it was made and never reaches the vault (ADR 0113 §2)", name) + } + if by == "" { + by = "the vault" + } + v.mu.Lock() + defer v.mu.Unlock() + operator, err := v.Store.OperatorKey() + if err != nil { + return Changed{}, err + } + if operator == "" { + return Changed{}, fmt.Errorf("%s is not kept: the vault has no operator key to seal its copy to (ADR 0113 §22) — the mesh writes it with the export once `operator key set` has been run", name) + } + before, held, err := v.Store.Read(name) + if err != nil { + return Changed{}, err + } + if held { + if old, err := seal.Open(before.AtRest, v.Own.PrivateKey); err == nil && old == value { + return Changed{Name: name, Generation: before.Generation, At: before.ChangedAt, By: before.ChangedBy, Unchanged: true}, nil + } + } + atRest, err := seal.Seal(value, v.Own.PublicKey) + if err != nil { + return Changed{}, err + } + forOperator, err := operatorSeal(operator, []byte(value)) + if err != nil { + return Changed{}, err + } + now := v.Now().UTC() + rec := Record{Name: name, Generation: 1, AtRest: atRest, + Operator: OperatorCopy{Box: OperatorBox, Key: operator, Sealed: forOperator}, + MadeAt: now, ChangedAt: now, ChangedBy: by} + if held { + rec.Generation, rec.MadeAt = before.Generation+1, before.MadeAt + } + if err := v.Store.Write(rec); err != nil { + return Changed{}, err + } + c := Changed{Name: name, Generation: rec.Generation, At: now, By: by} + v.announce(c) + return c, nil +} + +// Mint makes a fresh value for name and keeps it: 30 random bytes, base64url without padding — 40 +// characters, the controller's shape, which fits the tightest backend a minted password reaches. +func (v *Vault) Mint(name, by string) (Changed, error) { + raw := make([]byte, 30) + if _, err := rand.Read(raw); err != nil { + return Changed{}, err + } + return v.Put(name, base64.RawURLEncoding.EncodeToString(raw), by) +} + +// announce raises `secret-changed`: who, when, which and its generation — never a value or a sealed copy. +func (v *Vault) announce(c Changed) { + if v.Emit == nil { + return + } + body := map[string]any{"name": c.Name, "by": c.By, "at": c.At.Format(time.RFC3339Nano), "generation": c.Generation} + if err := v.Emit(ChangedEvent, body); err != nil { + v.log("%s is kept at generation %d, but %s could not be said: %v", c.Name, c.Generation, ChangedEvent, err) + } +} + +// recordHanding keeps who asked for what, and what they got, in the vault's own store, and says loudly +// when the record had to be rolled to make room. +func (v *Vault) recordHanding(h Handing) error { + h.At = v.Now().UTC() + rolled, err := v.Store.RecordHanding(h) + if err != nil { + v.log("the record of %s asking for %s could not be written: %v", h.To, h.Name, err) + return err + } + if rolled { + v.log("THE RECORD OF HAND-OUTS WAS ROLLED (%d times since the vault started): it reached its bound of %d bytes; "+ + "the oldest of %d kept files is dropped. Many asks in little time is worth a look: secret_handed and vault_status", + v.Store.Rolled(), v.Store.HandedBound(), HandedKept) + } + return nil +} + +// busUser is the shape of a caller the runtime proves (the SDK's): the one shape a refusal is recorded for. +var busUser = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`) + +// Get answers caller with the secret name sealed to publicKey, if caller may ask for it. The caller is the +// bus user the runtime proved, never an argument; an unproven call is refused. +func (v *Vault) Get(caller, name, publicKey string) (Answer, error) { + if caller == "" || caller == stdio.Unproven { + return Answer{}, errors.New("the vault answers only a caller the runtime proved") + } + askers := v.Askers + if askers == nil { + askers = NoAsker{} + } + if !busUser.MatchString(caller) || len(caller) > 200 { + return Answer{}, errors.New("the vault answers only a caller the runtime proved") + } + // A name is judged before anything is recorded, so an arbitrary argument fills no record. + if err := validName(name); err != nil { + return Answer{}, err + } + if ok, why := askers.May(caller, name); !ok { + v.log("refused %s to %s: %s", name, caller, why) + _ = v.recordHanding(Handing{To: caller, Name: name, Outcome: "refused", Why: why}) + return Answer{}, fmt.Errorf("%s may not ask for %s: %s", caller, name, why) + } + key, err := askerKey(publicKey) + if err != nil { + return Answer{}, err + } + rec, held, err := v.Store.Read(name) + if err != nil { + return Answer{}, err + } + if !held { + return Answer{}, fmt.Errorf("the vault holds no secret named %s", name) + } + value, err := seal.Open(rec.AtRest, v.Own.PrivateKey) + if err != nil { + return Answer{}, fmt.Errorf("%s does not open with the vault's key: %w", name, err) + } + sealed, err := seal.Seal(value, key) + if err != nil { + return Answer{}, err + } + if err := v.recordHanding(Handing{To: caller, Name: name, Generation: rec.Generation, Outcome: "handed"}); err != nil { + return Answer{}, fmt.Errorf("%s is not handed out: who was handed it could not be recorded: %w", name, err) + } + v.log("handed %s generation %d to %s, sealed to its key", name, rec.Generation, caller) + return Answer{Name: name, Generation: rec.Generation, ChangedAt: rec.ChangedAt, Sealed: sealed}, nil +} + +// SetOperatorKey makes key (SPKI PEM) the operator key, then seals again to it every operator copy that is +// not yet — see Reseal. It returns how many copies it sealed again. +// +// The key is recorded first and each record says which key its copy is sealed to, so that record is the +// progress: a failure partway leaves the rest for the next call, and the same key given again (every minute +// from the mesh's export) finishes them. Nothing returns early while a copy is sealed to another key. +func (v *Vault) SetOperatorKey(key, by string) (int, error) { + raw, err := checkOperatorKey(key) + if err != nil { + return 0, err + } + v.mu.Lock() + defer v.mu.Unlock() + current, err := v.Store.OperatorKey() + if err != nil { + return 0, err + } + if current != raw { + if err := v.Store.setOperatorKey(raw, by, v.Now().UTC()); err != nil { + return 0, err + } + v.log("the operator key is now %s (by %s); every operator copy is sealed again to it", raw, by) + } + return v.reseal(raw) +} + +// Reseal seals again to the current operator key every copy sealed to another one, and says how many it +// did. Called every minute, so a copy a failure left behind is not left for ever. +func (v *Vault) Reseal() (int, error) { + v.mu.Lock() + defer v.mu.Unlock() + key, err := v.Store.OperatorKey() + if err != nil || key == "" { + return 0, err + } + return v.reseal(key) +} + +func (v *Vault) reseal(key string) (int, error) { + all, err := v.Store.List() + if err != nil { + return 0, err + } + n := 0 + var failed []error + for _, rec := range all { + if rec.Operator.Key == key && rec.Operator.Box == OperatorBox { + continue + } + value, err := seal.Open(rec.AtRest, v.Own.PrivateKey) + if err != nil { + failed = append(failed, fmt.Errorf("%s does not open with the vault's key: %w", rec.Name, err)) + continue + } + sealed, err := operatorSeal(key, []byte(value)) + if err == nil { + rec.Operator = OperatorCopy{Box: OperatorBox, Key: key, Sealed: sealed} + err = v.Store.Write(rec) + } + if err != nil { + failed = append(failed, fmt.Errorf("%s: %w", rec.Name, err)) + continue + } + n++ + } + if len(failed) > 0 { + err := fmt.Errorf("%d operator copies are still sealed to an earlier key, tried again on the next pass: %w", + len(failed), errors.Join(failed...)) + v.log("%v", err) + return n, err + } + return n, nil +} + +// StillSealedToAnother is how many operator copies are not yet sealed to the current operator key. +func (v *Vault) StillSealedToAnother() (int, error) { + key, err := v.Store.OperatorKey() + if err != nil { + return 0, err + } + all, err := v.Store.List() + if err != nil { + return 0, err + } + n := 0 + for _, rec := range all { + if rec.Operator.Key != key { + n++ + } + } + return n, nil +} diff --git a/modules/mesh-vault/cmd/mesh-vault/vault_test.go b/modules/mesh-vault/cmd/mesh-vault/vault_test.go new file mode 100644 index 00000000..6c7d43e0 --- /dev/null +++ b/modules/mesh-vault/cmd/mesh-vault/vault_test.go @@ -0,0 +1,730 @@ +package main + +// ADR 0349's vault test list, the part this package (V1, issue 674) answers: no bucket holds a secret; no +// private key is ever sent to the vault; every secret has a copy sealed to the operator key; `get` answers +// only an allowed asker, sealed to its key; `secret-changed` carries no value. The rest of that list (a +// consumer binding remakes the provider's credential, an accepted value is not remade, issued and typed +// values) is V2's and V5's. + +import ( + "crypto/ecdh" + "crypto/rand" + "crypto/x509" + "encoding/base64" + "encoding/json" + "encoding/pem" + "os" + "path/filepath" + "regexp" + "strings" + "sync" + "testing" + "time" + + stdio "git.novox.be/novox/mesh-sdk/go" + "git.novox.be/novox/mesh-sdk/go/seal" + "golang.org/x/crypto/nacl/box" +) + +// published is everything the vault said on the bus during a test, as it said it. +type published struct { + mu sync.Mutex + events []map[string]any + raw []string +} + +func (p *published) emit(event string, body map[string]any) error { + p.mu.Lock() + defer p.mu.Unlock() + b, _ := json.Marshal(map[string]any{"event": event, "body": body}) + p.raw = append(p.raw, string(b)) + p.events = append(p.events, map[string]any{"event": event, "body": body}) + return nil +} + +// operatorKey is an operator key the way the controller's `operator key make` makes one: NaCl, raw X25519, +// base64 — the private half kept by the test, as the operator keeps it off the mesh. +func operatorKey(t *testing.T) (public, private string) { + t.Helper() + pub, priv, err := box.GenerateKey(rand.Reader) + if err != nil { + t.Fatal(err) + } + return base64.StdEncoding.EncodeToString(pub[:]), base64.StdEncoding.EncodeToString(priv[:]) +} + +// asPEM is a raw operator key as the SPKI PEM the vault takes. +func asPEM(t *testing.T, raw string) string { + t.Helper() + p, err := publicPEM(raw) + if err != nil { + t.Fatal(err) + } + return p +} + +// openOperator opens an operator copy the way `secret recover --from-export` does. +func openOperator(t *testing.T, public, private, sealed string) string { + t.Helper() + var pub, priv [32]byte + p, _ := base64.StdEncoding.DecodeString(public) + q, _ := base64.StdEncoding.DecodeString(private) + copy(pub[:], p) + copy(priv[:], q) + blob, err := base64.StdEncoding.DecodeString(sealed) + if err != nil { + t.Fatal(err) + } + out, ok := box.OpenAnonymous(nil, blob, &pub, &priv) + if !ok { + t.Fatal("the operator copy does not open with the operator's key") + } + return string(out) +} + +type testVault struct { + *Vault + dir string + bus *published + operatorPublic string + operatorPrivate string +} + +func newVault(t *testing.T, askers Askers) testVault { + t.Helper() + dir := t.TempDir() + store, err := OpenStore(filepath.Join(dir, "store")) + if err != nil { + t.Fatal(err) + } + own, err := OwnKey(filepath.Join(dir, "state"), store) + if err != nil { + t.Fatal(err) + } + bus := &published{} + clock := time.Date(2026, 10, 14, 9, 0, 0, 0, time.UTC) + v := &Vault{Store: store, Own: own, Askers: askers, Emit: bus.emit, + Now: func() time.Time { clock = clock.Add(time.Second); return clock }, Log: t.Logf} + pub, priv := operatorKey(t) + if _, err := v.SetOperatorKey(asPEM(t, pub), "the test"); err != nil { + t.Fatal(err) + } + return testVault{Vault: v, dir: dir, bus: bus, operatorPublic: pub, operatorPrivate: priv} +} + +// filesUnder is every file's content under dir. +func filesUnder(t *testing.T, dir string) map[string]string { + t.Helper() + out := map[string]string{} + _ = filepath.Walk(dir, func(path string, info os.FileInfo, err error) error { + if err == nil && !info.IsDir() { + raw, _ := os.ReadFile(path) + out[path] = string(raw) + } + return nil + }) + return out +} + +// ---- no bucket holds a secret --------------------------------------------------------------------------- + +// The manifest declares no bucket, and the code opens none: the vault's only place for a secret is its own +// store on its own disk (ADR 0349 §7, ADR 0201 §13). +func TestNoBucketHoldsASecret(t *testing.T) { + raw, err := os.ReadFile("../../module.json") + if err != nil { + t.Fatal(err) + } + var manifest map[string]any + if err := json.Unmarshal(raw, &manifest); err != nil { + t.Fatal(err) + } + if s, ok := manifest["state"]; ok { + t.Errorf("the manifest declares buckets %v: a secret is in no bucket, sealed or not (ADR 0201 §13)", s) + } + sources, _ := filepath.Glob("*.go") + bucket := regexp.MustCompile(`stdio\.State\(|nats-io|jetstream|KeyValue`) + for _, f := range sources { + if strings.HasSuffix(f, "_test.go") { + continue + } + code, _ := os.ReadFile(f) + if m := bucket.Find(code); m != nil { + t.Errorf("%s opens a bucket (%s): the vault keeps every secret in its own store only", f, m) + } + } + + // And what the vault publishes while it holds, changes and hands out secrets carries none of them, in + // the clear or sealed. + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + value := "the-password-nobody-may-see" + if _, err := v.Put("pair/postgres/gitea", value, "the test"); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + answer, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", asker.PublicKey) + if err != nil { + t.Fatal(err) + } + rec, _, _ := v.Store.Read("pair/postgres/gitea") + for _, said := range v.bus.raw { + for _, secret := range []string{value, answer.Sealed.Ct, rec.AtRest.Ct, rec.Operator.Sealed} { + if strings.Contains(said, secret) { + t.Errorf("the vault published %s, which carries a secret or a sealed copy of one", said) + } + } + } +} + +// Every secret is encrypted at rest: no file the vault writes holds a value in the clear, and each opens +// only with the vault's own key. +func TestEverySecretIsEncryptedAtRest(t *testing.T) { + v := newVault(t, nil) + values := map[string]string{"own/g14/dunst/token": "tok-1-plain", "pair/postgres/gitea": "pw-2-plain"} + for name, value := range values { + if _, err := v.Put(name, value, "the test"); err != nil { + t.Fatal(err) + } + } + for path, content := range filesUnder(t, filepath.Join(v.dir, "store")) { + for _, value := range values { + if strings.Contains(content, value) { + t.Errorf("%s holds a value in the clear", path) + } + } + if info, _ := os.Stat(path); info.Mode().Perm()&0o077 != 0 { + t.Errorf("%s is readable by others: %v", path, info.Mode()) + } + } + for name, value := range values { + rec, ok, err := v.Store.Read(name) + if err != nil || !ok { + t.Fatalf("%s: %v %v", name, ok, err) + } + got, err := seal.Open(rec.AtRest, v.Own.PrivateKey) + if err != nil || got != value { + t.Errorf("%s at rest opens to %q (%v), not its value", name, got, err) + } + } +} + +// ---- no private key is accepted ------------------------------------------------------------------------- + +func TestNoPrivateKeyIsAccepted(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + keys, _ := seal.GenerateKeyPair() + ssh := "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAA\n-----END OPENSSH PRIVATE KEY-----\n" + rsa := "-----BEGIN RSA PRIVATE KEY-----\nMIIBOgIBAAJBAK\n-----END RSA PRIVATE KEY-----\n" + + // Not as a secret to hold: a private key stays where it was made (ADR 0113 §2). + for _, value := range []string{keys.PrivateKey, ssh, rsa, "prefix " + keys.PrivateKey} { + if _, err := v.Put("own/g14/sshd/host", value, "the test"); err == nil { + t.Errorf("the vault took a private key as a secret: %.40q", value) + } + } + if _, ok, _ := v.Store.Read("own/g14/sshd/host"); ok { + t.Error("a refused private key was kept all the same") + } + + // Not as the key to seal an answer to. + if _, err := v.Put("pair/postgres/gitea", "pw", "the test"); err != nil { + t.Fatal(err) + } + raw32 := base64.StdEncoding.EncodeToString(make([]byte, 32)) + for _, key := range []string{keys.PrivateKey, ssh, raw32, "", "not a key"} { + if _, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", key); err == nil { + t.Errorf("get sealed an answer to %.40q, which is not a PEM public key", key) + } + } + + // Not as the operator key: neither a PEM private key nor a raw key, which looks the same as a raw + // private one. + rawPub, rawPriv := operatorKey(t) + for _, key := range []string{keys.PrivateKey, rawPriv, rawPub} { + if _, err := v.SetOperatorKey(key, "the test"); err == nil { + t.Errorf("the vault took %.20q as the operator key", key) + } + } + if k, _ := v.Store.OperatorKey(); k != v.operatorPublic { + t.Error("a refused operator key replaced the operator key") + } + + // And nothing it publishes or writes names one that it refused. + for _, said := range v.bus.raw { + if strings.Contains(said, "PRIVATE KEY") { + t.Errorf("the vault published a private key: %s", said) + } + } + for path, content := range filesUnder(t, filepath.Join(v.dir, "store")) { + if strings.Contains(content, "PRIVATE KEY") { + t.Errorf("%s holds a private key", path) + } + } +} + +// ---- every secret has an operator-sealed copy ----------------------------------------------------------- + +func TestEverySecretHasAnOperatorSealedCopy(t *testing.T) { + v := newVault(t, nil) + values := map[string]string{"own/g14/dunst/token": "a", "pair/postgres/gitea": "b"} + for name, value := range values { + if _, err := v.Put(name, value, "the test"); err != nil { + t.Fatal(err) + } + } + if _, err := v.Mint("pair/postgres/keycloak", "the test"); err != nil { + t.Fatal(err) + } + all, err := v.Store.List() + if err != nil { + t.Fatal(err) + } + if len(all) != 3 { + t.Fatalf("the store lists %d secrets, not 3", len(all)) + } + for _, rec := range all { + if rec.Operator.Key != v.operatorPublic || rec.Operator.Box != OperatorBox { + t.Errorf("%s is sealed to %q in %q, not to the operator key", rec.Name, rec.Operator.Key, rec.Operator.Box) + } + got := openOperator(t, v.operatorPublic, v.operatorPrivate, rec.Operator.Sealed) + want, _ := seal.Open(rec.AtRest, v.Own.PrivateKey) + if got != want || got == "" { + t.Errorf("%s: the operator copy opens to %q, the vault's to %q", rec.Name, got, want) + } + } + + // A new operator key: every copy is sealed again to it, and none stays sealed to the old one. + pub, priv := operatorKey(t) + n, err := v.SetOperatorKey(asPEM(t, pub), "the test") + if err != nil || n != 3 { + t.Fatalf("a new operator key sealed %d copies again (%v), not 3", n, err) + } + all, _ = v.Store.List() + for _, rec := range all { + if rec.Operator.Key != pub { + t.Errorf("%s is still sealed to the earlier operator key", rec.Name) + } + openOperator(t, pub, priv, rec.Operator.Sealed) + } +} + +// Without an operator key the vault keeps nothing: a secret only it could open is what ADR 0113 §22 rules +// out, so the refusal is loud rather than a copy quietly missing. +func TestNoOperatorKeyKeepsNothing(t *testing.T) { + dir := t.TempDir() + store, _ := OpenStore(filepath.Join(dir, "store")) + own, _ := OwnKey(filepath.Join(dir, "state"), store) + v := &Vault{Store: store, Own: own, Emit: (&published{}).emit, Now: time.Now, Log: t.Logf} + if _, err := v.Put("own/g14/dunst/token", "x", "the test"); err == nil || !strings.Contains(err.Error(), "operator key") { + t.Fatalf("a secret was kept with no operator key to seal its copy to: %v", err) + } + if all, _ := store.List(); len(all) != 0 { + t.Fatalf("the store holds %d secrets", len(all)) + } +} + +// ---- get answers only an allowed asker, sealed to its key ----------------------------------------------- + +func TestGetAnswersOnlyAnAllowedAskerSealedToItsKey(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + if _, err := v.Put("pair/postgres/gitea", "the-value", "the test"); err != nil { + t.Fatal(err) + } + if _, err := v.Put("own/ace/dunst/token", "another", "the test"); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + + answer, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", asker.PublicKey) + if err != nil { + t.Fatal(err) + } + if got, err := seal.Open(answer.Sealed, asker.PrivateKey); err != nil || got != "the-value" { + t.Fatalf("the answer opens to %q (%v)", got, err) + } + other, _ := seal.GenerateKeyPair() + if _, err := seal.Open(answer.Sealed, other.PrivateKey); err == nil { + t.Error("the answer opens with a key it was not sealed to") + } + if _, err := seal.Open(answer.Sealed, v.Own.PrivateKey); err == nil { + t.Error("the answer opens with the vault's own key") + } + if answer.Generation != 1 || answer.Name != "pair/postgres/gitea" { + t.Errorf("the answer says %+v", answer) + } + + refusals := []struct{ caller, name, why string }{ + {"ace.node-provisioner", "pair/postgres/gitea", "an asker not on the list"}, + {"g14.node-provisioner", "own/ace/dunst/token", "a secret the asker is not allowed"}, + {stdio.Unproven, "pair/postgres/gitea", "a call whose caller the runtime did not prove"}, + {"", "pair/postgres/gitea", "no caller at all"}, + {"g14.node-provisioner", "pair/postgres/nothing", "a secret the vault does not hold"}, + } + for _, r := range refusals { + if a, err := v.Get(r.caller, r.name, asker.PublicKey); err == nil { + t.Errorf("get answered %s: %+v", r.why, a) + } + } +} + +// Until who may ask is decided by a node's declaration and a module's own account (V4, P3), the vault +// that runs answers nobody. +func TestTheRunningVaultAnswersNobodyYet(t *testing.T) { + v := newVault(t, nil) // what main builds: no list + if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + for _, caller := range []string{"g14.node-provisioner", "novox.mesh-provisioner", "controller", "person.jochen", "novox.node-tools"} { + if _, err := v.Get(caller, "pair/postgres/gitea", asker.PublicKey); err == nil { + t.Errorf("the running vault answered %s", caller) + } + } +} + +// The node-engine's sealing key (mesh-host internal/identity/sealing.go: raw X25519, base64, the NaCl +// anonymous box) is the same curve as the ADR 0206 box. A node's client asks with that key's public half as +// SPKI PEM, and opens the answer with its private half as PKCS #8 — no second key on the node, and no NaCl +// in the vault's answers. This is the path V4 builds on. +func TestANodesSealingKeyOpensTheAnswer(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + if _, err := v.Put("pair/postgres/gitea", "for-the-node", "the test"); err != nil { + t.Fatal(err) + } + // As mesh-host's GenerateSealingKey makes it. + priv, _ := ecdh.X25519().GenerateKey(rand.Reader) + nodePublic := base64.StdEncoding.EncodeToString(priv.PublicKey().Bytes()) + nodePrivate := base64.StdEncoding.EncodeToString(priv.Bytes()) + + rawPub, _ := base64.StdEncoding.DecodeString(nodePublic) + pubKey, _ := ecdh.X25519().NewPublicKey(rawPub) + spki, _ := x509.MarshalPKIXPublicKey(pubKey) + askWith := string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: spki})) + + answer, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", askWith) + if err != nil { + t.Fatal(err) + } + rawPriv, _ := base64.StdEncoding.DecodeString(nodePrivate) + privKey, _ := ecdh.X25519().NewPrivateKey(rawPriv) + pkcs8, _ := x509.MarshalPKCS8PrivateKey(privKey) + openWith := string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: pkcs8})) + if got, err := seal.Open(answer.Sealed, openWith); err != nil || got != "for-the-node" { + t.Fatalf("the node's own sealing key does not open the answer: %q %v", got, err) + } +} + +// ---- secret-changed carries no value -------------------------------------------------------------------- + +func TestSecretChangedCarriesNoValue(t *testing.T) { + v := newVault(t, nil) + if _, err := v.Put("pair/postgres/gitea", "first-value", "g14.mesh-vault-test"); err != nil { + t.Fatal(err) + } + if _, err := v.Put("pair/postgres/gitea", "second-value", "person.jochen"); err != nil { + t.Fatal(err) + } + if _, err := v.Mint("own/g14/dunst/token", "the vault"); err != nil { + t.Fatal(err) + } + var changed []map[string]any + for _, e := range v.bus.events { + if e["event"] == ChangedEvent { + changed = append(changed, e["body"].(map[string]any)) + } + } + if len(changed) != 3 { + t.Fatalf("%d secret-changed events, not 3: %v", len(changed), v.bus.events) + } + for i, body := range changed { + keys := make([]string, 0, len(body)) + for k := range body { + keys = append(keys, k) + } + for _, k := range keys { + switch k { + case "name", "by", "at", "generation": + default: + t.Errorf("secret-changed carries %q, beyond who, when, which and its generation", k) + } + } + if body["by"] == "" || body["at"] == "" { + t.Errorf("secret-changed %d says not who or when: %v", i, body) + } + } + if changed[1]["by"] != "person.jochen" || changed[1]["generation"] != 2 { + t.Errorf("the second change says %v", changed[1]) + } + minted, _, _ := v.Store.Read("own/g14/dunst/token") + mintedValue, _ := seal.Open(minted.AtRest, v.Own.PrivateKey) + for _, said := range v.bus.raw { + for _, value := range []string{"first-value", "second-value", mintedValue} { + if strings.Contains(said, value) { + t.Errorf("an event carries a value: %s", said) + } + } + } +} + +// The same value again is no change: no new generation, no event. +func TestTheSameValueAgainIsNoChange(t *testing.T) { + v := newVault(t, nil) + for i := 0; i < 2; i++ { + if _, err := v.Put("pair/postgres/gitea", "same", "the test"); err != nil { + t.Fatal(err) + } + } + rec, _, _ := v.Store.Read("pair/postgres/gitea") + if rec.Generation != 1 { + t.Errorf("the same value twice is generation %d", rec.Generation) + } + n := 0 + for _, e := range v.bus.events { + if e["event"] == ChangedEvent { + n++ + } + } + if n != 1 { + t.Errorf("%d secret-changed events for one value", n) + } +} + +// A secret's name is a name, not a path. +func TestASecretsNameIsNotAPath(t *testing.T) { + v := newVault(t, nil) + for _, name := range []string{"../etc/passwd", "/abs", "a//b", "A/b", "a/../b", "", strings.Repeat("a", 300)} { + if _, err := v.Put(name, "x", "the test"); err == nil { + t.Errorf("the vault kept a secret named %q", name) + } + } +} + +// The vault's own key is made once, kept beside it, and the same on the next start. +func TestTheVaultsOwnKeyIsMadeOnce(t *testing.T) { + dir := t.TempDir() + a, err := OwnKey(dir, nil) + if err != nil { + t.Fatal(err) + } + b, err := OwnKey(dir, nil) + if err != nil { + t.Fatal(err) + } + if a != b { + t.Error("the vault's key changed between two starts") + } + info, _ := os.Stat(filepath.Join(dir, OwnKeyFile)) + if info.Mode().Perm() != 0o600 { + t.Errorf("the vault's key is %v", info.Mode()) + } +} + +// The vault's key gone while its store holds secrets: no new key is made, and the vault does not start. +func TestAMissingKeyBesideSecretsIsRefused(t *testing.T) { + v := newVault(t, nil) + if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil { + t.Fatal(err) + } + if err := os.Remove(filepath.Join(v.dir, "state", OwnKeyFile)); err != nil { + t.Fatal(err) + } + if _, err := OwnKey(filepath.Join(v.dir, "state"), v.Store); err == nil || !strings.Contains(err.Error(), "MISSING") { + t.Fatalf("a new key was made beside secrets sealed to the lost one: %v", err) + } + if _, err := os.Stat(filepath.Join(v.dir, "state", OwnKeyFile)); err == nil { + t.Error("a key file was written all the same") + } +} + +// A new operator key that fails partway leaves no copy sealed to the earlier key for good: the next call +// with the same key — every minute, from the mesh's export — finishes the rest. +func TestANewOperatorKeyFailingPartwayIsFinished(t *testing.T) { + v := newVault(t, nil) + for _, name := range []string{"a/one", "b/two", "c/three"} { + if _, err := v.Mint(name, "the test"); err != nil { + t.Fatal(err) + } + } + pub, priv := operatorKey(t) + v.Store.failWrite = func(r Record) error { + if r.Name == "b/two" { + return os.ErrPermission + } + return nil + } + n, err := v.SetOperatorKey(asPEM(t, pub), "the test") + if err == nil || n != 2 { + t.Fatalf("a failure partway said %d sealed again, err %v", n, err) + } + if left, _ := v.StillSealedToAnother(); left != 1 { + t.Fatalf("%d copies left on the earlier key, not 1", left) + } + v.Store.failWrite = nil + // The path the minute takes while the export holds the key: the same key again. + if n, err := v.SetOperatorKey(asPEM(t, pub), "the test"); err != nil || n != 1 { + t.Fatalf("the same key again sealed %d again (%v), not the 1 left", n, err) + } + if n, err := v.SetOperatorKey(asPEM(t, pub), "the test"); err != nil || n != 0 { + t.Fatalf("the same key a third time sealed %d (%v)", n, err) + } + all, _ := v.Store.List() + for _, rec := range all { + if rec.Operator.Key != pub { + t.Errorf("%s is still sealed to an earlier key", rec.Name) + } + openOperator(t, pub, priv, rec.Operator.Sealed) + } +} + +// Who was handed which secret, and when, and who was refused, is in the vault's own store. +func TestWhoWasHandedWhatIsRecordedInTheStore(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + if _, err := v.Put("pair/postgres/gitea", "the-value", "the test"); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + if _, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", asker.PublicKey); err != nil { + t.Fatal(err) + } + _, _ = v.Get("ace.node-provisioner", "pair/postgres/gitea", asker.PublicKey) + handed, err := v.Store.Handed(10) + if err != nil || len(handed) != 2 { + t.Fatalf("%v %v", handed, err) + } + if handed[1].To != "g14.node-provisioner" || handed[1].Outcome != "handed" || handed[1].Generation != 1 || handed[1].At.IsZero() { + t.Errorf("the hand-out is recorded as %+v", handed[1]) + } + if handed[0].To != "ace.node-provisioner" || handed[0].Outcome != "refused" || handed[0].Why == "" { + t.Errorf("the refusal is recorded as %+v", handed[0]) + } + raw, _ := os.ReadFile(filepath.Join(v.dir, "store", "handed.jsonl")) + if strings.Contains(string(raw), "the-value") { + t.Error("the record of hand-outs holds a value") + } +} + +// The vault's operator copies reach the operator's recovery: secret_vault_export is a document +// `secret recover --from-export` reads, found by node "vault", module "mesh-vault" and the secret's name, +// and opened with the operator's private key as that command opens it (mesh-controller +// cmd/mesh-controller/secret.go keptFromExport, internal/secrets Open). +func TestTheOperatorRecoversAVaultSecretFromItsExport(t *testing.T) { + v := newVault(t, nil) + if _, err := v.Put("pair/postgres/gitea", "recover-me", "the test"); err != nil { + t.Fatal(err) + } + answer, err := toolNamed(t, definitions(&Env{Vault: v.Vault}), "secret_vault_export").Run(map[string]any{}) + if err != nil { + t.Fatal(err) + } + file := filepath.Join(t.TempDir(), "vault-export.json") + raw, _ := json.Marshal(answer) + _ = os.WriteFile(file, raw, 0o600) + + // What `secret recover vault mesh-vault pair/postgres/gitea --from-export ` reads. + var e struct { + Kept []struct { + Node, Module, Name, Sealed, Key string + } `json:"kept"` + } + read, _ := os.ReadFile(file) + if err := json.Unmarshal(read, &e); err != nil { + t.Fatal(err) + } + var sealed string + for _, k := range e.Kept { + if k.Node == "vault" && k.Module == "mesh-vault" && k.Name == "pair/postgres/gitea" { + sealed = k.Sealed + if k.Key != v.operatorPublic { + t.Errorf("the export names key %s, not the operator's", k.Key) + } + } + } + if sealed == "" { + t.Fatalf("the export holds no copy of the secret: %s", read) + } + // secrets.Open: the public half derived from the private key, then the anonymous box. + privRaw, _ := base64.StdEncoding.DecodeString(v.operatorPrivate) + key, _ := ecdh.X25519().NewPrivateKey(privRaw) + if got := openOperator(t, base64.StdEncoding.EncodeToString(key.PublicKey().Bytes()), v.operatorPrivate, sealed); got != "recover-me" { + t.Fatalf("recovered %q", got) + } + if strings.Contains(string(read), "recover-me") { + t.Error("the export holds the value in the clear") + } +} + +// With no key in the export the minute calls Reseal, which finishes what is pending too. +func TestResealFinishesWhatIsPending(t *testing.T) { + v := newVault(t, nil) + if _, err := v.Mint("a/one", "the test"); err != nil { + t.Fatal(err) + } + pub, _ := operatorKey(t) + v.Store.failWrite = func(Record) error { return os.ErrPermission } + if _, err := v.SetOperatorKey(asPEM(t, pub), "the test"); err == nil { + t.Fatal("a failing write was not said") + } + v.Store.failWrite = nil + if n, err := v.Reseal(); err != nil || n != 1 { + t.Fatalf("%d %v", n, err) + } +} + +// A secret is not handed out when who was handed it cannot be recorded. +func TestNoHandOutWithoutItsRecord(t *testing.T) { + v := newVault(t, AllowList{"g14.node-provisioner": {"pair/postgres/gitea"}}) + if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil { + t.Fatal(err) + } + if err := os.Mkdir(filepath.Join(v.dir, "store", "handed.jsonl"), 0o700); err != nil { + t.Fatal(err) + } + asker, _ := seal.GenerateKeyPair() + if a, err := v.Get("g14.node-provisioner", "pair/postgres/gitea", asker.PublicKey); err == nil { + t.Fatalf("handed out with no record of it: %+v", a) + } +} + +// What an asker cannot shape fills no record: a name that is no secret's name, a caller that is no bus user. +func TestAnArbitraryAskFillsNoRecord(t *testing.T) { + v := newVault(t, nil) + asker, _ := seal.GenerateKeyPair() + for _, ask := range [][2]string{{"g14.x", strings.Repeat("junk ", 1000)}, {"g14.x", "../../etc"}, + {"not a bus user!", "pair/a"}, {strings.Repeat("a", 300), "pair/a"}} { + _, _ = v.Get(ask[0], ask[1], asker.PublicKey) + } + if h, _ := v.Store.Handed(10); len(h) != 0 { + t.Errorf("recorded %d arbitrary asks: %+v", len(h), h) + } +} + +// The record of hand-outs is bounded: rolled at its size into a fixed number of files, the oldest dropped. +func TestTheRecordOfHandOutsIsBounded(t *testing.T) { + v := newVault(t, nil) // refuses everyone, as the running vault does + v.Store.HandedMax = 2000 + asker, _ := seal.GenerateKeyPair() + for i := 0; i < 500; i++ { + _, _ = v.Get("g14.flood", "pair/postgres/gitea", asker.PublicKey) + } + var total int64 + files, _ := filepath.Glob(filepath.Join(v.dir, "store", "handed.jsonl*")) + if len(files) != HandedKept+1 { + t.Errorf("%d files of the record, not %d: %v", len(files), HandedKept+1, files) + } + for _, f := range files { + info, _ := os.Stat(f) + if info.Size() > 2000 { + t.Errorf("%s is %d bytes, over its bound", f, info.Size()) + } + total += info.Size() + } + if total > int64(HandedKept+1)*2000 { + t.Errorf("the record holds %d bytes", total) + } + if v.Store.Rolled() == 0 { + t.Error("rolling was not counted") + } + if h, _ := v.Store.Handed(5); len(h) != 5 || h[0].To != "g14.flood" { + t.Errorf("the newest are not read back: %+v", h) + } +} diff --git a/modules/mesh-vault/go.mod b/modules/mesh-vault/go.mod new file mode 100644 index 00000000..454acc9d --- /dev/null +++ b/modules/mesh-vault/go.mod @@ -0,0 +1,10 @@ +module mesh-vault + +go 1.25.0 + +require ( + git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009 + golang.org/x/crypto v0.50.0 +) + +require golang.org/x/sys v0.43.0 // indirect diff --git a/modules/mesh-vault/go.sum b/modules/mesh-vault/go.sum new file mode 100644 index 00000000..afe914a8 --- /dev/null +++ b/modules/mesh-vault/go.sum @@ -0,0 +1,6 @@ +git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009 h1:I9rm8HhYRJ0+mzQQz+VAH0kgv2YJWkvctEc67Yd0PX8= +git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY= +golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI= +golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q= +golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= +golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/modules/mesh-vault/index.ts b/modules/mesh-vault/index.ts deleted file mode 100644 index 07aee59f..00000000 --- a/modules/mesh-vault/index.ts +++ /dev/null @@ -1,31 +0,0 @@ -// mesh-vault's events entrypoint, loaded by the per-node tool host (the provisioner runs in the same -// process — ADR 0052). The lifecycle events are EMITTED from the provisioner, where custody -// actually changes (novox/hq ADR 0041/0042): -// mesh-vault.provisioned — a consumer was granted a secret -// mesh-vault.rotated — that consumer's value changed (`rotate secret`) -// mesh-vault.deprovisioned — the consumer went away and its secret was withdrawn -// Here the vault reacts to them, keeping a lightweight audit line of who holds what and when it -// moved — the audit an owner of secrets is best placed to log. Fingerprints, never values. - -import { on } from "@novox/mesh-sdk/events"; - -interface SecretEvent { - as: string; - consumer?: string; - fingerprint?: string; - rotations?: number; -} - -await on("provisioned", async (e) => { - console.log(`[mesh-vault] secret provisioned for ${e.body?.as} on ${e.body?.consumer} (${e.body?.fingerprint})`); -}); - -await on("rotated", async (e) => { - console.log(`[mesh-vault] secret rotated for ${e.body?.as} — rotation ${e.body?.rotations} (${e.body?.fingerprint})`); -}); - -await on("deprovisioned", async (e) => { - console.log(`[mesh-vault] secret withdrawn from ${e.body?.as}`); -}); - -console.log("[mesh-vault] auditing secret lifecycle events"); diff --git a/modules/mesh-vault/module.json b/modules/mesh-vault/module.json index 87b6e6d7..29c631f5 100644 --- a/modules/mesh-vault/module.json +++ b/modules/mesh-vault/module.json @@ -10,18 +10,10 @@ } } ], - "capabilities": [ - "container-runtime" - ], "emits": [ "provisioned", "rotated", - "deprovisioned" - ], - "consumes": [ - "mesh-vault.provisioned", - "mesh-vault.rotated", - "mesh-vault.deprovisioned" + "secret-changed" ], "receives": { "secret": "${dir:grants}/mesh.json" @@ -49,6 +41,12 @@ "path": "${dir:root}", "class": "valuable", "why": "the vault's root material" + }, + { + "id": "store", + "path": "${dir:store}", + "class": "valuable", + "why": "every secret the vault holds, sealed to its own key and to the operator key" } ], "consumers": { @@ -80,6 +78,11 @@ "id": "root", "type": "directory", "mode": "0700" + }, + { + "id": "store", + "type": "directory", + "mode": "0700" } ], "build": { @@ -87,29 +90,42 @@ { "name": "code", "kind": "bundle", - "language": "typescript", - "entrypoints": [ - "index.js", - "tools/index.js", - "provisioner/index.js" - ], + "language": "go", + "system": "arch", + "from": "cmd/mesh-vault", + "binary": "mesh-vault", "loads": [ - "index.js", - "tools/index.js", - "provisioner/index.js" + "mesh-vault" ], "env": { "MESH_RECEIVES": "${dir:grants}/mesh.json", "MESH_VAULT_LEDGER": "${dir:ledger}", - "MESH_VAULT_ROOT": "${dir:root}" + "MESH_VAULT_ROOT": "${dir:root}", + "MESH_VAULT_STORE": "${dir:store}", + "MESH_VAULT_STATE": "${dir:state}" } } ] }, + "tools": [ + "secret_export", + "secret_holders", + "secret_verify", + "secret_handed", + "secret_vault_export", + "provisioner_retirement", + "provisioner_retire_approve", + "provisioner_retire_reject", + "provisioner_delete", + "vault_status" + ], "claims": [ { "name": "mesh-vault", - "scope": "mesh" + "scope": "mesh", + "serves": [ + "get" + ] } ] } diff --git a/modules/mesh-vault/package.json b/modules/mesh-vault/package.json deleted file mode 100644 index 585de85b..00000000 --- a/modules/mesh-vault/package.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "name": "@novox/module-mesh-vault", - "version": "0.1.0", - "description": "mesh-vault — provides the mesh `secret` interface: a module's own secret as an ordinary pair credential, held, audited and rotated like any other (novox/hq ADR 0085). Its ledger, provisioner, tools and events live here (ADR 0039).", - "type": "module", - "private": true, - "scripts": { - "test": "node --test --experimental-strip-types 'test/*.test.ts'" - }, - "dependencies": { - "@novox/mesh-sdk": "^0.1.0" - }, - "devDependencies": { - "@types/node": "^22.0.0", - "typescript": "^5.6.0" - } -} diff --git a/modules/mesh-vault/provisioner/index.ts b/modules/mesh-vault/provisioner/index.ts deleted file mode 100644 index 37808a1e..00000000 --- a/modules/mesh-vault/provisioner/index.ts +++ /dev/null @@ -1,48 +0,0 @@ -// mesh-vault's provisioner — the adapter that makes vault a provider of the mesh `secret` interface. The -// reconcile loop, the contributions file, and reading the mesh's minted value are the sdk harness's; -// this writes only the per-service half (novox/hq ADR 0039/0040/0048) — and for a vault that half is -// taking custody, not creating anything. -// -// The `secret` interface (ADR 0085, design 24): a consumer requires a value for its own use — the -// password of a store it runs privately, an internal token — and reads it from the file the mesh -// writes on its machine. There is no server to create a login on. **The value is the pair -// credential itself**: the controller minted it, sealed it to both nodes, and delivered each its -// copy. What makes it *owned* is this: the vault records who holds it and its fingerprint, notices -// when `rotate secret` delivers a different one, and says so on the mesh. Rotation is not new -// machinery — it is the machinery that already moves a database password, pointed at a secret the -// vault provides (design 13). - -import { runProvisioner, type Provision } from "@novox/mesh-sdk/provisioner"; -import { emit } from "@novox/mesh-sdk/events"; -import { Ledger } from "../client.js"; - -const ledger = Ledger.fromEnv(); - -/** Emit a lifecycle event without letting a broker hiccup fail the custody itself. */ -async function announce(type: string, body: Record): Promise { - try { - await emit(type, body); - } catch (err) { - console.error(`[provisioner:secret] emit ${type} failed: ${err}`); - } -} - -runProvisioner("secret", { - async create(p: Provision): Promise { - const { held, outcome } = ledger.record(p.as, p.consumer ?? "", p.password); - if (outcome === "unchanged") return; // the harness re-runs create on restart; nothing happened - console.log(`[mesh-vault] ${outcome}: ${held.as} (${held.fingerprint.slice(0, 19)}…, rotations ${held.rotations})`); - await announce(`module.mesh-vault.secret.${outcome === "granted" ? "provisioned" : "rotated"}`, { - consumer: held.consumer, - as: held.as, - fingerprint: held.fingerprint, - rotations: held.rotations, - }); - }, - - async remove(p: { as: string }): Promise { - if (!ledger.withdraw(p.as)) return; - console.log(`[mesh-vault] withdrawn: ${p.as}`); - await announce("deprovisioned", { as: p.as }); - }, -}); diff --git a/modules/mesh-vault/test/ledger.test.ts b/modules/mesh-vault/test/ledger.test.ts deleted file mode 100644 index e2ca648f..00000000 --- a/modules/mesh-vault/test/ledger.test.ts +++ /dev/null @@ -1,80 +0,0 @@ -import { test } from "node:test"; -import assert from "node:assert/strict"; -import { mkdtempSync, readFileSync, readdirSync, statSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; - -import { Ledger, fingerprint, contributions, deliveredFingerprint } from "../client.ts"; - -function fresh(): Ledger { - return new Ledger(mkdtempSync(join(tmpdir(), "vault-ledger-"))); -} - -test("a first delivery is a grant, the same value again is nothing, a new value is a rotation", () => { - const ledger = fresh(); - const t0 = new Date("2026-09-20T10:00:00Z"); - const t1 = new Date("2026-09-21T10:00:00Z"); - - const granted = ledger.record("anchor-redis", "anchor", "first-value", t0); - assert.equal(granted.outcome, "granted"); - assert.equal(granted.held.rotations, 0); - assert.equal(granted.held.since, t0.toISOString()); - assert.equal(granted.held.fingerprint, fingerprint("first-value")); - - assert.equal(ledger.record("anchor-redis", "anchor", "first-value", t1).outcome, "unchanged"); - assert.equal(ledger.get("anchor-redis")!.rotations, 0, "an unchanged delivery counted as a rotation"); - - const rotated = ledger.record("anchor-redis", "anchor", "second-value", t1); - assert.equal(rotated.outcome, "rotated"); - assert.equal(rotated.held.rotations, 1); - assert.equal(rotated.held.since, t0.toISOString(), "a rotation reset the grant date"); - assert.equal(rotated.held.changed, t1.toISOString()); - assert.equal(rotated.held.fingerprint, fingerprint("second-value")); - assert.deepEqual(rotated.held.history, [{ fingerprint: fingerprint("first-value"), until: t1.toISOString() }]); -}); - -test("the ledger holds fingerprints and never the value, in files nobody else can read", () => { - const dir = mkdtempSync(join(tmpdir(), "vault-ledger-")); - const ledger = new Ledger(dir); - ledger.record("anchor-redis", "anchor", "the-actual-password", new Date()); - ledger.record("anchor-redis", "anchor", "the-rotated-password", new Date()); - for (const name of readdirSync(dir)) { - const raw = readFileSync(join(dir, name), "utf8"); - assert.doesNotMatch(raw, /the-actual-password|the-rotated-password/, `${name} holds a value`); - assert.equal(statSync(join(dir, name)).mode & 0o777, 0o600, `${name} is readable by others`); - } -}); - -test("withdrawing forgets a holder, and listing is by login", () => { - const ledger = fresh(); - ledger.record("b-app", "b", "x", new Date()); - ledger.record("a-app", "a", "y", new Date()); - assert.deepEqual(ledger.list().map((h) => h.as), ["a-app", "b-app"]); - assert.equal(ledger.withdraw("a-app"), true); - assert.equal(ledger.withdraw("a-app"), false, "withdrawing twice said it found something"); - assert.deepEqual(ledger.list().map((h) => h.as), ["b-app"]); -}); - -test("a login is a name, not a path", () => { - const ledger = fresh(); - assert.throws(() => ledger.record("../etc/passwd", "n", "v"), /a login is a name/); -}); - -test("what the mesh delivers is read from the contributions file and fingerprinted, never returned", () => { - const dir = mkdtempSync(join(tmpdir(), "vault-grants-")); - const secret = join(dir, "anchor.redis.secret"); - writeFileSync(secret, "minted-value\n"); // the host may leave a trailing newline; the value has none - const receives = join(dir, "mesh.json"); - writeFileSync(receives, JSON.stringify({ - requirement: "secret", - given: [ - { from: "redis", node: "anchor", as: "anchor-redis", secret }, - { from: "offer-only", node: "anchor" }, // a contribution with no login grants nothing - ], - })); - const asked = contributions(receives); - assert.deepEqual(asked.map((c) => c.as), ["anchor-redis"]); - const seen = deliveredFingerprint(asked[0]); - assert.deepEqual(seen, { fingerprint: fingerprint("minted-value"), length: "minted-value".length }); - assert.match(JSON.stringify(deliveredFingerprint({ as: "x", secret: join(dir, "missing") })), /not readable/); -}); diff --git a/modules/mesh-vault/tools/index.ts b/modules/mesh-vault/tools/index.ts deleted file mode 100644 index a6437e60..00000000 --- a/modules/mesh-vault/tools/index.ts +++ /dev/null @@ -1,131 +0,0 @@ -// mesh-vault's tools — vault's own code (novox/hq ADR 0039), served through the sdk's tool harness. They -// return structured data about the secrets this vault provides, and **never a value**: a holder is -// identified by its login and a value by its fingerprint. That is the rule the source mesh's -// secret_locate / secret_verify were built on, after a secret printed into a transcript. - -import { readFileSync } from "node:fs"; -import { join } from "node:path"; -import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools"; -import { Ledger, contributions, deliveredFingerprint, type Held } from "../client.js"; - -/** The mesh's export of every operator-sealed secret, as the mesh wrote it into the root dir. */ -interface KeptExport { - export: number; - "operator-key": string; - fingerprint: string; - kept: { node: string; module: string; name: string; origin: string; sealed: string; key: string; "made-at": string }[]; - unrecoverable?: { node: string; module: string; name: string }[]; -} - -export function getVaultTools(ledger: Ledger, receives: string | undefined, root: string | undefined): ToolDefinition[] { - return [ - { - name: "secret_export", - description: - "The mesh's root secrets as this vault keeps them: every secret a module holds for itself, " + - "sealed to the operator's key (novox/hq ADR 0085, amended). Ciphertext — nothing here can " + - "open a line of it; the operator, holding the private key off the mesh, recovers one with " + - "`nox secret recover --from-export` on the control-node. Also lists what is NOT recoverable: secrets " + - "made before the mesh had an operator key.", - input: { - sealed: { - type: "boolean", - description: "include the sealed blobs (default true); false lists holders and the key only", - }, - }, - run: async (args) => { - if (!root) return { available: false, error: "this vault keeps no root secrets (MESH_VAULT_ROOT is not set)" }; - let doc: KeptExport; - try { - doc = JSON.parse(readFileSync(join(root, "export.json"), "utf8")) as KeptExport; - } catch (err) { - return { available: false, error: `the mesh has not written an export here yet: ${err}` }; - } - const withBlobs = args.sealed !== false; - return { - available: true, - export: doc.export, - "operator-key": doc["operator-key"], - fingerprint: doc.fingerprint, - count: doc.kept.length, - kept: doc.kept.map((k) => (withBlobs ? k : { node: k.node, module: k.module, name: k.name, origin: k.origin, "made-at": k["made-at"] })), - unrecoverable: doc.unrecoverable ?? [], - }; - }, - }, - { - name: "secret_holders", - description: - "Who holds a secret from this vault: each consumer's login, node and module, when it was " + - "granted, how many times it has been rotated and when, and the fingerprint of the current " + - "value. Fingerprints only — the value is never returned.", - input: {}, - run: async () => { - const asked = receives ? contributions(receives) : []; - const holders = ledger.list().map((h) => ({ - ...h, - module: asked.find((c) => c.as === h.as)?.from ?? null, - asked: asked.some((c) => c.as === h.as), - })); - return { holders, count: holders.length }; - }, - }, - { - name: "secret_verify", - description: - "Check one holder's secret without seeing it: the fingerprint the vault recorded against " + - "the fingerprint of the value the mesh currently delivers here, and optionally against a " + - "fingerprint computed on the holder's own machine (sha256 of the file, as `sha256:`). " + - "Two ends agreeing proves they agree, not that either works — the login itself is the test.", - input: { - as: { type: "string", description: "the holder's login, e.g. anchor-redis" }, - fingerprint: { - type: "string", - description: "optional: sha256: of the value as the holder reads it, computed there — never the value", - }, - }, - run: async (args) => { - const as = String(args.as ?? ""); - const recorded = ledger.get(as); - if (!recorded) return { as, known: false, error: `this vault holds nothing for ${as}` }; - const asked = receives ? contributions(receives).find((c) => c.as === as) : undefined; - const delivered = asked ? deliveredFingerprint(asked) : { error: "the mesh does not currently ask this vault to serve that login" }; - const given = args.fingerprint ? String(args.fingerprint) : undefined; - return verdict(recorded, delivered, given); - }, - }, - ]; -} - -function verdict( - recorded: Held, - delivered: { fingerprint: string; length: number } | { error: string }, - given: string | undefined, -): Record { - const deliveredMatches = "fingerprint" in delivered ? delivered.fingerprint === recorded.fingerprint : null; - const givenMatches = given === undefined ? null : given === recorded.fingerprint; - return { - as: recorded.as, - known: true, - recorded: recorded.fingerprint, - rotations: recorded.rotations, - changed: recorded.changed, - delivered: "fingerprint" in delivered ? delivered.fingerprint : null, - deliveredError: "error" in delivered ? delivered.error : null, - deliveredMatchesRecorded: deliveredMatches, - given: given ?? null, - givenMatchesRecorded: givenMatches, - givenIsAnEarlierValue: given === undefined ? null : recorded.history.some((h) => h.fingerprint === given), - ok: deliveredMatches !== false && givenMatches !== false, - }; -} - -// The tools exist only when the ledger can be reached from the environment; without it, vault -// contributes none rather than failing the whole tool runtime. -registerModuleTools("mesh-vault", (env) => { - try { - return getVaultTools(Ledger.fromEnv(env), env.MESH_RECEIVES, env.MESH_VAULT_ROOT); - } catch { - return []; - } -}); diff --git a/modules/mesh-vault/tsconfig.json b/modules/mesh-vault/tsconfig.json deleted file mode 100644 index 51f4046a..00000000 --- a/modules/mesh-vault/tsconfig.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "compilerOptions": { - "target": "ES2022", - "module": "NodeNext", - "moduleResolution": "NodeNext", - "strict": true, - "esModuleInterop": true, - "skipLibCheck": true, - "noEmit": true - }, - "include": ["client.ts", "index.ts", "provisioner/index.ts", "tools/index.ts"] -}