novox/mesh-catalog
13
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Feature 57 (config and provisioning): modules declare config and provisions, their files rendered from templates; synced with main (issues of feature 57, 874)

Browse Source
This commit is contained in:
mesh-tools
2026-10-16 07:37:43 +00:00
parent 1a2c917abf
commit 9652b2d5ce
771 changed files with 52063 additions and 1774 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/mesh-vault/README.md
+132 -17
View File
@@ -76,17 +76,19 @@ Two boxes on the mesh today, both over X25519:
What the vault uses:
- **Its answers to `get` are the ADR 0206 box, for every asker, a node included.** The asker sends its
public key as SPKI PEM. A node's provisioner client (V4) needs no second key: its node's sealing key is
X25519 already, so it sends that key's public half wrapped as SPKI and opens the answer with the private
half wrapped as PKCS #8 (`TestANodesSealingKeyOpensTheAnswer` does exactly this). A raw 32-byte key is
refused: a raw private key looks the same, and the vault takes no private key.
public key as SPKI PEM. A node's provisioner client (V4) asks with a key of its own, made when it starts
and kept in its memory alone: it runs under an account of its own and cannot read the node-engine's sealing
key, which is root's. A node's sealing key would open an answer too, being X25519 (wrapped as SPKI and PKCS #8,
`TestANodesSealingKeyOpensTheAnswer`), for the node-engine's own asks. A raw 32-byte key is refused: a raw
private key looks the same, and the vault takes no private key.
- **Its operator copies are the NaCl anonymous box**, to the operator key as it is made today, so the
operator's recovery opens them with the key and the command it already has.
- **The at-rest copy is the ADR 0206 box**, to the vault's own key.
The NaCl box stays on the node-engine only for what the controller seals into declarations, until the last
module has moved (ADR 0349 §17). For the core's own credentials that travel in a declaration sealed by the
vault (ADR 0349 §16), V4 adds opening the ADR 0206 box with the same sealing key to the node-engine.
vault (ADR 0349 §16), the node-engine is to open the ADR 0206 box with the same sealing key; that is not built
yet (issue 647 leaves it to its own issue).
## The seat's verb
@@ -102,15 +104,107 @@ bounded: rolled at 1 MiB into `handed.jsonl.1` … `.3`, the oldest dropped, eac
journal and counted in `vault_status`. `secret_handed` reads it.
`vault_status` says what needs a person: operator copies still sealed to an earlier key, an operator key in
the export the vault refused, and how often the record was rolled. **It is not yet a condition**: a health
check is declared on a long-running resource (ADR 0240), and the vault is a bundle the tool runner loads, so
nothing asks it on its own; until it runs as a resource of its own (V3's review) these reach a person
through the journal and this tool only.
the export the vault refused, and how often the record was rolled.
**Who may ask, today: nobody.** The rule (ADR 0349 §11) is a node's provisioner client under a bus account
of its own, for what its node's declaration names, and a module under its own account (issue 569). Neither
exists yet, so the running vault refuses every asker; the tests set a stand-in list. The real rule goes
behind the `Askers` interface in `askers.go` (design 57's V4 and P3).
**A failing re-seal is a condition** (ADR 0366, issue 738). The vault re-seals its operator copies to the
operator key once a minute. `vault_health` answers unhealthy once that has failed on three passes in a row,
saying what failed and why, and healthy again after one pass succeeds. The manifest declares it as the
health of the vault's own runtime (`runtime-health`), which the controller puts on `mesh-vault.own-runtime`;
the node-engine asks it every minute and the controller raises `module.mesh-vault.<machine>.unhealthy` from
it, cleared on the first healthy answer. An operator key in the export the vault refuses fails the pass too.
How often the record was rolled still reaches a person through the journal and `vault_status` only.
**Who may ask: a node's provisioner client, for what its plan names** (ADR 0349 §11, design 57 V4, issue 647;
`askers.go`). The caller is `provisioner.<node>`, the client's own account (never the tool runner's, which an agent
on that machine shares), and the secret is `own/<node>/<module>/<name>` for that same machine, where the
controller's assignment `<node>.<module>` (the assignments mirror, which the manifest `reads`) carries the
provisioner's plan and the plan's `secrets` name it; or a value a provider's software issued for one holder,
`provision/<provider>/<provision>/<node>/<module>`, where that plan's `provisions` name the provision from that
provider with a credential's `name` (issue 757; an entry naming only the provider serving a provision, issue 760,
hands nothing). The vault cannot read a node's declaration, so the plan the
controller carries beside it is what the declaration names, as the vault reads it. A provision's one credential,
`provision/<provider>/<provision>`, is handed to the machines the vault made it for (below) while the holder there
is handed to the provisioner: the provider's module, or a consumer whose plan still names it from that provider. A
module not handed to the provisioner, another machine's secret or holder value, and every other caller are refused,
and each refusal is recorded with why. A module's own account (issue 569) is not answered yet.
**The controller, for the core's own credentials, sealed only to their machine** (ADR 0349 §16, issue 763;
`core.go`). The bus and mesh-provisioner need their files before any client runs, so their credentials travel in
the declaration sealed by the vault to their machine, which the controller carries and cannot open. The caller
`controller` is answered `own/<node>/<module>/<name>` only for a module on the core's fixed list (`nats`,
`mesh-provisioner`, `mesh-provisioner-client`, the controller's list too), and only when the key it names is that
machine's sealing key as its own node-engine stated it in its own name, in `move-in`'s first step (`sealing_key`).
A machine whose engine stated none, any other key, any other module and any other name are refused and recorded.
## Taking values in: `accept`, `secret-ask`, `issued`, `rotate`
Design 57's V2 (issue 675; ADR 0349 clauses 8 to 10, ADR 0277 §22–§27 and ADR 0305 with the vault in the
controller's place). `intake.go`.
**From the controller alone; the vault asks the desk and the provider itself.** The controller alone asks these
verbs, as the runtime proves its caller. Each value comes in sealed to an X25519 key the vault made for that one
call (raw, base64 — the NaCl anonymous box the desk's prompt and a provider's tool already seal to), and opens with
the private half, which lives in this process only and is forgotten once used or after two minutes. A value sealed
to any other key is refused, and a call is answered once.
**The vault's own account** (issue 723). The vault runs as `mesh-vault` in a runtime of its own (`runs-as`, ADR
0259 §8), on a bus account of its own (`<node>.mesh-vault`, from its own secret `broker`). The bus lets that
account call the desks' prompt (`node-launcher.secret` on any machine) and each issuing provider's tool in its own
name, and nothing else outside its seat (mesh-controller's `internal/broker` grants, proven on a real server). So
the prompt and the tool know the vault asks, and the controller carries neither the key nor the sealed value. The
controller still asks them itself for a module filled the old way, until R1 (issue 682) removes that path; they
answer both until then. Everything the vault keeps is its account's: its directories, the export it keeps, the
grant files the controller still writes for it.
- `accept` — two steps: node, module, name (a module's own secret) or provider, provision (a provision's one
credential), and `by`; then `call` and `sealed`. The controller's terminal seals the value it read to the call's
key before it leaves the terminal. Kept with origin `accepted`; an accepted provision credential is never remade
(`rotate` refuses it); a value a provider's software issued is not taken by `accept`.
- `secret-ask` — one step: node, module, name, desk and `asked_by` (who asked the controller, as the bus named its
caller). The ask is recorded in `asks.jsonl` before anything else, and refused while another ask for the same
secret is open (two minutes at most) or when three were made in the last hour. The vault then asks the desk's
prompt itself, naming who asked as the record has it, with `timeout_seconds` 20. The answer is how it ended
(`given`, dismissed, timed-out, empty, refused, failed) and who asked; how each ask ended is recorded.
`secret_asks` reads the record. The hand-act log stays the controller's: it records the act once the vault has
taken the value.
- `issued` — one step: provision, provider (machine), module and tool. The vault asks that tool on the provider's
machine itself; the value is kept as the provision's one credential with origin `issued`. The same value again
writes nothing, says nothing and keeps the generation (ADR 0305 §5).
- `issued` for each holder apart (issue 749; ADR 0349 clause 11, ADR 0365 clause 4: the forge's read token per agent
account). The controller names the provision's holders from that provider (`holders`, each a node and a consuming
module), and the vault asks the tool with them. The tool answers either one value (`sealed`, kept as above) or one
per holder (`holders`: node, module and `sealed` each, all sealed to the call's key). Each is kept as
`provision/<provider node>/<provision>/<holder node>/<holder module>`, origin `issued`, and the same value again for
a holder writes nothing for it. The answer is taken whole or not at all: a holder not asked about, one named twice, a
value not sealed to the call or an empty one keeps nothing. A holder asked about and not answered is said
(`not_answered`) and keeps nothing. **`get` hands a holder's value to that holder alone**, its machine's provisioner
client (`provisioner.<node>`) or the module under an account of its own (`<node>.<module>`), whatever the rule for
who may ask allows; every other caller is refused and the refusal recorded.
`TestOnARealBusTwoHoldersOnTwoMachinesEachGetOnlyTheirOwnIssuedValueAndNoneCrossesInTheClear` proves it on a NATS
server with a stand-in forge.
- `rotate` — a module's own secret or a provision's credential, made anew. Refused for an `issued` value (the
software rotates it, ADR 0305 §9) and an accepted provision credential (clause 8). From the controller alone,
which knows whether the module's secrets are filled from the vault yet, and with a `why`, said in the vault's
log. Who asked is the `by` the controller names, beside the controller; both are kept (`changed-by`,
`changed-via`). The controller's `rotate <provision>` forwards here for a provider filled the new way (issue
767): a shared value is made anew for every holder, never for one consumer alone; for a provider whose consumers
keep a login each, `consumer` and `consumer_module` (`--consumer`, `--module`) name the one consumer whose own
login is remade, and no other is touched.
Names in the store: `own/<node>/<module>/<name>`, `provision/<provider node>/<provision>` and, for a value issued for
one holder, `provision/<provider node>/<provision>/<holder node>/<holder module>`. V3 copies the
controller's rows in under these names.
**Which modules reach the vault this way.** The controller sends a module's verbs here only when its secrets are
filled the new way (its machine's provisioner client asks the vault for them, V4); every other module keeps the
controller's path, so no value lands where its machine never looks (ADR 0349 clause 17; the release lead's decision
on issue 675).
**What the issuing provider's test is here.** `TestAnIssuingProviderAnswersOnlyTheVaultsCallSealedToTheVault` and
`TestTheVaultAsksTheIssuingToolItselfAndOnlyASealedValueCrosses` use a stand-in provider that answers the vault's
account (and, until R1, the controller) and no other caller, and only sealed to the call's key. plex's own tool
answers the controller alone today; it must also answer `<node>.mesh-vault` when it moves with the media catalogue,
after release 57.
## Events
@@ -149,11 +243,28 @@ and a holder's ledger file removed only through a person's delete, as here. The
lives: the harness kept it in its process, so a restart forgot it (ADR 0230 §13); here it is in the ledger
file.
**The credential a provider shares with its consumers** (ADR 0349 clause 8, ADR 0158 §4–§6, design 57 V5, issue
677; `pair.go`). One value per provider assignment and provision, `provision/<provider>/<provision>`. Once a minute
the vault reads every plan in the assignments mirror; each plan's `provisions` names the provider serving each
provision its module requires (issue 760). A consumer binds when an entry appears and unbinds when it goes or its
assignment goes; on either, the vault makes one new value for every holder together and says `secret-changed`, and
each holder's client asks again. What the move brought in (V3) is adopted when every pair row of a provider and
provision shares one generation stamp and one value (ADR 0158's shared form), with its origin, and nothing is
remade. Rows without a stamp (ADR 0048, a login per consumer) are never made one value (ADR 0158 §11): `rotate` of
the provision remakes each consumer's own apart, and each consumer's machine is handed only its own row,
`pair/<provider>/<provision>/<consumer>/<module>[/<local>]` (the provider's machine all of them). A provider the
vault holds no row of gets its first value in the form each plan entry names (`credential-form`, from the provider's
offer, issue 768): `shared`, one value for every holder; `pair`, a login of its own for each consumer bound; `issued`
or `given` are never made. No form named, plans that disagree, or a form it does not know: nothing is made. An accepted or issued value is never remade, only
its holders recorded. A provision whose plans name a credential per holder (issue 757) has no shared value. **No
value is made that a holder would never get:** while the provider's assignment, or a consumer the move brought in,
is still filled the old way (no plan), every make, remake and rotation is held back and the reason said in the
log. Who holds each value is kept in the store's `shared/` directory, never a value.
## What comes later
Taking values in (`secret accept`, `secret-ask`, `issued`, `rotate`) is V2; the controller's secrets move
in with V3; nodes ask with V4; pair credentials are minted here with V5. Until V3 the vault's store is
empty and the controller still serves every secret.
Taking values in is built (above); the controller's secrets move in with V3; nodes' provisioner clients ask with V4,
for each module handed to the provisioner; providers answer their consumers with B1–B3.
## Tests
@@ -161,4 +272,8 @@ empty and the controller still serves every secret.
`vault_test.go` is ADR 0349's vault test list as far as this package goes: no bucket holds a secret; no
private key is accepted; every secret has an operator-sealed copy; `get` answers only an allowed asker,
sealed to its key; `secret-changed` carries no value.
sealed to its key; `secret-changed` carries no value. `intake_test.go` is ADR 0349's `secret-ask` test as far as
the vault answers it (recorded before the prompt, one open ask per secret and three an hour, the asker named as
the bus named it, the typed value opened only here), the issuing provider's test with a stand-in provider (ADR 0305
§13's checks that belong to the vault: nothing rewritten for a value held, `rotate` refused, no other way in), and
accept and rotate. `followups_test.go` holds V1's review follow-ups.
modules/mesh-vault/cmd/mesh-vault/askers.go
+147 -9
View File
@@ -1,12 +1,42 @@
package main
// Who may ask `get` for which secret: the seam the real rule fills (novox/hq ADR 0349 §11).
import (
"encoding/json"
"fmt"
"strings"
)
// Who may ask `get` for which secret (novox/hq ADR 0349 §11, design 57 V4, issue 647).
//
// The rule is: a node's provisioner client, running as a service under a bus account of its own, for a
// secret that node's declaration names; or a module, for a secret its manifest declares, only once its
// code runs under a bus account of its own (issue 569). Neither exists yet — the client and its account are
// design 57's P3, the vault's answer to a node V4 — so the vault that runs allows nobody (NoAsker), and the
// tests set a stand-in list (AllowList). V4 and P3 put the real rule behind this interface.
// The rule is: a node's provisioner client, running as a service under a bus account of its own
// (`provisioner.<node>`, design 57 P3), for a secret that node's declaration names; or a module, for a secret its
// manifest declares, only once its code runs under a bus account of its own (issue 569). The second does not exist
// yet, so the vault answers the first alone (Declared), and refuses every other caller.
//
// **What a node's declaration names, as the vault can read it.** The vault cannot read a node's declaration, which
// the node-engine keeps for root. The controller carries the provisioner's plan of each module it handed to the
// provisioner in that module's assignment, `<node>.<module>` of the assignments mirror (mesh-controller internal/
// inventory Provisioning), the same value the node's client fills from; its `secrets` are the module's own secrets
// the declaration names for its files there. So a client is handed `own/<node>/<module>/<name>` only when the
// asker is that node's client, the module is on that node and handed to the provisioner, and its plan names the
// secret. A module only pulled in by another has a value there of its own, which names what requires it (issue 765),
// and is read the same way. Everything else is refused, and the refusal recorded with why (vault.go).
//
// **A value a provider's software issued for one holder** (issue 757): the vault keeps it as
// `provision/<provider>/<provision>/<node>/<module>` (perholder.go, issue 749). A node's client is handed it when
// the asker is that node's client (perholder.go's rule, before this one) and the module's plan there names that
// provision from that provider among its `provisions`: the credentials of what it requires that its machine's
// client fills from the vault.
//
// **The provision's one credential** (issue 677): `provision/<provider>/<provision>` is handed to a node's client
// when the vault made, adopted or recorded it for a holder on that node (pair.go) and that holder is still there,
// handed to the provisioner: the provider's module on its machine, or a consumer whose plan there names the
// provision from that provider (without a name, which is a credential per holder).
//
// **A consumer's own login** (issue 677's note 12, ADR 0048, ADR 0158 §11): `pair/<provider>/<provision>/<consumer>/
// <module>[/<local>]` is handed to that consumer's machine while its plan there names the provision from that
// provider under that local name, and to the provider's machine while the provider is handed there; never to
// another consumer.
// Askers decides whether caller (the bus user the runtime proved) may have the secret name, and if not,
// why not.
@@ -14,13 +44,12 @@ type Askers interface {
May(caller, name string) (bool, string)
}
// NoAsker allows nobody: the vault's rule until who may ask is decided by a declaration and an account.
// NoAsker allows nobody: what a vault with no rule given answers.
type NoAsker struct{}
// May refuses every caller.
func (NoAsker) May(string, string) (bool, string) {
return false, "the vault hands a secret to no asker yet: a node's provisioner client under its own account, " +
"for what its declaration names, is the first it will answer (novox/hq ADR 0349 §11, issue 647)"
return false, "the vault hands a secret to no asker without a rule of who may ask (novox/hq ADR 0349 §11)"
}
// AllowList is a stand-in rule: each caller, and the names it may have.
@@ -35,3 +64,112 @@ func (l AllowList) May(caller, name string) (bool, string) {
}
return false, "not on the list of who may ask for it"
}
// ProvisionerPrefix begins a node's provisioner client's bus user, `provisioner.<node>` (mesh-controller
// internal/broker, KindProvisioner).
const ProvisionerPrefix = "provisioner."
// AssignmentsRead is the controller's assignments mirror as this module reads it (its manifest's `reads`).
const AssignmentsRead = "mesh-controller.assignments"
// AssignmentReader reads one value of the assignments mirror, `<node>.<module>`: nil when there is none.
type AssignmentReader func(key string) ([]byte, error)
// Declared is the rule the running vault keeps: a node's provisioner client, for a secret of a module on its node
// that the module's plan there names.
type Declared struct {
Read AssignmentReader
// Store is the vault's store, whose record of who holds a provision's one credential (pair.go) the rule reads.
Store *Store
}
// plannedSecrets is as much of an assignment's value as the rule reads: the plan's secrets, by name.
type plannedSecrets struct {
Node string `json:"node"`
Module string `json:"module"`
Provisioning *struct {
Secrets []struct {
Name string `json:"name"`
} `json:"secrets"`
// Provisions are what the module requires, each with its provider (issue 760); one with a name is a
// credential the machine's client fills from the vault (issue 757).
Provisions []struct {
Name string `json:"name"`
Provision string `json:"provision"`
Provider string `json:"provider"`
} `json:"provisions"`
} `json:"provisioning"`
}
// May allows caller the secret name when caller is `provisioner.<node>` and name is `own/<node>/<module>/<secret>`
// that the plan of module on node names, or `provision/<provider>/<provision>/<node>/<module>` whose provision from
// that provider the plan names (issue 757).
func (d Declared) May(caller, name string) (bool, string) {
node, ok := strings.CutPrefix(caller, ProvisionerPrefix)
if !ok || !namePart.MatchString(node) {
return false, "the vault hands a secret only to a node's provisioner client under its own account " +
"(provisioner.<node>); a module's code is answered once it runs under an account of its own (issue 569)"
}
parts := strings.Split(name, "/")
if len(parts) == 3 && parts[0] == "provision" {
if why := sharedRefusal(d.Store, d.Read, node, parts[1], parts[2]); why != "" {
return false, why
}
return true, ""
}
if (len(parts) == 5 || len(parts) == 6) && parts[0] == "pair" {
if why := pairRowRefusal(d.Read, node, parts); why != "" {
return false, why
}
return true, ""
}
var holderNode, module, secret, provider string
switch {
case len(parts) == 4 && parts[0] == "own":
holderNode, module, secret = parts[1], parts[2], parts[3]
case len(parts) == 5 && parts[0] == "provision":
provider, secret, holderNode, module = parts[1], parts[2], parts[3], parts[4]
default:
return false, fmt.Sprintf("a node's provisioner client is handed a module's own secret, own/%s/<module>/<name>, "+
"the value a provider issued for one of its modules, provision/<provider>/<provision>/%s/<module>, or "+
"a provision's one credential, provision/<provider>/<provision>; %s is none of them", node, node, name)
}
if holderNode != node {
return false, fmt.Sprintf("%s is a secret of machine %s, and %s asks for machine %s alone", name, holderNode,
caller, node)
}
if d.Read == nil {
return false, "the vault cannot read the assignments, so it cannot tell what " + node + "'s declaration names"
}
raw, err := d.Read(node + "." + module)
if err != nil {
return false, fmt.Sprintf("the assignment %s.%s could not be read, so what it names is not known: %v", node,
module, err)
}
if raw == nil {
return false, fmt.Sprintf("%s is not on %s: neither assigned there nor required by a module there", module,
node)
}
var a plannedSecrets
if err := json.Unmarshal(raw, &a); err != nil || a.Node != node || a.Module != module {
return false, fmt.Sprintf("the assignment %s.%s is not the controller's", node, module)
}
if a.Provisioning == nil {
return false, fmt.Sprintf("%s on %s is not handed to the provisioner, so its secrets reach it in its "+
"declaration (novox/hq ADR 0349 §17)", module, node)
}
if provider != "" {
for _, p := range a.Provisioning.Provisions {
if p.Name != "" && p.Provision == secret && p.Provider == provider {
return true, ""
}
}
return false, fmt.Sprintf("%s's plan on %s names no credential of %s from %s", module, node, secret, provider)
}
for _, s := range a.Provisioning.Secrets {
if s.Name == secret {
return true, ""
}
}
return false, fmt.Sprintf("%s's plan on %s names no secret %s", module, node, secret)
}
modules/mesh-vault/cmd/mesh-vault/core.go
+142
View File
@@ -0,0 +1,142 @@
package main
// The core's own credentials (novox/hq ADR 0349 §16, issue 763). The bus, the node-engine, the controller and
// mesh-provisioner need their files before any provisioner client runs, so their credentials travel in the
// declaration as values the vault sealed to the node they run on, which the controller carries and cannot open.
//
// **The controller is answered for a core module's own secret alone, and only sealed to that node's key.** The
// controller asks `get` for own/<node>/<core module>/<name> with the key to seal to; the vault seals only when that
// key is the sealing key the node's own engine stated in its own name (`node.<node>`, at the move: move.go). The
// controller names the key, so that a key it knows to be stale is refused loudly rather than answered with a box
// nobody opens; but it can never choose one, so a value it asks for is never sealed to a key it holds.
//
// **Where the node's key comes from.** Only from the node-engine of that machine, never from the controller's
// record of it, which is the asker's own word. A machine whose engine stated none is answered nothing.
import (
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"time"
)
// CoreModules are the core's modules whose own secrets the controller carries sealed by the vault: the same fixed
// list as the controller's (ADR 0349 §16). The node-engine and the controller have no own secrets of their own here.
var CoreModules = []string{"mesh-provisioner", "mesh-provisioner-client", "nats"}
// nodeKeyFile is a machine's sealing key as its engine stated it.
type nodeKeyFile struct {
Node string `json:"node"`
Key string `json:"key"`
By string `json:"by"`
Since time.Time `json:"since"`
}
func (s *Store) nodeKeyPath(node string) string {
return filepath.Join(s.dir, "nodes", node+".json")
}
// NodeKey is the sealing key node's engine stated (a raw X25519 public key in base64), and whether there is one.
func (s *Store) NodeKey(node string) (string, bool, error) {
if !namePart.MatchString(node) {
return "", false, fmt.Errorf("%q is not a machine's name", node)
}
raw, err := os.ReadFile(s.nodeKeyPath(node))
if errors.Is(err, os.ErrNotExist) {
return "", false, nil
}
if err != nil {
return "", false, err
}
var f nodeKeyFile
if err := json.Unmarshal(raw, &f); err != nil || f.Node != node || f.Key == "" {
return "", false, fmt.Errorf("the record of %s's sealing key is not readable", node)
}
return f.Key, true, nil
}
func (s *Store) setNodeKey(node, key, by string, at time.Time) error {
if err := os.MkdirAll(filepath.Join(s.dir, "nodes"), 0o700); err != nil {
return err
}
raw, _ := json.MarshalIndent(nodeKeyFile{Node: node, Key: key, By: by, Since: at}, "", " ")
return writeWhole(s.nodeKeyPath(node), append(raw, '\n'))
}
// sealingKey checks a sealing key a node-engine states: one raw X25519 public key in base64, never a PEM, which a
// private key could be.
func sealingKey(key string) (string, error) {
key = strings.TrimSpace(key)
if privateKeyIn(key) || strings.Contains(key, "-----") {
return "", errors.New("a machine's sealing key is stated as its raw X25519 public key in base64, never a PEM " +
"and never a private key; nothing was taken")
}
raw, err := base64.StdEncoding.DecodeString(key)
if err != nil || len(raw) != 32 {
return "", errors.New("the sealing key stated is not a raw X25519 public key in base64; nothing was taken")
}
return key, nil
}
// recordNodeKey keeps the sealing key node's engine stated, saying so when it replaces another.
func (v *Vault) recordNodeKey(node, key, caller string) error {
held, had, err := v.Store.NodeKey(node)
if err != nil {
return err
}
if had && held == key {
return nil
}
if err := v.Store.setNodeKey(node, key, caller, v.Now().UTC()); err != nil {
return fmt.Errorf("%s's sealing key could not be recorded: %w", node, err)
}
if had {
v.log("%s's sealing key is now %s, stated by %s (it was %s): the core's credentials are sealed to it from now on",
node, key, caller, held)
} else {
v.log("%s's sealing key is %s, stated by %s", node, key, caller)
}
return nil
}
// coreRefusal says why the controller may not have name sealed to key, or "" when it may: name is a core module's
// own secret and key is its machine's sealing key as that machine's engine stated it.
func (v *Vault) coreRefusal(name, key string) string {
parts := strings.Split(name, "/")
if len(parts) != 4 || parts[0] != "own" {
return fmt.Sprintf("the controller is handed a core module's own secret alone, own/<node>/<%s>/<name>, sealed "+
"to its machine (ADR 0349 §16); %s is not one", strings.Join(CoreModules, "|"), name)
}
node, module := parts[1], parts[2]
if !slices.Contains(CoreModules, module) {
return fmt.Sprintf("%s is not one of the core's modules (%s): its secrets reach its machine through the "+
"provisioner or its declaration, never through the controller's ask (ADR 0349 §16)", module,
strings.Join(CoreModules, ", "))
}
stated, held, err := v.Store.NodeKey(node)
if err != nil {
return fmt.Sprintf("%s's sealing key cannot be read: %v", node, err)
}
if !held {
return fmt.Sprintf("%s's node-engine has stated no sealing key to the vault (it does so at the move), so "+
"there is no key of that machine's own to seal %s to", node, name)
}
want, err := publicPEM(stated)
if err != nil {
return fmt.Sprintf("%s's sealing key as recorded is not usable: %v", node, err)
}
named, err := askerKey(key)
if err != nil {
return err.Error()
}
if named != want {
return fmt.Sprintf("%s is sealed only to %s's sealing key as its node-engine stated it, and the key named "+
"is another", name, node)
}
return ""
}
modules/mesh-vault/cmd/mesh-vault/core_test.go
+278
View File
@@ -0,0 +1,278 @@
package main
// Issue 763 (ADR 0349 §16, the lead's decision on issue 755): the core's own credentials travel in the declaration
// sealed by the vault to their node, which the controller carries and cannot open. The vault answers the controller
// for own/<node>/<core module>/<name> alone, sealed only to the sealing key that node's engine stated in its own name
// at the move; any other module, any other key and any other caller are refused.
import (
"crypto/ecdh"
"crypto/rand"
"crypto/x509"
"encoding/base64"
"encoding/json"
"encoding/pem"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
)
// aSealingKey is a machine's sealing key as its node-engine makes it: the public half raw in base64, the form it
// states, and the private half as the PKCS #8 PEM the ADR 0206 box opens with.
func aSealingKey(t *testing.T) (public, privatePEM string) {
t.Helper()
k, err := ecdh.X25519().GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
der, err := x509.MarshalPKCS8PrivateKey(k)
if err != nil {
t.Fatal(err)
}
return base64.StdEncoding.EncodeToString(k.PublicKey().Bytes()),
string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der}))
}
func pemOf(t *testing.T, raw string) string {
t.Helper()
p, err := publicPEM(raw)
if err != nil {
t.Fatal(err)
}
return p
}
const brokerValue = "pv-anchors-provisioner-broker-password-2025"
func clientsBroker(sealingKey string) MoveOf {
return MoveOf{Node: "anchor", Module: "mesh-provisioner-client", Name: "provisioner-broker", Origin: "made",
Generation: "g-1", SealingKey: sealingKey}
}
// The list is the controller's (mesh-controller's core list, ADR 0349 §16): named here so a change to either is seen.
func TestTheCoreIsTheBusTheProvisionerAndItsClient(t *testing.T) {
if want := []string{"mesh-provisioner", "mesh-provisioner-client", "nats"}; !reflect.DeepEqual(CoreModules, want) {
t.Fatalf("the core's modules are %v, want %v", CoreModules, want)
}
}
// A node-engine states its machine's sealing key in its own name at the move, and only its own machine's.
func TestAMoveRecordsTheSealingKeyItsNodeEngineStates(t *testing.T) {
v := newVault(t, nil)
pub, _ := aSealingKey(t)
if _, err := moveIn(t, v, "node.anchor", clientsBroker(pub), brokerValue); err != nil {
t.Fatal(err)
}
if got, held, err := v.Store.NodeKey("anchor"); err != nil || !held || got != pub {
t.Fatalf("anchor's key is %q, %v, %v", got, held, err)
}
if _, held, _ := v.Store.NodeKey("laptop"); held {
t.Error("a key was recorded for a machine that stated none")
}
// A PEM, which a private key could be, is never taken as a sealing key; nor anything not 32 bytes.
_, private := aSealingKey(t)
for _, bad := range []string{private, pemOf(t, pub), "c2hvcnQ="} {
if _, err := v.MoveOpen("node.anchor", clientsBroker(bad)); err == nil {
t.Errorf("the move took %q as a sealing key", bad[:8])
}
}
if got, _, _ := v.Store.NodeKey("anchor"); got != pub {
t.Error("a refused key replaced the one stated")
}
}
// coreOnABus is a vault holding anchor's core credential and a leaf module's secret, both moved in by anchor's
// node-engine, which stated anchor's sealing key; the controller asks it on a real bus. Answers the controller's
// ask (the box as answered), and every byte that crossed.
func coreOnABus(t *testing.T) (ask func(name, key string) (Answer, error), anchorPub, anchorPriv string,
heard func() string) {
t.Helper()
const conf = `listen: 127.0.0.1:-1
authorization {
users = [
{ user: "novox.mesh-vault", password: "pw", permissions: {
publish: { deny: [">"] }
subscribe: { allow: ["mesh.seat.mesh-vault.call.get.*"] }
allow_responses: true } }
{ user: "controller", password: "pw", permissions: {
publish: { allow: ["mesh.seat.mesh-vault.call.get.controller"] }
subscribe: { allow: ["_INBOX.controller.>"] } } }
{ user: "tap", password: "pw", permissions: { publish: { deny: [">"] }, subscribe: { allow: [">"] } } }
]
}
`
file := filepath.Join(t.TempDir(), "bus.conf")
if err := os.WriteFile(file, []byte(conf), 0o600); err != nil {
t.Fatal(err)
}
opts, err := server.ProcessConfigFile(file)
if err != nil {
t.Fatal(err)
}
opts.NoLog, opts.NoSigs = true, true
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(s.Shutdown)
url := s.ClientURL()
var mu sync.Mutex
var crossed []string
tap := connectAs(t, url, "tap")
if _, err := tap.Subscribe(">", func(m *nats.Msg) {
mu.Lock()
crossed = append(crossed, m.Subject+" "+string(m.Data))
mu.Unlock()
}); err != nil {
t.Fatal(err)
}
_ = tap.Flush()
v := newVault(t, Declared{Read: assignments{}.read})
anchorPub, anchorPriv = aSealingKey(t)
if _, err := moveIn(t, v, "node.anchor", clientsBroker(anchorPub), brokerValue); err != nil {
t.Fatal(err)
}
if _, err := moveIn(t, v, "node.anchor", superuser(), movedValue); err != nil {
t.Fatal(err)
}
vaultConn := connectAs(t, url, "novox.mesh-vault")
if _, err := vaultConn.Subscribe("mesh.seat.mesh-vault.call.get.*", func(m *nats.Msg) {
parts := strings.Split(m.Subject, ".")
var args struct {
Name string `json:"name"`
PublicKey string `json:"public_key"`
}
_ = json.Unmarshal(m.Data, &args)
answer := busAnswer{}
got, err := v.Get(untoken(parts[len(parts)-1]), args.Name, args.PublicKey)
if err != nil {
answer.Error = err.Error()
} else {
answer.Result, _ = json.Marshal(got)
}
raw, _ := json.Marshal(answer)
_ = m.Respond(raw)
}); err != nil {
t.Fatal(err)
}
_ = vaultConn.Flush()
controller := connectAs(t, url, "controller")
ask = func(name, key string) (Answer, error) {
raw, _ := json.Marshal(map[string]string{"name": name, "public_key": key})
msg, err := controller.Request("mesh.seat.mesh-vault.call.get.controller", raw, 3*time.Second)
if err != nil {
return Answer{}, err
}
var a busAnswer
if err := json.Unmarshal(msg.Data, &a); err != nil {
t.Fatal(err)
}
if a.Error != "" {
return Answer{}, errString(a.Error)
}
var got Answer
if err := json.Unmarshal(a.Result, &got); err != nil {
t.Fatal(err)
}
return got, nil
}
heard = func() string {
_ = tap.Flush()
time.Sleep(100 * time.Millisecond)
mu.Lock()
defer mu.Unlock()
return strings.Join(crossed, "\n")
}
return ask, anchorPub, anchorPriv, heard
}
type errString string
func (e errString) Error() string { return string(e) }
func TestOnARealBusTheControllerGetsACoreCredentialSealedOnlyToItsNodesKey(t *testing.T) {
ask, anchorPub, anchorPriv, heard := coreOnABus(t)
got, err := ask("own/anchor/mesh-provisioner-client/provisioner-broker", pemOf(t, anchorPub))
if err != nil {
t.Fatal(err)
}
if got.Name != "own/anchor/mesh-provisioner-client/provisioner-broker" || got.Generation != 1 {
t.Errorf("answered %+v", got)
}
opened, err := seal.Open(got.Sealed, anchorPriv)
if err != nil || opened != brokerValue {
t.Fatalf("anchor's key opens %q, %v", opened, err)
}
// The controller's own key, or any key but anchor's, is refused: it carries the box and never opens it.
mine, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
if got, err := ask("own/anchor/mesh-provisioner-client/provisioner-broker", mine.PublicKey); err == nil {
t.Errorf("the controller was handed anchor's core credential sealed to its own key: %+v", got)
} else if !strings.Contains(err.Error(), "sealing key") {
t.Errorf("refused for another reason: %v", err)
}
// A module that is not the core's is refused, though anchor's key is the one named.
if got, err := ask("own/anchor/postgres/superuser-password", pemOf(t, anchorPub)); err == nil {
t.Errorf("the controller was handed a leaf module's secret: %+v", got)
} else if !strings.Contains(err.Error(), "core") {
t.Errorf("refused for another reason: %v", err)
}
// A machine whose engine stated no key: nothing to seal to, so nothing is handed.
laptopPub, _ := aSealingKey(t)
if _, err := ask("own/laptop/nats/broker", pemOf(t, laptopPub)); err == nil {
t.Error("the controller was handed a core credential for a machine whose engine stated no key")
}
all := heard()
if !strings.Contains(all, "mesh.seat.mesh-vault.call.get.controller") {
t.Fatalf("the listener heard nothing of the asks:\n%s", all)
}
for _, v := range []string{brokerValue, movedValue} {
for _, form := range []string{v, base64.StdEncoding.EncodeToString([]byte(v))} {
if strings.Contains(all, form) {
t.Fatal("a value crossed the bus in the clear")
}
}
}
}
// Only the controller is answered by this rule: a machine's client asking for a core credential, which no plan
// names, is still refused by the running rule, and a node-engine is not an asker.
func TestTheCoreRuleAnswersTheControllerAlone(t *testing.T) {
v := newVault(t, Declared{Read: assignments{}.read})
pub, _ := aSealingKey(t)
if _, err := moveIn(t, v, "node.anchor", clientsBroker(pub), brokerValue); err != nil {
t.Fatal(err)
}
for _, caller := range []string{"provisioner.anchor", "node.anchor", "anchor.mesh-provisioner-client"} {
if _, err := v.Get(caller, "own/anchor/mesh-provisioner-client/provisioner-broker", pemOf(t, pub)); err == nil {
t.Errorf("%s was handed a core credential", caller)
}
}
// The controller is answered for nothing else: not a pair credential, not a provision's.
for _, name := range []string{"pair/anchor/postgres/anchor/gitea", "provision/anchor/postgres"} {
if _, err := v.Get(ControllerCaller, name, pemOf(t, pub)); err == nil {
t.Errorf("the controller was handed %s", name)
}
}
}
modules/mesh-vault/cmd/mesh-vault/followups_test.go
+162
View File
@@ -0,0 +1,162 @@
package main
// The four follow-ups V1's third review (review 263 on mesh-catalog#286) left for this package: the tool layer
// passes an act's words through; a rolled record of hand-outs is said in the journal; the export's fingerprint
// and vault_status are what they say; and an act's unproven `by` is kept beside the proven caller.
import (
"fmt"
"sort"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/seal"
)
func anyOf(list []string) []any {
out := make([]any, len(list))
for i, s := range list {
out[i] = s
}
return out
}
// (1) and (4): the retirement tools pass consumers, why, by, via and confirm through to the custody ledger, and
// a `by` naming somebody other than the proven caller is kept beside it.
func TestTheRetirementToolsPassTheActThrough(t *testing.T) {
c := newCustody(t)
c.ask(apps(5))
c.passes(1)
c.ask(keepOnly(apps(5), 1))
c.stable()
set := waitingNames(c.Retirement())
sort.Strings(set)
defs := definitions(&Env{Custody: c.Custody})
approve := toolNamed(t, defs, "provisioner_retire_approve")
if _, err := approve.runAs(ControllerCaller, map[string]any{"consumers": anyOf(set[:2]), "why": "x"}); err == nil {
t.Error("the tool took a set other than the one waiting")
}
got, err := approve.runAs(ControllerCaller, map[string]any{"consumers": anyOf(set), "why": "the apps are gone",
"by": "person.jochen", "via": "telegram"})
if err != nil {
t.Fatal(err)
}
if done := got.(map[string]any)["retired"].([]string); len(done) != 4 {
t.Errorf("retired %v", done)
}
last := c.bus.events[len(c.bus.events)-5]["body"].(map[string]any)
for _, e := range c.bus.events {
if b := e["body"].(map[string]any); b["change"] == "approved" {
last = b
}
}
if last["why"] != "the apps are gone" || last["by"] != "person.jochen" || last["via"] != "telegram, called by controller" {
t.Errorf("the approval said %v", last)
}
// Delete: consumer and confirm through, the act's words with them.
r := retiredC(t)
del := toolNamed(t, definitions(&Env{Custody: r.Custody}), "provisioner_delete")
if _, err := del.runAs("person.jochen", map[string]any{"consumer": "d-app", "confirm": "c-app", "why": "x"}); err == nil {
t.Error("a confirm naming another was taken through the tool")
}
if _, err := del.runAs("person.jochen", map[string]any{"consumer": "d-app", "confirm": "d-app", "why": "gone"}); err != nil {
t.Fatal(err)
}
body := r.bus.events[len(r.bus.events)-1]["body"].(map[string]any)
if body["change"] != "deleted" || body["by"] != "person.jochen" || body["via"] != nil || body["why"] != "gone" {
t.Errorf("the deletion said %v", body)
}
// Reject, by the proven caller alone: nothing more is added to via.
k := newCustody(t)
k.ask(apps(5))
k.passes(1)
k.ask(keepOnly(apps(5), 1))
k.stable()
waiting := waitingNames(k.Retirement())
if _, err := toolNamed(t, definitions(&Env{Custody: k.Custody}), "provisioner_retire_reject").runAs("person.jochen",
map[string]any{"consumers": anyOf(waiting), "why": "they come back"}); err != nil {
t.Fatal(err)
}
if body := k.bus.events[len(k.bus.events)-1]["body"].(map[string]any); body["by"] != "person.jochen" || body["via"] != nil {
t.Errorf("the rejection said %v", body)
}
}
// (2): a roll of the record of hand-outs is said in the journal, loudly.
func TestARolledRecordOfHandOutsIsSaid(t *testing.T) {
v := newVault(t, nil)
var mu sync.Mutex
var said []string
v.Log = func(format string, args ...any) {
mu.Lock()
defer mu.Unlock()
said = append(said, fmt.Sprintf(format, args...))
}
v.Store.HandedMax = 600
asker, _ := seal.GenerateKeyPair()
for i := 0; i < 20; i++ {
_, _ = v.Get("g14.flood", "pair/postgres/gitea", asker.PublicKey)
}
rolled := 0
for _, s := range said {
if strings.Contains(s, "THE RECORD OF HAND-OUTS WAS ROLLED") {
rolled++
}
}
if rolled == 0 || rolled != v.Store.Rolled() {
t.Errorf("rolled %d times, said %d times", v.Store.Rolled(), rolled)
}
}
// (3): the vault's export names the operator key's fingerprint as the controller's secrets.Fingerprint says it,
// and vault_status says what needs a person.
func TestTheExportsFingerprintAndVaultStatus(t *testing.T) {
v := newVault(t, nil)
if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil {
t.Fatal(err)
}
answer, err := toolNamed(t, definitions(&Env{Vault: v.Vault}), "secret_vault_export").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
doc := answer.(map[string]any)
// sha256 of the key, its first eight bytes in hex: the controller's secrets.Fingerprint.
if fp := doc["fingerprint"].(string); !strings.HasPrefix(fp, "sha256:") || len(fp) != len("sha256:")+16 ||
fp != keyFingerprint(v.operatorPublic) || doc["operator-key"] != v.operatorPublic {
t.Errorf("the export says %v", doc)
}
if keyFingerprint("") != "" {
t.Error("no key has a fingerprint")
}
env := &Env{Vault: v.Vault}
status := func() map[string]any {
got, err := toolNamed(t, definitions(env), "vault_status").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
return got.(map[string]any)
}
if s := status(); s["ok"] != true || s["copies_on_an_earlier_key"] != 0 || s["operator_key"] != v.operatorPublic {
t.Errorf("a vault in order says %v", s)
}
env.refuseKey("a private key was offered")
if s := status(); s["ok"] != false || s["operator_key_refused"] != "a private key was offered" {
t.Errorf("a refused key says %v", s)
}
env.refuseKey("")
// A new operator key whose resealing fails leaves copies on the earlier key, and the status says so.
pub, _ := operatorKey(t)
v.Store.failWrite = func(Record) error { return fmt.Errorf("disk full") }
_, _ = v.SetOperatorKey(asPEM(t, pub), "the test")
v.Store.failWrite = nil
if s := status(); s["ok"] != false || s["copies_on_an_earlier_key"] != 1 {
t.Errorf("a copy on an earlier key says %v", s)
}
if s, _ := toolNamed(t, definitions(&Env{}), "vault_status").Run(map[string]any{}); s.(map[string]any)["ready"] != false {
t.Errorf("no vault says %v", s)
}
}
modules/mesh-vault/cmd/mesh-vault/form_test.go
+221
View File
@@ -0,0 +1,221 @@
package main
// A provider the vault holds no row of yet gets its first value in the form its offer declares (issue 768): the
// controller names the form in each consumer's plan entry (`credential-form`, from the provider's manifest), and
// the vault reads it strictly. A provider the vault already holds rows of is unchanged by it.
import (
"reflect"
"strings"
"testing"
)
func TestANewSharedProviderGetsOneFirstValueForEveryHolder(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in+shared"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in+shared"),
}
v := pairVault(t, mirror)
if o := pass(t, v, standIn); o.Outcome != PairMade || o.Generation != 1 {
t.Fatalf("a new shared provider: %+v", o)
}
value, _, err := asks(v, "provisioner.anchor", standIn)
if err != nil || value == "" {
t.Fatalf("the provider's machine was refused its first value: %v", err)
}
for _, c := range []string{"provisioner.g14", "provisioner.ace"} {
if got, _, err := asks(v, c, standIn); err != nil || got != value {
t.Fatalf("%s got %q (%v), not the one value", c, got, err)
}
}
if changedSaid(t, v, standIn, value) != 1 {
t.Fatal("secret-changed was not said once for the first value")
}
rows, _ := v.pairRows()
if len(rows[pairKey{"anchor", "stand-in-api"}]) != 0 {
t.Fatal("a login per consumer was made for a provider whose offer shares one credential")
}
if o := pass(t, v, standIn); o.Outcome != PairUnchanged {
t.Fatalf("the pass after the first value: %+v", o)
}
}
func TestANewPerLoginProviderGetsAFirstLoginOfItsOwnForEachConsumer(t *testing.T) {
const pg = "provision/novox/postgres-database"
mirror := assignments{
"novox.postgres": handedWith("novox", "postgres"),
"g14.gitea": handedWith("g14", "gitea", "postgres-database@novox/postgres+pair"),
"ace.umami": handedWith("ace", "umami", "postgres-database@novox/postgres#stats+pair"),
}
v := pairVault(t, mirror)
if o := pass(t, v, pg); o.Outcome != PairMade {
t.Fatalf("a new per-login provider: %+v", o)
}
if _, held, _ := v.Store.Read(pg); held {
t.Fatal("one shared value was made for a provider whose consumers each keep a login of their own")
}
gitea, umami := "pair/novox/postgres-database/g14/gitea", "pair/novox/postgres-database/ace/umami/stats"
ofGitea, _, err := asks(v, "provisioner.g14", gitea)
if err != nil || ofGitea == "" {
t.Fatalf("gitea's machine was refused its first login: %v", err)
}
ofUmami, _, err := asks(v, "provisioner.ace", umami)
if err != nil || ofUmami == "" {
t.Fatalf("umami's machine was refused its first login: %v", err)
}
if ofGitea == ofUmami {
t.Fatal("two consumers of a per-login provider were given one value")
}
if _, _, err := asks(v, "provisioner.g14", umami); err == nil {
t.Fatal("gitea's machine was handed umami's login")
}
for name, want := range map[string]string{gitea: ofGitea, umami: ofUmami} {
if got, _, err := asks(v, "provisioner.novox", name); err != nil || got != want {
t.Fatalf("the provider's machine got %q for %s (%v)", got, name, err)
}
}
// The logins are rows now: the next pass remakes none of them.
if o := pass(t, v, pg); o.Outcome != PairPerPair {
t.Fatalf("the pass after the first logins: %+v", o)
}
if got, gen, _ := asks(v, "provisioner.g14", gitea); got != ofGitea || gen != 1 {
t.Fatal("a consumer's login was remade by the next pass")
}
}
// A provider the vault holds rows of keeps what it had, whatever form the plans name.
func TestAProviderWithRowsIsUnchangedByTheFormItsPlansName(t *testing.T) {
t.Run("stamped rows", func(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in+shared"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "the-value-they-share", "stamp-1", OriginMade)
if o := pass(t, v, standIn); o.Outcome != PairAdopted {
t.Fatalf("stamped rows: %+v", o)
}
if got, gen, _ := asks(v, "provisioner.g14", standIn); got != "the-value-they-share" || gen != 1 {
t.Fatal("the adopted value is not the move's")
}
})
t.Run("logins of their own", func(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in+shared"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in+pair"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "login-of-app", "", OriginMade)
movedRow(t, v, "ace", "other", "login-of-other", "", OriginMade)
if o := pass(t, v, standIn); o.Outcome != PairPerPair {
t.Fatalf("unstamped rows: %+v", o)
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatal("a shared value was made for a provider whose rows are logins of their own")
}
if got, gen, _ := asks(v, "provisioner.g14", "pair/anchor/stand-in-api/g14/app"); got != "login-of-app" ||
gen != 1 {
t.Fatal("a consumer's own login changed")
}
})
}
// A form the vault does not make (issued by the provider's software, or given by a person), a form it does not know,
// or plans that disagree: nothing is made, and the pass says why. The rule that no holder is left behind still holds.
func TestTheVaultMakesNothingForAFormItDoesNotMakeOrCannotTrust(t *testing.T) {
for form, said := range map[string]string{
"issued": "issued by the provider's software",
"given": "given by a person",
"one-for-all": "which is not a credential form",
"shared|pair": "disagree",
"shared|": "no plan says whether",
"shared@unmoved": "stand-in on anchor",
} {
t.Run(form, func(t *testing.T) {
provider := handedWith("anchor", "stand-in")
first, second := form, form
if a, b, ok := strings.Cut(form, "|"); ok {
first, second = a, b
}
if a, _, ok := strings.Cut(form, "@"); ok {
first, second, provider = a, a, unmoved("anchor", "stand-in")
}
entry := func(f string) string {
if f == "" {
return "stand-in-api@anchor/stand-in"
}
return "stand-in-api@anchor/stand-in+" + f
}
mirror := assignments{
"anchor.stand-in": provider,
"g14.app": handedWith("g14", "app", entry(first)),
"ace.other": handedWith("ace", "other", entry(second)),
}
v := pairVault(t, mirror)
if o := pass(t, v, standIn); o.Outcome != PairHeld || !strings.Contains(o.Why, said) {
t.Fatalf("form %s: %+v", form, o)
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatalf("form %s: a shared value was made", form)
}
if rows, _ := v.pairRows(); len(rows[pairKey{"anchor", "stand-in-api"}]) != 0 {
t.Fatalf("form %s: a login was made", form)
}
})
}
}
// A consumer that binds after the move to a per-login provider the vault already holds rows of gets a login of its
// own (issue 771): the rows that exist are never touched, and `secret-changed` is said for the new login only.
func TestANewConsumerOfAPerLoginProviderWithRowsGetsALoginOfItsOwn(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in+pair"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in+pair"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "login-of-app", "", OriginMade)
movedRow(t, v, "ace", "other", "login-of-other", "", OriginMade)
app, other := "pair/anchor/stand-in-api/g14/app", "pair/anchor/stand-in-api/ace/other"
third := "pair/anchor/stand-in-api/novox/third"
before, said := map[string]Record{}, map[string]int{}
for _, n := range []string{app, other} {
r, _, _ := v.Store.Read(n)
before[n], said[n] = r, changedSaid(t, v, n, "")
}
mirror["novox.third"] = handedWith("novox", "third", "stand-in-api@anchor/stand-in+pair")
if o := pass(t, v, standIn); o.Outcome != PairMade {
t.Fatalf("a third consumer binding: %+v", o)
}
got, _, err := asks(v, "provisioner.novox", third)
if err != nil || got == "" || got == "login-of-app" || got == "login-of-other" {
t.Fatalf("the third consumer has no login of its own: %q (%v)", got, err)
}
if got, _, err := asks(v, "provisioner.anchor", third); err != nil || got == "" {
t.Fatalf("the provider's machine was refused the new login: %v", err)
}
for n, was := range before {
now, _, _ := v.Store.Read(n)
if !reflect.DeepEqual(now, was) {
t.Fatalf("%s was touched: %+v, was %+v", n, now, was)
}
if changedSaid(t, v, n, "") != said[n] {
t.Fatalf("secret-changed was said for %s", n)
}
}
if changedSaid(t, v, third, "") != 1 {
t.Fatal("secret-changed was not said once for the new login")
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatal("a shared value was made")
}
if o := pass(t, v, standIn); o.Outcome != PairPerPair {
t.Fatalf("the pass after the new login: %+v", o)
}
if got2, _, _ := asks(v, "provisioner.novox", third); got2 != got {
t.Fatal("the new login was remade by the next pass")
}
}
modules/mesh-vault/cmd/mesh-vault/intake.go
+760
View File
@@ -0,0 +1,760 @@
package main
// What the vault takes in, and how it rotates (novox/hq ADR 0349 clauses 8 to 10; ADR 0277 §22–§27 and ADR 0305
// with the vault in the controller's place; design 57's V2, issue 675).
//
// Four verbs of the seat: `accept` (a value given at the controller's terminal), `secret-ask` (a value the
// operator types at a desk), `issued` (a value a provider's software issued) and `rotate`.
//
// **A value comes in sealed to a key the vault made for that one call, and opens only here.** Each call opens with
// the public half of a key made for it (`seal_to`, a raw X25519 key in base64, the NaCl anonymous box the desk's
// prompt and a provider's tool already seal to) and ends with what was sealed to that key, which the vault opens
// with the private half: it never leaves this process and is forgotten once used or after two minutes.
//
// **The vault asks the desk and the provider itself** (issue 723). The desk's prompt and a provider's issuing tool
// are asked by the vault, in its own name, under the bus account its own runtime runs on (ADR 0259 §8's runs-as):
// the bus lets that account alone call them naming the vault, so the prompt and the tool know the vault asks, and
// what they seal to the call's key opens here and nowhere else (ADR 0349 clauses 9 and 10, ADR 0277 §23). The
// controller asks `secret-ask` and `issued` once, carrying no key and no sealed value; it reaches the desk and the
// tool itself only for a module still filled the old way, until R1 (issue 682). Only `accept` stays two steps:
// the value is given at the controller's terminal, which seals it there to the call's key.
//
// Which secret each one is, by name in the vault's store:
//
// own/<node>/<module>/<name> a module's own secret on one machine
// provision/<provider node>/<provision> the one credential of a provider assignment and provision (clause 8)
import (
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"regexp"
"slices"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"golang.org/x/crypto/nacl/box"
)
// ControllerCaller is the controller's bus user: the one caller that carries a value in (broker.ControllerName).
const ControllerCaller = "controller"
// Where a value came from.
const (
OriginMade = "made"
OriginAccepted = "accepted"
OriginTyped = "typed"
OriginIssued = "issued"
)
// CallOpenFor is how long a call's key waits for its sealed value: longer than any prompt or tool waits.
const CallOpenFor = 2 * time.Minute
// The bounds of asking for one secret (ADR 0277 §24, the controller's inventory.SecretAsk* until now).
const (
// SecretAskOpenFor is how long an ask that has not ended counts as open.
SecretAskOpenFor = 2 * time.Minute
// SecretAsksPerHour is how many asks for one secret an hour takes.
SecretAsksPerHour = 3
)
var namePart = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]{0,62}$`)
// OwnSecret is the vault's name for a module's own secret on one machine.
func OwnSecret(node, module, name string) (string, error) {
for what, v := range map[string]string{"the machine": node, "the module": module, "the secret": name} {
if !namePart.MatchString(v) {
return "", fmt.Errorf("%s %q is not a name the mesh writes", what, v)
}
}
return "own/" + node + "/" + module + "/" + name, nil
}
// ProvisionSecret is the vault's name for the one credential of a provider assignment and provision.
func ProvisionSecret(provider, provision string) (string, error) {
for what, v := range map[string]string{"the provider's machine": provider, "the provision": provision} {
if !namePart.MatchString(v) {
return "", fmt.Errorf("%s %q is not a name the mesh writes", what, v)
}
}
return "provision/" + provider + "/" + provision, nil
}
// callKey is one call's key and what the call takes in. Never written anywhere.
type callKey struct {
verb, secret, by string
// origin is what an `accept` call keeps the value as: accepted, or issued for a covered account (issue 787).
origin string
// ask is the ask's record for `secret-ask`, which the call's id is.
ask *AskRecord
// move is what `move-in` takes: the row of the controller's store (move.go).
move *MoveOf
public [32]byte
private [32]byte
until time.Time
}
// Opened is a first step's answer: the call, and the key to seal its value to. Never a value.
type Opened struct {
Call string `json:"call"`
SealTo string `json:"seal_to"`
Secret string `json:"secret"`
AskedBy string `json:"asked_by,omitempty"`
}
// fromTheController refuses every caller but the controller: only it asks the vault to take a value in, and only
// it carries a value given at its terminal.
func fromTheController(caller, verb string) error {
if caller != ControllerCaller {
if caller == "" {
caller = stdio.Unproven
}
return fmt.Errorf("%s takes a value from the controller alone, which carries it sealed to this vault's key "+
"for one call; %s is not the controller", verb, caller)
}
return nil
}
// openCall makes a key for one call and keeps its private half here until the call is taken or two minutes
// have passed.
func (v *Vault) openCall(verb, secret, by string, ask *AskRecord) (string, string, error) {
pub, priv, err := box.GenerateKey(rand.Reader)
if err != nil {
return "", "", err
}
idRaw := make([]byte, 12)
if _, err := rand.Read(idRaw); err != nil {
return "", "", err
}
id := hex.EncodeToString(idRaw)
if ask != nil {
id = ask.ID
}
v.callsMu.Lock()
defer v.callsMu.Unlock()
if v.pending == nil {
v.pending = map[string]*callKey{}
}
now := v.Now()
for k, c := range v.pending {
if now.After(c.until) {
delete(v.pending, k)
}
}
v.pending[id] = &callKey{verb: verb, secret: secret, by: by, ask: ask, public: *pub, private: *priv,
until: now.Add(CallOpenFor)}
return id, base64.StdEncoding.EncodeToString(pub[:]), nil
}
// takeCall ends a call: its key is forgotten whatever follows.
func (v *Vault) takeCall(verb, id string) (*callKey, error) {
v.callsMu.Lock()
defer v.callsMu.Unlock()
c, ok := v.pending[id]
if !ok || c.verb != verb {
return nil, fmt.Errorf("no %s call %q is open in the vault: it ended, it was never opened, or the vault "+
"started again; open it anew", verb, id)
}
delete(v.pending, id)
if v.Now().After(c.until) {
return nil, fmt.Errorf("the %s call %q waited longer than %s and is closed; open it anew", verb, id, CallOpenFor)
}
return c, nil
}
// open opens what was sealed to the call's key, and forgets the key.
func (c *callKey) open(sealed string) ([]byte, error) {
defer func() { c.private = [32]byte{} }()
return c.openOne(sealed)
}
// openOne opens one value sealed to the call's key and keeps the key, for a call that takes several (one per
// holder, perholder.go); the caller forgets the key.
func (c *callKey) openOne(sealed string) ([]byte, error) {
blob, err := base64.StdEncoding.DecodeString(strings.TrimSpace(sealed))
if err != nil {
return nil, errors.New("the value is not sealed to this call; nothing was taken")
}
out, ok := box.OpenAnonymous(nil, blob, &c.public, &c.private)
if !ok {
// Never the value, never what failed to open: only that it was not sealed to this call.
return nil, errors.New("the value is not sealed to this call; nothing was taken")
}
return out, nil
}
func zero(b []byte) {
for i := range b {
b[i] = 0
}
}
// ---- accept -------------------------------------------------------------------------------------------------
// CoveredSecrets are the names, in a module's own secrets, of the bus accounts the controller mints for a core module
// or a module marked moved and hands to the vault (novox/hq ADR 0374, issue 787): the module's account on the bus
// (`broker`) and, for the machine's provisioner client, its own (`provisioner-broker`).
var CoveredSecrets = []string{"broker", "provisioner-broker"}
// Covered says whether secret is a covered account: a module's own secret named one of CoveredSecrets. The vault
// cannot tell core or moved from a module's manifest; the controller, which can, is the only caller that may hand
// one in, and the name keeps what it hands to these.
func Covered(secret string) bool {
parts := strings.Split(secret, "/")
return len(parts) == 4 && parts[0] == "own" && slices.Contains(CoveredSecrets, parts[3])
}
// AcceptOpen opens a call to take a value given at the controller's terminal for secret, which the terminal
// seals to the call's key before it leaves it (ADR 0349 clause 8: the vault alone opens an accepted value).
//
// **With origin issued it is a covered account the controller minted** (ADR 0374, issue 787): the controller alone
// hands it, for a covered secret alone, and a re-issue replaces the value in place. Without origin it is a value
// given at the terminal, as before.
func (v *Vault) AcceptOpen(caller, secret, by, origin string) (Opened, error) {
if err := fromTheController(caller, "accept"); err != nil {
return Opened{}, err
}
if err := validName(secret); err != nil {
return Opened{}, err
}
kept := OriginAccepted
switch origin {
case "":
if rec, held, err := v.Store.Read(secret); err != nil {
return Opened{}, err
} else if held && rec.Origin == OriginIssued && !Covered(secret) {
return Opened{}, fmt.Errorf("%s is issued by its provider's software, and `secret issued` is the only way "+
"it enters the vault (ADR 0305 §10); nothing is taken", secret)
}
case OriginIssued:
if !Covered(secret) {
return Opened{}, fmt.Errorf("%s is not a bus account the controller mints for a core or moved module "+
"(%s), so it is not handed with origin issued; nothing is taken", secret, strings.Join(CoveredSecrets, ", "))
}
kept = OriginIssued
default:
return Opened{}, fmt.Errorf("the origin %q is not one accept takes (issued, or none for a value given at the "+
"terminal); nothing is taken", origin)
}
id, key, err := v.openCall("accept", secret, by, nil)
if err != nil {
return Opened{}, err
}
v.callsMu.Lock()
if c := v.pending[id]; c != nil {
c.origin = kept
}
v.callsMu.Unlock()
return Opened{Call: id, SealTo: key, Secret: secret}, nil
}
// AcceptTake opens the value sealed to the call's key and keeps it, accepted: never remade by the vault.
func (v *Vault) AcceptTake(caller, call, sealed string) (Changed, error) {
if err := fromTheController(caller, "accept"); err != nil {
return Changed{}, err
}
c, err := v.takeCall("accept", call)
if err != nil {
return Changed{}, err
}
value, err := c.open(sealed)
if err != nil {
return Changed{}, err
}
defer zero(value)
if strings.TrimSpace(string(value)) == "" {
return Changed{}, errors.New("the value given is empty; nothing was taken")
}
origin := c.origin
if origin == "" {
origin = OriginAccepted
}
return v.keep(c.secret, string(value), someoneOr(c.by, caller), caller, origin)
}
// ---- secret-ask ---------------------------------------------------------------------------------------------
// AskOf is what an ask is for, and where.
type AskOf struct {
Node, Module, Name, Desk string
// AskedBy is who asked the controller, as the bus named its caller; the controller's word, taken from the
// controller alone.
AskedBy string
}
// AskOpen records an ask for a module's own secret before any prompt opens, holding its bounds — one open ask
// per secret, three an hour (ADR 0277 §24, §25) — and answers the key the desk's prompt seals what is typed to,
// and who asked as the record says it, which the prompt names (§26).
func (v *Vault) AskOpen(caller string, a AskOf) (Opened, error) {
if err := fromTheController(caller, "secret-ask"); err != nil {
return Opened{}, err
}
secret, err := OwnSecret(a.Node, a.Module, a.Name)
if err != nil {
return Opened{}, fmt.Errorf("nobody was asked to type anything: %w", err)
}
if !namePart.MatchString(a.Desk) {
return Opened{}, fmt.Errorf("nobody was asked to type anything: the desk %q is not a machine's name", a.Desk)
}
askedBy := askedByName(a.AskedBy)
v.Store.asksMu.Lock()
defer v.Store.asksMu.Unlock()
now := v.Now().UTC()
recent, err := v.Store.AsksSince(now.Add(-time.Hour))
if err != nil {
return Opened{}, fmt.Errorf("nobody was asked to type anything: the record of asks cannot be read: %w", err)
}
ended := map[string]bool{}
for _, r := range recent {
if r.Event == AskEnded {
ended[r.ID] = true
}
}
lastHour, openBy := 0, ""
for _, r := range recent {
if r.Event != AskOpened || r.Secret != secret {
continue
}
lastHour++
if !ended[r.ID] && r.At.After(now.Add(-SecretAskOpenFor)) {
openBy = r.AskedBy
}
}
if openBy != "" {
return Opened{}, fmt.Errorf("an ask for %s of %s on %s is still open (asked by %s): one prompt at a time for a "+
"secret, and this one ends within %s", a.Name, a.Module, a.Node, openBy, SecretAskOpenFor)
}
if lastHour >= SecretAsksPerHour {
return Opened{}, fmt.Errorf("%s of %s on %s was asked for %d times in the last hour, and an hour takes %d asks "+
"for one secret: the operator is not kept at a prompt", a.Name, a.Module, a.Node, lastHour, SecretAsksPerHour)
}
idRaw := make([]byte, 12)
if _, err := rand.Read(idRaw); err != nil {
return Opened{}, err
}
rec := AskRecord{ID: hex.EncodeToString(idRaw), At: now, Event: AskOpened, Secret: secret, Node: a.Node,
Module: a.Module, Name: a.Name, Desk: a.Desk, AskedBy: askedBy, Caller: caller}
if err := v.recordAsk(rec); err != nil {
return Opened{}, fmt.Errorf("nobody was asked to type anything: the ask could not be recorded: %w", err)
}
id, key, err := v.openCall("secret-ask", secret, askedBy, &rec)
if err != nil {
_ = v.recordAsk(AskRecord{ID: rec.ID, At: v.Now().UTC(), Event: AskEnded, Secret: secret, Outcome: "failed"})
return Opened{}, err
}
return Opened{Call: id, SealTo: key, Secret: secret, AskedBy: askedBy}, nil
}
// The outcomes an ask ends with, as the controller's record had them.
var askOutcomes = map[string]bool{"dismissed": true, "timed-out": true, "empty": true, "refused": true, "failed": true}
// AskEnd ends an ask: with what the desk sealed to its key, which the vault opens and keeps as typed, or with
// the outcome the desk answered instead. How it ended is recorded whatever happens.
func (v *Vault) AskEnd(caller, call, sealed, outcome string) (Changed, error) {
if err := fromTheController(caller, "secret-ask"); err != nil {
return Changed{}, err
}
if sealed == "" && !askOutcomes[outcome] {
return Changed{}, fmt.Errorf("an ask ends with what was sealed to its key, or with one of dismissed, " +
"timed-out, empty, refused or failed")
}
c, err := v.takeCall("secret-ask", call)
if err != nil {
return Changed{}, err
}
end := func(how string) {
v.Store.asksMu.Lock()
defer v.Store.asksMu.Unlock()
if err := v.recordAsk(AskRecord{ID: c.ask.ID, At: v.Now().UTC(), Event: AskEnded, Secret: c.secret,
Outcome: how}); err != nil {
v.log("the end of ask %s (%s) could not be recorded: %v", c.ask.ID, how, err)
}
}
if sealed == "" {
end(outcome)
return Changed{Name: c.secret, Unchanged: true}, nil
}
opened, err := c.open(sealed)
if err != nil {
end("failed")
return Changed{}, err
}
value := strings.TrimRight(string(opened), "\r\n")
zero(opened)
if strings.TrimSpace(value) == "" {
end("empty")
return Changed{}, fmt.Errorf("%w: the prompt on %s was answered empty", ErrNothingGiven, c.ask.Desk)
}
changed, err := v.keep(c.secret, value, c.ask.AskedBy, caller, OriginTyped)
value = ""
if err != nil {
end("failed")
return Changed{}, err
}
end("given")
return changed, nil
}
// recordAsk appends one line to the record of asks. The caller holds Store.asksMu, so no two writers roll the
// record at once and no ask is counted against its bounds while another line is half written.
func (v *Vault) recordAsk(r AskRecord) error {
rolled, err := v.Store.RecordAsk(r)
if rolled {
v.log("THE RECORD OF ASKS WAS ROLLED: it reached its bound of %d bytes; the oldest of %d kept files is "+
"dropped. Many asks in little time is worth a look: secret_asks", v.Store.HandedBound(), HandedKept)
}
return err
}
// askedByName is who asked as the prompt and the record name it: the first clause of what the bus said, in the
// characters a name has, at most 80 (the controller's own rule, ADR 0277 §26), so no words of the caller's own
// reach the prompt.
func askedByName(caller string) string {
first, _, _ := strings.Cut(caller, ",")
var b strings.Builder
for _, r := range strings.TrimSpace(first) {
if b.Len() >= 80 {
break
}
switch {
case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '.', r == '_', r == '/', r == '@',
r == '-', r == ' ':
b.WriteRune(r)
default:
b.WriteRune('-')
}
}
name := strings.TrimSpace(b.String())
if name == "" || strings.HasPrefix(name, "-") {
return "an unnamed caller"
}
return name
}
// ---- issued -------------------------------------------------------------------------------------------------
// IssuedOpen opens a call to take the credential a provider's software issued for a provision (ADR 0349 clause
// 10, ADR 0305 with the vault in the controller's place): the controller asks the provider's tool, on the
// provider's machine, with the call's key, and passes on what it answered.
func (v *Vault) IssuedOpen(caller, provision, provider, module string) (Opened, error) {
if err := fromTheController(caller, "issued"); err != nil {
return Opened{}, err
}
secret, err := ProvisionSecret(provider, provision)
if err != nil {
return Opened{}, err
}
if !namePart.MatchString(module) {
return Opened{}, fmt.Errorf("the provider's module %q is not a name the mesh writes", module)
}
id, key, err := v.openCall("issued", secret, module+" on "+provider, nil)
if err != nil {
return Opened{}, err
}
return Opened{Call: id, SealTo: key, Secret: secret}, nil
}
// IssuedTake opens what the provider's tool sealed to the call's key and keeps it as the provision's one
// credential. The same value again writes nothing (ADR 0305 §5): every holder that has it is left as it is.
func (v *Vault) IssuedTake(caller, call, sealed string) (Changed, error) {
if err := fromTheController(caller, "issued"); err != nil {
return Changed{}, err
}
c, err := v.takeCall("issued", call)
if err != nil {
return Changed{}, err
}
opened, err := c.open(sealed)
if err != nil {
return Changed{}, err
}
value := strings.TrimSpace(string(opened))
zero(opened)
if value == "" {
return Changed{}, fmt.Errorf("%s answered an empty value; nothing was taken", c.by)
}
return v.keep(c.secret, value, c.by, caller, OriginIssued)
}
// ---- the vault's own calls (issue 723) -----------------------------------------------------------------------
// DeskPromptFor is how long the desk's prompt waits for the operator when the vault asks: under the thirty seconds
// a call through the runtime waits, and under the controller's wait for `secret-ask`.
const DeskPromptFor = 20
// DeskPrompt is the desk's secret prompt as the vault asks it: the seat's verb on the desk's machine.
func DeskPrompt(desk string) string { return "seat:node-launcher.secret@" + desk }
// Asked is how an ask at a desk ended: what was kept, or why nothing was. Never the value.
type Asked struct {
Secret string `json:"secret"`
Generation int `json:"generation,omitempty"`
AskedBy string `json:"asked_by"`
// Outcome is given, dismissed, timed-out, empty, refused or failed.
Outcome string `json:"outcome"`
Unchanged bool `json:"unchanged,omitempty"`
}
// AskAtDesk is `secret-ask` (ADR 0349 clause 9, ADR 0277 §22–§27): asked by the controller alone, which names who
// asked it. The ask is recorded and its bounds held before any prompt opens; the vault asks the desk's prompt
// itself, in its own name, naming the asker as recorded and giving a key made for this one call; what is typed
// comes back sealed to that key and opens here. How the prompt ended is the answer and the record's.
func (v *Vault) AskAtDesk(caller string, a AskOf) (Asked, error) {
if v.Ask == nil {
return Asked{}, errors.New("nobody was asked to type anything: this vault reaches no desk")
}
opened, err := v.AskOpen(caller, a)
if err != nil {
return Asked{}, err
}
out := Asked{Secret: opened.Secret, AskedBy: opened.AskedBy}
end := func(outcome string) (Asked, error) {
out.Outcome = outcome
if _, err := v.AskEnd(caller, opened.Call, "", outcome); err != nil {
return out, err
}
return out, nil
}
raw, err := v.Ask(DeskPrompt(a.Desk), map[string]any{"module": a.Module, "secret": a.Name, "node": a.Node,
"asked_by": opened.AskedBy, "seal_to": opened.SealTo, "timeout_seconds": DeskPromptFor})
if err != nil {
if _, ended := end("refused"); ended != nil {
v.log("%v", ended)
}
return Asked{}, fmt.Errorf("the desk on %s could not be asked: %w", a.Desk, err)
}
var answer struct {
Sealed string `json:"sealed"`
Cancelled bool `json:"cancelled"`
TimedOut bool `json:"timed_out"`
}
if err := json.Unmarshal(raw, &answer); err != nil {
return end("failed")
}
switch {
case answer.TimedOut:
return end("timed-out")
case answer.Cancelled:
return end("dismissed")
case answer.Sealed == "":
return end("failed")
}
changed, err := v.AskEnd(caller, opened.Call, answer.Sealed, "")
if err != nil {
// AskEnd recorded how it ended; an empty answer is the operator's, not a failure.
if errors.Is(err, ErrNothingGiven) {
out.Outcome = "empty"
return out, nil
}
return Asked{}, err
}
out.Outcome, out.Generation, out.Unchanged = "given", changed.Generation, changed.Unchanged
return out, nil
}
// ErrNothingGiven is a prompt answered empty: the operator's answer, not a failure.
var ErrNothingGiven = errors.New("nothing was given")
// Issued is what the vault kept of a value a provider's software issued. Never the value. For a tool that answers a
// value per holder (perholder.go), Secret is the provision's name, Holders what was kept for each and NotAnswered
// the holders asked about that the tool gave nothing for; Unchanged is then whether no holder's value changed.
type Issued struct {
Secret string `json:"secret"`
Generation int `json:"generation"`
Unchanged bool `json:"unchanged,omitempty"`
Holders []IssuedHolder `json:"holders,omitempty"`
NotAnswered []Holder `json:"not_answered,omitempty"`
}
// TakeIssued is `issued` (ADR 0349 clause 10, ADR 0305 with the vault in the controller's place): asked by the
// controller alone, naming the provision and the provider's machine, module and tool. The vault asks that tool
// itself, in its own name, with a key made for this one call; the tool answers its software's value sealed to that
// key, which opens here and is kept as the provision's one credential. The bus lets the vault's account call no
// tool but an issuing one, so a tool named here that is not one is refused there.
func (v *Vault) TakeIssued(caller, provision, provider, module, tool string) (Issued, error) {
return v.TakeIssuedFor(caller, provision, provider, module, tool, nil)
}
// TakeIssuedFor is TakeIssued naming the provision's holders from that provider, as the controller knows them (issue
// 749): the tool is asked with them, and may answer one value for the provision (`sealed`, kept as its one
// credential, as before) or one per holder (`holders`: node, module and sealed each, kept as that holder's own and
// handed to it alone, perholder.go). Never both.
func (v *Vault) TakeIssuedFor(caller, provision, provider, module, tool string, holders []Holder) (Issued, error) {
if v.Ask == nil {
return Issued{}, errors.New("nothing was taken: this vault reaches no provider")
}
if !toolName.MatchString(tool) {
return Issued{}, fmt.Errorf("the provider's tool %q is not a tool's name", tool)
}
if err := checkHolders(holders); err != nil {
return Issued{}, fmt.Errorf("nothing was asked: %w", err)
}
opened, err := v.IssuedOpen(caller, provision, provider, module)
if err != nil {
return Issued{}, err
}
args := map[string]any{"provision": provision, "seal_to": opened.SealTo}
if len(holders) > 0 {
args["holders"] = holders
}
raw, err := v.Ask(module+"."+tool+"@"+provider, args)
if err != nil {
v.dropCall("issued", opened.Call)
return Issued{}, fmt.Errorf("its tool %s could not be asked: %w", tool, err)
}
var given struct {
Sealed string `json:"sealed"`
Holders []sealedFor `json:"holders"`
}
if err := json.Unmarshal(raw, &given); err != nil || (given.Sealed == "") == (len(given.Holders) == 0) {
v.dropCall("issued", opened.Call)
return Issued{}, fmt.Errorf("its tool %s answered no sealed value, or both one value and a value per holder", tool)
}
if len(given.Holders) > 0 {
return v.takePerHolder(caller, opened.Call, provider, provision, holders, given.Holders)
}
changed, err := v.IssuedTake(caller, opened.Call, given.Sealed)
if err != nil {
return Issued{}, err
}
return Issued{Secret: changed.Name, Generation: changed.Generation, Unchanged: changed.Unchanged}, nil
}
var toolName = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]{0,62}$`)
// dropCall forgets a call's key that will not be answered.
func (v *Vault) dropCall(verb, id string) {
v.callsMu.Lock()
defer v.callsMu.Unlock()
if c, ok := v.pending[id]; ok && c.verb == verb {
c.private = [32]byte{}
delete(v.pending, id)
}
}
// ---- rotate -------------------------------------------------------------------------------------------------
// Rotate makes a secret anew (ADR 0114, ADR 0228). Refused: a secret the vault does not hold; a value a
// provider's software issued, which the software rotates (ADR 0305 §9); and a provision's accepted credential,
// which the vault does not remake (ADR 0349 clause 8, ADR 0158 §8). A module's own value given by hand is
// rotated like a made one (ADR 0228). Who asked is the person the controller names, beside the controller that
// carried the act — both are kept — and why is said in the vault's log.
//
// **From the controller alone, and with a why** (the review of issue 675): only the controller knows whether a
// module's secrets are filled from the vault yet (ADR 0349 clause 17); a rotation asked of the vault for a module
// still filled the old way would change a value its machine never reads, and drift from the one the controller
// holds. And a rotation is an act by hand, which says why.
func (v *Vault) Rotate(caller, secret, by, why string) (Changed, error) {
return v.RotateFor(caller, secret, Narrow{}, by, why)
}
// Narrow is the one consumer a rotation of a per-login provider's credential is for (issue 767): the controller's
// `rotate <provision> --consumer <machine> --module <module>`, either or both. Empty is every consumer.
type Narrow struct {
Consumer string `json:"consumer,omitempty"`
ConsumerModule string `json:"consumer_module,omitempty"`
}
func (n Narrow) empty() bool { return n.Consumer == "" && n.ConsumerModule == "" }
func (n Narrow) holds(r pairRow) bool {
return (n.Consumer == "" || r.consumer == n.Consumer) && (n.ConsumerModule == "" || r.module == n.ConsumerModule)
}
func (n Narrow) String() string {
switch {
case n.Consumer == "":
return n.ConsumerModule
case n.ConsumerModule == "":
return n.Consumer
}
return n.ConsumerModule + " on " + n.Consumer
}
// RotateFor is Rotate, narrowed to one consumer's own login of a per-login provider (ADR 0158 §11, issue 767). A
// value shared by every holder (ADR 0158) is one value, so it is never rotated for one consumer alone: refused, and
// nothing changes. A module's own secret names no consumer.
func (v *Vault) RotateFor(caller, secret string, one Narrow, by, why string) (Changed, error) {
if caller != ControllerCaller {
return Changed{}, fmt.Errorf("the vault rotates for the controller alone, which knows whether the module's "+
"secrets are filled from the vault yet; %s is not the controller: `secret rotate` at the controller", caller)
}
if strings.TrimSpace(why) == "" {
return Changed{}, errors.New("not rotated: a rotation says why it was asked (`why`)")
}
if err := validName(secret); err != nil {
return Changed{}, err
}
rec, held, err := v.Store.Read(secret)
if err != nil {
return Changed{}, err
}
parts := strings.Split(secret, "/")
shared := len(parts) == 3 && parts[0] == "provision"
switch {
case !one.empty() && !shared:
return Changed{}, fmt.Errorf("not rotated: %s is not a provision's credential, and only a provider's "+
"consumers are named in a rotation", secret)
case !one.empty() && held:
return Changed{}, fmt.Errorf("not rotated: %s is one value every holder shares (ADR 0158), so it cannot be "+
"rotated for %s alone; rotated without a consumer, it is made anew for every holder", secret, one)
case !held && shared:
// A provider whose consumers keep logins of their own: each one's own is remade, never one shared value.
changed, err := v.rotatePerPair(secret, one, someoneOr(by, caller))
if err == nil {
v.log("rotated each consumer's own login of %s (%d), asked by %s via %s: %s", secret, len(changed.Each),
someoneOr(by, caller), caller, strings.TrimSpace(why))
}
return changed, err
case !held:
return Changed{}, fmt.Errorf("not rotated: the vault holds no secret named %s", secret)
case Covered(secret):
return Changed{}, fmt.Errorf("not rotated: %s is a bus account the controller mints and hands here (ADR 0374 "+
"clause 4): only a password whose hash the bus's user list holds can log in, so the vault makes no new "+
"value for it; `module issue` again hands a new one and replaces this in place", secret)
case rec.Origin == OriginIssued:
return Changed{}, fmt.Errorf("not rotated: %s is issued by its provider's software, which the mesh cannot "+
"make or rotate (ADR 0305 §9): change it in that software, and `secret issued` takes the value it holds now",
secret)
case rec.Origin == OriginAccepted && strings.HasPrefix(secret, "provision/"):
return Changed{}, fmt.Errorf("not rotated: %s was accepted, and an accepted credential is not remade by the "+
"vault (ADR 0349 clause 8): accept its new value instead", secret)
}
if shared || (parts[0] == "pair" && (len(parts) == 5 || len(parts) == 6)) {
if why := v.rotationHeldBack(strings.Join(append([]string{"provision"}, parts[1:3]...), "/")); why != "" {
return Changed{}, fmt.Errorf("not rotated: %s", why)
}
}
if by == "" {
by = caller
}
raw := make([]byte, 30)
if _, err := rand.Read(raw); err != nil {
return Changed{}, err
}
changed, err := v.keep(secret, base64.RawURLEncoding.EncodeToString(raw), by, caller, OriginMade)
if err == nil && shared {
if sh, recorded, rerr := v.Store.ReadShared(secret); rerr == nil && recorded {
sh.Generation, sh.At = changed.Generation, v.Now().UTC()
if werr := v.Store.WriteShared(sh); werr != nil {
v.log("%s was rotated, but the record of who holds it keeps the earlier generation: %v", secret, werr)
}
}
}
if err == nil {
v.log("rotated %s (generation %d), asked by %s via %s: %s", secret, changed.Generation, by, caller,
strings.TrimSpace(why))
}
return changed, err
}
func someoneOr(by, caller string) string {
if strings.TrimSpace(by) == "" {
return caller
}
return by
}
modules/mesh-vault/cmd/mesh-vault/intake_test.go
+552
View File
@@ -0,0 +1,552 @@
package main
// ADR 0349's tests for what the vault takes in (design 57's V2, issue 675): the vault's `secret-ask` test, the
// issuing provider's test with a stand-in provider (plex's own tool moves with the media catalogue, after
// release 57), an accepted value opened in the vault alone and never remade, and rotate.
//
// The two-step tests below drive the steps the vault's own calls are made of (owncalls_test.go, issue 723): "the
// controller" there is a test function that holds no private key, standing where the vault's call to the desk or the
// provider stands, and carries only what was sealed to the key the vault made for the call.
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
"golang.org/x/crypto/nacl/box"
)
// sealToCall seals value to a call's key as the desk's prompt and a provider's tool do: the NaCl anonymous box.
func sealToCall(t *testing.T, sealTo, value string) string {
t.Helper()
raw, err := base64.StdEncoding.DecodeString(sealTo)
if err != nil || len(raw) != 32 {
t.Fatalf("seal_to %q is not a key", sealTo)
}
var key [32]byte
copy(key[:], raw)
sealed, err := box.SealAnonymous(nil, []byte(value), &key, rand.Reader)
if err != nil {
t.Fatal(err)
}
return base64.StdEncoding.EncodeToString(sealed)
}
// handedTo opens what `get` hands an allowed asker: the value the vault holds.
func handedTo(t *testing.T, v testVault, asker, name string) string {
t.Helper()
key, _ := seal.GenerateKeyPair()
a, err := v.Get(asker, name, key.PublicKey)
if err != nil {
t.Fatal(err)
}
value, err := seal.Open(a.Sealed, key.PrivateKey)
if err != nil {
t.Fatal(err)
}
return value
}
// everything is all the vault said and wrote during a test, as text.
func everything(t *testing.T, v testVault, more ...any) string {
t.Helper()
var b strings.Builder
for _, said := range v.bus.raw {
b.WriteString(said)
}
for _, content := range filesUnder(t, v.dir) {
b.WriteString(content)
}
for _, m := range more {
raw, _ := json.Marshal(m)
b.Write(raw)
}
return b.String()
}
const typedAtTheDesk = "123456789:AAEhBP0av28P4XFQnIuR-o-7Xnz1kkUzW3g"
func telegramAsk(by string) AskOf {
return AskOf{Node: "anchor", Module: "telegram", Name: "telegram-token", Desk: "laptop", AskedBy: by}
}
// ---- the vault's secret-ask test (ADR 0349 "How it is checked") ---------------------------------------------
// An ask is recorded in the vault's store before the prompt opens; its prompt names the asker as the bus named
// it; the typed value opens only in the vault; the controller carries ciphertext only.
func TestAnAskIsRecordedBeforeThePromptAndTheTypedValueOpensOnlyInTheVault(t *testing.T) {
v := newVault(t, AllowList{"anchor.node-provisioner": {"own/anchor/telegram/telegram-token"}})
opened, err := v.AskOpen(ControllerCaller, telegramAsk("g14/claude-code, via the console"))
if err != nil {
t.Fatal(err)
}
// Recorded before any prompt opens: the record holds the ask now, with no end.
asks, _ := v.Store.Asks(10)
if len(asks) != 1 || asks[0].Event != AskOpened || asks[0].ID != opened.Call || asks[0].AskedBy != "g14/claude-code" ||
asks[0].Caller != ControllerCaller || asks[0].Desk != "laptop" || asks[0].Secret != "own/anchor/telegram/telegram-token" {
t.Fatalf("the record before the prompt: %+v", asks)
}
// The prompt names the asker as the bus named it, cut to a name: what the vault answers is what it recorded.
if opened.AskedBy != "g14/claude-code" {
t.Errorf("the prompt would name %q", opened.AskedBy)
}
// The desk seals what is typed to the call's key; the controller passes it on.
changed, err := v.AskEnd(ControllerCaller, opened.Call, sealToCall(t, opened.SealTo, typedAtTheDesk+"\n"), "")
if err != nil {
t.Fatal(err)
}
if changed.Generation != 1 || changed.By != "g14/claude-code" {
t.Errorf("kept as %+v", changed)
}
if got := handedTo(t, v, "anchor.node-provisioner", "own/anchor/telegram/telegram-token"); got != typedAtTheDesk {
t.Errorf("the vault holds %q, not what was typed with its line ending taken off", got)
}
rec, _, _ := v.Store.Read("own/anchor/telegram/telegram-token")
if rec.Origin != OriginTyped || rec.ChangedBy != "g14/claude-code" || rec.ChangedVia != ControllerCaller {
t.Errorf("the record says origin %q, by %q, via %q", rec.Origin, rec.ChangedBy, rec.ChangedVia)
}
asks, _ = v.Store.Asks(10)
if len(asks) != 2 || asks[0].Event != AskEnded || asks[0].Outcome != "given" || asks[0].ID != opened.Call {
t.Errorf("how it ended: %+v", asks)
}
if strings.Contains(everything(t, v, opened, changed, asks), typedAtTheDesk) {
t.Fatal("the typed value is in something the vault said, answered or wrote in the clear")
}
}
// One open ask per secret, and three an hour (ADR 0277 §24), read from the vault's own record.
func TestAnAskIsBoundedToOneOpenPerSecretAndThreeAnHour(t *testing.T) {
v := newVault(t, nil)
first, err := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code"))
if err != nil {
t.Fatal(err)
}
if _, err := v.AskOpen(ControllerCaller, telegramAsk("ace.claude-code")); err == nil ||
!strings.Contains(err.Error(), "still open (asked by g14.claude-code)") {
t.Fatalf("a second ask while one is open: %v", err)
}
// Another secret is not held back by it.
other := telegramAsk("ace.claude-code")
other.Name = "webhook-secret"
if _, err := v.AskOpen(ControllerCaller, other); err != nil {
t.Fatalf("another secret's ask was held back: %v", err)
}
if _, err := v.AskEnd(ControllerCaller, first.Call, "", "dismissed"); err != nil {
t.Fatal(err)
}
for i := 0; i < 2; i++ {
o, err := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code"))
if err != nil {
t.Fatalf("ask %d of the hour: %v", i+2, err)
}
if _, err := v.AskEnd(ControllerCaller, o.Call, "", "timed-out"); err != nil {
t.Fatal(err)
}
}
if _, err := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code")); err == nil ||
!strings.Contains(err.Error(), "3 times in the last hour") {
t.Fatalf("a fourth ask in the hour: %v", err)
}
asks, _ := v.Store.Asks(50)
outcomes := map[string]int{}
for _, a := range asks {
outcomes[a.Outcome]++
}
if outcomes["dismissed"] != 1 || outcomes["timed-out"] != 2 {
t.Errorf("the record's outcomes: %v", outcomes)
}
// An ask left open counts as open for SecretAskOpenFor only: a prompt that died does not hold the secret.
w := newVault(t, nil)
clock := time.Date(2026, 10, 14, 9, 0, 0, 0, time.UTC)
w.Now = func() time.Time { return clock }
if _, err := w.AskOpen(ControllerCaller, telegramAsk("g14.claude-code")); err != nil {
t.Fatal(err)
}
clock = clock.Add(SecretAskOpenFor + time.Second)
if _, err := w.AskOpen(ControllerCaller, telegramAsk("g14.claude-code")); err != nil {
t.Errorf("an ask left open past its time still held the secret: %v", err)
}
}
// Only the controller reaches a desk's prompt, so only it opens and ends an ask; a value not sealed to the
// call's key, or a call that waited too long, keeps nothing.
func TestAnAskIsTakenFromTheControllerAloneAndOnlySealedToItsCall(t *testing.T) {
v := newVault(t, nil)
for _, caller := range []string{"g14.node-tools", "unproven", "", "person.jochen"} {
if _, err := v.AskOpen(caller, telegramAsk("x")); err == nil {
t.Errorf("%q opened an ask", caller)
}
}
if asks, _ := v.Store.Asks(10); len(asks) != 0 {
t.Fatalf("a refused caller filled the record: %+v", asks)
}
opened, err := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code"))
if err != nil {
t.Fatal(err)
}
if _, err := v.AskEnd("g14.node-tools", opened.Call, sealToCall(t, opened.SealTo, "x"), ""); err == nil {
t.Error("another caller ended the ask")
}
other, _, _ := box.GenerateKey(rand.Reader)
if _, err := v.AskEnd(ControllerCaller, opened.Call, sealToCall(t, base64.StdEncoding.EncodeToString(other[:]), "x"), ""); err == nil ||
!strings.Contains(err.Error(), "not sealed to this call") {
t.Errorf("a value sealed to another key: %v", err)
}
if _, held, _ := v.Store.Read("own/anchor/telegram/telegram-token"); held {
t.Error("a value not sealed to the call was kept")
}
// The call's key is forgotten once used: the same call cannot be answered twice.
if _, err := v.AskEnd(ControllerCaller, opened.Call, sealToCall(t, opened.SealTo, "x"), ""); err == nil {
t.Error("a call was answered twice")
}
asks, _ := v.Store.Asks(1)
if asks[0].Outcome != "failed" {
t.Errorf("the ask that failed ended %+v", asks[0])
}
// A call that waited longer than its time is closed.
clock := time.Date(2026, 10, 14, 9, 0, 0, 0, time.UTC)
v.Now = func() time.Time { return clock }
late, err := v.IssuedOpen(ControllerCaller, "plex-api", "anchor", "plex")
if err != nil {
t.Fatal(err)
}
clock = clock.Add(CallOpenFor + time.Second)
if _, err := v.IssuedTake(ControllerCaller, late.Call, sealToCall(t, late.SealTo, "x")); err == nil {
t.Error("a call that waited too long was taken")
}
// An empty answer gives nothing and says so.
clock = clock.Add(time.Hour)
empty, _ := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code"))
if _, err := v.AskEnd(ControllerCaller, empty.Call, sealToCall(t, empty.SealTo, " \n"), ""); err == nil {
t.Error("an empty answer was kept")
}
if asks, _ := v.Store.Asks(1); asks[0].Outcome != "empty" {
t.Errorf("an empty answer ended %+v", asks[0])
}
}
// ---- the issuing provider's test, with a stand-in provider -------------------------------------------------
// standInProvider is a provider's tool for an `issued-by: software` offer, as plex's must answer (ADR 0305 §2
// with the vault in the controller's place): it answers the vault's own account, which asks it itself (issue 723),
// and the controller, whose path for a provider filled the old way stays until R1 (issue 682); it refuses every
// other caller, and answers only the software's value sealed to the call's key.
func standInProvider(software *string) func(caller string, args map[string]any) (map[string]any, error) {
return func(caller string, args map[string]any) (map[string]any, error) {
if caller != ControllerCaller && !strings.HasSuffix(caller, ".mesh-vault") {
return nil, errors.New("this tool answers the vault and, until R1, the controller alone")
}
key, _ := args["seal_to"].(string)
raw, err := base64.StdEncoding.DecodeString(key)
if err != nil || len(raw) != 32 {
return nil, errors.New("seal_to is not a sealing key")
}
var to [32]byte
copy(to[:], raw)
sealed, err := box.SealAnonymous(nil, []byte(*software), &to, rand.Reader)
if err != nil {
return nil, err
}
return map[string]any{"sealed": base64.StdEncoding.EncodeToString(sealed)}, nil
}
}
// issue is the controller's `secret issued` for one provider: one ask of the vault, which asks the provider's
// tool itself, in its own name.
func issue(t *testing.T, v testVault, provider func(string, map[string]any) (map[string]any, error)) (Changed, error) {
t.Helper()
v.Ask = (&aMesh{tools: map[string]func(string, map[string]any) (map[string]any, error){
"plex.plex_issued_credential@anchor": provider}}).ask
got, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", "plex_issued_credential")
if err != nil {
return Changed{}, err
}
rec, _, _ := v.Store.Read(got.Secret)
return Changed{Name: got.Secret, Generation: got.Generation, By: rec.ChangedBy, Unchanged: got.Unchanged}, nil
}
func TestAnIssuingProviderAnswersOnlyTheVaultsCallSealedToTheVault(t *testing.T) {
v := newVault(t, AllowList{"anchor.node-provisioner": {"provision/anchor/plex-api"}})
software := "plex-token-from-the-vendor"
provider := standInProvider(&software)
// The provider's tool refuses every caller but the vault's account and the controller, whatever it asks.
for _, caller := range []string{"anchor.node-tools", "g14.claude-code", "unproven", ""} {
if _, err := provider(caller, map[string]any{"provision": "plex-api", "seal_to": "x"}); err == nil {
t.Errorf("the provider's tool answered %q", caller)
}
}
// The vault takes the steps from the controller alone.
if _, err := v.IssuedOpen("anchor.node-tools", "plex-api", "anchor", "plex"); err == nil {
t.Error("the vault opened an issued call for another caller")
}
changed, err := issue(t, v, provider)
if err != nil {
t.Fatal(err)
}
if changed.Name != "provision/anchor/plex-api" || changed.Generation != 1 || changed.By != "plex on anchor" {
t.Errorf("kept as %+v", changed)
}
if got := handedTo(t, v, "anchor.node-provisioner", "provision/anchor/plex-api"); got != software {
t.Errorf("the vault holds %q", got)
}
rec, _, _ := v.Store.Read("provision/anchor/plex-api")
if rec.Origin != OriginIssued || rec.ChangedVia != ControllerCaller {
t.Errorf("the record: origin %q via %q", rec.Origin, rec.ChangedVia)
}
// Nothing rewritten for a value every holder has (ADR 0305 §5): no new generation, no write, nothing said.
path := filepath.Join(v.dir, "store", "secrets", "provision%2Fanchor%2Fplex-api.json")
before, _ := os.ReadFile(path)
said := len(v.bus.raw)
again, err := issue(t, v, provider)
if err != nil {
t.Fatal(err)
}
after, _ := os.ReadFile(path)
if !again.Unchanged || again.Generation != 1 || string(before) != string(after) || len(v.bus.raw) != said {
t.Errorf("the same value again rewrote something: %+v", again)
}
// The software changes its value: the next ask keeps it as a new generation, and says so without it.
software = "plex-token-renewed"
renewed, err := issue(t, v, provider)
if err != nil || renewed.Generation != 2 {
t.Fatalf("%+v %v", renewed, err)
}
if got := handedTo(t, v, "anchor.node-provisioner", "provision/anchor/plex-api"); got != software {
t.Errorf("after the change the vault holds %q", got)
}
// A value sealed to any other key is refused, and nothing changes.
opened, _ := v.IssuedOpen(ControllerCaller, "plex-api", "anchor", "plex")
stranger, _, _ := box.GenerateKey(rand.Reader)
if _, err := v.IssuedTake(ControllerCaller, opened.Call, sealToCall(t, base64.StdEncoding.EncodeToString(stranger[:]), "forged")); err == nil {
t.Error("a value not sealed to the call was taken")
}
// The software rotates it, not the mesh (ADR 0305 §9); and no other path puts it into the vault (§10).
if _, err := v.Rotate(ControllerCaller, "provision/anchor/plex-api", "person.jochen", "leaked"); err == nil ||
!strings.Contains(err.Error(), "software") {
t.Errorf("rotate of an issued value: %v", err)
}
if _, err := v.AcceptOpen(ControllerCaller, "provision/anchor/plex-api", "person.jochen", ""); err == nil {
t.Error("an issued value was taken by accept")
}
if strings.Contains(everything(t, v), "plex-token") {
t.Fatal("the issued value is in something the vault said or wrote in the clear")
}
}
// ---- accept and rotate -------------------------------------------------------------------------------------
// A value given at the controller's terminal is sealed there to the vault's call and opened here alone; an
// accepted provision's credential is never remade; a module's own given value rotates as a made one (ADR 0228).
func TestAnAcceptedValueOpensInTheVaultAloneAndIsNotRemade(t *testing.T) {
v := newVault(t, AllowList{"g14.node-provisioner": {"provision/novox/postgres", "own/g14/dunst/token"}})
if _, err := v.AcceptOpen("g14.claude-code", "provision/novox/postgres", "", ""); err == nil {
t.Error("accept was opened for a caller that is not the controller")
}
for name, value := range map[string]string{"provision/novox/postgres": "pw-of-last-year", "own/g14/dunst/token": "tok"} {
opened, err := v.AcceptOpen(ControllerCaller, name, "jochen", "")
if err != nil {
t.Fatal(err)
}
changed, err := v.AcceptTake(ControllerCaller, opened.Call, sealToCall(t, opened.SealTo, value))
if err != nil {
t.Fatal(err)
}
if changed.By != "jochen" {
t.Errorf("%s accepted by %q", name, changed.By)
}
if got := handedTo(t, v, "g14.node-provisioner", name); got != value {
t.Errorf("%s holds %q", name, got)
}
rec, _, _ := v.Store.Read(name)
if rec.Origin != OriginAccepted || rec.ChangedVia != ControllerCaller {
t.Errorf("%s: origin %q via %q", name, rec.Origin, rec.ChangedVia)
}
}
if _, err := v.Rotate(ControllerCaller, "provision/novox/postgres", "jochen", "leaked"); err == nil ||
!strings.Contains(err.Error(), "not remade") {
t.Errorf("rotate of an accepted credential: %v", err)
}
rotated, err := v.Rotate(ControllerCaller, "own/g14/dunst/token", "jochen", "leaked")
if err != nil || rotated.Generation != 2 {
t.Fatalf("rotate of a module's own given value: %+v %v", rotated, err)
}
if got := handedTo(t, v, "g14.node-provisioner", "own/g14/dunst/token"); got == "tok" || len(got) != 40 {
t.Errorf("rotated to %q", got)
}
if strings.Contains(everything(t, v), "pw-of-last-year") {
t.Fatal("an accepted value is in something the vault said or wrote in the clear")
}
}
// Rotate: refused for a secret not held and for an unproven caller; who asked is the proven caller, or the
// person the controller names when it carries the act — both kept.
func TestRotateKeepsWhoAskedAndWhoCarriedIt(t *testing.T) {
v := newVault(t, nil)
if _, err := v.Rotate(ControllerCaller, "own/g14/dunst/token", "jochen", "leaked"); err == nil {
t.Error("a secret not held was rotated")
}
if _, err := v.Mint("own/g14/dunst/token", "the test"); err != nil {
t.Fatal(err)
}
if _, err := v.Rotate("unproven", "own/g14/dunst/token", "", "leaked"); err == nil {
t.Error("an unproven caller rotated")
}
if _, err := v.Rotate(ControllerCaller, "own/g14/dunst/token", "jochen", "leaked"); err != nil {
t.Fatal(err)
}
rec, _, _ := v.Store.Read("own/g14/dunst/token")
if rec.ChangedBy != "jochen" || rec.ChangedVia != ControllerCaller || rec.Generation != 2 {
t.Errorf("by %q via %q generation %d", rec.ChangedBy, rec.ChangedVia, rec.Generation)
}
}
// Rotate is the controller's alone, and says why (the review of issue 675): only the controller knows whether a
// module is filled from the vault yet, so a rotation asked by any other caller — a proven one included — would
// drift from the value the controller holds; and a rotation without a reason is refused. Neither changes anything.
func TestRotateIsTheControllersAloneAndSaysWhy(t *testing.T) {
v := newVault(t, nil)
if _, err := v.Mint("own/g14/dunst/token", "the test"); err != nil {
t.Fatal(err)
}
for _, caller := range []string{"g14.claude-code", "g14.node-provisioner", "unproven", ""} {
if _, err := v.Rotate(caller, "own/g14/dunst/token", "jochen", "leaked"); err == nil ||
!strings.Contains(err.Error(), "controller alone") {
t.Errorf("%q rotated: %v", caller, err)
}
}
for _, why := range []string{"", " "} {
if _, err := v.Rotate(ControllerCaller, "own/g14/dunst/token", "jochen", why); err == nil ||
!strings.Contains(err.Error(), "why") {
t.Errorf("a rotation without a why (%q): %v", why, err)
}
}
if rec, _, _ := v.Store.Read("own/g14/dunst/token"); rec.Generation != 1 {
t.Fatalf("a refused rotation changed the secret: generation %d", rec.Generation)
}
}
// The names the vault keeps them under are names, never paths.
func TestTheNamesOfWhatIsTakenIn(t *testing.T) {
if n, err := OwnSecret("g14", "dunst", "token"); err != nil || n != "own/g14/dunst/token" {
t.Errorf("%q %v", n, err)
}
if n, err := ProvisionSecret("novox", "postgres"); err != nil || n != "provision/novox/postgres" {
t.Errorf("%q %v", n, err)
}
for _, bad := range [][3]string{{"..", "x", "y"}, {"g14", "a/b", "y"}, {"g14", "x", ""}, {"G14", "x", "y"}} {
if _, err := OwnSecret(bad[0], bad[1], bad[2]); err == nil {
t.Errorf("%v was taken", bad)
}
}
}
// ---- the verbs as the runtime calls them ------------------------------------------------------------------
// Each verb takes its caller from the runtime, never from its arguments, and names its secret one way only.
func TestTheVerbsTakeTheirCallerFromTheRuntime(t *testing.T) {
v := newVault(t, nil)
env := &Env{Vault: v.Vault}
defs := definitions(env)
var asked []map[string]any
software := "tok"
v.Ask = (&aMesh{desk: aDesk("typed", &asked), tools: map[string]func(string, map[string]any) (map[string]any, error){
"plex.plex_issued_credential@anchor": standInProvider(&software)}}).ask
ask := toolNamed(t, defs, Seat+".secret-ask")
args := map[string]any{"node": "anchor", "module": "telegram", "name": "telegram-token", "desk": "laptop",
"asked_by": "g14.claude-code", "caller": ControllerCaller}
if _, err := ask.Run(args); err == nil {
t.Error("an ask opened for a caller named in the arguments")
}
answer, err := ask.runAs(ControllerCaller, args)
if err != nil || answer.(Asked).Outcome != "given" || len(asked) != 1 {
t.Fatalf("secret-ask: %+v %v", answer, err)
}
listed, err := toolNamed(t, defs, "secret_asks").Run(map[string]any{})
if err != nil || len(listed.([]AskRecord)) != 2 {
t.Fatalf("secret_asks: %v %v", listed, err)
}
accept := toolNamed(t, defs, Seat+".accept")
if _, err := accept.runAs(ControllerCaller, map[string]any{"node": "g14", "provision": "postgres"}); err == nil {
t.Error("accept took a name both ways")
}
answer, err = accept.runAs(ControllerCaller, map[string]any{"provider": "novox", "provision": "postgres", "by": "jochen"})
if err != nil {
t.Fatal(err)
}
o := answer.(Opened)
if _, err := accept.runAs(ControllerCaller, map[string]any{"call": o.Call, "sealed": sealToCall(t, o.SealTo, "pw")}); err != nil {
t.Fatal(err)
}
rotate := toolNamed(t, defs, Seat+".rotate")
if _, err := rotate.runAs("g14.claude-code", map[string]any{"node": "g14", "module": "dunst", "name": "token"}); err == nil {
t.Error("a secret not held was rotated through the verb")
}
issued := toolNamed(t, defs, Seat+".issued")
answer, err = issued.runAs(ControllerCaller, map[string]any{"provision": "plex-api", "provider": "anchor",
"module": "plex", "tool": "plex_issued_credential"})
if err != nil || answer.(Issued).Secret != "provision/anchor/plex-api" {
t.Fatalf("issued: %+v %v", answer, err)
}
broken := &Env{Broken: errors.New("no store")}
if _, err := toolNamed(t, definitions(broken), Seat+".rotate").runAs(ControllerCaller, map[string]any{}); err == nil ||
err.Error() != "no store" {
t.Errorf("a broken vault said %v", err)
}
}
// How an ask ended is written under the record's lock (the review of issue 675), as its opening is: two writers
// never roll asks.jsonl at once, and an ask opened meanwhile never counts its bounds from a half-written record.
// With the lock held elsewhere, AskEnd waits, and writes nothing until it is let go.
func TestTheEndOfAnAskIsWrittenUnderTheRecordsLock(t *testing.T) {
v := newVault(t, nil)
opened, err := v.AskOpen(ControllerCaller, telegramAsk("g14.claude-code"))
if err != nil {
t.Fatal(err)
}
ended := func() int {
t.Helper()
raw, err := os.ReadFile(filepath.Join(v.Store.dir, "asks.jsonl"))
if err != nil {
t.Fatal(err)
}
return strings.Count(string(raw), `"`+string(AskEnded)+`"`)
}
v.Store.asksMu.Lock()
done := make(chan error, 1)
go func() {
_, err := v.AskEnd(ControllerCaller, opened.Call, "", "dismissed")
done <- err
}()
select {
case err := <-done:
v.Store.asksMu.Unlock()
t.Fatalf("the end of the ask was written while another writer held the record (%v)", err)
case <-time.After(200 * time.Millisecond):
}
if n := ended(); n != 0 {
v.Store.asksMu.Unlock()
t.Fatalf("%d end(s) written under another writer's lock", n)
}
v.Store.asksMu.Unlock()
if err := <-done; err != nil {
t.Fatal(err)
}
if n := ended(); n != 1 {
t.Fatalf("%d end(s) written once the lock was let go", n)
}
}
modules/mesh-vault/cmd/mesh-vault/issued_account_test.go
+99
View File
@@ -0,0 +1,99 @@
package main
import (
"strings"
"testing"
)
// A bus account the controller minted for a core or moved module is handed to the vault with origin issued (novox/hq
// ADR 0374, issue 787): kept as that module's own secret on that machine, served to the controller's core get and to
// the moved module's client, replaced in place on a re-issue with `secret-changed`, and never made anew by the vault.
func handIssued(t *testing.T, v testVault, caller, secret, value string) (Changed, error) {
t.Helper()
opened, err := v.AcceptOpen(caller, secret, "module issue", OriginIssued)
if err != nil {
return Changed{}, err
}
return v.AcceptTake(caller, opened.Call, sealToCall(t, opened.SealTo, value))
}
func TestAHandedBusAccountIsKeptServedReplacedAndNeverMadeByTheVault(t *testing.T) {
const moved = "own/g14/moved-bot/broker"
v := newVault(t, AllowList{"provisioner.g14": {moved}})
first, err := handIssued(t, v, ControllerCaller, moved, `{"password":"first-pw"}`)
if err != nil || first.Generation != 1 {
t.Fatalf("the handed account: %+v %v", first, err)
}
rec, held, _ := v.Store.Read(moved)
if !held || rec.Origin != OriginIssued || rec.ChangedVia != ControllerCaller {
t.Fatalf("kept as origin %q via %q (held %v), want issued via the controller", rec.Origin, rec.ChangedVia, held)
}
if got := handedTo(t, v, "provisioner.g14", moved); got != `{"password":"first-pw"}` {
t.Errorf("the moved module's client is handed %q", got)
}
// A re-issue replaces it in place, a new generation, and the vault says it changed.
before := len(v.bus.events)
second, err := handIssued(t, v, ControllerCaller, moved, `{"password":"second-pw"}`)
if err != nil || second.Generation != 2 || second.Unchanged {
t.Fatalf("the re-issue: %+v %v", second, err)
}
if got := handedTo(t, v, "provisioner.g14", moved); got != `{"password":"second-pw"}` {
t.Errorf("after the re-issue the client is handed %q: a stale value stayed", got)
}
changedEvents := 0
for _, e := range v.bus.events[before:] {
if e["event"] == ChangedEvent {
changedEvents++
}
}
if changedEvents != 1 {
t.Errorf("the re-issue raised %d secret-changed events, want 1", changedEvents)
}
// The vault makes no value for it: not a rotation, and not an accept without origin over it either way round.
if _, err := v.Rotate(ControllerCaller, moved, "jochen", "leaked"); err == nil || !strings.Contains(err.Error(), "ADR 0374") {
t.Errorf("the vault rotated a covered account: %v", err)
}
if _, err := v.Rotate(ControllerCaller, "own/g14/moved-bot/provisioner-broker", "jochen", "leaked"); err == nil {
t.Error("the vault made a first value for a covered account by rotating it")
}
if got := handedTo(t, v, "provisioner.g14", moved); got != `{"password":"second-pw"}` {
t.Errorf("a refused rotation changed the value to %q", got)
}
// A secret that is not a covered account is not handed with origin issued; and only the controller hands one.
if _, err := v.AcceptOpen(ControllerCaller, "own/g14/moved-bot/token", "x", OriginIssued); err == nil {
t.Error("origin issued was taken for a secret that is not a bus account")
}
if _, err := v.AcceptOpen(ControllerCaller, "provision/anchor/postgres", "x", OriginIssued); err == nil {
t.Error("origin issued was taken for a provision's credential")
}
if _, err := v.AcceptOpen("g14.moved-bot", moved, "x", OriginIssued); err == nil {
t.Error("a caller that is not the controller handed a covered account")
}
if _, err := v.AcceptOpen(ControllerCaller, moved, "x", "made"); err == nil {
t.Error("an origin accept does not take was taken")
}
if strings.Contains(everything(t, v), "first-pw") || strings.Contains(everything(t, v), "second-pw") {
t.Fatal("a handed account is in something the vault said or wrote in the clear")
}
}
// A core module's handed account is the one the controller's core get is answered, sealed to its machine's key.
func TestAHandedCoreAccountIsKeptAsTheModulesOwnSecret(t *testing.T) {
const natsBroker = "own/anchor/nats/broker"
v := newVault(t, nil)
if _, err := handIssued(t, v, ControllerCaller, natsBroker, `{"password":"core-pw"}`); err != nil {
t.Fatal(err)
}
rec, held, _ := v.Store.Read(natsBroker)
if !held || rec.Origin != OriginIssued {
t.Fatalf("the core's account is held %v as %q", held, rec.Origin)
}
if !Covered(natsBroker) || Covered("own/anchor/nats/token") || Covered("pair/a/b/c/broker") {
t.Error("the covered names are not the module's broker and provisioner-broker own secrets")
}
}
modules/mesh-vault/cmd/mesh-vault/main.go
+32 -8
View File
@@ -8,6 +8,7 @@
package main
import (
"errors"
"fmt"
"os"
"time"
@@ -45,17 +46,24 @@ func build() *Env {
say("%v", env.Broken)
return env
}
// Askers is left nil: the vault that runs answers nobody until who may ask arrives (askers.go).
env.Vault = &Vault{Store: store, Own: own, Emit: emit, Now: time.Now, Log: say}
// Who may ask (askers.go): a node's provisioner client, for what its module's plan there names.
// Where it sees a consumer bind a provider (pair.go): the same mirror, every key.
env.Vault = &Vault{Store: store, Own: own, Askers: Declared{Read: readAssignment, Store: store},
Assignments: Mirror{Keys: assignmentKeys, Read: readAssignment}, Emit: emit, Ask: stdio.Ask,
Now: time.Now, Log: say}
return env
}
// adoptOperatorKey takes the operator key from the export the mesh writes to the vault's root (what
// `operator key set` recorded; the controller keeps only a public half there), and seals again any
// operator copy not yet sealed to the current key. Run every minute, so a failure partway is finished.
func adoptOperatorKey(env *Env) {
// operator copy not yet sealed to the current key. Run every minute, so a failure partway is finished. It
// returns why the pass did not re-seal everything, or nil.
func adoptOperatorKey(env *Env) error {
if env.Vault == nil {
return
if env.Broken != nil {
return env.Broken
}
return errors.New("the vault was not built")
}
if raw := exportedOperatorKey(env.Root); env.Root != "" && raw != "" {
key, err := publicPEM(raw)
@@ -67,20 +75,36 @@ func adoptOperatorKey(env *Env) {
env.refuseKey(err.Error())
say("the operator key in the mesh's export is not taken, or not every copy is sealed to it yet: %v", err)
}
return
return err
}
if _, err := env.Vault.Reseal(); err != nil {
_, err := env.Vault.Reseal()
if err != nil {
say("%v", err)
}
return err
}
// readAssignment is one value of the controller's assignments mirror, read through the runtime under the vault's
// own account (its manifest `reads` it), or nil when there is none.
func readAssignment(key string) ([]byte, error) {
e, err := stdio.State(AssignmentsRead).Get(key)
if err != nil || e == nil {
return nil, err
}
return e.Value, nil
}
// assignmentKeys is every key of the controller's assignments mirror, read the same way.
func assignmentKeys() ([]string, error) { return stdio.State(AssignmentsRead).Keys() }
func main() {
env := build()
go func() {
tick := time.NewTicker(time.Minute)
defer tick.Stop()
for {
adoptOperatorKey(env)
env.pass()
env.pairPass()
if env.Custody != nil {
env.Custody.Pass()
}
modules/mesh-vault/cmd/mesh-vault/move.go
+369
View File
@@ -0,0 +1,369 @@
package main
// The move of the controller's secrets into the vault (novox/hq ADR 0349 clause 7; design 57's V3, issue 676;
// the operator's decision of 2026-10-14 on that issue).
//
// **Only a machine the value was sealed to can open it.** The controller's store holds every value sealed to a
// machine's sealing key (and, for some, to the operator key); the controller cannot open one. So each machine's
// node-engine opens what was sealed to it and hands it in, in the two steps of the intake (intake.go): `move-in`
// names the secret, where it came from and its generation in the controller's store, and is answered a call and
// a key made for it; the second step carries the value sealed to that key and a key of the node-engine's own,
// and is answered what the vault kept, read back from its own store and sealed to that key — the node-engine
// opens it and compares it with what it handed in, which is the move's proof.
//
// **From the node-engine of a machine the value was sealed to, and no other caller.** The caller is the bus user
// the runtime proved, `node.<machine>` (the controller's broker names a node-engine so): a module's own secret
// was sealed to its one machine, a pair credential to both its ends.
//
// **Copied as it is, and never over a value the vault holds.** Nothing is rotated (ADR 0349's rule for a move):
// the value keeps its origin, the controller's generation and when it was made. The same value again changes
// nothing — the other end of a pair, or the move run again — and a different one is refused, so a value the
// vault holds, rotated or given since, is never put back to the controller's older one.
//
// The controller's record of asks at a desk holds no value, and the controller copies it as it is (`move-asks`).
//
// Which secret each row of the controller's store is, by name in the vault's store:
//
// own/<node>/<module>/<name> a row of module_secret
// pair/<provider>/<provision>/<consumer>/<module>[/<local>] a row of secret, its local name last when it has one
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"slices"
"strconv"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/seal"
"golang.org/x/crypto/nacl/box"
)
// NodeEnginePrefix starts the bus user of a machine's node-engine: `node.<machine>`.
const NodeEnginePrefix = "node."
// MovedFrom is where a value the move took came from: the node-engine that handed it in, and the generation the
// controller's store stamped it with (ADR 0158, ADR 0305 §5), kept as it was. Never a value.
type MovedFrom struct {
By string `json:"by"`
At time.Time `json:"at"`
Generation string `json:"generation,omitempty"`
}
// MoveOf is what one row of the controller's store is: a module's own secret (node, module, name) or a pair
// credential (provider, provision, consumer, consumer module, local name), where it came from, its generation
// and when it was made.
type MoveOf struct {
Node, Module, Name string
Provider, Provision, Consumer, ConsumerModule, Local string
Origin, Generation string
MadeAt time.Time
// SealingKey is the caller's machine's sealing key as its node-engine states it (a raw X25519 public key in
// base64), kept as that machine's: what the core's credentials are sealed to (core.go, issue 763). Optional.
SealingKey string
}
// MovedIn is what the vault kept, and that value read back from its store sealed to the node-engine's key.
type MovedIn struct {
Name string `json:"name"`
Generation int `json:"generation"`
Unchanged bool `json:"unchanged,omitempty"`
// ReadBack is the value the vault's store holds now, sealed to the key the second step sent (NaCl anonymous
// box, base64).
ReadBack string `json:"read_back"`
}
// PairSecret is the vault's name for one row of the controller's pair credentials.
func PairSecret(provider, provision, consumer, consumerModule, local string) (string, error) {
parts := map[string]string{"the provider's machine": provider, "the provision": provision,
"the consumer's machine": consumer, "the consuming module": consumerModule}
if local != "" {
parts["the local name"] = local
}
for what, v := range parts {
if !namePart.MatchString(v) {
return "", fmt.Errorf("%s %q is not a name the mesh writes", what, v)
}
}
name := "pair/" + provider + "/" + provision + "/" + consumer + "/" + consumerModule
if local != "" {
name += "/" + local
}
return name, nil
}
// secret is the row's name in the vault and the machines it was sealed to.
func (m MoveOf) secret() (string, []string, error) {
own := m.Node != "" || m.Module != "" || m.Name != ""
pair := m.Provider != "" || m.Provision != "" || m.Consumer != "" || m.ConsumerModule != "" || m.Local != ""
switch {
case own && !pair:
name, err := OwnSecret(m.Node, m.Module, m.Name)
return name, []string{m.Node}, err
case pair && !own:
name, err := PairSecret(m.Provider, m.Provision, m.Consumer, m.ConsumerModule, m.Local)
return name, []string{m.Provider, m.Consumer}, err
}
return "", nil, errors.New("name a module's own secret (node, module, name) or a pair credential (provider, " +
"provision, consumer, consumer_module and local when it has one), one of the two")
}
// movedOrigins are the controller's words for where a value came from, as the vault's.
var movedOrigins = map[string]string{"made": OriginMade, "accepted": OriginAccepted, "issued": OriginIssued}
// nodeEngineOf is the machine whose node-engine caller is, if it is one.
func nodeEngineOf(caller string) (string, bool) {
node, ok := strings.CutPrefix(caller, NodeEnginePrefix)
return node, ok && namePart.MatchString(node) && !strings.Contains(node, ".")
}
// MoveOpen opens a call to take one value of the controller's store from the node-engine of a machine it was
// sealed to.
func (v *Vault) MoveOpen(caller string, m MoveOf) (Opened, error) {
node, ok := nodeEngineOf(caller)
if !ok {
if caller == "" {
caller = "an unproven caller"
}
return Opened{}, fmt.Errorf("move-in takes a value from a machine's node-engine alone, which opens what was "+
"sealed to its machine; %s is not a node-engine", caller)
}
secret, sealedTo, err := m.secret()
if err != nil {
return Opened{}, err
}
if !slices.Contains(sealedTo, node) {
return Opened{}, fmt.Errorf("%s was sealed to %s, not to %s: only a machine it was sealed to moves it in",
secret, strings.Join(sealedTo, " and "), node)
}
if movedOrigins[m.Origin] == "" {
return Opened{}, fmt.Errorf("the origin %q is not one the controller's store holds (made, accepted or "+
"issued); nothing is taken", m.Origin)
}
if len(m.Generation) > 120 || strings.ContainsAny(m.Generation, "\r\n\x00") {
return Opened{}, errors.New("the generation is not one short line; nothing is taken")
}
if m.SealingKey != "" {
stated, err := sealingKey(m.SealingKey)
if err != nil {
return Opened{}, err
}
if err := v.recordNodeKey(node, stated, caller); err != nil {
return Opened{}, err
}
}
id, key, err := v.openCall("move-in", secret, caller, nil)
if err != nil {
return Opened{}, err
}
v.callsMu.Lock()
if c := v.pending[id]; c != nil {
moved := m
c.move = &moved
}
v.callsMu.Unlock()
return Opened{Call: id, SealTo: key, Secret: secret}, nil
}
// readBackKey is the key a read-back is sealed to: a raw X25519 public key in base64, the box the node-engine's
// own sealing key opens. A PEM, which a private key could be, is refused.
func readBackKey(key string) (*[32]byte, error) {
if privateKeyIn(key) || strings.Contains(key, "-----") {
return nil, errors.New("the key to read back to is a raw X25519 public key in base64, never a PEM and never " +
"a private key; nothing was taken")
}
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(key))
if err != nil || len(raw) != 32 {
return nil, errors.New("the second step names no key to read back to (`read_back_to`, a raw X25519 public " +
"key in base64); nothing was taken")
}
var pub [32]byte
copy(pub[:], raw)
return &pub, nil
}
// MoveTake opens the value sealed to the call's key and keeps it as it is, unless the vault holds another; then
// answers what the store holds, read back and sealed to readBackTo.
func (v *Vault) MoveTake(caller, call, sealed, readBackTo string) (MovedIn, error) {
if _, ok := nodeEngineOf(caller); !ok {
return MovedIn{}, fmt.Errorf("move-in takes a value from a machine's node-engine alone; %q is not one", caller)
}
to, err := readBackKey(readBackTo)
if err != nil {
return MovedIn{}, err
}
c, err := v.takeCall("move-in", call)
if err != nil {
return MovedIn{}, err
}
if c.by != caller || c.move == nil {
return MovedIn{}, fmt.Errorf("the move-in call %q was opened by another caller; nothing was taken", call)
}
value, err := c.open(sealed)
if err != nil {
return MovedIn{}, err
}
defer zero(value)
if len(value) == 0 {
return MovedIn{}, errors.New("the value handed in is empty; nothing was taken")
}
m := c.move
changed, err := v.keepWith(c.secret, string(value), caller, "", movedOrigins[m.Origin], keepOptions{
neverReplace: true, madeAt: m.MadeAt, movedFrom: &MovedFrom{By: caller, Generation: m.Generation}})
if err != nil {
return MovedIn{}, err
}
rec, held, err := v.Store.Read(c.secret)
if err != nil || !held {
return MovedIn{}, fmt.Errorf("%s was kept but cannot be read back from the vault's store: %v", c.secret, err)
}
back, err := seal.Open(rec.AtRest, v.Own.PrivateKey)
if err != nil {
return MovedIn{}, fmt.Errorf("%s was kept but does not open with the vault's key: %w", c.secret, err)
}
boxed, err := box.SealAnonymous(nil, []byte(back), to, rand.Reader)
back = ""
if err != nil {
return MovedIn{}, err
}
v.log("moved in %s (generation %d) from %s, origin %s, the controller's generation %q", c.secret,
changed.Generation, caller, movedOrigins[m.Origin], m.Generation)
return MovedIn{Name: c.secret, Generation: rec.Generation, Unchanged: changed.Unchanged,
ReadBack: base64.StdEncoding.EncodeToString(boxed)}, nil
}
// ---- move-asks ----------------------------------------------------------------------------------------------
// MovedAsk is one row of the controller's record of asks at a desk (its secret_ask table): no value.
type MovedAsk struct {
ID int64 `json:"id"`
Node string `json:"node"`
Module string `json:"module"`
Name string `json:"name"`
Desk string `json:"desk"`
AskedBy string `json:"asked_by"`
OpenedAt time.Time `json:"opened_at"`
EndedAt *time.Time `json:"ended_at,omitempty"`
Outcome string `json:"outcome,omitempty"`
}
// AsksMoved is what a copy of the record of asks did.
type AsksMoved struct {
Copied int `json:"copied"`
Already int `json:"already"`
}
// MovedAsksAtOnce bounds one call: the controller sends its record in parts.
const MovedAsksAtOnce = 500
// movedAskID is the vault's id of a row of the controller's record.
func movedAskID(id int64) string { return "controller-" + strconv.FormatInt(id, 10) }
// MoveAsks copies rows of the controller's record of asks into the vault's, as they are: each ask's opening and,
// when it ended, its end — once, so a copy run again adds nothing. From the controller alone, whose record it is.
// Every row is judged before any is written.
func (v *Vault) MoveAsks(caller string, asks []MovedAsk) (AsksMoved, error) {
if err := fromTheController(caller, "move-asks"); err != nil {
return AsksMoved{}, err
}
if len(asks) > MovedAsksAtOnce {
return AsksMoved{}, fmt.Errorf("%d asks at once is more than %d; send them in parts", len(asks), MovedAsksAtOnce)
}
type lines struct{ opened, ended AskRecord }
var all []lines
for _, a := range asks {
secret, err := OwnSecret(a.Node, a.Module, a.Name)
if err != nil {
return AsksMoved{}, fmt.Errorf("ask %d: %w; nothing was copied", a.ID, err)
}
if !namePart.MatchString(a.Desk) || a.OpenedAt.IsZero() || a.ID <= 0 {
return AsksMoved{}, fmt.Errorf("ask %d names no desk, no time it opened or no id; nothing was copied", a.ID)
}
l := lines{opened: AskRecord{ID: movedAskID(a.ID), At: a.OpenedAt.UTC(), Event: AskOpened, Secret: secret,
Node: a.Node, Module: a.Module, Name: a.Name, Desk: a.Desk, AskedBy: askedByName(a.AskedBy),
Caller: ControllerCaller}}
if a.EndedAt != nil {
if a.Outcome != "given" && !askOutcomes[a.Outcome] {
return AsksMoved{}, fmt.Errorf("ask %d ended %q, which is no outcome the record knows; nothing was "+
"copied", a.ID, a.Outcome)
}
l.ended = AskRecord{ID: movedAskID(a.ID), At: a.EndedAt.UTC(), Event: AskEnded, Secret: secret,
Outcome: a.Outcome}
}
all = append(all, l)
}
v.Store.asksMu.Lock()
defer v.Store.asksMu.Unlock()
held, err := v.Store.asks()
if err != nil {
return AsksMoved{}, fmt.Errorf("the record of asks cannot be read: %w; nothing was copied", err)
}
seen := map[string]bool{}
for _, r := range held {
if r.Event == AskOpened {
seen[r.ID] = true
}
}
out := AsksMoved{}
for _, l := range all {
if seen[l.opened.ID] {
out.Already++
continue
}
if err := v.recordAsk(l.opened); err != nil {
return out, fmt.Errorf("ask %s could not be copied, after %d: %w", l.opened.ID, out.Copied, err)
}
if l.ended.ID != "" {
if err := v.recordAsk(l.ended); err != nil {
return out, fmt.Errorf("the end of ask %s could not be copied, after %d: %w", l.opened.ID, out.Copied, err)
}
}
seen[l.opened.ID] = true
out.Copied++
}
return out, nil
}
// StateNodeKey records the sealing key the node-engine of caller's machine states in its own name (novox/hq ADR 0372,
// issue 786), at its start and whenever the key changes, whether or not it ever moved a secret in. The machine is the
// caller's, as the bus proved it (`node.<machine>`), never an argument. A first key is handled like a changed one: the
// machine's core secrets, sealed to no key until now or to the old one, are said changed so the controller asks again
// (ADR 0349 §16). The same key again changes nothing and says nothing.
func (v *Vault) StateNodeKey(caller, key string) error {
node, ok := nodeEngineOf(caller)
if !ok {
if caller == "" || caller == stdio.Unproven {
caller = "an unproven caller"
}
return fmt.Errorf("node-key takes a machine's sealing key from that machine's node-engine alone, in its own "+
"name; %s is not a node-engine", caller)
}
stated, err := sealingKey(key)
if err != nil {
return err
}
held, had, err := v.Store.NodeKey(node)
if err != nil {
return err
}
if had && held == stated {
return nil
}
if err := v.recordNodeKey(node, stated, caller); err != nil {
return err
}
all, err := v.Store.List()
if err != nil {
return fmt.Errorf("%s's key is recorded, but its core secrets could not be listed to say them changed: %w", node, err)
}
for _, r := range all {
parts := strings.Split(r.Name, "/")
if len(parts) == 4 && parts[0] == "own" && parts[1] == node && slices.Contains(CoreModules, parts[2]) {
v.announce(Changed{Name: r.Name, Generation: r.Generation, At: v.Now().UTC(), By: caller})
}
}
return nil
}
modules/mesh-vault/cmd/mesh-vault/move_test.go
+321
View File
@@ -0,0 +1,321 @@
package main
// The move of the controller's secrets into the vault (design 57's V3, issue 676, the operator's decision of
// 2026-10-14): each machine's node-engine opens what was sealed to it and hands it in, sealed to a key the vault
// made for the call, keeping where it came from and its generation; the vault takes that step only from the
// node-engine of a machine the value was sealed to, never replaces a value it holds, and answers what it kept
// read back from its own store, sealed to a key the node-engine sent. The controller copies its record of asks
// as it is. Nothing is rotated.
import (
"crypto/rand"
"encoding/base64"
"strings"
"testing"
"time"
"golang.org/x/crypto/nacl/box"
)
// aReadBackKey is a key the node-engine makes for one read-back: its public half to send, its private to open.
func aReadBackKey(t *testing.T) (string, *[32]byte, *[32]byte) {
t.Helper()
pub, priv, err := box.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
return base64.StdEncoding.EncodeToString(pub[:]), pub, priv
}
// openReadBack opens what the vault read back, as the node-engine does.
func openReadBack(t *testing.T, sealed string, pub, priv *[32]byte) string {
t.Helper()
blob, err := base64.StdEncoding.DecodeString(sealed)
if err != nil {
t.Fatal(err)
}
out, ok := box.OpenAnonymous(nil, blob, pub, priv)
if !ok {
t.Fatal("the read-back does not open with the key sent for it")
}
return string(out)
}
// moveIn is one node-engine's two steps for one value.
func moveIn(t *testing.T, v testVault, caller string, m MoveOf, value string) (MovedIn, error) {
t.Helper()
opened, err := v.MoveOpen(caller, m)
if err != nil {
return MovedIn{}, err
}
key, _, _ := aReadBackKey(t)
return v.MoveTake(caller, opened.Call, sealToCall(t, opened.SealTo, value), key)
}
const movedValue = "Xq3-the-superusers-password-made-in-2025"
func superuser() MoveOf {
return MoveOf{Node: "anchor", Module: "postgres", Name: "superuser-password", Origin: "made",
Generation: "g-2025-11-02T10:00:00Z-1", MadeAt: time.Date(2025, 11, 2, 10, 0, 0, 0, time.UTC)}
}
// A machine's own value moves as it is: opened in the vault alone, kept with its origin, the controller's
// generation and when it was made, and read back from the vault's store to the node-engine, sealed to its key.
func TestANodeEngineMovesInAValueSealedToItsMachineAsItIs(t *testing.T) {
v := newVault(t, nil)
opened, err := v.MoveOpen("node.anchor", superuser())
if err != nil {
t.Fatal(err)
}
if opened.Secret != "own/anchor/postgres/superuser-password" || opened.SealTo == "" || opened.Call == "" {
t.Fatalf("opened %+v", opened)
}
key, pub, priv := aReadBackKey(t)
moved, err := v.MoveTake("node.anchor", opened.Call, sealToCall(t, opened.SealTo, movedValue), key)
if err != nil {
t.Fatal(err)
}
if moved.Name != opened.Secret || moved.Generation != 1 || moved.Unchanged {
t.Errorf("moved %+v", moved)
}
if got := openReadBack(t, moved.ReadBack, pub, priv); got != movedValue {
t.Errorf("the read-back opens to %q", got)
}
rec, held, _ := v.Store.Read(opened.Secret)
if !held || rec.Origin != OriginMade || rec.ChangedBy != "node.anchor" || rec.MovedFrom == nil ||
rec.MovedFrom.Generation != "g-2025-11-02T10:00:00Z-1" || rec.MovedFrom.By != "node.anchor" ||
!rec.MadeAt.Equal(superuser().MadeAt) {
t.Errorf("the record: %+v moved from %+v", rec, rec.MovedFrom)
}
// Kept as every value is: a copy for the operator.
if got := openOperator(t, v.operatorPublic, v.operatorPrivate, rec.Operator.Sealed); got != movedValue {
t.Errorf("the operator copy opens to %q", got)
}
if strings.Contains(everything(t, v, opened, moved), movedValue) {
t.Fatal("the value is in something the vault said, answered or wrote in the clear")
}
}
// Each origin the controller's store holds is kept as the vault's word for it; any other is refused.
func TestAMovedValueKeepsWhereItCameFrom(t *testing.T) {
v := newVault(t, nil)
for origin, want := range map[string]string{"made": OriginMade, "accepted": OriginAccepted, "issued": OriginIssued} {
m := superuser()
m.Name, m.Origin = "from-"+origin, origin
if _, err := moveIn(t, v, "node.anchor", m, "value-"+origin); err != nil {
t.Fatalf("%s: %v", origin, err)
}
rec, _, _ := v.Store.Read("own/anchor/postgres/from-" + origin)
if rec.Origin != want {
t.Errorf("%s was kept as %q", origin, rec.Origin)
}
}
m := superuser()
m.Origin = "invented"
if _, err := v.MoveOpen("node.anchor", m); err == nil || !strings.Contains(err.Error(), "origin") {
t.Errorf("an unknown origin: %v", err)
}
}
// Only the node-engine of a machine the value was sealed to takes the step: not the controller, not a module's
// tools, not another machine's node-engine.
func TestOnlyTheNodeEngineOfAMachineTheValueWasSealedToMovesIt(t *testing.T) {
v := newVault(t, nil)
for _, caller := range []string{ControllerCaller, "anchor.node-tools", "anchor.mesh-vault", "node.shanks", "",
"node.", "node.anchor.x"} {
if _, err := v.MoveOpen(caller, superuser()); err == nil {
t.Errorf("%q opened a move of anchor's value", caller)
}
}
// A pair credential was sealed to both its ends: either may move it, and nobody else.
pair := MoveOf{Provider: "anchor", Provision: "database", Consumer: "shanks", ConsumerModule: "gitea",
Origin: "made", Generation: "g-1"}
for _, caller := range []string{"node.anchor", "node.shanks"} {
if _, err := v.MoveOpen(caller, pair); err != nil {
t.Errorf("%s, an end of the pair, was refused: %v", caller, err)
}
}
if _, err := v.MoveOpen("node.g14", pair); err == nil {
t.Error("a machine at neither end opened a move of the pair")
}
// The second step is the first step's caller's alone.
opened, err := v.MoveOpen("node.anchor", superuser())
if err != nil {
t.Fatal(err)
}
key, _, _ := aReadBackKey(t)
if _, err := v.MoveTake("node.shanks", opened.Call, sealToCall(t, opened.SealTo, movedValue), key); err == nil {
t.Error("another machine's node-engine took the second step")
}
if _, held, _ := v.Store.Read("own/anchor/postgres/superuser-password"); held {
t.Error("a value was kept from the wrong caller")
}
}
// A pair credential is named by both ends and its local name, so every row of the controller's store is one
// secret in the vault, kept as it is.
func TestAPairCredentialMovesUnderTheNameOfItsRow(t *testing.T) {
v := newVault(t, nil)
pair := MoveOf{Provider: "anchor", Provision: "database", Consumer: "shanks", ConsumerModule: "gitea",
Local: "replica", Origin: "accepted", Generation: "g-7"}
moved, err := moveIn(t, v, "node.shanks", pair, "the-pair-value")
if err != nil {
t.Fatal(err)
}
if moved.Name != "pair/anchor/database/shanks/gitea/replica" {
t.Errorf("named %s", moved.Name)
}
pair.Local = ""
if moved, err = moveIn(t, v, "node.anchor", pair, "the-ordinary-value"); err != nil ||
moved.Name != "pair/anchor/database/shanks/gitea" {
t.Errorf("the ordinary one: %+v, %v", moved, err)
}
}
// The move never replaces a value: the same value again changes nothing (the other end of a pair, or a move run
// twice), and a different one is refused, so a value the vault holds is never put back to an older one.
func TestTheMoveNeverReplacesAValueTheVaultHolds(t *testing.T) {
v := newVault(t, nil)
pair := MoveOf{Provider: "anchor", Provision: "database", Consumer: "shanks", ConsumerModule: "gitea",
Origin: "made", Generation: "g-1"}
if _, err := moveIn(t, v, "node.anchor", pair, "the-pair-value"); err != nil {
t.Fatal(err)
}
opened, err := v.MoveOpen("node.shanks", pair)
if err != nil {
t.Fatal(err)
}
key, pub, priv := aReadBackKey(t)
again, err := v.MoveTake("node.shanks", opened.Call, sealToCall(t, opened.SealTo, "the-pair-value"), key)
if err != nil || !again.Unchanged || again.Generation != 1 {
t.Fatalf("the other end's same value: %+v, %v", again, err)
}
if got := openReadBack(t, again.ReadBack, pub, priv); got != "the-pair-value" {
t.Errorf("read back %q", got)
}
if _, err := v.Rotate(ControllerCaller, "pair/anchor/database/shanks/gitea", "jochen", "a test"); err != nil {
t.Fatal(err)
}
if _, err := moveIn(t, v, "node.anchor", pair, "the-pair-value"); err == nil ||
!strings.Contains(err.Error(), "never replaces") {
t.Fatalf("a move over a value the vault changed since: %v", err)
}
rec, _, _ := v.Store.Read("pair/anchor/database/shanks/gitea")
if rec.Generation != 2 || rec.Origin != OriginMade {
t.Errorf("the vault's own value was touched: %+v", rec)
}
}
// A second step that is not sealed to its call, or carries no key to read back to, takes nothing.
func TestAMoveTakesNothingItCannotOpenOrReadBack(t *testing.T) {
v := newVault(t, nil)
opened, err := v.MoveOpen("node.anchor", superuser())
if err != nil {
t.Fatal(err)
}
for _, key := range []string{"", "not a key", "-----BEGIN PRIVATE KEY-----"} {
if _, err := v.MoveTake("node.anchor", opened.Call, sealToCall(t, opened.SealTo, movedValue), key); err == nil {
t.Errorf("read-back key %q was taken", key)
}
}
// A refused key leaves the call open for the right one.
other, _ := v.MoveOpen("node.anchor", superuser())
key, _, _ := aReadBackKey(t)
if _, err := v.MoveTake("node.anchor", other.Call, "bm90IHNlYWxlZA==", key); err == nil {
t.Error("a value not sealed to the call was taken")
}
if _, held, _ := v.Store.Read("own/anchor/postgres/superuser-password"); held {
t.Error("something was kept")
}
if moved, err := v.MoveTake("node.anchor", opened.Call, sealToCall(t, opened.SealTo, movedValue), key); err != nil ||
moved.Generation != 1 {
t.Errorf("the call after a refused key: %+v, %v", moved, err)
}
}
// The controller's record of asks is copied as it is, once: each ask's opening and its end, who asked and at which
// desk, and the bounds then count them. From the controller alone.
func TestTheRecordOfAsksIsCopiedAsItIsOnce(t *testing.T) {
v := newVault(t, nil)
now := v.Now()
ended := now.Add(-30 * time.Second)
asks := []MovedAsk{
{ID: 41, Node: "anchor", Module: "telegram", Name: "telegram-token", Desk: "laptop", AskedBy: "g14/claude-code",
OpenedAt: now.Add(-50 * time.Minute), EndedAt: &ended, Outcome: "given"},
{ID: 42, Node: "anchor", Module: "telegram", Name: "telegram-token", Desk: "laptop", AskedBy: "g14/claude-code",
OpenedAt: now.Add(-40 * time.Minute), EndedAt: &ended, Outcome: "dismissed"},
{ID: 43, Node: "anchor", Module: "telegram", Name: "telegram-token", Desk: "laptop", AskedBy: "ace/agent",
OpenedAt: now.Add(-20 * time.Minute)},
}
if _, err := v.MoveAsks("node.anchor", asks); err == nil {
t.Error("a node-engine copied the record of asks")
}
copied, err := v.MoveAsks(ControllerCaller, asks)
if err != nil || copied.Copied != 3 || copied.Already != 0 {
t.Fatalf("copied %+v, %v", copied, err)
}
again, err := v.MoveAsks(ControllerCaller, asks)
if err != nil || again.Copied != 0 || again.Already != 3 {
t.Fatalf("copied again %+v, %v", again, err)
}
all, _ := v.Store.Asks(100)
if len(all) != 5 {
t.Fatalf("%d lines: %+v", len(all), all)
}
opened := map[string]AskRecord{}
for _, r := range all {
if r.Event == AskOpened {
opened[r.ID] = r
}
}
if r := opened["controller-41"]; r.Secret != "own/anchor/telegram/telegram-token" || r.AskedBy != "g14/claude-code" ||
r.Desk != "laptop" || !r.At.Equal(asks[0].OpenedAt) || r.Caller != ControllerCaller {
t.Errorf("ask 41: %+v", r)
}
// Three asks in the hour, from the controller's record: the vault's bound holds them.
if _, err := v.AskOpen(ControllerCaller, telegramAsk("g14/claude-code")); err == nil ||
!strings.Contains(err.Error(), "in the last hour") {
t.Errorf("an ask after three copied in the hour: %v", err)
}
bad := []MovedAsk{{ID: 44, Node: "anchor", Module: "telegram", Name: "x", Desk: "laptop", AskedBy: "a",
OpenedAt: now, EndedAt: &ended, Outcome: "whatever"}}
if _, err := v.MoveAsks(ControllerCaller, bad); err == nil {
t.Error("an ask that ended in no outcome the record knows was copied")
}
}
// The two steps as the runtime calls the tool: the caller is the one it proved, never an argument.
func TestMoveInThroughTheTool(t *testing.T) {
v := newVault(t, nil)
tool := toolNamed(t, definitions(&Env{Vault: v.Vault}), Seat+".move-in")
first := map[string]any{"node": "anchor", "module": "postgres", "name": "superuser-password", "origin": "made",
"generation": "g-1", "made_at": "2025-11-02T10:00:00Z"}
if _, err := tool.runAs("controller", first); err == nil {
t.Fatal("the controller opened a move")
}
got, err := tool.runAs("node.anchor", first)
if err != nil {
t.Fatal(err)
}
opened := got.(Opened)
key, pub, priv := aReadBackKey(t)
got, err = tool.runAs("node.anchor", map[string]any{"call": opened.Call,
"sealed": sealToCall(t, opened.SealTo, movedValue), "read_back_to": key})
if err != nil {
t.Fatal(err)
}
if back := openReadBack(t, got.(MovedIn).ReadBack, pub, priv); back != movedValue {
t.Errorf("read back %q", back)
}
rec, _, _ := v.Store.Read("own/anchor/postgres/superuser-password")
if rec.MadeAt.Year() != 2025 || rec.MovedFrom == nil || rec.MovedFrom.Generation != "g-1" {
t.Errorf("kept %+v", rec)
}
asks := toolNamed(t, definitions(&Env{Vault: v.Vault}), Seat+".move-asks")
moved, err := asks.runAs("controller", map[string]any{"asks": []any{map[string]any{"id": 7, "node": "anchor",
"module": "telegram", "name": "telegram-token", "desk": "laptop", "asked_by": "g14/claude-code",
"opened_at": "2026-10-01T10:00:00Z", "ended_at": "2026-10-01T10:00:20Z", "outcome": "given"}}})
if err != nil || moved.(AsksMoved).Copied != 1 {
t.Fatalf("asks through the tool: %+v, %v", moved, err)
}
}
modules/mesh-vault/cmd/mesh-vault/named_pair_test.go
+44
View File
@@ -0,0 +1,44 @@
package main
// A consumer's own login, kept by the vault as its pair row, is named on the consumer's plan entry when the
// controller hands the consumer's secrets file to its machine's client (issue 779): the client asks for the row, and
// the rule hands it to that consumer's machine alone.
import "testing"
func TestANamedPairEntryHandsTheConsumersMachineItsOwnLoginAndNoOtherConsumers(t *testing.T) {
const pg = "provision/novox/postgres-database"
mirror := assignments{
"novox.postgres": handedWith("novox", "postgres"),
"g14.gitea": handedWith("g14", "gitea", "postgres-database@novox/postgres=database+pair"),
"ace.umami": handedWith("ace", "umami", "postgres-database@novox/postgres#stats=database-stats+pair"),
"shanks.other": handedWith("shanks", "other", "postgres-database@novox/postgres=database+pair"),
}
v := pairVault(t, mirror)
if o := pass(t, v, pg); o.Outcome != PairMade {
t.Fatalf("a named pair entry held the provider back: %+v", o)
}
gitea, umami := "pair/novox/postgres-database/g14/gitea", "pair/novox/postgres-database/ace/umami/stats"
ofGitea, _, err := asks(v, "provisioner.g14", gitea)
if err != nil || ofGitea == "" {
t.Fatalf("the consumer's machine was refused its own login: %v", err)
}
if ofUmami, _, err := asks(v, "provisioner.ace", umami); err != nil || ofUmami == "" || ofUmami == ofGitea {
t.Fatalf("the second consumer's login: %q (%v)", ofUmami, err)
}
if _, _, err := asks(v, "provisioner.g14", umami); err == nil {
t.Fatal("a consumer's machine was handed another consumer's login")
}
if _, _, err := asks(v, "provisioner.shanks", gitea); err == nil {
t.Fatal("a machine was handed the login of a module that is not its own")
}
// A named entry of another form is still not a pair row's.
other := assignments{
"novox.postgres": handedWith("novox", "postgres"),
"g14.gitea": handedWith("g14", "gitea", "postgres-database@novox/postgres=database+issued"),
}
w := pairVault(t, other)
if _, _, err := asks(w, "provisioner.g14", gitea); err == nil {
t.Fatal("a named entry of the issued form was handed a pair row")
}
}
modules/mesh-vault/cmd/mesh-vault/nodekey_test.go
+191
View File
@@ -0,0 +1,191 @@
package main
// Issue 786 (ADR 0372): a machine's node-engine states its own sealing key to the vault, in its own name, at start and
// when it changes, whether or not it ever moved a secret in. The caller is the bus user the runtime proved; a changed
// key (or a first one) raises `secret-changed` for the machine's core secrets, the same key says nothing.
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
)
func changedNames(p *published) []string {
p.mu.Lock()
defer p.mu.Unlock()
var out []string
for _, e := range p.events {
if e["event"] == ChangedEvent {
out = append(out, e["body"].(map[string]any)["name"].(string))
}
}
return out
}
func TestANodeEngineStatesItsOwnKeyAndAChangeRaisesItsCoreSecrets(t *testing.T) {
v := newVault(t, nil)
tool := toolNamed(t, definitions(&Env{Vault: v.Vault}), Seat+".node-key")
// ace holds two core secrets and a leaf module's; g14 holds a core secret too.
old, _ := aSealingKey(t)
for _, m := range []MoveOf{
{Node: "ace", Module: "mesh-provisioner-client", Name: "provisioner-broker", Origin: "made", Generation: "g-1", SealingKey: old},
{Node: "ace", Module: "nats", Name: "broker", Origin: "made", Generation: "g-1"},
{Node: "ace", Module: "postgres", Name: "superuser-password", Origin: "made", Generation: "g-1"},
{Node: "g14", Module: "nats", Name: "broker", Origin: "made", Generation: "g-1"},
} {
if _, err := moveIn(t, v, "node."+m.Node, m, movedValue); err != nil {
t.Fatal(err)
}
}
// ace never stated a key in the moves above except through the first row; forget it to be a machine that never moved.
if err := os.Remove(v.Store.nodeKeyPath("ace")); err != nil {
t.Fatal(err)
}
before := len(changedNames(v.bus))
pub, _ := aSealingKey(t)
if _, err := tool.runAs("controller", map[string]any{"key": pub}); err == nil {
t.Error("the controller stated a machine's key")
}
if _, err := tool.runAs("", map[string]any{"key": pub}); err == nil {
t.Error("an unproven caller stated a key")
}
if _, err := tool.runAs("node.ace", map[string]any{"key": "c2hvcnQ="}); err == nil {
t.Error("a short key was taken")
}
if _, err := tool.runAs("node.ace", map[string]any{"node": "g14", "key": pub}); err != nil {
t.Fatal(err)
}
// The machine is the caller's; an argument naming another one changes nothing.
if got, held, _ := v.Store.NodeKey("ace"); !held || got != pub {
t.Errorf("ace's key is %q, %v", got, held)
}
if _, held, _ := v.Store.NodeKey("g14"); held {
t.Error("a key was recorded for g14 by ace's engine")
}
got := changedNames(v.bus)[before:]
want := []string{"own/ace/mesh-provisioner-client/provisioner-broker", "own/ace/nats/broker"}
if strings.Join(got, ",") != strings.Join(want, ",") {
t.Errorf("secret-changed was raised for %v, want %v", got, want)
}
// The same key again changes nothing and says nothing.
if _, err := tool.runAs("node.ace", map[string]any{"key": pub}); err != nil {
t.Fatal(err)
}
if n := len(changedNames(v.bus)); n != before+2 {
t.Errorf("the same key raised %d more", n-before-2)
}
// A different key raises them again.
other, _ := aSealingKey(t)
if _, err := tool.runAs("node.ace", map[string]any{"key": other}); err != nil {
t.Fatal(err)
}
if n := len(changedNames(v.bus)); n != before+4 {
t.Errorf("a changed key raised %d, want 2", n-before-2)
}
if got, _, _ := v.Store.NodeKey("ace"); got != other {
t.Error("the changed key was not recorded")
}
}
const nodeKeyBus = `listen: 127.0.0.1:-1
authorization {
users = [
{ user: "novox.mesh-vault", password: "pw", permissions: {
publish: { deny: [">"] }
subscribe: { allow: ["mesh.seat.mesh-vault.call.node-key.*"] }
allow_responses: true } }
{ user: "node.ace", password: "pw", permissions: {
publish: { allow: ["mesh.seat.mesh-vault.call.node-key.node_ace"] }
subscribe: { allow: ["_INBOX.node.ace.>"] } } }
{ user: "node.g14", password: "pw", permissions: {
publish: { allow: ["mesh.seat.mesh-vault.call.node-key.node_g14"] }
subscribe: { allow: ["_INBOX.node.g14.>"] } } }
]
}
`
// On a real bus whose accounts may publish only their own subject: node.ace's statement records ace's key; node.g14
// cannot speak in ace's name, and what it states is g14's.
func TestOnARealBusANodeEngineStatesOnlyItsOwnKey(t *testing.T) {
file := filepath.Join(t.TempDir(), "bus.conf")
if err := os.WriteFile(file, []byte(nodeKeyBus), 0o600); err != nil {
t.Fatal(err)
}
opts, err := server.ProcessConfigFile(file)
if err != nil {
t.Fatal(err)
}
opts.NoLog, opts.NoSigs = true, true
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(s.Shutdown)
v := newVault(t, nil)
tool := toolNamed(t, definitions(&Env{Vault: v.Vault}), Seat+".node-key")
vaultConn := connectAs(t, s.ClientURL(), "novox.mesh-vault")
if _, err := vaultConn.Subscribe("mesh.seat.mesh-vault.call.node-key.*", func(m *nats.Msg) {
parts := strings.Split(m.Subject, ".")
var args map[string]any
_ = json.Unmarshal(m.Data, &args)
answer := busAnswer{}
if got, err := tool.runAs(untoken(parts[len(parts)-1]), args); err != nil {
answer.Error = err.Error()
} else {
answer.Result, _ = json.Marshal(got)
}
raw, _ := json.Marshal(answer)
_ = m.Respond(raw)
}); err != nil {
t.Fatal(err)
}
_ = vaultConn.Flush()
say := func(user, subject, key string) (string, error) {
raw, _ := json.Marshal(map[string]string{"key": key})
msg, err := connectAs(t, s.ClientURL(), user).Request(subject, raw, 3*time.Second)
if err != nil {
return "", err
}
var a busAnswer
_ = json.Unmarshal(msg.Data, &a)
if a.Error != "" {
return a.Error, errString(a.Error)
}
return "", nil
}
aceKey, _ := aSealingKey(t)
g14Key, _ := aSealingKey(t)
if _, err := say("node.ace", "mesh.seat.mesh-vault.call.node-key.node_ace", aceKey); err != nil {
t.Fatal(err)
}
if got, held, _ := v.Store.NodeKey("ace"); !held || got != aceKey {
t.Errorf("ace's key is %q, %v", got, held)
}
// g14 may not publish on ace's subject: the bus refuses it, so the vault never hears it.
if _, err := say("node.g14", "mesh.seat.mesh-vault.call.node-key.node_ace", g14Key); err == nil {
t.Error("node.g14 spoke in ace's name")
}
if got, _, _ := v.Store.NodeKey("ace"); got != aceKey {
t.Error("ace's key was replaced by g14's")
}
if _, err := say("node.g14", "mesh.seat.mesh-vault.call.node-key.node_g14", g14Key); err != nil {
t.Fatal(err)
}
if got, held, _ := v.Store.NodeKey("g14"); !held || got != g14Key {
t.Errorf("g14's key is %q, %v", got, held)
}
}
modules/mesh-vault/cmd/mesh-vault/owncalls_test.go
+230
View File
@@ -0,0 +1,230 @@
package main
// The vault's own calls (issue 723): it asks the desk's prompt and an issuing provider's tool itself, in its own
// name, under the bus account its own runtime runs on. That the bus lets that account alone make these calls, and
// nothing else, is mesh-controller's test on a real server (internal/broker vault_calls_live_test.go); here, what
// the vault sends and takes back on those calls carries keys and sealed values only.
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"golang.org/x/crypto/nacl/box"
)
// vaultsAccount is the vault's bus user as its own runtime names it to a tool it calls.
const vaultsAccount = "novox.mesh-vault"
// aMesh stands in for the bus the vault's runtime calls on: each call is answered by the desk's prompt or a
// provider's tool, named by the caller the vault's own account is, and everything that crosses is kept.
type aMesh struct {
mu sync.Mutex
desk func(caller string, args map[string]any) (map[string]any, error)
tools map[string]func(caller string, args map[string]any) (map[string]any, error)
keys []string
crossed []string
}
func (m *aMesh) ask(key string, body any) (json.RawMessage, error) {
m.mu.Lock()
m.keys = append(m.keys, key)
m.mu.Unlock()
raw, _ := json.Marshal(body)
var args map[string]any
_ = json.Unmarshal(raw, &args)
var answer map[string]any
var err error
switch {
case strings.HasPrefix(key, "seat:node-launcher.secret@") && m.desk != nil:
answer, err = m.desk(vaultsAccount, args)
case m.tools[key] != nil:
answer, err = m.tools[key](vaultsAccount, args)
default:
err = errors.New("503 no responders")
}
out, _ := json.Marshal(answer)
m.mu.Lock()
m.crossed = append(m.crossed, string(raw), string(out))
m.mu.Unlock()
if err != nil {
return nil, err
}
return out, nil
}
func (m *aMesh) said() string {
m.mu.Lock()
defer m.mu.Unlock()
return strings.Join(m.crossed, "\n")
}
// aDesk is the desk's prompt (rofi's node-launcher.secret): it answers what the operator types, sealed to the key
// it is given, and keeps what it was asked.
func aDesk(typed string, asked *[]map[string]any) func(string, map[string]any) (map[string]any, error) {
return func(caller string, args map[string]any) (map[string]any, error) {
*asked = append(*asked, args)
switch typed {
case "":
return map[string]any{"cancelled": true}, nil
case "\x00timeout":
return map[string]any{"cancelled": true, "timed_out": true}, nil
}
key, _ := args["seal_to"].(string)
raw, err := base64.StdEncoding.DecodeString(key)
if err != nil || len(raw) != 32 {
return nil, errors.New("seal_to is not a sealing key")
}
var to [32]byte
copy(to[:], raw)
sealed, err := box.SealAnonymous(nil, []byte(typed+"\n"), &to, rand.Reader)
if err != nil {
return nil, err
}
return map[string]any{"sealed": base64.StdEncoding.EncodeToString(sealed)}, nil
}
}
// The vault asks the desk itself: recorded first, the prompt named by the vault's call and naming the asker, what
// is typed opened here alone; what crossed is a key and a sealed box.
func TestTheVaultAsksTheDesksPromptItselfAndOnlyASealedValueCrosses(t *testing.T) {
v := newVault(t, AllowList{"anchor.node-provisioner": {"own/anchor/telegram/telegram-token"}})
var asked []map[string]any
mesh := &aMesh{desk: aDesk(typedAtTheDesk, &asked)}
v.Ask = mesh.ask
got, err := v.AskAtDesk(ControllerCaller, telegramAsk("g14/claude-code, via the console"))
if err != nil {
t.Fatal(err)
}
if got.Outcome != "given" || got.Generation != 1 || got.AskedBy != "g14/claude-code" ||
got.Secret != "own/anchor/telegram/telegram-token" {
t.Errorf("answered %+v", got)
}
if len(mesh.keys) != 1 || mesh.keys[0] != "seat:node-launcher.secret@laptop" {
t.Fatalf("the vault asked %v", mesh.keys)
}
p := asked[0]
if p["asked_by"] != "g14/claude-code" || p["module"] != "telegram" || p["secret"] != "telegram-token" ||
p["node"] != "anchor" || p["seal_to"] == "" || p["timeout_seconds"] != float64(DeskPromptFor) {
t.Errorf("the prompt was asked %v", p)
}
if v := handedTo(t, v, "anchor.node-provisioner", "own/anchor/telegram/telegram-token"); v != typedAtTheDesk {
t.Errorf("the vault holds %q", v)
}
rec, _, _ := v.Store.Read("own/anchor/telegram/telegram-token")
if rec.Origin != OriginTyped || rec.ChangedBy != "g14/claude-code" || rec.ChangedVia != ControllerCaller {
t.Errorf("the record: origin %q by %q via %q", rec.Origin, rec.ChangedBy, rec.ChangedVia)
}
asks, _ := v.Store.Asks(10)
if len(asks) != 2 || asks[1].Event != AskOpened || asks[0].Outcome != "given" {
t.Errorf("the record of asks: %+v", asks)
}
if strings.Contains(mesh.said(), typedAtTheDesk) || strings.Contains(everything(t, v, got, asks), typedAtTheDesk) {
t.Fatal("the typed value crossed the bus, or is in something the vault said or wrote, in the clear")
}
// Dismissed and timed out end the ask so, and keep nothing.
for typed, outcome := range map[string]string{"": "dismissed", "\x00timeout": "timed-out"} {
w := newVault(t, nil)
var a []map[string]any
w.Ask = (&aMesh{desk: aDesk(typed, &a)}).ask
got, err := w.AskAtDesk(ControllerCaller, telegramAsk("g14.claude-code"))
if err != nil || got.Outcome != outcome {
t.Errorf("%s: %+v %v", outcome, got, err)
}
if asks, _ := w.Store.Asks(1); asks[0].Outcome != outcome {
t.Errorf("%s recorded %+v", outcome, asks[0])
}
if _, held, _ := w.Store.Read("own/anchor/telegram/telegram-token"); held {
t.Errorf("%s kept a value", outcome)
}
}
// Answered empty: the operator's answer, said so, nothing kept.
e := newVault(t, nil)
var ea []map[string]any
e.Ask = (&aMesh{desk: aDesk(" ", &ea)}).ask
if got, err := e.AskAtDesk(ControllerCaller, telegramAsk("g14.claude-code")); err != nil || got.Outcome != "empty" {
t.Errorf("empty: %+v %v", got, err)
}
// The desk cannot be asked: refused, recorded so.
r := newVault(t, nil)
r.Ask = (&aMesh{}).ask
if _, err := r.AskAtDesk(ControllerCaller, telegramAsk("g14.claude-code")); err == nil {
t.Error("an ask nobody answered was said to be done")
}
if asks, _ := r.Store.Asks(1); asks[0].Outcome != "refused" {
t.Errorf("an unanswered desk recorded %+v", asks[0])
}
// Only the controller asks the vault to ask: anybody else is refused before any prompt.
n := newVault(t, nil)
var na []map[string]any
n.Ask = (&aMesh{desk: aDesk(typedAtTheDesk, &na)}).ask
for _, caller := range []string{"g14.node-tools", "g14.claude-code", "unproven", ""} {
if _, err := n.AskAtDesk(caller, telegramAsk("x")); err == nil {
t.Errorf("%q had the vault ask a desk", caller)
}
}
if len(na) != 0 {
t.Fatal("a desk was asked for a caller that is not the controller")
}
}
// The vault asks an issuing provider's tool itself: by its name on the provider's machine, with a key made for the
// call; the tool answers the vault's account; the value opens here; what crossed is a key and a sealed box.
func TestTheVaultAsksTheIssuingToolItselfAndOnlyASealedValueCrosses(t *testing.T) {
v := newVault(t, AllowList{"anchor.node-provisioner": {"provision/anchor/plex-api"}})
software := "plex-token-from-the-vendor"
mesh := &aMesh{tools: map[string]func(string, map[string]any) (map[string]any, error){
"plex.plex_issued_credential@anchor": standInProvider(&software)}}
v.Ask = mesh.ask
got, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", "plex_issued_credential")
if err != nil {
t.Fatal(err)
}
if got.Secret != "provision/anchor/plex-api" || got.Generation != 1 || got.Unchanged {
t.Errorf("answered %+v", got)
}
if got := handedTo(t, v, "anchor.node-provisioner", "provision/anchor/plex-api"); got != software {
t.Errorf("the vault holds %q", got)
}
if strings.Contains(mesh.said(), software) || strings.Contains(everything(t, v, got), software) {
t.Fatal("the issued value crossed the bus, or is in something the vault said or wrote, in the clear")
}
if !strings.Contains(mesh.said(), `"provision":"plex-api"`) || !strings.Contains(mesh.said(), `"seal_to"`) {
t.Errorf("the tool was asked %s", mesh.said())
}
// The same value again writes nothing.
again, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", "plex_issued_credential")
if err != nil || !again.Unchanged || again.Generation != 1 {
t.Errorf("again: %+v %v", again, err)
}
// Only the controller asks the vault to take it; a tool's name that is not one is refused; a tool that is not
// answered, or answers no sealed value, keeps nothing.
for _, caller := range []string{"anchor.node-tools", "g14.claude-code", "unproven", ""} {
if _, err := v.TakeIssued(caller, "plex-api", "anchor", "plex", "plex_issued_credential"); err == nil {
t.Errorf("%q had the vault ask a provider", caller)
}
}
for _, bad := range []string{"", "plex.other", "x@y", "a b"} {
if _, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", bad); err == nil {
t.Errorf("the tool %q was asked", bad)
}
}
if _, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", "plex_status"); err == nil {
t.Error("a tool nobody answers gave a value")
}
mesh.tools["plex.plex_issued_credential@anchor"] = func(string, map[string]any) (map[string]any, error) {
return map[string]any{"value": "in-the-clear"}, nil
}
if _, err := v.TakeIssued(ControllerCaller, "plex-api", "anchor", "plex", "plex_issued_credential"); err == nil {
t.Error("a tool answering no sealed value gave a value")
}
if len(v.pending) != 0 {
t.Errorf("%d calls' keys are still kept", len(v.pending))
}
}
modules/mesh-vault/cmd/mesh-vault/pair.go
+814
View File
@@ -0,0 +1,814 @@
package main
// The credential a provider shares with its consumers (novox/hq ADR 0349 clause 8; ADR 0158 §4–§6, §8; design 57's
// V5, issue 677): one value per provider assignment and provision, `provision/<provider node>/<provision>`, minted
// and remade by the vault for all its holders together when a consumer binds or unbinds, or when it is rotated.
//
// **Where the vault sees a bind.** A recorded need names no provider (issue 677's note 7). The controller names, in
// the plan of each module it handed to the provisioner, each provision the module requires with the provider
// resolution chose (issue 760): `provisions` in `<node>.<module>` of the assignments mirror, the plan the module's
// client records its needs from. A consumer binds when such an entry appears and unbinds when it goes, or when its
// assignment goes. The vault reads every plan on each pass (once a minute, and when it starts).
//
// **Every holder can get it, or nothing is made** (the lead's rule on issue 677, note 10). The holders are the
// provider on its machine and every consumer bound to it. Only a module handed to the provisioner gets a value from
// the vault; one still filled the old way gets the controller's (ADR 0349 clause 17). So the vault mints or remakes
// a provider's value only when the provider's assignment is handed to the provisioner, and no consumer the vault
// knows to hold it is still filled the old way: a consumer whose pair row the move brought in (V3) and whose
// assignment is still there with no plan. Otherwise it makes nothing, and says why on each pass that changes what
// it says. A consumer filled the old way that bound after the move names no provider on the bus at all; it is the
// provider's move (design 57 B2, B3) that ends that case.
//
// **What the move brought in is adopted, and nothing is rotated early.** The move (V3) copied each pair row as
// `pair/<provider>/<provision>/<consumer>/<module>[/<local>]`. Rows a provider shares (ADR 0158: its own secret and
// every consumer's row made in one act) carry one generation stamp; when every row of a provider and provision
// carries the same stamp and the same value, the vault keeps that value as the provision's one credential, with its
// origin, and remakes nothing. Rows of a provider with a login per consumer (ADR 0048) carry no stamp and differ
// per pair: each consumer keeps a value of its own for ever (ADR 0158 §11, issue 677's note 12). The vault never
// makes them one shared value; a rotation (`rotate` of `provision/<provider>/<provision>`) remakes each consumer's
// own apart, and each consumer's machine is handed only its own.
//
// **A provider the vault holds no row of gets its first value in its offer's form** (issue 768). Each plan entry
// names the form the provider's offer declares (`credential-form`, from the offer's `credential` in the provider's
// manifest): `shared`, one value for every holder (ADR 0158), made as `provision/<provider>/<provision>`; `pair`, a
// login of its own for each consumer (ADR 0048, ADR 0158 §11), made as each consumer's pair row; `issued` by the
// provider's software (ADR 0305) or `given` by a person (ADR 0337), which the vault never makes. An entry naming no
// form (a controller from before), plans that disagree, or a form the vault does not know: nothing is made, rather
// than a guess. A provider the vault already holds rows of is unchanged by the form: it is adopted or kept per pair
// as above.
//
// **An accepted or issued value is never remade** (ADR 0158 §8, ADR 0305 §9): the vault records who holds it and
// hands it to them, and leaves the value alone. A provision whose plans name a credential per holder (issue 757,
// an entry with a name) has no one shared value and is not the vault's to make here.
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// Mirror is the controller's assignments mirror as the vault reads it: every key, and one key's value.
type Mirror struct {
Keys func() ([]string, error)
Read AssignmentReader
}
// Shared is what the vault last made, adopted or recorded the provision's one credential for: its holders and the
// generation they hold. Never a value.
type Shared struct {
Name string `json:"name"`
Provider string `json:"provider"`
Provision string `json:"provision"`
ProviderModule string `json:"provider-module,omitempty"`
// Consumers are the binds holding it: `<node>/<module>`, and `/<local>` when bound under a local name.
Consumers []string `json:"consumers"`
Generation int `json:"generation"`
At time.Time `json:"at"`
// Adopted is the controller's generation stamp of the pair rows the value was adopted from, when it was.
Adopted string `json:"adopted,omitempty"`
}
// holderNodes are the machines holding it: the provider's and every consumer's.
func (s Shared) holderNodes() []string {
out := []string{s.Provider}
for _, c := range s.Consumers {
out = append(out, strings.SplitN(c, "/", 2)[0])
}
return out
}
func (s *Store) sharedPath(name string) string {
return filepath.Join(s.dir, "shared", url.PathEscape(name)+".json")
}
// ReadShared is the record of who holds the provision's one credential name, and whether there is one.
func (s *Store) ReadShared(name string) (Shared, bool, error) {
if err := validName(name); err != nil {
return Shared{}, false, err
}
raw, err := os.ReadFile(s.sharedPath(name))
if errors.Is(err, os.ErrNotExist) {
return Shared{}, false, nil
}
if err != nil {
return Shared{}, false, err
}
var sh Shared
if err := json.Unmarshal(raw, &sh); err != nil {
return Shared{}, false, fmt.Errorf("the record of who holds %s is not readable: %w", name, err)
}
return sh, true, nil
}
// WriteShared keeps the record of who holds a provision's one credential.
func (s *Store) WriteShared(sh Shared) error {
if err := validName(sh.Name); err != nil {
return err
}
if err := os.MkdirAll(filepath.Join(s.dir, "shared"), 0o700); err != nil {
return err
}
sort.Strings(sh.Consumers)
raw, err := json.MarshalIndent(sh, "", " ")
if err != nil {
return err
}
return writeWhole(s.sharedPath(sh.Name), append(raw, '\n'))
}
// pairKey is one provider assignment and provision.
type pairKey struct{ provider, provision string }
func (k pairKey) name() string { return "provision/" + k.provider + "/" + k.provision }
// binds is what the plans say of one provider and provision: its module, who binds it, whether a plan names a
// credential per holder for it, and the credential forms the plans name for it ("" for an entry naming none).
type binds struct {
providerModules map[string]bool
consumers map[string]bool
perHolder bool
forms map[string]bool
}
// The credential forms a plan entry names (issue 768), as the provider's offer declares them.
const (
// FormShared is one value every holder shares (ADR 0158): the offer's credential is one of its own secrets.
FormShared = "shared"
// FormPair is a login of its own for each consumer (ADR 0048, ADR 0158 §11): the offer says no credential.
FormPair = "pair"
// FormIssued is a value the provider's software issues (ADR 0305): the mesh never makes it.
FormIssued = "issued"
// FormGiven is a value only a person gives (ADR 0337): the mesh never makes it.
FormGiven = "given"
)
// credentialForms are every form a plan entry may name.
var credentialForms = []string{FormShared, FormPair, FormIssued, FormGiven}
// form is the one form the plans name for k, or why the vault cannot trust one.
func (b *binds) form(k pairKey) (string, string) {
if b == nil || len(b.forms) == 0 || b.forms[""] {
return "", unknownForm(k)
}
var named []string
for f := range b.forms {
if !slices.Contains(credentialForms, f) {
return "", fmt.Sprintf("a plan names %q for %s from %s, which is not a credential form the vault knows "+
"(%s); it makes nothing", f, k.provision, k.provider, strings.Join(credentialForms, ", "))
}
named = append(named, f)
}
if len(named) > 1 {
sort.Strings(named)
return "", fmt.Sprintf("the plans binding %s from %s disagree on its credential form (%s); it makes nothing",
k.provision, k.provider, strings.Join(named, ", "))
}
return named[0], ""
}
// pairRow is one row the move brought in.
type pairRow struct {
consumer, module string
rec Record
}
// bindings is what one pass read: every plan's bound provisions, which assignments there are, and which of them
// are handed to the provisioner.
type bindings struct {
bound map[pairKey]*binds
assigned map[string]bool
handed map[string]bool
}
// plannedProvisions is as much of an assignment as the pass reads.
type plannedProvisions struct {
Node string `json:"node"`
Module string `json:"module"`
Provisioning *struct {
Provisions []struct {
Name string `json:"name"`
Provision string `json:"provision"`
Local string `json:"local"`
Provider string `json:"provider"`
ProviderModule string `json:"provider-module"`
CredentialForm string `json:"credential-form"`
// Contributes is what the consumer tells its provider (issue 776); the vault does not act on it.
Contributes map[string]any `json:"contributes"`
} `json:"provisions"`
} `json:"provisioning"`
}
// readBindings reads every assignment. An assignment that cannot be read stops the pass: a bind missed would read
// as an unbind, and remake a value its holder still uses.
func readBindings(m Mirror) (bindings, error) {
b := bindings{bound: map[pairKey]*binds{}, assigned: map[string]bool{}, handed: map[string]bool{}}
if m.Keys == nil || m.Read == nil {
return b, errors.New("the vault cannot read the assignments, so it cannot see who binds a provider")
}
keys, err := m.Keys()
if err != nil {
return b, fmt.Errorf("the assignments' keys could not be read: %w", err)
}
for _, key := range keys {
raw, err := m.Read(key)
if err != nil {
return b, fmt.Errorf("the assignment %s could not be read: %w", key, err)
}
if raw == nil {
continue
}
var a plannedProvisions
if err := json.Unmarshal(raw, &a); err != nil || a.Node+"."+a.Module != key {
return b, fmt.Errorf("the assignment %s is not the controller's", key)
}
b.assigned[key] = true
if a.Provisioning == nil {
continue
}
b.handed[key] = true
for _, p := range a.Provisioning.Provisions {
if !namePart.MatchString(p.Provider) || !namePart.MatchString(p.Provision) {
continue
}
k := pairKey{p.Provider, p.Provision}
if b.bound[k] == nil {
b.bound[k] = &binds{providerModules: map[string]bool{}, consumers: map[string]bool{},
forms: map[string]bool{}}
}
b.bound[k].forms[p.CredentialForm] = true
if p.ProviderModule != "" {
b.bound[k].providerModules[p.ProviderModule] = true
}
// A name on an entry of the login-per-consumer form is that login's file (issue 779), not a value issued
// for one holder: the provider still gets its logins as rows.
if p.Name != "" && p.CredentialForm != FormPair {
b.bound[k].perHolder = true
}
b.bound[k].consumers[consumerOf(a.Node, a.Module, p.Local)] = true
}
}
return b, nil
}
func consumerOf(node, module, local string) string {
if local == "" {
return node + "/" + module
}
return node + "/" + module + "/" + local
}
// pairRows is every row the move brought in, by provider and provision.
func (v *Vault) pairRows() (map[pairKey][]pairRow, error) {
all, err := v.Store.List()
if err != nil {
return nil, err
}
out := map[pairKey][]pairRow{}
for _, rec := range all {
parts := strings.Split(rec.Name, "/")
if parts[0] != "pair" || (len(parts) != 5 && len(parts) != 6) {
continue
}
k := pairKey{parts[1], parts[2]}
out[k] = append(out[k], pairRow{consumer: parts[3], module: parts[4], rec: rec})
}
return out, nil
}
// PairOutcome is what one pass did, or did not do, for one provider and provision.
type PairOutcome struct {
Name string `json:"name"`
Outcome string `json:"outcome"`
Generation int `json:"generation,omitempty"`
Why string `json:"why,omitempty"`
}
// What a pass did for a provider and provision.
const (
PairMade = "made"
PairRemade = "remade"
PairAdopted = "adopted"
PairRecorded = "recorded"
PairUnchanged = "unchanged"
PairHeld = "held"
PairPerPair = "waits-for-rotation"
PairPerHolder = "per-holder"
)
// sharedHolders is who holds a provider's value now: its provider module and the consumers bound to it.
func sharedHolders(k pairKey, b *binds) (string, []string) {
var modules, consumers []string
if b != nil {
for m := range b.providerModules {
modules = append(modules, m)
}
for c := range b.consumers {
consumers = append(consumers, c)
}
}
sort.Strings(modules)
sort.Strings(consumers)
return strings.Join(modules, ","), consumers
}
// heldBack says why the vault may not make a value for k now, or "": a holder that would never get it.
func heldBack(k pairKey, b *binds, rows []pairRow, seen bindings) string {
if b == nil || len(b.providerModules) == 0 {
return fmt.Sprintf("no plan names the module serving %s on %s, so the vault cannot tell the provider gets "+
"a value it makes", k.provision, k.provider)
}
for m := range b.providerModules {
if !seen.handed[k.provider+"."+m] {
return fmt.Sprintf("the provider %s on %s is not handed to the provisioner, so it would never get a "+
"value the vault makes (ADR 0349 clause 17)", m, k.provider)
}
}
for _, r := range rows {
key := r.consumer + "." + r.module
if seen.assigned[key] && !seen.handed[key] {
return fmt.Sprintf("%s on %s holds %s from %s and is not handed to the provisioner, so it would never "+
"get a value the vault makes (ADR 0349 clause 17)", r.module, r.consumer, k.provision, k.provider)
}
}
return ""
}
// oneStamp is the generation stamp every row shares, with their one origin and value, when they all share one; or
// why not.
func (v *Vault) oneStamp(rows []pairRow) (stamp, origin, value string, madeAt time.Time, why string) {
for i, r := range rows {
if r.rec.MovedFrom == nil || r.rec.MovedFrom.Generation == "" {
return "", "", "", time.Time{}, fmt.Sprintf("%s carries no generation stamp: a login of its own (ADR 0048)",
r.rec.Name)
}
opened, err := seal.Open(r.rec.AtRest, v.Own.PrivateKey)
if err != nil {
return "", "", "", time.Time{}, fmt.Sprintf("%s does not open with the vault's key: %v", r.rec.Name, err)
}
o := r.rec.Origin
if o == "" {
o = OriginMade
}
if i == 0 {
stamp, origin, value, madeAt = r.rec.MovedFrom.Generation, o, opened, r.rec.MadeAt
continue
}
if r.rec.MovedFrom.Generation != stamp || opened != value || o != origin {
return "", "", "", time.Time{}, fmt.Sprintf("the rows of one provider and provision carry different "+
"generations, values or origins (%s among them): logins of their own (ADR 0048)", r.rec.Name)
}
if r.rec.MadeAt.Before(madeAt) {
madeAt = r.rec.MadeAt
}
}
return stamp, origin, value, madeAt, ""
}
// PairPass looks at every provider and provision the plans bind or the move brought in, and makes, remakes or
// adopts each one's shared value as the rules above say. It returns what it did for each; an error is a pass that
// could not read what it needs, and did nothing.
func (v *Vault) PairPass() ([]PairOutcome, error) {
seen, err := readBindings(v.Assignments)
if err != nil {
return nil, err
}
rows, err := v.pairRows()
if err != nil {
return nil, err
}
keys := map[pairKey]bool{}
for k := range seen.bound {
keys[k] = true
}
for k := range rows {
keys[k] = true
}
all, err := v.Store.List()
if err != nil {
return nil, err
}
for _, rec := range all {
parts := strings.Split(rec.Name, "/")
if len(parts) == 3 && parts[0] == "provision" {
if _, recorded, _ := v.Store.ReadShared(rec.Name); recorded {
keys[pairKey{parts[1], parts[2]}] = true
}
}
}
ordered := make([]pairKey, 0, len(keys))
for k := range keys {
ordered = append(ordered, k)
}
sort.Slice(ordered, func(i, j int) bool { return ordered[i].name() < ordered[j].name() })
var out []PairOutcome
var failed []error
for _, k := range ordered {
o, err := v.pairOne(k, seen.bound[k], rows[k], seen)
if err != nil {
failed = append(failed, fmt.Errorf("%s: %w", k.name(), err))
o = PairOutcome{Name: k.name(), Outcome: PairHeld, Why: err.Error()}
}
out = append(out, o)
}
return out, errors.Join(failed...)
}
// pairOne is one provider and provision's part of a pass.
func (v *Vault) pairOne(k pairKey, b *binds, rows []pairRow, seen bindings) (PairOutcome, error) {
name := k.name()
if b != nil && b.perHolder {
return PairOutcome{Name: name, Outcome: PairPerHolder,
Why: "a plan names a credential per holder for it (issue 757): there is no one shared value"}, nil
}
rec, held, err := v.Store.Read(name)
if err != nil {
return PairOutcome{}, err
}
sh, recorded, err := v.Store.ReadShared(name)
if err != nil {
return PairOutcome{}, err
}
// The provider's module, when no plan names it any more (its last consumer unbound), is the one recorded.
if b == nil {
b = &binds{providerModules: map[string]bool{}, consumers: map[string]bool{}}
}
if len(b.providerModules) == 0 && recorded && sh.ProviderModule != "" {
for _, m := range strings.Split(sh.ProviderModule, ",") {
b.providerModules[m] = true
}
}
providerModule, consumers := sharedHolders(k, b)
record := func(generation int, adopted string) error {
return v.Store.WriteShared(Shared{Name: name, Provider: k.provider, Provision: k.provision,
ProviderModule: providerModule, Consumers: consumers, Generation: generation, At: v.Now().UTC(),
Adopted: adopted})
}
switch {
case !held && len(rows) > 0:
stamp, origin, value, madeAt, why := v.oneStamp(rows)
if why != "" {
// A consumer bound since the move, that has no row yet, gets a login of its own; the rows there are
// never touched (issue 771).
if form, formWhy := b.form(k); formWhy == "" && form == FormPair {
have := map[string]bool{}
for _, r := range rows {
have[strings.Join(strings.Split(r.rec.Name, "/")[3:], "/")] = true
}
var missing []string
for _, c := range consumers {
if !have[c] {
missing = append(missing, c)
}
}
if len(missing) > 0 {
if held := heldBack(k, b, rows, seen); held != "" {
return PairOutcome{Name: name, Outcome: PairHeld, Why: held}, nil
}
return v.firstLogins(k, missing)
}
}
return PairOutcome{Name: name, Outcome: PairPerPair, Why: why + "; each consumer keeps a value of its " +
"own for ever (ADR 0048, ADR 0158 §11), and a rotation remakes each one's own, never one shared value"}, nil
}
// Adopted as it is: the holders the rows name hold it, and a bind since then is a remake on the next pass.
changed, err := v.keepWith(name, value, "the vault", "", origin, keepOptions{neverReplace: true, madeAt: madeAt,
movedFrom: &MovedFrom{By: "the move's pair rows", Generation: stamp}})
value = ""
if err != nil {
return PairOutcome{}, err
}
var fromRows []string
for _, r := range rows {
parts := strings.Split(r.rec.Name, "/")
fromRows = append(fromRows, strings.Join(parts[3:], "/"))
}
consumers = fromRows
if err := record(changed.Generation, stamp); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: name, Outcome: PairAdopted, Generation: changed.Generation}, nil
case !held:
if len(consumers) == 0 {
return PairOutcome{Name: name, Outcome: PairUnchanged}, nil
}
if why := heldBack(k, b, rows, seen); why != "" {
return PairOutcome{Name: name, Outcome: PairHeld, Why: why}, nil
}
form, why := b.form(k)
switch {
case why != "":
return PairOutcome{Name: name, Outcome: PairHeld, Why: why}, nil
case form == FormShared:
changed, err := v.mintShared(name)
if err != nil {
return PairOutcome{}, err
}
if err := record(changed.Generation, ""); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: name, Outcome: PairMade, Generation: changed.Generation}, nil
case form == FormPair:
return v.firstLogins(k, consumers)
case form == FormIssued:
return PairOutcome{Name: name, Outcome: PairHeld, Why: "its credential is issued by the provider's " +
"software (ADR 0305), which the vault never makes; `secret issued` brings it in"}, nil
}
return PairOutcome{Name: name, Outcome: PairHeld, Why: "its credential is given by a person (ADR 0337), " +
"which the vault never makes; `secret accept` brings it in"}, nil
case rec.Origin == OriginAccepted || rec.Origin == OriginIssued:
// Never remade: who holds it is recorded, so each gets it on request.
if recorded && slices.Equal(sh.Consumers, consumers) && sh.ProviderModule == providerModule {
return PairOutcome{Name: name, Outcome: PairUnchanged, Generation: rec.Generation}, nil
}
if err := record(rec.Generation, sh.Adopted); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: name, Outcome: PairRecorded, Generation: rec.Generation,
Why: "a value " + rec.Origin + " is not remade (ADR 0158 §8, ADR 0305 §9); its holders are recorded"}, nil
case !recorded:
// Held but never recorded (made by a rotation before any plan bound it): who binds it now holds it.
if err := record(rec.Generation, ""); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: name, Outcome: PairRecorded, Generation: rec.Generation}, nil
case slices.Equal(sh.Consumers, consumers):
return PairOutcome{Name: name, Outcome: PairUnchanged, Generation: rec.Generation}, nil
}
if why := heldBack(k, b, rows, seen); why != "" {
return PairOutcome{Name: name, Outcome: PairHeld, Generation: rec.Generation, Why: why}, nil
}
changed, err := v.mintShared(name)
if err != nil {
return PairOutcome{}, err
}
if err := record(changed.Generation, ""); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: name, Outcome: PairRemade, Generation: changed.Generation}, nil
}
// mintShared makes the provision's one credential anew, for all its holders in one act: a new generation and
// `secret-changed`, which tells every holder to ask again.
func (v *Vault) mintShared(name string) (Changed, error) {
raw := make([]byte, 30)
if _, err := rand.Read(raw); err != nil {
return Changed{}, err
}
return v.keep(name, base64.RawURLEncoding.EncodeToString(raw), "the vault", "", OriginMade)
}
// firstLogins makes each bound consumer of a per-login provider the vault holds no row of its first login of its
// own (ADR 0048, ADR 0158 §11, issue 768): `pair/<provider>/<provision>/<consumer>/<module>[/<local>]`, a value
// apart for each, never one shared value. Once made they are rows, and every later pass keeps them per pair.
func (v *Vault) firstLogins(k pairKey, consumers []string) (PairOutcome, error) {
var failed []error
for _, c := range consumers {
parts := strings.SplitN(c, "/", 3)
local := ""
if len(parts) == 3 {
local = parts[2]
}
row, err := PairSecret(k.provider, k.provision, parts[0], parts[1], local)
if err != nil {
failed = append(failed, err)
continue
}
raw := make([]byte, 30)
if _, err := rand.Read(raw); err != nil {
return PairOutcome{}, err
}
if _, err := v.keepWith(row, base64.RawURLEncoding.EncodeToString(raw), "the vault", "", OriginMade,
keepOptions{neverReplace: true}); err != nil {
failed = append(failed, fmt.Errorf("%s: %w", row, err))
}
}
if err := errors.Join(failed...); err != nil {
return PairOutcome{}, err
}
return PairOutcome{Name: k.name(), Outcome: PairMade, Why: fmt.Sprintf("a first login of its own for each of "+
"%d consumers (ADR 0048, ADR 0158 §11)", len(consumers))}, nil
}
// unknownForm is why the vault makes nothing for a provider and provision it has no row of while no plan names its
// offer's credential form: whether the offer shares one credential (ADR 0158) or keeps a login per consumer (ADR
// 0048) is the offer's, and a plan from a controller before issue 768 does not say which.
func unknownForm(k pairKey) string {
return fmt.Sprintf("the vault holds no row of %s from %s, and no plan says whether its offer shares one credential "+
"(ADR 0158) or keeps a login per consumer (ADR 0048, §11); it makes nothing until it knows", k.provision,
k.provider)
}
// rotatePerPair is the rotation of a provider whose consumers each keep a login of their own (ADR 0048, ADR 0158
// §11): every consumer's own value is remade apart, each a value of its own, and no shared value is made. A value
// accepted or issued is left as it is. Refused whole while any holder would never get its new value. Narrowed to one
// consumer, only that consumer's own logins are remade (issue 767), and a consumer that holds none is refused.
func (v *Vault) rotatePerPair(name string, one Narrow, by string) (Changed, error) {
parts := strings.Split(name, "/")
k := pairKey{parts[1], parts[2]}
rows, err := v.pairRows()
if err != nil {
return Changed{}, err
}
if len(rows[k]) == 0 {
return Changed{}, fmt.Errorf("not rotated: the vault holds no secret named %s, and %s", name, unknownForm(k))
}
if _, _, _, _, why := v.oneStamp(rows[k]); why == "" {
return Changed{}, fmt.Errorf("not rotated: the rows of %s share one value not adopted yet; the next pass "+
"adopts it, then rotate it", name)
}
if why := v.rotationHeldBack(name); why != "" {
return Changed{}, fmt.Errorf("not rotated: %s", why)
}
if !slices.ContainsFunc(rows[k], one.holds) {
return Changed{}, fmt.Errorf("not rotated: %s holds no login of %s from %s", one, k.provision, k.provider)
}
out := Changed{Name: name, At: v.Now().UTC(), By: by}
var failed []error
for _, r := range rows[k] {
if !one.holds(r) {
continue
}
if r.rec.Origin == OriginAccepted || r.rec.Origin == OriginIssued {
continue
}
raw := make([]byte, 30)
if _, err := rand.Read(raw); err != nil {
return out, err
}
changed, err := v.keep(r.rec.Name, base64.RawURLEncoding.EncodeToString(raw), by, ControllerCaller, OriginMade)
if err != nil {
failed = append(failed, fmt.Errorf("%s: %w", r.rec.Name, err))
continue
}
out.Each = append(out.Each, changed)
}
return out, errors.Join(failed...)
}
// rotationHeldBack says why a rotation of a provision's one credential would leave a holder behind, or "".
func (v *Vault) rotationHeldBack(name string) string {
parts := strings.Split(name, "/")
if len(parts) != 3 || parts[0] != "provision" || v.Assignments.Keys == nil {
return ""
}
k := pairKey{parts[1], parts[2]}
seen, err := readBindings(v.Assignments)
if err != nil {
return err.Error()
}
rows, err := v.pairRows()
if err != nil {
return err.Error()
}
return heldBack(k, seen.bound[k], rows[k], seen)
}
// sharedRefusal says why caller (`provisioner.<node>`) may not have the provision's one credential name, or "": it
// is handed to the machines the vault made it for, the provider's and each bound consumer's, while the provider's
// assignment there is handed to the provisioner, or a consumer's plan there still names it from that provider.
func sharedRefusal(store *Store, read AssignmentReader, node, provider, provision string) string {
name := "provision/" + provider + "/" + provision
if store == nil {
return "the vault keeps no record of who holds " + name
}
sh, ok, err := store.ReadShared(name)
if err != nil {
return err.Error()
}
if !ok {
return fmt.Sprintf("the vault made %s for nobody yet: it is handed to the machines its provider and its "+
"consumers run on once their plans bind it (ADR 0349 clause 8)", name)
}
if !slices.Contains(sh.holderNodes(), node) {
return fmt.Sprintf("%s is held by %s, not by a module on %s", name, strings.Join(sh.holderNodes(), ", "), node)
}
if read == nil {
return "the vault cannot read the assignments, so it cannot tell what " + node + "'s plans name"
}
if node == provider && sh.ProviderModule != "" {
for _, m := range strings.Split(sh.ProviderModule, ",") {
if raw, err := read(node + "." + m); err == nil && raw != nil {
var a plannedProvisions
if json.Unmarshal(raw, &a) == nil && a.Node == node && a.Module == m && a.Provisioning != nil {
return ""
}
}
}
}
for _, c := range sh.Consumers {
parts := strings.SplitN(c, "/", 3)
if parts[0] != node {
continue
}
raw, err := read(node + "." + parts[1])
if err != nil || raw == nil {
continue
}
var a plannedProvisions
if json.Unmarshal(raw, &a) != nil || a.Node != node || a.Module != parts[1] || a.Provisioning == nil {
continue
}
for _, p := range a.Provisioning.Provisions {
if p.Name == "" && p.Provision == provision && p.Provider == provider {
return ""
}
}
}
return fmt.Sprintf("no module on %s that the vault made %s for is handed to the provisioner and bound to it now",
node, name)
}
// pairPass is one minute's PairPass, saying each outcome when it differs from the last pass's, so a provider held
// back is said once and again when it changes, not every minute.
func (e *Env) pairPass() {
if e.Vault == nil {
return
}
outcomes, err := e.Vault.PairPass()
e.mu.Lock()
defer e.mu.Unlock()
if err != nil && err.Error() != e.lastPairFailure {
say("the pass over the providers' shared credentials did not finish: %v", err)
}
e.lastPairFailure = ""
if err != nil {
e.lastPairFailure = err.Error()
}
if e.pairsSaid == nil {
e.pairsSaid = map[string]string{}
}
for _, o := range outcomes {
said := o.Outcome + ": " + o.Why
if o.Outcome == PairUnchanged || e.pairsSaid[o.Name] == said {
continue
}
e.pairsSaid[o.Name] = said
if o.Why == "" {
say("%s %s (generation %d)", o.Name, o.Outcome, o.Generation)
} else {
say("%s %s: %s", o.Name, o.Outcome, o.Why)
}
}
}
// pairRowRefusal says why caller's machine node may not have the pair row name (`pair/<provider>/<provision>/
// <consumer>/<module>[/<local>]`, a consumer's own login, ADR 0048), or "": it is handed to that consumer's machine
// while the consumer's plan there names the provision from that provider under that local name, and to the
// provider's machine while the provider module that plan names is handed to the provisioner there. Another
// consumer, on the same machine or not, is refused: each gets only its own.
func pairRowRefusal(read AssignmentReader, node string, parts []string) string {
provider, provision, consumer, module, local := parts[1], parts[2], parts[3], parts[4], ""
if len(parts) == 6 {
local = parts[5]
}
if node != consumer && node != provider {
return fmt.Sprintf("%s is the login of %s on %s from %s, and %s is neither", strings.Join(parts, "/"),
module, consumer, provider, node)
}
if read == nil {
return "the vault cannot read the assignments, so it cannot tell what " + node + "'s plans name"
}
raw, err := read(consumer + "." + module)
if err != nil || raw == nil {
return fmt.Sprintf("%s is not on %s", module, consumer)
}
var a plannedProvisions
if json.Unmarshal(raw, &a) != nil || a.Node != consumer || a.Module != module {
return fmt.Sprintf("the assignment %s.%s is not the controller's", consumer, module)
}
if a.Provisioning == nil {
return fmt.Sprintf("%s on %s is not handed to the provisioner, so its login reaches it in its declaration "+
"(ADR 0349 clause 17)", module, consumer)
}
for _, p := range a.Provisioning.Provisions {
// An unnamed entry, or a named one of the same form (the consumer's login file, issue 779): one rule.
if (p.Name != "" && p.CredentialForm != FormPair) || p.Provision != provision ||
p.Provider != provider || p.Local != local {
continue
}
if node == consumer {
return ""
}
if p.ProviderModule != "" {
if praw, err := read(provider + "." + p.ProviderModule); err == nil && praw != nil {
var pa plannedProvisions
if json.Unmarshal(praw, &pa) == nil && pa.Node == provider && pa.Provisioning != nil {
return ""
}
}
}
return fmt.Sprintf("the provider of %s on %s is not handed to the provisioner", provision, provider)
}
return fmt.Sprintf("%s's plan on %s does not name %s from %s under that name", module, consumer, provision, provider)
}
modules/mesh-vault/cmd/mesh-vault/pair_test.go
+487
View File
@@ -0,0 +1,487 @@
package main
// ADR 0349's provider test, the vault's part (design 57 V5, issue 677), with a stand-in provider: a consumer binding
// to it remakes its one credential without the provider asking, and a consumer unbinding remakes it for the holders
// left; every holder gets it on request after `secret-changed`; an accepted or issued value is not remade; what the
// move brought in is adopted when its rows share a generation stamp, and otherwise waits for the provider's
// rotation; and no value is made that a holder still filled the old way would never get.
import (
"errors"
"sort"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// keys is every key of a stand-in mirror.
func (a assignments) keys() ([]string, error) {
var out []string
for k := range a {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
// handedWith is an assignment handed to the provisioner, its plan naming each provision given as
// provision@provider/provider-module[#local][=name][+credential-form].
func handedWith(node, module string, provisions ...string) string {
var entries []string
for _, p := range provisions {
form := ""
if i := strings.Index(p, "+"); i >= 0 {
p, form = p[:i], p[i+1:]
}
name := ""
if i := strings.Index(p, "="); i >= 0 {
p, name = p[:i], p[i+1:]
}
local := ""
if i := strings.Index(p, "#"); i >= 0 {
p, local = p[:i], p[i+1:]
}
provision, rest, _ := strings.Cut(p, "@")
provider, providerModule, _ := strings.Cut(rest, "/")
e := `{"provision":"` + provision + `","provider":"` + provider + `","provider-module":"` + providerModule + `"`
if local != "" {
e += `,"local":"` + local + `"`
}
if name != "" {
e += `,"name":"` + name + `","path":"/s/` + name + `"`
}
if form != "" {
e += `,"credential-form":"` + form + `"`
}
entries = append(entries, e+"}")
}
return `{"node":"` + node + `","module":"` + module + `","provisioning":{"templates":{"source":"x","digest":"y"},` +
`"files":[],"provisions":[` + strings.Join(entries, ",") + `]}}`
}
// unmoved is an assignment the controller has not handed to the provisioner.
func unmoved(node, module string) string {
return `{"node":"` + node + `","module":"` + module + `"}`
}
const standIn = "provision/anchor/stand-in-api"
// pairVault is a vault reading mirror for its binds and its rule.
func pairVault(t *testing.T, mirror assignments) testVault {
t.Helper()
v := newVault(t, nil)
v.Askers = Declared{Read: mirror.read, Store: v.Store}
v.Assignments = Mirror{Keys: mirror.keys, Read: mirror.read}
return v
}
// pass runs one pass and is the outcome for name.
func pass(t *testing.T, v testVault, name string) PairOutcome {
t.Helper()
outcomes, err := v.PairPass()
if err != nil {
t.Fatal(err)
}
for _, o := range outcomes {
if o.Name == name {
return o
}
}
t.Fatalf("the pass said nothing of %s: %+v", name, outcomes)
return PairOutcome{}
}
// asks is what caller is handed for name, opened, or the refusal.
func asks(v testVault, caller, name string) (string, int, error) {
key, _ := seal.GenerateKeyPair()
a, err := v.Get(caller, name, key.PublicKey)
if err != nil {
return "", 0, err
}
got, err := seal.Open(a.Sealed, key.PrivateKey)
return got, a.Generation, err
}
// changedSaid is how many times `secret-changed` was said for name, and that none carried a value.
func changedSaid(t *testing.T, v testVault, name, value string) int {
t.Helper()
v.bus.mu.Lock()
defer v.bus.mu.Unlock()
n := 0
for i, e := range v.bus.events {
if value != "" && strings.Contains(v.bus.raw[i], value) {
t.Fatalf("the bus carried the value: %s", v.bus.raw[i])
}
if e["event"] == ChangedEvent && e["body"].(map[string]any)["name"] == name {
n++
}
}
return n
}
func TestAConsumerBindingOrUnbindingRemakesTheProvidersOneCredentialForEveryHolder(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "shared-from-the-move", "stamp-1", OriginMade)
// The move's shared value is adopted, and said to have changed in the vault.
if o := pass(t, v, standIn); o.Outcome != PairAdopted || o.Generation != 1 {
t.Fatalf("the first pass: %+v", o)
}
first, _, err := asks(v, "provisioner.anchor", standIn)
if err != nil {
t.Fatalf("the provider's machine was refused its credential: %v", err)
}
if got, _, err := asks(v, "provisioner.g14", standIn); err != nil || got != first {
t.Fatalf("the consumer got %q (%v), the provider %q", got, err, first)
}
if first != "shared-from-the-move" || changedSaid(t, v, standIn, "") != 1 {
t.Fatal("the adopted value is not the move's, or secret-changed was not said once for it")
}
if _, _, err := asks(v, "provisioner.ace", standIn); err == nil {
t.Fatal("a machine holding nothing of it was handed the credential")
}
// Nothing changed: nothing is remade.
if o := pass(t, v, standIn); o.Outcome != PairUnchanged {
t.Fatalf("a pass with no bind or unbind: %+v", o)
}
// A second consumer binds: the vault remakes it without the provider asking, one new value for all three.
mirror["ace.other"] = handedWith("ace", "other", "stand-in-api@anchor/stand-in")
if o := pass(t, v, standIn); o.Outcome != PairRemade || o.Generation != 2 {
t.Fatalf("a consumer binding: %+v", o)
}
second, gen, _ := asks(v, "provisioner.anchor", standIn)
if second == first || gen != 2 {
t.Fatal("the value was not remade when a consumer bound")
}
for _, c := range []string{"provisioner.g14", "provisioner.ace"} {
if got, _, err := asks(v, c, standIn); err != nil || got != second {
t.Fatalf("%s got %q (%v) after the bind, not the one new value", c, got, err)
}
}
if changedSaid(t, v, standIn, "") != 2 {
t.Fatal("secret-changed was not said for the remade value")
}
// It unbinds (its plan no longer names it): remade for the holders left, and it is handed nothing more.
mirror["ace.other"] = handedWith("ace", "other")
if o := pass(t, v, standIn); o.Outcome != PairRemade || o.Generation != 3 {
t.Fatalf("a consumer unbinding: %+v", o)
}
third, _, _ := asks(v, "provisioner.anchor", standIn)
if got, _, _ := asks(v, "provisioner.g14", standIn); third == second || got != third {
t.Fatal("the holders left do not share one new value after the unbind")
}
if _, _, err := asks(v, "provisioner.ace", standIn); err == nil {
t.Fatal("a consumer that unbound was still handed the credential")
}
// Its assignment goes: an unbind too.
delete(mirror, "g14.app")
if o := pass(t, v, standIn); o.Outcome != PairRemade {
t.Fatalf("a consumer taken off its machine: %+v", o)
}
if _, _, err := asks(v, "provisioner.g14", standIn); err == nil {
t.Fatal("a machine whose consumer left was still handed the credential")
}
}
// Two local names of one provision on one consumer are two binds of the same value; a credential per holder (issue
// 757) has no one shared value.
func TestLocalNamesAreBindsAndAPerHolderCredentialIsNotShared(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in#reader"),
"novox.gitea": handedWith("novox", "gitea"),
"g14.claude-code": handedWith("g14", "claude-code", "gitea-read@novox/gitea=gitea-read"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "shared", "stamp-1", OriginMade, "reader")
pass(t, v, standIn)
mirror["g14.app"] = handedWith("g14", "app", "stand-in-api@anchor/stand-in#reader",
"stand-in-api@anchor/stand-in#writer")
if o := pass(t, v, standIn); o.Outcome != PairRemade {
t.Fatalf("a second local name binding: %+v", o)
}
if o := pass(t, v, "provision/novox/gitea-read"); o.Outcome != PairPerHolder {
t.Fatalf("a credential per holder: %+v", o)
}
if _, held, _ := v.Store.Read("provision/novox/gitea-read"); held {
t.Fatal("a shared value was made for a credential issued per holder")
}
}
// An accepted value is opened by the vault alone and not remade (ADR 0158 §8), nor is one the provider's software
// issued (ADR 0305 §9): a bind records its holder, who gets the value as it is.
func TestAnAcceptedOrIssuedValueIsNotRemadeOnABind(t *testing.T) {
for _, origin := range []string{OriginAccepted, OriginIssued} {
t.Run(origin, func(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
if _, err := v.keep(standIn, "given-by-a-person", "person.jochen", ControllerCaller, origin); err != nil {
t.Fatal(err)
}
if o := pass(t, v, standIn); o.Outcome != PairRecorded || o.Generation != 1 {
t.Fatalf("the first pass: %+v", o)
}
mirror["ace.other"] = handedWith("ace", "other", "stand-in-api@anchor/stand-in")
if o := pass(t, v, standIn); o.Outcome != PairRecorded || o.Generation != 1 {
t.Fatalf("a bind to a value %s: %+v", origin, o)
}
for _, c := range []string{"provisioner.anchor", "provisioner.g14", "provisioner.ace"} {
if got, gen, err := asks(v, c, standIn); err != nil || got != "given-by-a-person" || gen != 1 {
t.Fatalf("%s got %q generation %d (%v)", c, got, gen, err)
}
}
})
}
}
// movedRow is a pair row as the move (V3) keeps it: its value, the controller's generation stamp and origin.
func movedRow(t *testing.T, v testVault, consumer, module, value, stamp, origin string, local ...string) {
t.Helper()
name, err := PairSecret("anchor", "stand-in-api", consumer, module, strings.Join(local, ""))
if err != nil {
t.Fatal(err)
}
if _, err := v.keepWith(name, value, "node."+consumer, "", origin, keepOptions{neverReplace: true,
madeAt: time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC),
movedFrom: &MovedFrom{By: "node." + consumer, Generation: stamp}}); err != nil {
t.Fatal(err)
}
}
// Rows a provider shares carry one generation stamp (ADR 0158): the vault adopts their value as the provision's one
// credential, keeps its origin and when it was made, and remakes nothing, so the move rotates no credential.
func TestPairRowsSharingAGenerationStampAreAdoptedAndNothingIsRemade(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "the-value-they-share", "20260901T000000-1", OriginMade)
movedRow(t, v, "ace", "other", "the-value-they-share", "20260901T000000-1", OriginMade)
if o := pass(t, v, standIn); o.Outcome != PairAdopted || o.Generation != 1 {
t.Fatalf("stamped rows: %+v", o)
}
for _, c := range []string{"provisioner.anchor", "provisioner.g14", "provisioner.ace"} {
if got, _, err := asks(v, c, standIn); err != nil || got != "the-value-they-share" {
t.Fatalf("%s got %q (%v), not the value it had", c, got, err)
}
}
rec, _, _ := v.Store.Read(standIn)
if rec.Origin != OriginMade || !rec.MadeAt.Equal(time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)) ||
rec.MovedFrom == nil || rec.MovedFrom.Generation != "20260901T000000-1" {
t.Fatalf("the adopted value does not say where it came from: %+v", rec)
}
if o := pass(t, v, standIn); o.Outcome != PairUnchanged {
t.Fatalf("the pass after the adoption: %+v", o)
}
if got, gen, _ := asks(v, "provisioner.g14", standIn); got != "the-value-they-share" || gen != 1 {
t.Fatal("the adopted value was remade with no bind or unbind")
}
}
// Rows of a provider with a login per consumer (ADR 0048) carry no stamp: each consumer keeps a value of its own
// for ever (ADR 0158 §11, issue 677's note 12). A bind makes no shared value, and nor does a rotation.
func TestUnstampedPairRowsAreNeverMadeOneSharedValue(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "login-of-app", "", OriginMade)
movedRow(t, v, "ace", "other", "login-of-other", "", OriginMade)
for i := 0; i < 2; i++ {
if o := pass(t, v, standIn); o.Outcome != PairPerPair {
t.Fatalf("unstamped rows, pass %d: %+v", i, o)
}
}
mirror["novox.third"] = handedWith("novox", "third", "stand-in-api@anchor/stand-in")
if o := pass(t, v, standIn); o.Outcome != PairPerPair {
t.Fatalf("a bind to a provider with logins of their own: %+v", o)
}
if _, err := v.Rotate(ControllerCaller, standIn, "person.jochen", "its ordinary rotation"); err != nil {
t.Fatal(err)
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatal("a shared value was made for a provider whose consumers keep logins of their own")
}
if changedSaid(t, v, standIn, "") != 0 {
t.Fatal("secret-changed was said for a shared value")
}
}
// Two consumers of postgres-database (a login each, ADR 0048) hold different values after a rotation, each remade
// apart, and each consumer's machine is handed only its own; the provider's machine is handed both.
func TestARotationRemakesEachPostgresConsumersOwnLoginApart(t *testing.T) {
mirror := assignments{
"novox.postgres": handedWith("novox", "postgres"),
"g14.gitea": handedWith("g14", "gitea", "postgres-database@novox/postgres"),
"ace.umami": handedWith("ace", "umami", "postgres-database@novox/postgres"),
}
v := pairVault(t, mirror)
gitea, umami := "pair/novox/postgres-database/g14/gitea", "pair/novox/postgres-database/ace/umami"
for name, value := range map[string]string{gitea: "login-of-gitea", umami: "login-of-umami"} {
if _, err := v.keepWith(name, value, "node.novox", "", OriginMade, keepOptions{neverReplace: true,
movedFrom: &MovedFrom{By: "node.novox"}}); err != nil {
t.Fatal(err)
}
}
changed, err := v.Rotate(ControllerCaller, "provision/novox/postgres-database", "person.jochen", "yearly")
if err != nil || len(changed.Each) != 2 {
t.Fatalf("the rotation: %+v (%v)", changed, err)
}
ofGitea, _, err := asks(v, "provisioner.g14", gitea)
if err != nil {
t.Fatalf("gitea's machine was refused its own login: %v", err)
}
ofUmami, _, err := asks(v, "provisioner.ace", umami)
if err != nil {
t.Fatalf("umami's machine was refused its own login: %v", err)
}
if ofGitea == ofUmami || ofGitea == "login-of-gitea" || ofUmami == "login-of-umami" {
t.Fatalf("after the rotation gitea holds %q and umami %q: not two new values of their own", ofGitea, ofUmami)
}
if _, _, err := asks(v, "provisioner.g14", umami); err == nil {
t.Fatal("gitea's machine was handed umami's login")
}
if _, _, err := asks(v, "provisioner.ace", gitea); err == nil {
t.Fatal("umami's machine was handed gitea's login")
}
for name, want := range map[string]string{gitea: ofGitea, umami: ofUmami} {
if got, _, err := asks(v, "provisioner.novox", name); err != nil || got != want {
t.Fatalf("the provider's machine got %q for %s (%v)", got, name, err)
}
}
if _, held, _ := v.Store.Read("provision/novox/postgres-database"); held {
t.Fatal("a shared value was made for postgres-database")
}
}
// A provider and provision the vault holds no row of: whether its offer shares one credential or keeps a login per
// consumer is not said where the vault reads, so it makes nothing rather than guess.
func TestAProviderTheVaultHoldsNoRowOfIsNotGuessed(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
if o := pass(t, v, standIn); o.Outcome != PairHeld || !strings.Contains(o.Why, "no plan says whether") {
t.Fatalf("a provider with no row: %+v", o)
}
if _, err := v.Rotate(ControllerCaller, standIn, "person.jochen", "why not"); err == nil {
t.Fatal("a rotation guessed the form of a provider with no row")
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatal("a value was made for a provider whose form the vault does not know")
}
}
// The vault makes no value a holder would never get (the lead's rule on issue 677): a provider not handed to the
// provisioner, or a consumer the move brought in that is still filled the old way, holds every make and remake
// back, and a rotation too.
func TestNoValueIsMadeThatAHolderFilledTheOldWayWouldNeverGet(t *testing.T) {
t.Run("an unmoved provider", func(t *testing.T) {
mirror := assignments{
"anchor.stand-in": unmoved("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
if o := pass(t, v, standIn); o.Outcome != PairHeld || !strings.Contains(o.Why, "stand-in on anchor") {
t.Fatalf("an unmoved provider: %+v", o)
}
if _, held, _ := v.Store.Read(standIn); held {
t.Fatal("a value was made that the provider would never get")
}
if _, err := v.Rotate(ControllerCaller, standIn, "person.jochen", "why not"); err == nil {
t.Fatal("a rotation made a value the provider would never get")
}
})
t.Run("an unmoved consumer", func(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
"ace.old": unmoved("ace", "old"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "shared", "stamp-1", OriginMade)
movedRow(t, v, "ace", "old", "shared", "stamp-1", OriginMade)
if o := pass(t, v, standIn); o.Outcome != PairAdopted {
t.Fatalf("adopting changes no value, so it is not held back: %+v", o)
}
mirror["novox.new"] = handedWith("novox", "new", "stand-in-api@anchor/stand-in")
o := pass(t, v, standIn)
if o.Outcome != PairHeld || !strings.Contains(o.Why, "old on ace") {
t.Fatalf("a bind while a consumer is filled the old way: %+v", o)
}
if got, gen, _ := asks(v, "provisioner.g14", standIn); got != "shared" || gen != 1 {
t.Fatal("the value was remade while a consumer filled the old way holds it")
}
if _, err := v.Rotate(ControllerCaller, standIn, "person.jochen", "why not"); err == nil ||
!strings.Contains(err.Error(), "old on ace") {
t.Fatalf("a rotation while a consumer is filled the old way: %v", err)
}
// Once it moves, the remake the bind asked for follows.
mirror["ace.old"] = handedWith("ace", "old", "stand-in-api@anchor/stand-in")
if o := pass(t, v, standIn); o.Outcome != PairRemade {
t.Fatalf("after the consumer moved: %+v", o)
}
})
}
// A pass that cannot read every assignment makes nothing: a bind it missed would read as an unbind.
func TestAPassThatCannotReadTheAssignmentsMakesNothing(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "shared", "stamp-1", OriginMade)
pass(t, v, standIn)
mirror["g14.broken"] = "not json"
if _, err := v.PairPass(); err == nil {
t.Fatal("a pass over an unreadable assignment said nothing")
}
v.Assignments.Keys = func() ([]string, error) { return nil, errors.New("the bus is away") }
if _, err := v.PairPass(); err == nil {
t.Fatal("a pass that could not list the assignments said nothing")
}
if _, gen, _ := asks(v, "provisioner.g14", standIn); gen != 1 {
t.Fatal("a pass that could not read the assignments remade the value")
}
}
// A module only pulled in by another has a value of its own in the mirror (issue 765), naming what requires it: its
// plan binds like an assigned module's.
func TestAPulledInConsumerBindsLikeAnAssignedOne(t *testing.T) {
mirror := assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
}
v := pairVault(t, mirror)
movedRow(t, v, "g14", "app", "shared", "stamp-1", OriginMade)
pass(t, v, standIn)
mirror["ace.helper"] = strings.Replace(handedWith("ace", "helper", "stand-in-api@anchor/stand-in"),
`"module":"helper",`, `"module":"helper","required-by":"desk",`, 1)
if o := pass(t, v, standIn); o.Outcome != PairRemade {
t.Fatalf("a pulled-in consumer binding: %+v", o)
}
if _, _, err := asks(v, "provisioner.ace", standIn); err != nil {
t.Fatalf("the pulled-in consumer's machine was refused: %v", err)
}
}
modules/mesh-vault/cmd/mesh-vault/perholder.go
+201
View File
@@ -0,0 +1,201 @@
package main
// A value a provider's software issued for each holder apart (design 57's V2c, issue 749; ADR 0349 clauses 10 and
// 11). The forge mints one read token per agent account (ADR 0365 clause 4): one value per provision, sealed to
// every holder alike, cannot carry that. So an issuing provider's tool may answer one value per holder — a holder
// being a consuming module on one machine — and the vault keeps each as that holder's own and hands it to that
// holder alone. A tool that answers one value keeps the provision's one credential, as before.
//
// Names in the store: `provision/<provider node>/<provision>/<holder node>/<holder module>`.
import (
"errors"
"fmt"
"strings"
)
// Holder is who one issued value is for: a consuming module on one machine.
type Holder struct {
Node string `json:"node"`
Module string `json:"module"`
}
func (h Holder) String() string { return h.Module + " on " + h.Node }
// HolderSecret is the vault's name for the value a provider's software issued for one holder of a provision.
func HolderSecret(provider, provision string, h Holder) (string, error) {
base, err := ProvisionSecret(provider, provision)
if err != nil {
return "", err
}
if err := h.valid(); err != nil {
return "", err
}
return base + "/" + h.Node + "/" + h.Module, nil
}
func (h Holder) valid() error {
if !namePart.MatchString(h.Node) {
return fmt.Errorf("the holder's machine %q is not a name the mesh writes", h.Node)
}
if !namePart.MatchString(h.Module) {
return fmt.Errorf("the holder's module %q is not a name the mesh writes", h.Module)
}
return nil
}
// holderOf is the holder a secret's name is for, when it is one holder's issued value.
func holderOf(name string) (Holder, bool) {
parts := strings.Split(name, "/")
if len(parts) != 5 || parts[0] != "provision" {
return Holder{}, false
}
return Holder{Node: parts[3], Module: parts[4]}, true
}
// isTheHolder is whether caller, the bus user the runtime proved, is the holder: its machine's provisioner client
// (`provisioner.<node>`, an account of its own, mesh-controller internal/broker) or the module itself under an
// account of its own (`<node>.<module>`, ADR 0259 §8's runs-as). Any other caller is not, whatever else allows it.
func isTheHolder(caller string, h Holder) bool {
return caller == "provisioner."+h.Node || caller == h.Node+"."+h.Module
}
// notTheHolder says why caller may not have a holder's value, or "".
func notTheHolder(caller, name string) string {
h, per := holderOf(name)
if !per || isTheHolder(caller, h) {
return ""
}
return fmt.Sprintf("%s was issued for %s alone, and is handed only to that holder: provisioner.%s or %s.%s",
name, h, h.Node, h.Node, h.Module)
}
// checkHolders refuses a list of holders that is not names the mesh writes, each once.
func checkHolders(holders []Holder) error {
seen := map[Holder]bool{}
for _, h := range holders {
if err := h.valid(); err != nil {
return err
}
if seen[h] {
return fmt.Errorf("%s is named twice", h)
}
seen[h] = true
}
return nil
}
// IssuedHolder is what the vault kept for one holder. Never the value.
type IssuedHolder struct {
Holder
Secret string `json:"secret"`
Generation int `json:"generation"`
Unchanged bool `json:"unchanged,omitempty"`
}
// sealedFor is one holder's value as the tool answers it: sealed to the call's key.
type sealedFor struct {
Node string `json:"node"`
Module string `json:"module"`
Sealed string `json:"sealed"`
}
// takePerHolder opens every holder's value the tool sealed to the call's key and keeps each as that holder's own.
// It takes the answer whole or not at all: a holder not asked about, a holder twice, a value not sealed to the call
// or an empty one keeps nothing for anybody. A holder asked about and not answered is said, and keeps nothing.
func (v *Vault) takePerHolder(caller, call, provider, provision string, asked []Holder, answered []sealedFor) (Issued, error) {
c, err := v.takeCall("issued", call)
if err != nil {
return Issued{}, err
}
defer func() { c.private = [32]byte{} }()
if len(asked) == 0 {
return Issued{}, fmt.Errorf("%s answered a value per holder, and no holder was named to it; nothing was taken", c.by)
}
want := map[Holder]bool{}
for _, h := range asked {
want[h] = true
}
type opened struct {
holder Holder
secret string
value string
}
var all []opened
got := map[Holder]bool{}
for _, a := range answered {
h := Holder{Node: a.Node, Module: a.Module}
switch {
case !want[h]:
return Issued{}, fmt.Errorf("%s answered a value for %s, which it was not asked about; nothing was taken", c.by, h)
case got[h]:
return Issued{}, fmt.Errorf("%s answered %s twice; nothing was taken", c.by, h)
}
got[h] = true
raw, err := c.openOne(a.Sealed)
if err != nil {
return Issued{}, fmt.Errorf("%s's value: %w", h, err)
}
value := strings.TrimSpace(string(raw))
zero(raw)
if value == "" {
return Issued{}, fmt.Errorf("%s answered an empty value for %s; nothing was taken", c.by, h)
}
secret, err := HolderSecret(provider, provision, h)
if err != nil {
return Issued{}, err
}
all = append(all, opened{holder: h, secret: secret, value: value})
}
base, _ := ProvisionSecret(provider, provision)
out := Issued{Secret: base, Unchanged: true}
var failed []error
for i := range all {
changed, err := v.keep(all[i].secret, all[i].value, c.by, caller, OriginIssued)
all[i].value = ""
if err != nil {
failed = append(failed, fmt.Errorf("%s: %w", all[i].holder, err))
continue
}
out.Holders = append(out.Holders, IssuedHolder{Holder: all[i].holder, Secret: changed.Name,
Generation: changed.Generation, Unchanged: changed.Unchanged})
if !changed.Unchanged {
out.Unchanged = false
}
}
for _, h := range asked {
if !got[h] {
out.NotAnswered = append(out.NotAnswered, h)
}
}
if len(failed) > 0 {
return out, fmt.Errorf("not every holder's value was kept: %w", errors.Join(failed...))
}
return out, nil
}
// holdersArg reads the holders the controller names, refusing anything else.
func holdersArg(a map[string]any) ([]Holder, error) {
raw, ok := a["holders"]
if !ok || raw == nil {
return nil, nil
}
list, ok := raw.([]any)
if !ok {
return nil, errors.New("holders is a list of {node, module}")
}
out := make([]Holder, 0, len(list))
for _, item := range list {
m, ok := item.(map[string]any)
if !ok {
return nil, errors.New("holders is a list of {node, module}")
}
node, nok := m["node"].(string)
module, mok := m["module"].(string)
if !nok || !mok || len(m) != 2 {
return nil, errors.New("each holder is {node, module} and nothing else")
}
out = append(out, Holder{Node: strings.TrimSpace(node), Module: strings.TrimSpace(module)})
}
return out, nil
}
modules/mesh-vault/cmd/mesh-vault/perholder_bus_test.go
+284
View File
@@ -0,0 +1,284 @@
package main
// Issue 749's proof on a real bus: a NATS server whose users are the accounts of the mesh's parts (the vault's own,
// the provider's machine's runtime, each machine's provisioner client), each allowed only its own subjects, so the
// caller a tool sees is the one the bus proved — as the mesh's runtime does (subjects ending in the caller's token,
// which only that caller may publish; mesh-controller internal/broker). A listener on every subject keeps every byte
// that crosses.
import (
"encoding/base64"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
)
// callerToken is a caller's name as the last token of a subject.
func callerToken(user string) string { return strings.ReplaceAll(user, ".", "_") }
func untoken(token string) string { return strings.ReplaceAll(token, "_", ".") }
const busConf = `listen: 127.0.0.1:-1
authorization {
users = [
{ user: "novox.mesh-vault", password: "pw", permissions: {
publish: { allow: ["mesh.mod.gitea.call.gitea_agent_token.novox.novox_mesh-vault"] }
subscribe: { allow: ["mesh.seat.mesh-vault.call.get.*", "_INBOX.novox.mesh-vault.>"] }
allow_responses: true } }
{ user: "novox.node-tools", password: "pw", permissions: {
publish: { deny: [">"] }
subscribe: { allow: ["mesh.mod.gitea.call.gitea_agent_token.novox.*"] }
allow_responses: true } }
{ user: "provisioner.anchor", password: "pw", permissions: {
publish: { allow: ["mesh.seat.mesh-vault.call.get.provisioner_anchor"] }
subscribe: { allow: ["_INBOX.provisioner.anchor.>"] } } }
{ user: "provisioner.laptop", password: "pw", permissions: {
publish: { allow: ["mesh.seat.mesh-vault.call.get.provisioner_laptop"] }
subscribe: { allow: ["_INBOX.provisioner.laptop.>"] } } }
{ user: "tap", password: "pw", permissions: { publish: { deny: [">"] }, subscribe: { allow: [">"] } } }
]
}
`
type busAnswer struct {
Result json.RawMessage `json:"result,omitempty"`
Error string `json:"error,omitempty"`
}
func connectAs(t *testing.T, url, user string) *nats.Conn {
t.Helper()
conn, err := nats.Connect(url, nats.UserInfo(user, "pw"), nats.CustomInboxPrefix("_INBOX."+user),
nats.ErrorHandler(func(*nats.Conn, *nats.Subscription, error) {}))
if err != nil {
t.Fatalf("%s cannot connect: %v", user, err)
}
t.Cleanup(conn.Close)
return conn
}
// forgeOnABus is the forge's read tokens issued into a vault keeping rule, on a real bus: what a machine's client is
// answered when it asks over the bus with a key of its own (opened), and every byte that crossed.
func forgeOnABus(t *testing.T, rule Askers) (ask func(user, name string) (string, error), heard func() string) {
t.Helper()
conf := filepath.Join(t.TempDir(), "bus.conf")
if err := os.WriteFile(conf, []byte(busConf), 0o600); err != nil {
t.Fatal(err)
}
opts, err := server.ProcessConfigFile(conf)
if err != nil {
t.Fatal(err)
}
opts.NoLog, opts.NoSigs = true, true
s, err := server.NewServer(opts)
if err != nil {
t.Fatal(err)
}
go s.Start()
if !s.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(s.Shutdown)
url := s.ClientURL()
// Every byte that crosses.
var mu sync.Mutex
var crossed []string
tap := connectAs(t, url, "tap")
if _, err := tap.Subscribe(">", func(m *nats.Msg) {
mu.Lock()
crossed = append(crossed, m.Subject+" "+string(m.Data))
mu.Unlock()
}); err != nil {
t.Fatal(err)
}
_ = tap.Flush()
// The forge's machine's runtime serves the stand-in issuing tool, naming the caller the subject proves.
tool := perHolderProvider(agentTokens, nil)
runtime := connectAs(t, url, "novox.node-tools")
if _, err := runtime.Subscribe("mesh.mod.gitea.call.gitea_agent_token.novox.*", func(m *nats.Msg) {
parts := strings.Split(m.Subject, ".")
var args map[string]any
_ = json.Unmarshal(m.Data, &args)
out, err := tool(untoken(parts[len(parts)-1]), args)
answer := busAnswer{}
if err != nil {
answer.Error = err.Error()
} else {
answer.Result, _ = json.Marshal(out)
}
raw, _ := json.Marshal(answer)
_ = m.Respond(raw)
}); err != nil {
t.Fatal(err)
}
_ = runtime.Flush()
// The vault, on its own account: it asks the tool over the bus in its own name, and answers `get` to the caller
// the subject proves. Its rule for who may ask would let both clients ask for both values: its rule for a
// holder's value is what keeps each to its own.
v := newVault(t, rule)
vaultConn := connectAs(t, url, "novox.mesh-vault")
v.Ask = func(key string, body any) (json.RawMessage, error) {
call, node, _ := strings.Cut(key, "@")
module, tl, _ := strings.Cut(call, ".")
raw, _ := json.Marshal(body)
msg, err := vaultConn.Request(fmt.Sprintf("mesh.mod.%s.call.%s.%s.%s", module, tl, node,
callerToken("novox.mesh-vault")), raw, 5*time.Second)
if err != nil {
return nil, err
}
var a busAnswer
if err := json.Unmarshal(msg.Data, &a); err != nil {
return nil, err
}
if a.Error != "" {
return nil, fmt.Errorf("%s", a.Error)
}
return a.Result, nil
}
if _, err := vaultConn.Subscribe("mesh.seat.mesh-vault.call.get.*", func(m *nats.Msg) {
parts := strings.Split(m.Subject, ".")
var args struct {
Name string `json:"name"`
PublicKey string `json:"public_key"`
}
_ = json.Unmarshal(m.Data, &args)
answer := busAnswer{}
got, err := v.Get(untoken(parts[len(parts)-1]), args.Name, args.PublicKey)
if err != nil {
answer.Error = err.Error()
} else {
answer.Result, _ = json.Marshal(got)
}
raw, _ := json.Marshal(answer)
_ = m.Respond(raw)
}); err != nil {
t.Fatal(err)
}
_ = vaultConn.Flush()
// The controller's `secret issued` for the forge's read tokens, naming both holders.
issued, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents)
if err != nil {
t.Fatal(err)
}
if len(issued.Holders) != 2 {
t.Fatalf("kept %+v", issued)
}
// Each machine's client asks over the bus, with a key of its own.
ask = func(user, name string) (string, error) {
conn := connectAs(t, url, user)
key, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(map[string]string{"name": name, "public_key": key.PublicKey})
msg, err := conn.Request("mesh.seat.mesh-vault.call.get."+callerToken(user), raw, 3*time.Second)
if err != nil {
return "", err
}
var a busAnswer
if err := json.Unmarshal(msg.Data, &a); err != nil {
t.Fatal(err)
}
if a.Error != "" {
return "", fmt.Errorf("%s", a.Error)
}
var got Answer
if err := json.Unmarshal(a.Result, &got); err != nil {
t.Fatal(err)
}
return seal.Open(got.Sealed, key.PrivateKey)
}
// No client can ask in another's name: the bus refuses laptop's client anchor's subject.
laptop := connectAs(t, url, "provisioner.laptop")
key, _ := seal.GenerateKeyPair()
raw, _ := json.Marshal(map[string]string{"name": anchorsToken, "public_key": key.PublicKey})
if _, err := laptop.Request("mesh.seat.mesh-vault.call.get."+callerToken("provisioner.anchor"), raw,
time.Second); err == nil {
t.Error("laptop's client asked in anchor's name and was answered")
}
heard = func() string {
_ = tap.Flush()
time.Sleep(100 * time.Millisecond)
mu.Lock()
defer mu.Unlock()
return strings.Join(crossed, "\n")
}
return ask, heard
}
// noneInTheClear fails when an issued value, or its base64, crossed the bus.
func noneInTheClear(t *testing.T, all string) {
t.Helper()
if !strings.Contains(all, "mesh.mod.gitea.call.gitea_agent_token.novox.novox_mesh-vault") ||
!strings.Contains(all, "mesh.seat.mesh-vault.call.get.provisioner_anchor") {
t.Fatalf("the listener heard nothing of the calls:\n%s", all)
}
for _, token := range agentTokens {
for _, form := range []string{token, base64.StdEncoding.EncodeToString([]byte(token)),
base64.RawURLEncoding.EncodeToString([]byte(token))} {
if strings.Contains(all, form) {
t.Fatalf("an issued value crossed the bus in the clear")
}
}
}
}
func TestOnARealBusTwoHoldersOnTwoMachinesEachGetOnlyTheirOwnIssuedValueAndNoneCrossesInTheClear(t *testing.T) {
ask, heard := forgeOnABus(t, bothMayAsk)
if got, err := ask("provisioner.anchor", anchorsToken); err != nil || got != agentTokens["anchor/claude-code"] {
t.Errorf("anchor's client got %q, %v", got, err)
}
if got, err := ask("provisioner.laptop", laptopsToken); err != nil || got != agentTokens["laptop/claude-code"] {
t.Errorf("laptop's client got %q, %v", got, err)
}
if got, err := ask("provisioner.anchor", laptopsToken); err == nil || !strings.Contains(err.Error(), "alone") {
t.Errorf("anchor's client was handed laptop's value %q: %v", got, err)
}
if got, err := ask("provisioner.laptop", anchorsToken); err == nil || !strings.Contains(err.Error(), "alone") {
t.Errorf("laptop's client was handed anchor's value %q: %v", got, err)
}
noneInTheClear(t, heard())
}
// Issue 757's proof on a real bus, with the rule the running vault keeps: a machine's client is handed the value the
// forge issued for its claude-code because the module's plan there names gitea-read from the forge's machine, and
// is refused another machine's and another module's, sealed to its key and never in the clear.
func TestOnARealBusTheRunningRuleHandsAClientTheIssuedValueItsModulesPlanNamesAndNoOther(t *testing.T) {
plan := `{"templates":{"source":"x","digest":"y"},"files":[],"provisions":[{"name":"gitea-read",` +
`"provision":"gitea-read","provider":"novox","path":"/var/lib/claude-code/gitea-read.secret"}]}`
mirror := assignments{
"anchor.claude-code": `{"node":"anchor","module":"claude-code","provisioning":` + plan + `}`,
"laptop.claude-code": `{"node":"laptop","module":"claude-code","provisioning":` + plan + `}`,
"anchor.zsh": `{"node":"anchor","module":"zsh","provisioning":{"secrets":[{"name":"token"}]}}`,
}
ask, heard := forgeOnABus(t, Declared{Read: mirror.read})
if got, err := ask("provisioner.anchor", anchorsToken); err != nil || got != agentTokens["anchor/claude-code"] {
t.Errorf("anchor's client got %q, %v", got, err)
}
if got, err := ask("provisioner.laptop", laptopsToken); err != nil || got != agentTokens["laptop/claude-code"] {
t.Errorf("laptop's client got %q, %v", got, err)
}
for _, r := range []struct{ user, name, why string }{
{"provisioner.anchor", laptopsToken, "another machine's holder value"},
{"provisioner.laptop", anchorsToken, "another machine's holder value"},
{"provisioner.anchor", "provision/novox/gitea-read/anchor/zsh", "another module's, its plan naming it not"},
} {
if got, err := ask(r.user, r.name); err == nil {
t.Errorf("%s was handed %s, %s: %q", r.user, r.name, r.why, got)
}
}
noneInTheClear(t, heard())
}
modules/mesh-vault/cmd/mesh-vault/perholder_test.go
+291
View File
@@ -0,0 +1,291 @@
package main
// A value a provider's software issued for each holder apart (design 57's V2c, issue 749; ADR 0349 clauses 10 and
// 11; ADR 0365 clause 4, where the forge mints one read token per agent account): the vault asks the provider's tool
// naming the holders, keeps each value the tool answers as that holder's own, and hands it, sealed, to that holder
// alone. A provider that answers one value keeps the provision's one credential, as before.
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seal"
"golang.org/x/crypto/nacl/box"
)
// The two agents' tokens the stand-in forge mints: one per holder, each different.
var agentTokens = map[string]string{
"anchor/claude-code": "gta_anchor-agent-read-token-0001",
"laptop/claude-code": "gta_laptop-agent-read-token-0002",
}
// perHolderProvider is a provider's tool that issues one value per holder (the forge's read token per agent
// account): it answers the vault's account alone, each asked holder's value sealed to the call's key.
func perHolderProvider(tokens map[string]string, asked *[]map[string]any) func(string, map[string]any) (map[string]any, error) {
return func(caller string, args map[string]any) (map[string]any, error) {
if asked != nil {
*asked = append(*asked, args)
}
if !strings.HasSuffix(caller, ".mesh-vault") {
return nil, errors.New("this tool answers the vault alone")
}
key, _ := args["seal_to"].(string)
raw, err := base64.StdEncoding.DecodeString(key)
if err != nil || len(raw) != 32 {
return nil, errors.New("seal_to is not a sealing key")
}
var to [32]byte
copy(to[:], raw)
list, _ := args["holders"].([]any)
var out []map[string]any
for _, h := range list {
m, _ := h.(map[string]any)
node, _ := m["node"].(string)
module, _ := m["module"].(string)
value, ok := tokens[node+"/"+module]
if !ok {
continue
}
sealed, err := box.SealAnonymous(nil, []byte(value), &to, rand.Reader)
if err != nil {
return nil, err
}
out = append(out, map[string]any{"node": node, "module": module,
"sealed": base64.StdEncoding.EncodeToString(sealed)})
}
return map[string]any{"holders": out}, nil
}
}
var twoAgents = []Holder{{Node: "anchor", Module: "claude-code"}, {Node: "laptop", Module: "claude-code"}}
// The holders' secrets' names in the vault's store.
const (
anchorsToken = "provision/novox/gitea-read/anchor/claude-code"
laptopsToken = "provision/novox/gitea-read/laptop/claude-code"
)
// bothMayAsk is a stand-in rule that would let both machines' clients ask for both values: the vault's own rule
// for a holder's value is what keeps each to its own.
var bothMayAsk = AllowList{
"provisioner.anchor": {anchorsToken, laptopsToken},
"provisioner.laptop": {anchorsToken, laptopsToken},
"laptop.claude-code": {anchorsToken, laptopsToken},
}
func forgeMesh(provider func(string, map[string]any) (map[string]any, error)) *aMesh {
return &aMesh{tools: map[string]func(string, map[string]any) (map[string]any, error){
"gitea.gitea_agent_token@novox": provider}}
}
// Two holders on two machines: the vault asks the tool naming both, keeps a value for each, and hands each only to
// that holder (its machine's provisioner client, or the module under an account of its own), sealed to its key.
func TestAProviderIssuesAValueForEachHolderAndEachHolderGetsOnlyItsOwn(t *testing.T) {
v := newVault(t, bothMayAsk)
var asked []map[string]any
mesh := forgeMesh(perHolderProvider(agentTokens, &asked))
v.Ask = mesh.ask
got, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents)
if err != nil {
t.Fatal(err)
}
if len(asked) != 1 || fmt.Sprint(asked[0]["holders"]) != "[map[module:claude-code node:anchor] map[module:claude-code node:laptop]]" {
t.Fatalf("the tool was asked %v", asked)
}
if len(got.Holders) != 2 || got.Holders[0].Secret != anchorsToken || got.Holders[1].Secret != laptopsToken ||
got.Holders[0].Generation != 1 || got.Unchanged {
t.Fatalf("answered %+v", got)
}
if _, held, _ := v.Store.Read("provision/novox/gitea-read"); held {
t.Error("a value per holder was also kept as the provision's one credential")
}
// Each holder gets its own, by its machine's client and by itself under an account of its own.
if got := handedTo(t, v, "provisioner.anchor", anchorsToken); got != agentTokens["anchor/claude-code"] {
t.Errorf("anchor got %q", got)
}
if got := handedTo(t, v, "provisioner.laptop", laptopsToken); got != agentTokens["laptop/claude-code"] {
t.Errorf("laptop got %q", got)
}
if got := handedTo(t, v, "laptop.claude-code", laptopsToken); got != agentTokens["laptop/claude-code"] {
t.Errorf("laptop's agent got %q", got)
}
// Neither gets the other's, though the stand-in rule would allow it.
for caller, name := range map[string]string{"provisioner.laptop": anchorsToken, "provisioner.anchor": laptopsToken,
"laptop.claude-code": anchorsToken} {
if _, err := v.Get(caller, name, mustKey(t)); err == nil || !strings.Contains(err.Error(), "alone") {
t.Errorf("%s was handed %s: %v", caller, name, err)
}
}
handed, _ := v.Store.Handed(20)
refused := 0
for _, h := range handed {
if h.Outcome == "refused" {
refused++
}
}
if refused != 3 {
t.Errorf("%d refusals recorded, want 3: %+v", refused, handed)
}
for _, token := range agentTokens {
if strings.Contains(mesh.said(), token) || strings.Contains(everything(t, v, got, handed), token) {
t.Fatal("an issued value crossed the bus, or is in something the vault said or wrote, in the clear")
}
}
rec, _, _ := v.Store.Read(anchorsToken)
if rec.Origin != OriginIssued || rec.ChangedVia != ControllerCaller || rec.Operator.Sealed == "" {
t.Errorf("the record: %+v", rec)
}
// The same values again write nothing; one holder's new value changes that holder alone (ADR 0305 §5).
again, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents)
if err != nil || !again.Unchanged || again.Holders[0].Generation != 1 || again.Holders[1].Generation != 1 {
t.Errorf("again: %+v %v", again, err)
}
renewed := map[string]string{"anchor/claude-code": agentTokens["anchor/claude-code"],
"laptop/claude-code": "gta_laptop-agent-read-token-0003"}
v.Ask = forgeMesh(perHolderProvider(renewed, nil)).ask
moved, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents)
if err != nil || moved.Unchanged || !moved.Holders[0].Unchanged || moved.Holders[1].Unchanged ||
moved.Holders[1].Generation != 2 {
t.Errorf("renewed: %+v %v", moved, err)
}
// A holder's value is issued by the software: never rotated, never accepted.
if _, err := v.Rotate(ControllerCaller, laptopsToken, "jochen", "test"); err == nil {
t.Error("a holder's issued value was rotated")
}
if _, err := v.AcceptOpen(ControllerCaller, laptopsToken, "jochen", ""); err == nil {
t.Error("a holder's issued value was opened for accept")
}
}
// A holder the tool was not asked about, a holder twice, a value not sealed to the call, or an answer of both
// shapes: nothing is kept, for any holder. A holder the tool did not answer is said, not kept.
func TestAPerHolderAnswerIsTakenWholeOrNotAtAll(t *testing.T) {
sealedTo := func(args map[string]any, value string) string {
raw, _ := base64.StdEncoding.DecodeString(args["seal_to"].(string))
var to [32]byte
copy(to[:], raw)
s, _ := box.SealAnonymous(nil, []byte(value), &to, rand.Reader)
return base64.StdEncoding.EncodeToString(s)
}
var stranger [32]byte
_, _ = rand.Read(stranger[:])
notToTheCall, _ := box.SealAnonymous(nil, []byte("forged"), &stranger, rand.Reader)
answers := map[string]func(args map[string]any) map[string]any{
"a holder not asked about": func(args map[string]any) map[string]any {
return map[string]any{"holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "a")},
map[string]any{"node": "ace", "module": "claude-code", "sealed": sealedTo(args, "b")}}}
},
"a holder twice": func(args map[string]any) map[string]any {
return map[string]any{"holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "a")},
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "b")}}}
},
"a value not sealed to the call": func(args map[string]any) map[string]any {
return map[string]any{"holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "a")},
map[string]any{"node": "laptop", "module": "claude-code",
"sealed": base64.StdEncoding.EncodeToString(notToTheCall)}}}
},
"both shapes": func(args map[string]any) map[string]any {
return map[string]any{"sealed": sealedTo(args, "one"), "holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "a")}}}
},
"an empty value": func(args map[string]any) map[string]any {
return map[string]any{"holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, " ")}}}
},
}
for what, answer := range answers {
v := newVault(t, nil)
answer := answer
v.Ask = forgeMesh(func(_ string, args map[string]any) (map[string]any, error) { return answer(args), nil }).ask
if _, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents); err == nil {
t.Errorf("%s was taken", what)
}
if all, _ := v.Store.List(); len(all) != 0 {
t.Errorf("%s kept %d values", what, len(all))
}
if len(v.pending) != 0 {
t.Errorf("%s left a call's key kept", what)
}
}
// Values per holder when the controller named no holder: refused, nothing kept.
v := newVault(t, nil)
v.Ask = forgeMesh(func(_ string, args map[string]any) (map[string]any, error) {
return map[string]any{"holders": []any{
map[string]any{"node": "anchor", "module": "claude-code", "sealed": sealedTo(args, "a")}}}, nil
}).ask
if _, err := v.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", nil); err == nil {
t.Error("values per holder were taken for no holder named")
}
// A holder the tool did not answer is said and keeps nothing; the others are kept.
w := newVault(t, nil)
w.Ask = forgeMesh(perHolderProvider(map[string]string{"anchor/claude-code": "only-anchors"}, nil)).ask
got, err := w.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", twoAgents)
if err != nil || len(got.Holders) != 1 || len(got.NotAnswered) != 1 || got.NotAnswered[0] != twoAgents[1] {
t.Errorf("a holder not answered: %+v %v", got, err)
}
if _, held, _ := w.Store.Read(laptopsToken); held {
t.Error("a holder the tool did not answer holds a value")
}
// The holders the controller names are names the mesh writes, each once.
for _, bad := range [][]Holder{{{Node: "anchor", Module: ""}}, {{Node: "../x", Module: "claude-code"}},
{twoAgents[0], twoAgents[0]}} {
if _, err := w.TakeIssuedFor(ControllerCaller, "gitea-read", "novox", "gitea", "gitea_agent_token", bad); err == nil {
t.Errorf("holders %v were asked for", bad)
}
}
}
// A provider that answers one value keeps the provision's one credential, though the controller named holders:
// the per-provision path stays as it is.
func TestAProviderAnsweringOneValueKeepsTheProvisionsOneCredential(t *testing.T) {
v := newVault(t, AllowList{"provisioner.anchor": {"provision/anchor/plex-api"}})
software := "plex-token-from-the-vendor"
v.Ask = (&aMesh{tools: map[string]func(string, map[string]any) (map[string]any, error){
"plex.plex_issued_credential@anchor": standInProvider(&software)}}).ask
got, err := v.TakeIssuedFor(ControllerCaller, "plex-api", "anchor", "plex", "plex_issued_credential",
[]Holder{{Node: "anchor", Module: "kometa"}, {Node: "anchor", Module: "tautulli"}})
if err != nil || got.Secret != "provision/anchor/plex-api" || got.Generation != 1 || len(got.Holders) != 0 {
t.Fatalf("%+v %v", got, err)
}
if handedTo(t, v, "provisioner.anchor", "provision/anchor/plex-api") != software {
t.Error("the provision's one credential is not what the software issued")
}
}
// The seat's verb takes the holders as the controller sends them, and refuses a list it cannot read.
func TestTheIssuedVerbTakesTheHolders(t *testing.T) {
v := newVault(t, nil)
v.Ask = forgeMesh(perHolderProvider(agentTokens, nil)).ask
defs := definitions(&Env{Vault: v.Vault})
issued := toolNamed(t, defs, Seat+".issued")
answer, err := issued.runAs(ControllerCaller, map[string]any{"provision": "gitea-read", "provider": "novox",
"module": "gitea", "tool": "gitea_agent_token", "holders": []any{
map[string]any{"node": "anchor", "module": "claude-code"},
map[string]any{"node": "laptop", "module": "claude-code"}}})
if err != nil || len(answer.(Issued).Holders) != 2 {
t.Fatalf("%+v %v", answer, err)
}
for _, bad := range []any{"anchor", []any{"anchor"}, []any{map[string]any{"node": 1}}} {
if _, err := issued.runAs(ControllerCaller, map[string]any{"provision": "gitea-read", "provider": "novox",
"module": "gitea", "tool": "gitea_agent_token", "holders": bad}); err == nil {
t.Errorf("holders %v were read", bad)
}
}
}
func mustKey(t *testing.T) string {
t.Helper()
key, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
return key.PublicKey
}
modules/mesh-vault/cmd/mesh-vault/providers_test.go
+21
View File
@@ -0,0 +1,21 @@
package main
import "testing"
// A plan names, for each provision its module requires, the provider serving it (issue 760); only an entry with a
// name is a credential the machine's client fills (issue 757). The rule hands the value issued for the holder only
// for such an entry: naming a provider alone widens nothing.
func TestARuleHandsOnlyACredentialThePlanNamesNotAProviderAlone(t *testing.T) {
plan := `{"templates":{"source":"x","digest":"y"},"files":[],"provisions":[` +
`{"provision":"postgres-database","provider":"novox","provider-module":"postgres"},` +
`{"name":"gitea-read","provision":"gitea-read","provider":"novox","provider-module":"gitea",` +
`"path":"/s/gitea-read.secret"}]}`
mirror := assignments{"anchor.app": `{"node":"anchor","module":"app","provisioning":` + plan + `}`}
rule := Declared{Read: mirror.read}
if ok, why := rule.May("provisioner.anchor", "provision/novox/gitea-read/anchor/app"); !ok {
t.Fatalf("the credential the plan names was refused: %s", why)
}
if ok, _ := rule.May("provisioner.anchor", "provision/novox/postgres-database/anchor/app"); ok {
t.Fatal("a provision the plan names only with its provider was handed out")
}
}
modules/mesh-vault/cmd/mesh-vault/pulled_in_test.go
+45
View File
@@ -0,0 +1,45 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// A module only pulled in by another on a node has a value of its own in the assignments mirror (mesh-controller
// internal/inventory AssignmentValues, issue 765): its plan, what requires it, and no time it was assigned. The
// running rule hands that node's client what the plan names, as for an assigned module, and nothing else.
func TestTheRunningVaultAnswersAMachinesClientForWhatAPulledInModulesPlanNames(t *testing.T) {
mirror := assignments{
"g14.app": `{"node":"g14","module":"app","required-by":"desk","provisioning":` +
`{"templates":{"source":"x","digest":"y"},"files":[],"secrets":[{"name":"token"}],` +
`"provisions":[{"name":"gitea-read","provision":"gitea-read","provider":"novox"}]}}`,
"g14.kept": `{"node":"g14","module":"kept","required-by":"desk"}`,
}
v := newVault(t, Declared{Read: mirror.read})
for _, name := range []string{"own/g14/app/token", "own/g14/app/other", "own/g14/kept/token",
"provision/novox/gitea-read/g14/app"} {
if _, err := v.Put(name, "value-of-"+name, "the test"); err != nil {
t.Fatal(err)
}
}
asker, _ := seal.GenerateKeyPair()
for _, name := range []string{"own/g14/app/token", "provision/novox/gitea-read/g14/app"} {
answer, err := v.Get("provisioner.g14", name, asker.PublicKey)
if err != nil {
t.Fatalf("the machine's client was refused %s, which its pulled-in module's plan names: %v", name, err)
}
if got, err := seal.Open(answer.Sealed, asker.PrivateKey); err != nil || got != "value-of-"+name {
t.Fatalf("the answer opens to %q (%v)", got, err)
}
}
for _, r := range []struct{ caller, name, why string }{
{"provisioner.g14", "own/g14/app/other", "a secret the pulled-in module's plan does not name"},
{"provisioner.g14", "own/g14/kept/token", "a pulled-in module not handed to the provisioner"},
{"provisioner.ace", "own/g14/app/token", "another machine's client"},
} {
if a, err := v.Get(r.caller, r.name, asker.PublicKey); err == nil {
t.Errorf("get answered %s (%s asking %s): %+v", r.why, r.caller, r.name, a)
}
}
}
modules/mesh-vault/cmd/mesh-vault/reseal_health.go
+44
View File
@@ -0,0 +1,44 @@
package main
// The vault's runtime health (novox/hq ADR 0366 §4): the vault re-seals its operator copies every minute
// (adoptOperatorKey), and a vault that keeps failing to is loud before secrets move into it. The node-engine
// asks vault_health on the vault's own runtime (`runtime-health` in module.json); the controller raises
// module.mesh-vault.<machine>.unhealthy from it, with these words, and clears it on the first healthy answer.
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/health"
)
// FailedPassesUnhealthy is how many failed re-seal passes in a row make the vault unhealthy: one minute's
// pass can fail for a moment's reason and be finished by the next.
const FailedPassesUnhealthy = 3
// pass is one minute's re-seal, its outcome kept for vault_health.
func (e *Env) pass() {
err := adoptOperatorKey(e)
e.mu.Lock()
defer e.mu.Unlock()
if err == nil {
e.failedPasses, e.lastFailure = 0, ""
return
}
e.failedPasses++
e.lastFailure = err.Error()
}
// health is vault_health's answer.
func (e *Env) health() health.Answer {
e.mu.Lock()
defer e.mu.Unlock()
if e.failedPasses >= FailedPassesUnhealthy {
return health.Unhealthy(fmt.Sprintf("re-sealing the vault's operator copies failed on %d passes in a row "+
"(one a minute); the last said: %s", e.failedPasses, e.lastFailure))
}
if e.failedPasses > 0 {
return health.Healthy(fmt.Sprintf("the last %d re-seal pass(es) failed, tried again every minute; "+
"unhealthy after %d in a row", e.failedPasses, FailedPassesUnhealthy))
}
return health.Healthy("every re-seal pass succeeds")
}
modules/mesh-vault/cmd/mesh-vault/reseal_health_test.go
+82
View File
@@ -0,0 +1,82 @@
package main
import (
"encoding/json"
"fmt"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/health"
)
// novox/hq ADR 0366 §4: the vault's runtime health is a tool that says unhealthy once re-sealing has failed on
// three passes in a row, saying what failed and why, and healthy again after one pass succeeds.
func TestTheVaultIsUnhealthyAfterThreeFailedReSealPassesAndHealthyAfterOne(t *testing.T) {
v := newVault(t, nil)
if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil {
t.Fatal(err)
}
env := &Env{Vault: v.Vault}
answer := func() health.Answer {
t.Helper()
got, err := toolNamed(t, definitions(env), "vault_health").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
raw, err := json.Marshal(got)
if err != nil {
t.Fatal(err)
}
a, err := health.Read(raw)
if err != nil {
t.Fatalf("the node-engine cannot read %s: %v", raw, err)
}
return a
}
if a := answer(); !a.Healthy {
t.Fatalf("a vault before any pass says %+v", a)
}
// A new operator key whose copies cannot be written: every pass fails.
pub, _ := operatorKey(t)
v.Store.failWrite = func(Record) error { return fmt.Errorf("disk full") }
_, _ = v.SetOperatorKey(asPEM(t, pub), "the test")
for pass := 1; pass <= 3; pass++ {
env.pass()
a := answer()
if pass < 3 && !a.Healthy {
t.Fatalf("after %d failed pass(es) the vault says %+v; it is unhealthy after three", pass, a)
}
if pass == 3 && (a.Healthy || a.State != health.StateUnhealthy || !strings.Contains(a.Why, "disk full") ||
!strings.Contains(a.Why, "3 passes") || !strings.Contains(a.Why, "pair/postgres/gitea")) {
t.Fatalf("after three failed passes the vault says %+v", a)
}
}
// One pass that succeeds makes it healthy again, and the streak starts over.
v.Store.failWrite = nil
env.pass()
if a := answer(); !a.Healthy {
t.Fatalf("after a pass that re-sealed the vault says %+v", a)
}
v.Store.failWrite = func(Record) error { return fmt.Errorf("disk full") }
pub, _ = operatorKey(t)
_, _ = v.SetOperatorKey(asPEM(t, pub), "the test")
env.pass()
if a := answer(); !a.Healthy {
t.Fatalf("one failed pass after a good one says %+v", a)
}
}
// A vault that could not be built re-seals nothing: its passes fail, saying why.
func TestABrokenVaultsPassesFail(t *testing.T) {
env := &Env{Broken: fmt.Errorf("the vault could not open its store: no such directory")}
for i := 0; i < 3; i++ {
env.pass()
}
got, err := toolNamed(t, definitions(env), "vault_health").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
if a := got.(health.Answer); a.Healthy || !strings.Contains(a.Why, "could not open its store") {
t.Errorf("a broken vault says %+v", a)
}
}
modules/mesh-vault/cmd/mesh-vault/rotate_one_test.go
+119
View File
@@ -0,0 +1,119 @@
package main
// The controller's `rotate <provision> [--consumer <machine>] [--module <module>]` reaches the vault (issue 767): a
// provider's one shared value is rotated for every holder together, and it is never rotated for one consumer alone;
// a per-login provider (ADR 0158 §11) rotates the named consumer's own login and no other.
import (
"strings"
"testing"
)
// postgresLogins is a vault holding a login each for gitea on g14 and umami on ace, from postgres on novox.
func postgresLogins(t *testing.T) (testVault, string, string) {
t.Helper()
v := pairVault(t, assignments{
"novox.postgres": handedWith("novox", "postgres"),
"g14.gitea": handedWith("g14", "gitea", "postgres-database@novox/postgres"),
"ace.umami": handedWith("ace", "umami", "postgres-database@novox/postgres"),
})
gitea, umami := "pair/novox/postgres-database/g14/gitea", "pair/novox/postgres-database/ace/umami"
for name, value := range map[string]string{gitea: "login-of-gitea", umami: "login-of-umami"} {
if _, err := v.keepWith(name, value, "node.novox", "", OriginMade, keepOptions{neverReplace: true,
movedFrom: &MovedFrom{By: "node.novox"}}); err != nil {
t.Fatal(err)
}
}
return v, gitea, umami
}
func TestARotationNamingOneConsumerRemakesThatConsumersLoginAlone(t *testing.T) {
for _, one := range []Narrow{{Consumer: "g14"}, {ConsumerModule: "gitea"}, {Consumer: "g14", ConsumerModule: "gitea"}} {
v, gitea, umami := postgresLogins(t)
before := changedSaid(t, v, umami, "")
changed, err := v.RotateFor(ControllerCaller, "provision/novox/postgres-database", one, "person.jochen",
"suspected on g14")
if err != nil || len(changed.Each) != 1 || changed.Each[0].Name != gitea {
t.Fatalf("%+v: the rotation %+v (%v)", one, changed, err)
}
if got, _, err := asks(v, "provisioner.g14", gitea); err != nil || got == "login-of-gitea" {
t.Fatalf("%+v: gitea's login was not remade: %q (%v)", one, got, err)
}
if got, gen, err := asks(v, "provisioner.ace", umami); err != nil || got != "login-of-umami" || gen != 1 {
t.Fatalf("%+v: umami's login was touched: %q generation %d (%v)", one, got, gen, err)
}
if changedSaid(t, v, umami, "") != before {
t.Fatalf("%+v: secret-changed was said for umami's login", one)
}
}
}
func TestARotationNamingAConsumerThatHoldsNoLoginChangesNothing(t *testing.T) {
v, gitea, umami := postgresLogins(t)
_, err := v.RotateFor(ControllerCaller, "provision/novox/postgres-database", Narrow{Consumer: "g14",
ConsumerModule: "umami"}, "person.jochen", "suspected")
if err == nil || !strings.Contains(err.Error(), "umami on g14") {
t.Fatalf("a consumer holding no login was not refused by name: %v", err)
}
for name, want := range map[string]string{gitea: "login-of-gitea", umami: "login-of-umami"} {
if got, _, _ := asks(v, "provisioner.novox", name); got != want {
t.Fatalf("%s changed on a refused rotation", name)
}
}
}
// A shared value is one for every holder (ADR 0158): rotated, every holder gets the new one after secret-changed; it
// is never rotated for one consumer alone, which would change the value for all of them.
func TestASharedValueIsRotatedForEveryHolderAndNeverForOneConsumer(t *testing.T) {
v := pairVault(t, assignments{
"anchor.stand-in": handedWith("anchor", "stand-in"),
"g14.app": handedWith("g14", "app", "stand-in-api@anchor/stand-in"),
"ace.other": handedWith("ace", "other", "stand-in-api@anchor/stand-in"),
})
movedRow(t, v, "g14", "app", "the-value-they-share", "20260901T000000-1", OriginMade)
movedRow(t, v, "ace", "other", "the-value-they-share", "20260901T000000-1", OriginMade)
if o := pass(t, v, standIn); o.Outcome != PairAdopted {
t.Fatalf("stamped rows: %+v", o)
}
if _, err := v.RotateFor(ControllerCaller, standIn, Narrow{Consumer: "g14"}, "person.jochen", "suspected"); err == nil ||
!strings.Contains(err.Error(), "every holder") {
t.Fatalf("a shared value was rotated for one consumer: %v", err)
}
if got, gen, _ := asks(v, "provisioner.g14", standIn); got != "the-value-they-share" || gen != 1 {
t.Fatal("a refused rotation changed the shared value")
}
changed, err := v.RotateFor(ControllerCaller, standIn, Narrow{}, "person.jochen", "yearly")
if err != nil || changed.Generation != 2 {
t.Fatalf("the rotation: %+v (%v)", changed, err)
}
var seen string
for _, c := range []string{"provisioner.anchor", "provisioner.g14", "provisioner.ace"} {
got, gen, err := asks(v, c, standIn)
if err != nil || got == "the-value-they-share" || gen != 2 || (seen != "" && got != seen) {
t.Fatalf("%s got %q generation %d (%v): not the one new value", c, got, gen, err)
}
seen = got
}
if changedSaid(t, v, standIn, seen) == 0 {
t.Fatal("secret-changed was not said for the shared value")
}
}
func TestARotationOfAModulesOwnSecretNamesNoConsumer(t *testing.T) {
v := newVault(t, nil)
if _, err := v.RotateFor(ControllerCaller, "own/g14/app/key", Narrow{Consumer: "g14"}, "person.jochen", "x"); err == nil {
t.Fatal("a module's own secret was rotated with a consumer named")
}
}
func TestTheRotateToolTakesTheConsumerAndItsModule(t *testing.T) {
n, err := narrowNamed(map[string]any{"consumer": "g14", "consumer_module": "gitea"})
if err != nil || n != (Narrow{Consumer: "g14", ConsumerModule: "gitea"}) {
t.Fatalf("%+v (%v)", n, err)
}
if _, err := narrowNamed(map[string]any{"consumer": "g14/x"}); err == nil {
t.Fatal("a consumer that is not a name was taken")
}
}
modules/mesh-vault/cmd/mesh-vault/store.go
+108 -1
View File
@@ -39,6 +39,15 @@ type Record struct {
MadeAt time.Time `json:"made-at"`
ChangedAt time.Time `json:"changed-at"`
ChangedBy string `json:"changed-by"`
// ChangedVia is the caller the runtime proved, when it is not who ChangedBy names: the controller
// carrying a person's act (design 57's V2). Empty when they are the same.
ChangedVia string `json:"changed-via,omitempty"`
// Origin is where the value came from: made by the vault, accepted at the controller's terminal, typed at
// a desk, or issued by a provider's software (ADR 0349 clauses 8 to 10). Empty is made, as V1 wrote it.
Origin string `json:"origin,omitempty"`
// MovedFrom is, for a value the move took from the controller's store (design 57's V3), the node-engine that
// handed it in and the controller's generation for it, kept as it was.
MovedFrom *MovedFrom `json:"moved-from,omitempty"`
}
// OperatorCopy is a value sealed to the operator key named in Key.
@@ -58,6 +67,8 @@ type Store struct {
handedMu sync.Mutex
rolled int
// asksMu serialises the record of asks, so two asks for one secret at once do not both pass its bounds.
asksMu sync.Mutex
}
// The record of hand-outs is bounded: rolled at HandedMaxDefault into handed.jsonl.1 … .HandedKept, the
@@ -160,6 +171,7 @@ func (s *Store) RecordHanding(h Handing) (rolled bool, err error) {
if err := s.roll(path); err != nil {
return false, err
}
s.rolled++
rolled = true
}
return rolled, appendSynced(path, append(raw, '\n'))
@@ -176,7 +188,6 @@ func (s *Store) roll(path string) error {
if err := os.Rename(path, path+".1"); err != nil {
return err
}
s.rolled++
return syncDir(s.dir)
}
@@ -334,3 +345,99 @@ func syncDir(dir string) error {
defer d.Close()
return d.Sync()
}
// The two lines an ask leaves in the record: when it opened, and how it ended.
const (
AskOpened = "opened"
AskEnded = "ended"
)
// AskRecord is one line of the record of asks for a module's own secret at a desk (ADR 0277 §25, in the vault's
// store since ADR 0349 clause 9): who asked, as the bus named them, for what, at which desk, and how it ended.
// Never a value.
type AskRecord struct {
ID string `json:"id"`
At time.Time `json:"at"`
Event string `json:"event"`
Secret string `json:"secret"`
Node string `json:"node,omitempty"`
Module string `json:"module,omitempty"`
Name string `json:"name,omitempty"`
Desk string `json:"desk,omitempty"`
AskedBy string `json:"asked_by,omitempty"`
Caller string `json:"caller,omitempty"`
Outcome string `json:"outcome,omitempty"`
}
// RecordAsk appends one line to the record of asks, synced before it returns; rolled as the record of
// hand-outs is, and it says whether it was.
func (s *Store) RecordAsk(r AskRecord) (rolled bool, err error) {
raw, err := json.Marshal(r)
if err != nil {
return false, err
}
path := filepath.Join(s.dir, "asks.jsonl")
if info, err := os.Stat(path); err == nil && info.Mode().IsRegular() && info.Size()+int64(len(raw))+1 > s.HandedBound() {
if err := s.roll(path); err != nil {
return false, err
}
rolled = true
}
return rolled, appendSynced(path, append(raw, '\n'))
}
// asks is every line of the record of asks, oldest first, the rolled files before the current one.
func (s *Store) asks() ([]AskRecord, error) {
path := filepath.Join(s.dir, "asks.jsonl")
var out []AskRecord
for i := HandedKept; i >= 0; i-- {
file := path
if i > 0 {
file = fmt.Sprintf("%s.%d", path, i)
}
raw, err := os.ReadFile(file)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return nil, err
}
for _, line := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
var r AskRecord
if line != "" && json.Unmarshal([]byte(line), &r) == nil {
out = append(out, r)
}
}
}
return out, nil
}
// AsksSince is every line of the record of asks at or after since, oldest first.
func (s *Store) AsksSince(since time.Time) ([]AskRecord, error) {
all, err := s.asks()
if err != nil {
return nil, err
}
var out []AskRecord
for _, r := range all {
if !r.At.Before(since) {
out = append(out, r)
}
}
return out, nil
}
// Asks is the latest lines of the record of asks, newest first, at most limit.
func (s *Store) Asks(limit int) ([]AskRecord, error) {
s.asksMu.Lock()
defer s.asksMu.Unlock()
all, err := s.asks()
if err != nil {
return nil, err
}
out := []AskRecord{}
for i := len(all) - 1; i >= 0 && len(out) < limit; i-- {
out = append(out, all[i])
}
return out, nil
}
modules/mesh-vault/cmd/mesh-vault/tools.go
+265 -1
View File
@@ -16,6 +16,7 @@ import (
"path/filepath"
"strings"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
@@ -33,6 +34,14 @@ type Env struct {
refused string
// Broken is why the vault could not be built, said by `get`.
Broken error
// failedPasses is how many re-seal passes in a row failed, and lastFailure why the last one did
// (vault_health, novox/hq ADR 0366 §4).
failedPasses int
lastFailure string
// pairsSaid is what the last pass over the shared credentials said of each, and lastPairFailure why it last
// did not finish (pair.go): each said once, and again when it changes.
pairsSaid map[string]string
lastPairFailure string
}
func (e *Env) keyRefused() string {
@@ -83,6 +92,194 @@ func definitions(env *Env) []toolDef {
key, _ := a["public_key"].(string)
return env.Vault.Get(caller, strings.TrimSpace(name), key)
}},
{Name: Seat + ".accept",
Description: "Take a value given at the controller's terminal (ADR 0349 clause 8), in two steps from the " +
"controller alone: name the secret (node, module and name for a module's own; provider and provision " +
"for a provision's one credential) and get a call and a key made for it; then send the call and the " +
"value sealed to that key. The vault alone opens it, and never remakes an accepted credential. With " +
"origin issued it keeps a covered bus account the controller minted, replacing it in place.",
Input: intakeInput(map[string]string{"node": "the machine, for a module's own secret",
"module": "the module, for a module's own secret", "name": "the secret's name in the module",
"provider": "the provider's machine, for a provision's credential", "provision": "the provision",
"by": "who gave it at the terminal, as the controller names them",
"origin": "issued for a bus account the controller minted for a core or moved module and hands at " +
"once (ADR 0374, issue 787), kept as that module's own secret and replaced in place on a " +
"re-issue; left out for a value given at the terminal"}),
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
if call := arg(a, "call"); call != "" {
return v.AcceptTake(caller, call, arg(a, "sealed"))
}
secret, err := secretNamed(a)
if err != nil {
return nil, err
}
return v.AcceptOpen(caller, secret, arg(a, "by"), arg(a, "origin"))
}},
{Name: Seat + ".secret-ask",
Description: "Ask the operator for a module's own secret at a desk (ADR 0349 clause 9, ADR 0277 §22–§27), " +
"asked by the controller alone: node, module, name, desk and asked_by. The ask is recorded before any " +
"prompt opens — one open ask per secret and three an hour — and the vault asks the desk's prompt itself, " +
"in its own name, with a key made for the call; what is typed opens in the vault alone. Answers how the " +
"prompt ended (given, dismissed, timed-out, empty, refused or failed) and who asked as recorded; never " +
"the value.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"node": str("the machine the secret is for"), "module": str("the module"),
"name": str("the secret's name in the module"), "desk": str("the machine whose prompt asks"),
"asked_by": str("who asked the controller, as the bus named its caller")}},
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
return v.AskAtDesk(caller, AskOf{Node: arg(a, "node"), Module: arg(a, "module"), Name: arg(a, "name"),
Desk: arg(a, "desk"), AskedBy: arg(a, "asked_by")})
}},
{Name: Seat + ".issued",
Description: "Take the credential a provider's software issued for a provision (ADR 0349 clause 10, ADR 0305), " +
"asked by the controller alone: provision, provider (its machine), module and tool. The vault asks that " +
"tool itself, in its own name, with a key made for the call, and keeps what it sealed as the provision's " +
"one credential, or, when the tool answers a value per holder named in holders, each as that holder's " +
"own, handed to it alone; the same value again writes nothing. Never answers the value.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"provision": str("the provision"), "provider": str("the provider's machine"),
"module": str("the provider's module, whose software issued it"),
"tool": str("the provider's tool that answers it, as its offer names it"),
"holders": map[string]any{"type": "array", "description": "the provision's holders from that " +
"provider, each {node, module}: the tool may answer a value for each, kept as that holder's own " +
"and handed to it alone",
"items": map[string]any{"type": "object", "properties": map[string]any{
"node": str("the holder's machine"), "module": str("the consuming module")}}}}},
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
holders, err := holdersArg(a)
if err != nil {
return nil, err
}
return v.TakeIssuedFor(caller, arg(a, "provision"), arg(a, "provider"), arg(a, "module"), arg(a, "tool"),
holders)
}},
{Name: Seat + ".rotate",
Description: "Make a secret anew (ADR 0114, ADR 0228): node, module and name for a module's own, or provider " +
"and provision for a provision's credential. Refused for a value a provider's software issued, which " +
"the software rotates, and for an accepted credential, which the vault does not remake. Never answers " +
"the value. From the controller alone, and with a why.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"node": str("the machine, for a module's own secret"), "module": str("the module"),
"name": str("the secret's name in the module"), "provider": str("the provider's machine"),
"provision": str("the provision"),
"consumer": str("a provision's credential of a provider whose consumers keep a login each: only " +
"this consumer's machine's own"),
"consumer_module": str("with it or alone: only this consuming module's own login"),
"by": str("who asked, as the controller names them when it carries the act"),
"why": str("why it is rotated: required")}},
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
secret, err := secretNamed(a)
if err != nil {
return nil, err
}
one, err := narrowNamed(a)
if err != nil {
return nil, err
}
return v.RotateFor(caller, secret, one, arg(a, "by"), arg(a, "why"))
}},
{Name: Seat + ".move-in",
Description: "Take one value of the controller's store into the vault, as it is (design 57's V3): from the " +
"node-engine of a machine it was sealed to, and no other caller, in two steps — name the row (node, " +
"module and name for a module's own; provider, provision, consumer, consumer_module and local for a " +
"pair credential), its origin and the controller's generation, and get a call and a key made for it; " +
"then send the call, the value sealed to that key and read_back_to, a key of your own. Never replaces " +
"a value the vault holds; answers what it kept, read back from its store sealed to read_back_to.",
Input: intakeInput(map[string]string{"node": "a module's own secret: its machine", "module": "its module",
"name": "its name in the module", "provider": "a pair credential: the provider's machine",
"provision": "the provision", "consumer": "the consumer's machine",
"consumer_module": "the consuming module", "local": "the credential's local name, when it has one",
"origin": "where the value came from in the controller's store: made, accepted or issued",
"generation": "the controller's generation for the value, kept as it is",
"made_at": "when the controller made or took it (RFC 3339)",
"read_back_to": "the second step: a raw X25519 public key in base64 the read-back is sealed to",
"sealing_key": "the first step, optional: this machine's sealing key, a raw X25519 public key in " +
"base64, which the core's credentials are sealed to from then on (ADR 0349 §16)"}),
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
if call := arg(a, "call"); call != "" {
return v.MoveTake(caller, call, arg(a, "sealed"), arg(a, "read_back_to"))
}
m := MoveOf{Node: arg(a, "node"), Module: arg(a, "module"), Name: arg(a, "name"),
Provider: arg(a, "provider"), Provision: arg(a, "provision"), Consumer: arg(a, "consumer"),
ConsumerModule: arg(a, "consumer_module"), Local: arg(a, "local"), Origin: arg(a, "origin"),
Generation: arg(a, "generation"), SealingKey: arg(a, "sealing_key")}
if at := arg(a, "made_at"); at != "" {
if m.MadeAt, err = time.Parse(time.RFC3339Nano, at); err != nil {
return nil, fmt.Errorf("made_at %q is not a time (RFC 3339)", at)
}
}
return v.MoveOpen(caller, m)
}},
{Name: Seat + ".node-key",
Description: "State this machine's sealing key to the vault, in its own name (ADR 0372, issue 786): from the " +
"node-engine of the machine alone, at its start and whenever the key changes; the machine is the caller's, " +
"as the bus proved it. A first or changed key is recorded and `secret-changed` is raised for the machine's " +
"core secrets, so the controller asks again; the same key again changes nothing.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"key": str("this machine's sealing key, a raw X25519 public key in base64, as move-in's sealing_key carries it")}},
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
if err := v.StateNodeKey(caller, arg(a, "key")); err != nil {
return nil, err
}
return map[string]any{"recorded": true}, nil
}},
{Name: Seat + ".move-asks",
Description: "Copy rows of the controller's record of asks at a desk into the vault's, as they are and once " +
"(design 57's V3): from the controller alone, at most 500 at a time. No value is in any of them.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"asks": map[string]any{"type": "array", "description": "the controller's rows: id, node, module, name, " +
"desk, asked_by, opened_at, and ended_at and outcome when the ask ended",
"items": map[string]any{"type": "object"}}}},
run: func(caller string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
raw, err := json.Marshal(a["asks"])
if err != nil {
return nil, err
}
var asks []MovedAsk
if err := json.Unmarshal(raw, &asks); err != nil {
return nil, fmt.Errorf("asks is not the controller's rows: %w", err)
}
return v.MoveAsks(caller, asks)
}},
{Name: "secret_asks",
Description: "The record of asks for a module's own secret at a desk, newest first: who asked as the bus named " +
"them, for what, at which desk, when, and how each ended. Never a value.",
Input: map[string]any{"limit": map[string]any{"type": "number", "description": "how many lines (default 50)"}},
run: func(_ string, a map[string]any) (any, error) {
v, err := env.ready()
if err != nil {
return nil, err
}
return v.Store.Asks(limitOf(a))
}},
{Name: "secret_export",
Description: "The mesh's root secrets as this vault keeps them: every secret a module holds for itself, " +
"sealed to the operator's key (novox/hq ADR 0085, amended). Ciphertext — nothing here can open a line " +
@@ -204,6 +401,11 @@ func definitions(env *Env) []toolDef {
}
return map[string]any{"deleted": consumer, "freed_bytes": -1}, nil
}},
{Name: "vault_health",
Description: "The vault's runtime health, as the node-engine asks it (novox/hq ADR 0366): unhealthy once " +
"re-sealing the operator copies has failed on three passes in a row, saying what failed and why; " +
"healthy again after one pass succeeds.",
run: func(string, map[string]any) (any, error) { return env.health(), nil }},
{Name: "vault_status",
Description: "What needs a person's eye in the vault: operator copies still sealed to an earlier key, the operator " +
"key the mesh's export holds if the vault refused it, and how often the record of hand-outs was rolled.",
@@ -237,7 +439,8 @@ func actInput(set, one bool) map[string]any {
}
// actOf is an act's why, by and via. `by` is the person the controller names; with none, the caller the
// runtime proved.
// runtime proved. `by` is an argument, so it is not proven: when it names somebody other than the caller the
// runtime proved, the proven caller is kept beside it in via, so the record says both.
func actOf(caller string, a map[string]any) Act {
act := Act{}
act.Why, _ = a["why"].(string)
@@ -246,9 +449,70 @@ func actOf(caller string, a map[string]any) Act {
if act.By == "" {
act.By = caller
}
if act.By != caller {
proven := "called by " + caller
if act.Via == "" {
act.Via = proven
} else {
act.Via += ", " + proven
}
}
return act
}
// ready is the vault, or why there is none.
func (e *Env) ready() (*Vault, error) {
if e.Vault != nil {
return e.Vault, nil
}
if e.Broken != nil {
return nil, e.Broken
}
return nil, errors.New("the vault is not ready")
}
func arg(a map[string]any, key string) string {
s, _ := a[key].(string)
return strings.TrimSpace(s)
}
// secretNamed is the secret a call names: a module's own (node, module, name), or a provision's one credential
// (provider, provision) — one or the other.
func secretNamed(a map[string]any) (string, error) {
own := arg(a, "node") != "" || arg(a, "module") != "" || arg(a, "name") != ""
pair := arg(a, "provider") != "" || arg(a, "provision") != ""
switch {
case own && !pair:
return OwnSecret(arg(a, "node"), arg(a, "module"), arg(a, "name"))
case pair && !own:
return ProvisionSecret(arg(a, "provider"), arg(a, "provision"))
}
return "", errors.New("name a module's own secret (node, module, name) or a provision's credential " +
"(provider, provision), one of the two")
}
// narrowNamed is the one consumer a rotation names, if any (issue 767).
func narrowNamed(a map[string]any) (Narrow, error) {
n := Narrow{Consumer: arg(a, "consumer"), ConsumerModule: arg(a, "consumer_module")}
for what, v := range map[string]string{"the consumer's machine": n.Consumer, "the consuming module": n.ConsumerModule} {
if v != "" && !namePart.MatchString(v) {
return Narrow{}, fmt.Errorf("%s %q is not a name the mesh writes", what, v)
}
}
return n, nil
}
// intakeInput is a two-step verb's arguments: what its first step names, then the call and what was sealed.
func intakeInput(first map[string]string) map[string]any {
props := map[string]any{}
for k, d := range first {
props[k] = str(d)
}
props["call"] = str("the second step: the call the first step answered")
props["sealed"] = str("the second step: the value sealed to the call's key (NaCl anonymous box, base64) — never the value")
return map[string]any{"type": "object", "properties": props}
}
func stringsOf(v any) []string {
list, _ := v.([]any)
out := []string{}
modules/mesh-vault/cmd/mesh-vault/vault.go
+65 -5
View File
@@ -7,6 +7,7 @@ package main
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"regexp"
@@ -30,11 +31,21 @@ type Vault struct {
Own seal.KeyPair
// Askers decides who may ask `get` for what. Nil allows nobody.
Askers Askers
Emit func(event string, body map[string]any) error
Now func() time.Time
Log func(format string, args ...any)
// Assignments is the controller's assignments mirror, where the vault sees a consumer bind a provider and
// unbind (pair.go). Empty sees nothing, and makes no provision's credential.
Assignments Mirror
Emit func(event string, body map[string]any) error
// Ask calls a tool on the mesh as the vault, through the runtime that launched it, under the vault's own bus
// account (ADR 0259 §8's runs-as, issue 723): the desks' secret prompt and each issuing provider's tool, which
// the bus lets the vault's account alone call in its own name. Nil asks nobody.
Ask func(key string, body any) (json.RawMessage, error)
Now func() time.Time
Log func(format string, args ...any)
mu sync.Mutex
// callsMu guards pending: the keys of the calls open now (intake.go), never written anywhere.
callsMu sync.Mutex
pending map[string]*callKey
}
// Changed is what one change to a secret was. Never its value.
@@ -44,6 +55,8 @@ type Changed struct {
At time.Time `json:"at"`
By string `json:"by"`
Unchanged bool `json:"unchanged,omitempty"`
// Each is, for a rotation of a provider whose consumers keep logins of their own, each one's own value remade.
Each []Changed `json:"each,omitempty"`
}
// Answer is `get`'s reply: the secret sealed to the asker's key, and which generation it is.
@@ -65,6 +78,25 @@ func (v *Vault) log(format string, args ...any) {
// Refused: a private key (it stays where it was made), and any value while the vault has no operator key —
// a secret only the vault could open is what ADR 0113 §22 rules out.
func (v *Vault) Put(name, value, by string) (Changed, error) {
return v.keep(name, value, by, "", OriginMade)
}
// keep is Put with where the value came from (origin) and the caller the runtime proved when it is not who
// by names (via).
func (v *Vault) keep(name, value, by, via, origin string) (Changed, error) {
return v.keepWith(name, value, by, via, origin, keepOptions{})
}
// keepOptions are what the move keeps beside a value (move.go): that it never replaces another, when the value
// was first made, and where it was moved from.
type keepOptions struct {
neverReplace bool
madeAt time.Time
movedFrom *MovedFrom
}
// keepWith is keep with the move's options.
func (v *Vault) keepWith(name, value, by, via, origin string, opt keepOptions) (Changed, error) {
if err := validName(name); err != nil {
return Changed{}, err
}
@@ -94,6 +126,11 @@ func (v *Vault) Put(name, value, by string) (Changed, error) {
if old, err := seal.Open(before.AtRest, v.Own.PrivateKey); err == nil && old == value {
return Changed{Name: name, Generation: before.Generation, At: before.ChangedAt, By: before.ChangedBy, Unchanged: true}, nil
}
if opt.neverReplace {
return Changed{}, fmt.Errorf("the vault already holds another value for %s (generation %d, changed by %s "+
"at %s), and the move never replaces a value the vault holds: nothing was taken", name,
before.Generation, before.ChangedBy, before.ChangedAt.Format(time.RFC3339))
}
}
atRest, err := seal.Seal(value, v.Own.PublicKey)
if err != nil {
@@ -104,11 +141,21 @@ func (v *Vault) Put(name, value, by string) (Changed, error) {
return Changed{}, err
}
now := v.Now().UTC()
if via == by {
via = ""
}
rec := Record{Name: name, Generation: 1, AtRest: atRest,
Operator: OperatorCopy{Box: OperatorBox, Key: operator, Sealed: forOperator},
MadeAt: now, ChangedAt: now, ChangedBy: by}
MadeAt: now, ChangedAt: now, ChangedBy: by, ChangedVia: via, Origin: origin}
if held {
rec.Generation, rec.MadeAt = before.Generation+1, before.MadeAt
} else if !opt.madeAt.IsZero() {
rec.MadeAt = opt.madeAt.UTC()
}
if opt.movedFrom != nil {
moved := *opt.movedFrom
moved.At = now
rec.MovedFrom = &moved
}
if err := v.Store.Write(rec); err != nil {
return Changed{}, err
@@ -176,7 +223,20 @@ func (v *Vault) Get(caller, name, publicKey string) (Answer, error) {
if err := validName(name); err != nil {
return Answer{}, err
}
if ok, why := askers.May(caller, name); !ok {
// A value issued for one holder is that holder's alone, whatever else would allow it (issue 749).
if why := notTheHolder(caller, name); why != "" {
v.log("refused %s to %s: %s", name, caller, why)
_ = v.recordHanding(Handing{To: caller, Name: name, Outcome: "refused", Why: why})
return Answer{}, fmt.Errorf("%s may not ask for %s: %s", caller, name, why)
}
// The core's own credentials, to the controller alone, sealed only to their machine's key (ADR 0349 §16).
if caller == ControllerCaller {
if why := v.coreRefusal(name, publicKey); why != "" {
v.log("refused %s to %s: %s", name, caller, why)
_ = v.recordHanding(Handing{To: caller, Name: name, Outcome: "refused", Why: why})
return Answer{}, fmt.Errorf("%s may not ask for %s: %s", caller, name, why)
}
} else if ok, why := askers.May(caller, name); !ok {
v.log("refused %s to %s: %s", name, caller, why)
_ = v.recordHanding(Handing{To: caller, Name: name, Outcome: "refused", Why: why})
return Answer{}, fmt.Errorf("%s may not ask for %s: %s", caller, name, why)
modules/mesh-vault/cmd/mesh-vault/vault_test.go
+141 -7
View File
@@ -129,7 +129,8 @@ func filesUnder(t *testing.T, dir string) map[string]string {
// ---- no bucket holds a secret ---------------------------------------------------------------------------
// The manifest declares no bucket, and the code opens none: the vault's only place for a secret is its own
// store on its own disk (ADR 0349 §7, ADR 0201 §13).
// store on its own disk (ADR 0349 §7, ADR 0201 §13). It reads one bucket of another's, and only reads it: the
// controller's assignments, for what each node's plan names (askers.go, design 57 V4).
func TestNoBucketHoldsASecret(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
@@ -142,6 +143,9 @@ func TestNoBucketHoldsASecret(t *testing.T) {
if s, ok := manifest["state"]; ok {
t.Errorf("the manifest declares buckets %v: a secret is in no bucket, sealed or not (ADR 0201 §13)", s)
}
if r, _ := json.Marshal(manifest["reads"]); string(r) != `["mesh-controller.assignments"]` {
t.Errorf("the manifest reads %s: the vault reads the controller's assignments alone", r)
}
sources, _ := filepath.Glob("*.go")
bucket := regexp.MustCompile(`stdio\.State\(|nats-io|jetstream|KeyValue`)
for _, f := range sources {
@@ -149,6 +153,10 @@ func TestNoBucketHoldsASecret(t *testing.T) {
continue
}
code, _ := os.ReadFile(f)
// The reads it makes, and nothing that writes: Get and Keys alone, on the assignments (Keys for the
// binds it remakes a provider's credential on, pair.go).
code = []byte(strings.Replace(string(code), "stdio.State(AssignmentsRead).Get(", "", 1))
code = []byte(strings.Replace(string(code), "stdio.State(AssignmentsRead).Keys(", "", 1))
if m := bucket.Find(code); m != nil {
t.Errorf("%s opens a bucket (%s): the vault keeps every secret in its own store only", f, m)
}
@@ -367,17 +375,143 @@ func TestGetAnswersOnlyAnAllowedAskerSealedToItsKey(t *testing.T) {
}
}
// Until who may ask is decided by a node's declaration and a module's own account (V4, P3), the vault
// that runs answers nobody.
func TestTheRunningVaultAnswersNobodyYet(t *testing.T) {
v := newVault(t, nil) // what main builds: no list
// A vault given no rule answers nobody.
func TestAVaultWithNoRuleAnswersNobody(t *testing.T) {
v := newVault(t, nil)
if _, err := v.Put("pair/postgres/gitea", "x", "the test"); err != nil {
t.Fatal(err)
}
asker, _ := seal.GenerateKeyPair()
for _, caller := range []string{"g14.node-provisioner", "novox.mesh-provisioner", "controller", "person.jochen", "novox.node-tools"} {
for _, caller := range []string{"provisioner.g14", "novox.mesh-provisioner", "controller", "person.jochen", "novox.node-tools"} {
if _, err := v.Get(caller, "pair/postgres/gitea", asker.PublicKey); err == nil {
t.Errorf("the running vault answered %s", caller)
t.Errorf("a vault with no rule answered %s", caller)
}
}
}
// assignments stands in for the controller's assignments mirror as the vault reads it.
type assignments map[string]string
func (a assignments) read(key string) ([]byte, error) {
if v, ok := a[key]; ok {
return []byte(v), nil
}
return nil, nil
}
// The running vault's rule (ADR 0349 §11, design 57 V4): a node's provisioner client, under its own account, for a
// secret of a module on its node that the module's plan there names, and nobody else for anything else. Each
// refusal is recorded with why.
func TestTheRunningVaultAnswersAMachinesClientForWhatItsPlanNames(t *testing.T) {
mirror := assignments{
"g14.dunst": `{"node":"g14","module":"dunst","assigned":"2026-10-14T20:00:00Z","provisioning":` +
`{"templates":{"source":"x","digest":"y"},"files":[],"secrets":[{"name":"token"}]}}`,
"g14.picom": `{"node":"g14","module":"picom","assigned":"2026-10-14T20:00:00Z"}`,
"ace.dunst": `{"node":"ace","module":"dunst","assigned":"2026-10-14T20:00:00Z","provisioning":{"secrets":[{"name":"token"}]}}`,
"g14.liar": `{"node":"ace","module":"liar","provisioning":{"secrets":[{"name":"token"}]}}`,
"g14.unparsed": `not json`,
}
v := newVault(t, Declared{Read: mirror.read})
for _, name := range []string{"own/g14/dunst/token", "own/g14/dunst/other", "own/g14/picom/token",
"own/ace/dunst/token", "own/g14/liar/token", "own/g14/unparsed/token", "provision/novox/postgres",
"own/g14/absent/token"} {
if _, err := v.Put(name, "value-of-"+name, "the test"); err != nil {
t.Fatal(err)
}
}
asker, _ := seal.GenerateKeyPair()
answer, err := v.Get("provisioner.g14", "own/g14/dunst/token", asker.PublicKey)
if err != nil {
t.Fatalf("the machine's client was refused the secret its module's plan names: %v", err)
}
if got, err := seal.Open(answer.Sealed, asker.PrivateKey); err != nil || got != "value-of-own/g14/dunst/token" {
t.Fatalf("the answer opens to %q (%v)", got, err)
}
refusals := []struct{ caller, name, why string }{
{"provisioner.g14", "own/g14/dunst/other", "a secret the plan does not name"},
{"provisioner.g14", "own/g14/picom/token", "a module not handed to the provisioner"},
{"provisioner.g14", "own/ace/dunst/token", "another machine's secret"},
{"provisioner.ace", "own/g14/dunst/token", "another machine's client"},
{"provisioner.g14", "own/g14/absent/token", "a module not assigned there"},
{"provisioner.g14", "own/g14/liar/token", "an assignment whose value names another machine"},
{"provisioner.g14", "own/g14/unparsed/token", "an assignment that is not the controller's"},
{"provisioner.g14", "provision/novox/postgres", "a provision's credential the vault made for nobody there"},
{"g14.node-tools", "own/g14/dunst/token", "the tool runner's account, which an agent shares"},
{"g14.dunst", "own/g14/dunst/token", "a module's own account, until issue 569"},
{"controller", "own/g14/dunst/token", "the controller"},
{"provisioner.", "own/g14/dunst/token", "a client naming no machine"},
}
for _, r := range refusals {
if a, err := v.Get(r.caller, r.name, asker.PublicKey); err == nil {
t.Errorf("get answered %s (%s asking %s): %+v", r.why, r.caller, r.name, a)
}
}
handed, err := v.Store.Handed(100)
if err != nil {
t.Fatal(err)
}
refused := 0
for _, h := range handed {
if h.Outcome == "refused" {
if h.Why == "" {
t.Errorf("a refusal of %s to %s is recorded without why", h.Name, h.To)
}
refused++
}
}
// Every refusal but the last, whose caller is not a bus user's shape and so fills no record (vault.go).
if refused != len(refusals)-1 {
t.Errorf("%d refusals recorded, want %d", refused, len(refusals)-1)
}
}
// A value a provider's software issued for one holder (issue 757): the running rule hands
// provision/<provider>/<provision>/<node>/<module> to that node's client when the module's plan there names the
// provision from that provider, and refuses another node's, another module's, another provider's and the provision's
// one credential.
func TestTheRunningVaultAnswersAMachinesClientForAnIssuedValueItsPlanNames(t *testing.T) {
plan := `{"templates":{"source":"x","digest":"y"},"files":[],` +
`"provisions":[{"name":"gitea-read","provision":"gitea-read","provider":"novox","path":"/s/gitea-read.secret"}]}`
mirror := assignments{
"anchor.claude-code": `{"node":"anchor","module":"claude-code","provisioning":` + plan + `}`,
"laptop.claude-code": `{"node":"laptop","module":"claude-code","provisioning":` + plan + `}`,
"anchor.zsh": `{"node":"anchor","module":"zsh","provisioning":{"secrets":[{"name":"token"}]}}`,
"anchor.picom": `{"node":"anchor","module":"picom"}`,
}
v := newVault(t, Declared{Read: mirror.read})
for _, name := range []string{anchorsToken, laptopsToken, "provision/novox/gitea-read/anchor/zsh",
"provision/novox/gitea-read/anchor/picom", "provision/ace/gitea-read/anchor/claude-code",
"provision/novox/other/anchor/claude-code", "provision/novox/gitea-read",
"provision/novox/gitea-read/anchor/absent"} {
if _, err := v.Put(name, "value-of-"+name, "the test"); err != nil {
t.Fatal(err)
}
}
asker, _ := seal.GenerateKeyPair()
answer, err := v.Get("provisioner.anchor", anchorsToken, asker.PublicKey)
if err != nil {
t.Fatalf("the machine's client was refused the value issued for its module, which its plan names: %v", err)
}
if got, err := seal.Open(answer.Sealed, asker.PrivateKey); err != nil || got != "value-of-"+anchorsToken {
t.Fatalf("the answer opens to %q (%v)", got, err)
}
refusals := []struct{ caller, name, why string }{
{"provisioner.anchor", laptopsToken, "another machine's holder value"},
{"provisioner.laptop", anchorsToken, "another machine's holder value, the other way"},
{"provisioner.anchor", "provision/novox/gitea-read/anchor/zsh", "another module's, whose plan names it not"},
{"provisioner.anchor", "provision/novox/gitea-read/anchor/picom", "a module not handed to the provisioner"},
{"provisioner.anchor", "provision/novox/gitea-read/anchor/absent", "a module not assigned there"},
{"provisioner.anchor", "provision/ace/gitea-read/anchor/claude-code", "another provider's"},
{"provisioner.anchor", "provision/novox/other/anchor/claude-code", "a provision the plan names not"},
{"provisioner.anchor", "provision/novox/gitea-read", "the provision's one credential"},
{"anchor.node-tools", anchorsToken, "the tool runner's account, which an agent shares"},
{"anchor.claude-code", anchorsToken, "the module's own account, until issue 569"},
}
for _, r := range refusals {
if a, err := v.Get(r.caller, r.name, asker.PublicKey); err == nil {
t.Errorf("get answered %s (%s asking %s): %+v", r.why, r.caller, r.name, a)
}
}
}
modules/mesh-vault/go.mod
+13 -1
View File
@@ -4,7 +4,19 @@ go 1.25.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.51.0
golang.org/x/crypto v0.50.0
)
require golang.org/x/sys v0.43.0 // indirect
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/time v0.15.0 // indirect
)
modules/mesh-vault/go.sum
+21
View File
@@ -1,6 +1,27 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009 h1:I9rm8HhYRJ0+mzQQz+VAH0kgv2YJWkvctEc67Yd0PX8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.51.0 h1:ByW84XTz6W03GSSsygsZcA+xgKK8vPGaa/FCAAEHnAI=
github.com/nats-io/nats.go v1.51.0/go.mod h1:26HypzazeOkyO3/mqd1zZd53STJN0EjCYF9Uy2ZOBno=
github.com/nats-io/nkeys v0.4.15 h1:JACV5jRVO9V856KOapQ7x+EY8Jo3qw1vJt/9Jpwzkk4=
github.com/nats-io/nkeys v0.4.15/go.mod h1:CpMchTXC9fxA5zrMo4KpySxNjiDVvr8ANOSZdiNfUrs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
modules/mesh-vault/module.json
+43 -6
View File
@@ -1,6 +1,7 @@
{
"module": "mesh-vault",
"version": "1",
"runs-as": "mesh-vault",
"provides": [
{
"name": "secret",
@@ -15,6 +16,9 @@
"rotated",
"secret-changed"
],
"reads": [
"mesh-controller.assignments"
],
"receives": {
"secret": "${dir:grants}/mesh.json"
},
@@ -57,32 +61,56 @@
}
}
},
"own-secrets": {
"broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-vault",
"runtime-health": {
"kind": "tool",
"tool": "vault_health",
"interval": "60s",
"timeout": "10s",
"looks": 2,
"grace": "60s"
},
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-vault",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-vault"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-vault",
"place": "."
},
{
"id": "grants",
"type": "directory",
"mode": "0700"
"mode": "0700",
"owner": "mesh-vault"
},
{
"id": "ledger",
"type": "directory",
"mode": "0700"
"mode": "0700",
"owner": "mesh-vault"
},
{
"id": "root",
"type": "directory",
"mode": "0700"
"mode": "0700",
"owner": "mesh-vault"
},
{
"id": "store",
"type": "directory",
"mode": "0700"
"mode": "0700",
"owner": "mesh-vault"
}
],
"build": {
@@ -112,19 +140,28 @@
"secret_holders",
"secret_verify",
"secret_handed",
"secret_asks",
"secret_vault_export",
"provisioner_retirement",
"provisioner_retire_approve",
"provisioner_retire_reject",
"provisioner_delete",
"vault_status"
"vault_status",
"vault_health"
],
"claims": [
{
"name": "mesh-vault",
"scope": "mesh",
"serves": [
"get"
"get",
"accept",
"secret-ask",
"issued",
"rotate",
"move-in",
"node-key",
"move-asks"
]
}
]