280 lines
21 KiB
Markdown
280 lines
21 KiB
Markdown
# mesh-vault
|
||
|
||
Holds the `mesh-vault` seat: the mesh's shared secrets, each kept sealed, handed out only on request and
|
||
reply, sealed to whoever asked (novox/hq ADR 0085, ADR 0113, ADR 0349 §7 and §11). Go, one binary
|
||
(`cmd/mesh-vault`), launched by its node's runtime and speaking MCP to it through the Go SDK.
|
||
|
||
## Where the secrets are kept, and why there
|
||
|
||
In the vault's own store: one file per secret in `${dir:store}` on the vault's own disk, mode 0600 in a
|
||
0700 directory. Not a database, as most Go modules keep their data, because:
|
||
|
||
- the vault is installed before any other module built on the runtime base (ADR 0113 §14), and a database
|
||
login is itself a shared secret the vault makes — the vault cannot need one to start;
|
||
- every value in the store is ciphertext anyway, so a database would add nothing a file does not;
|
||
- no secret is in a bucket, sealed or not (ADR 0201 §13), so the module declares no `state`.
|
||
|
||
Each record holds the value twice, never in the clear:
|
||
|
||
- **at rest**, sealed to the vault's own key (the sealed box of ADR 0206, `mesh-sdk/go/seal`). The key is
|
||
an X25519 pair the vault makes once in `${dir:state}/vault-key.json`; its private half never leaves it.
|
||
The key is made only for an empty store: with the key file missing and secrets in the store, the vault
|
||
does not start and says so, rather than make a key that opens none of them.
|
||
- **for the operator**, sealed to the operator key (ADR 0113 §22), so nothing exists only inside the
|
||
vault. A secret is refused, loudly, while the vault has no operator key.
|
||
|
||
Every file is written whole, synced, renamed into place and its directory synced.
|
||
|
||
### What the encryption at rest protects against, and what not
|
||
|
||
The vault's private key lives in plaintext in `${dir:state}`, on the same disk as the store, readable by the
|
||
vault's account alone. So:
|
||
|
||
- **It protects** against a copy of the store directory alone — a `store` backup, a file sent somewhere,
|
||
a listing of `${dir:store}` by a tool that reads files — holding anything usable, and against a value
|
||
being found by searching the disk for it.
|
||
- **It does not protect** against anyone who can read the vault's state directory too: root on the
|
||
vault's node, the vault's own account, or a backup that holds both `state` and `store` (both are
|
||
class `valuable`, so the node's backup holds both). Against those, at-rest encryption adds nothing; the
|
||
protection is the node's own security. Keeping the key elsewhere (another directory class, a key the
|
||
operator unlocks) is for V3's review, before real secrets move in.
|
||
|
||
## Where the operator key comes from
|
||
|
||
From the export the mesh writes to the vault's root directory (`${dir:root}/export.json`, the manifest's
|
||
`keeps`): its `operator-key` is the public half `operator key set` recorded in the controller. The vault
|
||
checks it every minute; a new key makes the vault seal every operator copy again to it — it can, because
|
||
it opens its own copy. Each record names the key its copy is sealed to, so a failure partway is finished on
|
||
the next minute's pass, and nothing is left sealed to an earlier key for good.
|
||
|
||
The vault takes an operator key only as SPKI PEM (`SetOperatorKey`). A PEM private key is refused, and so
|
||
is a raw 32-byte key, because a raw private key looks the same. The one raw key it reads is the export's,
|
||
which the controller writes from what it recorded; the vault wraps it as SPKI before taking it. It keeps
|
||
the key raw in base64, the form of the operator's NaCl box. When the secrets move to the vault (design 57's
|
||
V3), the operator key moves with them.
|
||
|
||
### How the operator opens a copy
|
||
|
||
`mesh-vault.secret_vault_export` answers the vault's copies in the shape `secret export` writes. Saved to a
|
||
file, one is opened with the operator's private key by
|
||
|
||
nox secret recover vault mesh-vault <name> --key <operator key file> --from-export <file> --out <file>
|
||
|
||
(`--out` is needed: a name holds `/`). `TestTheOperatorRecoversAVaultSecretFromItsExport` reads the file
|
||
as that command does and opens it as it does. `secret_export` still answers the controller's export, as
|
||
before; until V3 the controller's secrets are there.
|
||
|
||
## The boxes and the key formats — settled
|
||
|
||
Two boxes on the mesh today, both over X25519:
|
||
|
||
| | format | keys | who uses it |
|
||
|---|---|---|---|
|
||
| NaCl anonymous box | XSalsa20-Poly1305, `box.SealAnonymous` | raw 32 bytes, base64 | the controller sealing into declarations; the node-engine's `sealing.key` (mesh-host `internal/identity/sealing.go`); the operator key and `secret recover --from-export` |
|
||
| the sealed box (ADR 0206) | X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}` | PEM: SPKI public, PKCS #8 private | the licence manager and `claude-code`; `mesh-sdk/go/seal` |
|
||
|
||
What the vault uses:
|
||
|
||
- **Its answers to `get` are the ADR 0206 box, for every asker, a node included.** The asker sends its
|
||
public key as SPKI PEM. A node's provisioner client (V4) asks with a key of its own, made when it starts
|
||
and kept in its memory alone: it runs under an account of its own and cannot read the node-engine's sealing
|
||
key, which is root's. A node's sealing key would open an answer too, being X25519 (wrapped as SPKI and PKCS #8,
|
||
`TestANodesSealingKeyOpensTheAnswer`), for the node-engine's own asks. A raw 32-byte key is refused: a raw
|
||
private key looks the same, and the vault takes no private key.
|
||
- **Its operator copies are the NaCl anonymous box**, to the operator key as it is made today, so the
|
||
operator's recovery opens them with the key and the command it already has.
|
||
- **The at-rest copy is the ADR 0206 box**, to the vault's own key.
|
||
|
||
The NaCl box stays on the node-engine only for what the controller seals into declarations, until the last
|
||
module has moved (ADR 0349 §17). For the core's own credentials that travel in a declaration sealed by the
|
||
vault (ADR 0349 §16), the node-engine is to open the ADR 0206 box with the same sealing key; that is not built
|
||
yet (issue 647 leaves it to its own issue).
|
||
|
||
## The seat's verb
|
||
|
||
`mesh-vault.get` — `name`, `public_key` (SPKI PEM). It answers `{name, generation, changed-at, sealed}`,
|
||
`sealed` being the secret sealed to that key, and only to an asker allowed for that secret, the asker being
|
||
the bus user the runtime proved — never an argument.
|
||
|
||
Every answer and every refusal is recorded in the vault's own store (`handed.jsonl`: who, which secret,
|
||
which generation, when, and why a refusal), synced before the answer is given — as ADR 0277 §25 records
|
||
asks; a secret whose hand-out cannot be recorded is not handed out. A refusal is recorded only for a caller
|
||
of a bus user's shape and a well-formed secret name, so an arbitrary argument fills nothing. The record is
|
||
bounded: rolled at 1 MiB into `handed.jsonl.1` … `.3`, the oldest dropped, each roll said loudly in the
|
||
journal and counted in `vault_status`. `secret_handed` reads it.
|
||
|
||
`vault_status` says what needs a person: operator copies still sealed to an earlier key, an operator key in
|
||
the export the vault refused, and how often the record was rolled.
|
||
|
||
**A failing re-seal is a condition** (ADR 0366, issue 738). The vault re-seals its operator copies to the
|
||
operator key once a minute. `vault_health` answers unhealthy once that has failed on three passes in a row,
|
||
saying what failed and why, and healthy again after one pass succeeds. The manifest declares it as the
|
||
health of the vault's own runtime (`runtime-health`), which the controller puts on `mesh-vault.own-runtime`;
|
||
the node-engine asks it every minute and the controller raises `module.mesh-vault.<machine>.unhealthy` from
|
||
it, cleared on the first healthy answer. An operator key in the export the vault refuses fails the pass too.
|
||
How often the record was rolled still reaches a person through the journal and `vault_status` only.
|
||
|
||
**Who may ask: a node's provisioner client, for what its plan names** (ADR 0349 §11, design 57 V4, issue 647;
|
||
`askers.go`). The caller is `provisioner.<node>`, the client's own account (never the tool runner's, which an agent
|
||
on that machine shares), and the secret is `own/<node>/<module>/<name>` for that same machine, where the
|
||
controller's assignment `<node>.<module>` (the assignments mirror, which the manifest `reads`) carries the
|
||
provisioner's plan and the plan's `secrets` name it; or a value a provider's software issued for one holder,
|
||
`provision/<provider>/<provision>/<node>/<module>`, where that plan's `provisions` name the provision from that
|
||
provider with a credential's `name` (issue 757; an entry naming only the provider serving a provision, issue 760,
|
||
hands nothing). The vault cannot read a node's declaration, so the plan the
|
||
controller carries beside it is what the declaration names, as the vault reads it. A provision's one credential,
|
||
`provision/<provider>/<provision>`, is handed to the machines the vault made it for (below) while the holder there
|
||
is handed to the provisioner: the provider's module, or a consumer whose plan still names it from that provider. A
|
||
module not handed to the provisioner, another machine's secret or holder value, and every other caller are refused,
|
||
and each refusal is recorded with why. A module's own account (issue 569) is not answered yet.
|
||
|
||
**The controller, for the core's own credentials, sealed only to their machine** (ADR 0349 §16, issue 763;
|
||
`core.go`). The bus and mesh-provisioner need their files before any client runs, so their credentials travel in
|
||
the declaration sealed by the vault to their machine, which the controller carries and cannot open. The caller
|
||
`controller` is answered `own/<node>/<module>/<name>` only for a module on the core's fixed list (`nats`,
|
||
`mesh-provisioner`, `mesh-provisioner-client`, the controller's list too), and only when the key it names is that
|
||
machine's sealing key as its own node-engine stated it in its own name, in `move-in`'s first step (`sealing_key`).
|
||
A machine whose engine stated none, any other key, any other module and any other name are refused and recorded.
|
||
|
||
## Taking values in: `accept`, `secret-ask`, `issued`, `rotate`
|
||
|
||
Design 57's V2 (issue 675; ADR 0349 clauses 8 to 10, ADR 0277 §22–§27 and ADR 0305 with the vault in the
|
||
controller's place). `intake.go`.
|
||
|
||
**From the controller alone; the vault asks the desk and the provider itself.** The controller alone asks these
|
||
verbs, as the runtime proves its caller. Each value comes in sealed to an X25519 key the vault made for that one
|
||
call (raw, base64 — the NaCl anonymous box the desk's prompt and a provider's tool already seal to), and opens with
|
||
the private half, which lives in this process only and is forgotten once used or after two minutes. A value sealed
|
||
to any other key is refused, and a call is answered once.
|
||
|
||
**The vault's own account** (issue 723). The vault runs as `mesh-vault` in a runtime of its own (`runs-as`, ADR
|
||
0259 §8), on a bus account of its own (`<node>.mesh-vault`, from its own secret `broker`). The bus lets that
|
||
account call the desks' prompt (`node-launcher.secret` on any machine) and each issuing provider's tool in its own
|
||
name, and nothing else outside its seat (mesh-controller's `internal/broker` grants, proven on a real server). So
|
||
the prompt and the tool know the vault asks, and the controller carries neither the key nor the sealed value. The
|
||
controller still asks them itself for a module filled the old way, until R1 (issue 682) removes that path; they
|
||
answer both until then. Everything the vault keeps is its account's: its directories, the export it keeps, the
|
||
grant files the controller still writes for it.
|
||
|
||
- `accept` — two steps: node, module, name (a module's own secret) or provider, provision (a provision's one
|
||
credential), and `by`; then `call` and `sealed`. The controller's terminal seals the value it read to the call's
|
||
key before it leaves the terminal. Kept with origin `accepted`; an accepted provision credential is never remade
|
||
(`rotate` refuses it); a value a provider's software issued is not taken by `accept`.
|
||
- `secret-ask` — one step: node, module, name, desk and `asked_by` (who asked the controller, as the bus named its
|
||
caller). The ask is recorded in `asks.jsonl` before anything else, and refused while another ask for the same
|
||
secret is open (two minutes at most) or when three were made in the last hour. The vault then asks the desk's
|
||
prompt itself, naming who asked as the record has it, with `timeout_seconds` 20. The answer is how it ended
|
||
(`given`, dismissed, timed-out, empty, refused, failed) and who asked; how each ask ended is recorded.
|
||
`secret_asks` reads the record. The hand-act log stays the controller's: it records the act once the vault has
|
||
taken the value.
|
||
- `issued` — one step: provision, provider (machine), module and tool. The vault asks that tool on the provider's
|
||
machine itself; the value is kept as the provision's one credential with origin `issued`. The same value again
|
||
writes nothing, says nothing and keeps the generation (ADR 0305 §5).
|
||
- `issued` for each holder apart (issue 749; ADR 0349 clause 11, ADR 0365 clause 4: the forge's read token per agent
|
||
account). The controller names the provision's holders from that provider (`holders`, each a node and a consuming
|
||
module), and the vault asks the tool with them. The tool answers either one value (`sealed`, kept as above) or one
|
||
per holder (`holders`: node, module and `sealed` each, all sealed to the call's key). Each is kept as
|
||
`provision/<provider node>/<provision>/<holder node>/<holder module>`, origin `issued`, and the same value again for
|
||
a holder writes nothing for it. The answer is taken whole or not at all: a holder not asked about, one named twice, a
|
||
value not sealed to the call or an empty one keeps nothing. A holder asked about and not answered is said
|
||
(`not_answered`) and keeps nothing. **`get` hands a holder's value to that holder alone**, its machine's provisioner
|
||
client (`provisioner.<node>`) or the module under an account of its own (`<node>.<module>`), whatever the rule for
|
||
who may ask allows; every other caller is refused and the refusal recorded.
|
||
`TestOnARealBusTwoHoldersOnTwoMachinesEachGetOnlyTheirOwnIssuedValueAndNoneCrossesInTheClear` proves it on a NATS
|
||
server with a stand-in forge.
|
||
- `rotate` — a module's own secret or a provision's credential, made anew. Refused for an `issued` value (the
|
||
software rotates it, ADR 0305 §9) and an accepted provision credential (clause 8). From the controller alone,
|
||
which knows whether the module's secrets are filled from the vault yet, and with a `why`, said in the vault's
|
||
log. Who asked is the `by` the controller names, beside the controller; both are kept (`changed-by`,
|
||
`changed-via`). The controller's `rotate <provision>` forwards here for a provider filled the new way (issue
|
||
767): a shared value is made anew for every holder, never for one consumer alone; for a provider whose consumers
|
||
keep a login each, `consumer` and `consumer_module` (`--consumer`, `--module`) name the one consumer whose own
|
||
login is remade, and no other is touched.
|
||
|
||
Names in the store: `own/<node>/<module>/<name>`, `provision/<provider node>/<provision>` and, for a value issued for
|
||
one holder, `provision/<provider node>/<provision>/<holder node>/<holder module>`. V3 copies the
|
||
controller's rows in under these names.
|
||
|
||
**Which modules reach the vault this way.** The controller sends a module's verbs here only when its secrets are
|
||
filled the new way (its machine's provisioner client asks the vault for them, V4); every other module keeps the
|
||
controller's path, so no value lands where its machine never looks (ADR 0349 clause 17; the release lead's decision
|
||
on issue 675).
|
||
|
||
**What the issuing provider's test is here.** `TestAnIssuingProviderAnswersOnlyTheVaultsCallSealedToTheVault` and
|
||
`TestTheVaultAsksTheIssuingToolItselfAndOnlyASealedValueCrosses` use a stand-in provider that answers the vault's
|
||
account (and, until R1, the controller) and no other caller, and only sealed to the call's key. plex's own tool
|
||
answers the controller alone today; it must also answer `<node>.mesh-vault` when it moves with the media catalogue,
|
||
after release 57.
|
||
|
||
## Events
|
||
|
||
- `secret-changed` — `name`, `by`, `at`, `generation`: which secret, who changed it, when. Never a value
|
||
or a sealed copy.
|
||
- `provisioned`, `rotated` — the custody ledger's, by fingerprint, as before.
|
||
- `provisioner.retirement` (ADR 0230 §35) — `waiting`, `settled`, `approved`, `rejected`, `retired`,
|
||
`reenabled`, `deleted`, naming holders, never a value.
|
||
|
||
## The tools
|
||
|
||
`secret_export`, `secret_holders`, `secret_verify` answer as the TypeScript vault's did, and its custody
|
||
ledger (`${dir:ledger}`, the same JSON files) is kept by a pass over the contributions file every minute.
|
||
`secret_handed` and `secret_vault_export` read the vault's own store. None returns a value.
|
||
|
||
**The ledger follows ADR 0230.** A contributions file that does not say who is asked for (unreadable, not
|
||
JSON, no `given` list or a null one, or for another requirement) is not "nobody": the pass records nothing,
|
||
retires nobody and starts its count again. A holder no longer asked for is retired — a mark with when and
|
||
why in its own ledger file, its history kept — only after the same set has gone unasked in five passes and
|
||
ten minutes; more than three at once, or more than half of those held, waits for a person
|
||
(`provisioner_retire_approve`, `provisioner_retire_reject`); asked for again it is re-enabled; it is deleted
|
||
only by `provisioner_delete`, a person's act with a why. `provisioner_retirement` shows it all. The vault is
|
||
mark-only: it holds no access to switch off.
|
||
|
||
The words are the controller's: the `provisioner.retirement` event is mesh-controller's `link.Retirement`
|
||
(`provider-node`, `consumers` as objects) and the `provisioner_retirement` answer its `link.RetirementState`
|
||
(`held`, `bound`, `retires_by: mark-only`), so `retire approve`, `retire reject` and `cleanup list` read the
|
||
vault and a set waiting for a person raises the controller's urgent condition. The acts take what the SDK's
|
||
harness takes: approve and reject repeat the set in `consumers` and refuse any other set; delete repeats the
|
||
name in `confirm`; each records the `by` and `via` the controller sends. `testdata/retirement-*.json` are
|
||
those types marshalled by the controller (feat/57 at dc36536), and the tests hold the vault's words to them.
|
||
|
||
What the TypeScript vault did and this one does not: it consumed its own events only to print an audit line,
|
||
which the pass now prints itself. Its retirement ran on the SDK's harness, which this one follows: mark-only,
|
||
and a holder's ledger file removed only through a person's delete, as here. The difference is where the mark
|
||
lives: the harness kept it in its process, so a restart forgot it (ADR 0230 §13); here it is in the ledger
|
||
file.
|
||
|
||
**The credential a provider shares with its consumers** (ADR 0349 clause 8, ADR 0158 §4–§6, design 57 V5, issue
|
||
677; `pair.go`). One value per provider assignment and provision, `provision/<provider>/<provision>`. Once a minute
|
||
the vault reads every plan in the assignments mirror; each plan's `provisions` names the provider serving each
|
||
provision its module requires (issue 760). A consumer binds when an entry appears and unbinds when it goes or its
|
||
assignment goes; on either, the vault makes one new value for every holder together and says `secret-changed`, and
|
||
each holder's client asks again. What the move brought in (V3) is adopted when every pair row of a provider and
|
||
provision shares one generation stamp and one value (ADR 0158's shared form), with its origin, and nothing is
|
||
remade. Rows without a stamp (ADR 0048, a login per consumer) are never made one value (ADR 0158 §11): `rotate` of
|
||
the provision remakes each consumer's own apart, and each consumer's machine is handed only its own row,
|
||
`pair/<provider>/<provision>/<consumer>/<module>[/<local>]` (the provider's machine all of them). A provider the
|
||
vault holds no row of gets its first value in the form each plan entry names (`credential-form`, from the provider's
|
||
offer, issue 768): `shared`, one value for every holder; `pair`, a login of its own for each consumer bound; `issued`
|
||
or `given` are never made. No form named, plans that disagree, or a form it does not know: nothing is made. An accepted or issued value is never remade, only
|
||
its holders recorded. A provision whose plans name a credential per holder (issue 757) has no shared value. **No
|
||
value is made that a holder would never get:** while the provider's assignment, or a consumer the move brought in,
|
||
is still filled the old way (no plan), every make, remake and rotation is held back and the reason said in the
|
||
log. Who holds each value is kept in the store's `shared/` directory, never a value.
|
||
|
||
## What comes later
|
||
|
||
Taking values in is built (above); the controller's secrets move in with V3; nodes' provisioner clients ask with V4,
|
||
for each module handed to the provisioner; providers answer their consumers with B1–B3.
|
||
|
||
## Tests
|
||
|
||
go test ./...
|
||
|
||
`vault_test.go` is ADR 0349's vault test list as far as this package goes: no bucket holds a secret; no
|
||
private key is accepted; every secret has an operator-sealed copy; `get` answers only an allowed asker,
|
||
sealed to its key; `secret-changed` carries no value. `intake_test.go` is ADR 0349's `secret-ask` test as far as
|
||
the vault answers it (recorded before the prompt, one open ask per secret and three an hour, the asker named as
|
||
the bus named it, the typed value opened only here), the issuing provider's test with a stand-in provider (ADR 0305
|
||
§13's checks that belong to the vault: nothing rewritten for a value held, `rotate` refused, no other way in), and
|
||
accept and rotate. `followups_test.go` holds V1's review follow-ups.
|