---
status: resolved
opened: 2026-09-04
located-in: [mesh-sdk, mesh-catalog]
fixed-by: mesh-sdk src/provisioner rework + redis/postgres/minio/umami adapters (ADR 0048)
amended-design: 0048-a-provider-creates-the-credential-the-mesh-minted.md
---

# A provider's provisioner seals with a key the mesh has no way to deliver — and does not need to

## What was observed

Building the vertical slice for the module runtime (the module runs its own code as its own
process under its own account), a **provider** module — one that stands up a per-consumer
resource and hands back a credential — was assigned to a node and run as a broker-bound
runtime. The runtime hosts the module's provisioner (the sdk's `runProvisioner`), and the
harness opens by reading a **seal key** from `$MESH_SEAL_KEY`, failing immediately without
one. Every credential it produces for a consumer is sealed to that key with the sdk's
symmetric `seal()` (AES-256-GCM, `mesh-sdk/src/primitives/index.ts`) before being written.

Nothing in the mesh sets `$MESH_SEAL_KEY`. It is read in exactly two places in the sdk and
set nowhere — no manifest, no control-plane code, no host code. So a provider runtime, as
delivered, aborts at start-up. The slice proved the mechanism only by setting a lab-local key
in the manifest by hand.

## What a trace of the credential path turned up

The seal key is not a missing delivery. **The whole symmetric-seal provisioner is orphaned,
and it duplicates — badly — a job the mesh already does.**

- `runProvisioner` reads request files named `*.grant.json`. **Nothing writes those.**
- It writes sealed credential files named `<consumer>.<resource>.credential`. **Nothing reads
  those** — not the host, not the control plane. The host reports applied-resource digests
  upward and never ships credentials; the control plane has no reference to that filename.
- No consumer ever calls the symmetric `unseal()`. Consumers receive **plaintext**.

Meanwhile the mesh already carries a provider→consumer credential across nodes, with **no
shared key anywhere**:

- The control plane mints the password once (`secrets.Make`) and seals it **twice,
  asymmetrically** — `ForConsumer` to the consumer node's X25519 public key, `ForProvider` to
  the provider node's (`mesh-control/internal/secrets/seal.go`, `mesh-host/internal/identity/
  sealing.go`, NaCl box).
- Each host opens its own copy with its own private key on the machine; the plaintext exists
  only for the length of one function call (`mesh-host/internal/apply/apply.go`, the
  `${secret:name}` substitution — ADR 0024's "the host is the only thing that ever holds
  both").
- `serves` carries no credential and says so; `receives`/`bound` tell each side *where* its
  sealed secret is, never the value.

The two models also **contradict** each other. The sdk's `seal()` comment says the key is "a
per-node passphrase the host holds"; the host holds no such passphrase — it holds an X25519
private key, and the control plane's own code refuses a shared symmetric key on principle:
"a key both ends hold is a key the mesh would have to distribute, which is this problem again
one level down" (`secrets/seal.go`). A symmetric `MESH_SEAL_KEY` shared between a provider
node and a consumer node is exactly the thing the mesh was built not to have.

And the provisioner's model is wrong in a second way: its adapter **generates its own
password** (`generatePassword()`) and creates the resource with it — a different password from
the one the mesh mints and hands the consumer. Even with a seal key delivered, a consumer
would authenticate with the mesh's password against a resource created with the provisioner's.

## Why it matters beyond this instance

This is not a four-module problem. The provider contract lives in **one place** — the sdk's
`runProvisioner(resource, adapter)` harness — and every provider is built on it. Four exist
today (redis, postgres, minio, umami); a mesh of any size ends up with many. Whatever the
provisioner harness does, every present and future provider inherits, so the orphaned
symmetric seal is a fault stamped into the interface, not into four adapters. That also sets
the cost of getting it wrong: a contract N providers depend on is N migrations to change
later, which is the argument for settling it deliberately now rather than patching around it.

As written, each provider carries a provisioner that cannot start (no key), and that, if it
did, would create resources with a password it invented — a *different* password from the one
the mesh minted and handed the consumer — and seal them for a reader that does not exist. The
rule the design states, "a consumer receives a sealed credential and unseals it," is enforced
by nothing: no consumer unseals, and no shared key exists to unseal with.

## The mesh already does this — confirmed

The premise the fix rests on is not a hope; it is in the control plane today. For a served
interface, `Inventory.SecretFor` mints one password per (consumer, provider) pair via
`secrets.Make`, sealing it to **both** node keys — `ForConsumer` and `ForProvider`.
`SecretsFrom(provider)` is documented as "every credential a provider node was issued, so it
can be told what to create," and `grantsFor` (plan.go) hands the provider node one `Grant` per
consumer carrying `Sealed: ForProvider`. The provider receives, at the path its `receives`
names, one `Contribution` per consumer: the login to create (`As`, derived by the mesh so both
ends agree — 04-ISSUES/023), the consumer's address (`At`) and requested `Values`, and
`Secret`, the file holding that consumer's password sealed to this provider and unsealed by
its host. Everything the provisioner needs is delivered. It reads the wrong files
(`*.grant.json`, which nothing writes) and invents a password instead of reading the one in
`Secret`.

## The fix this points to

A **one-place contract change in the sdk harness**, plus re-pointing today's adapters at it —
not per-provider surgery, and inherited correctly by every provider after them:

- `runProvisioner` reconciles the mesh-delivered `receives` contributions (not `*.grant.json`):
  for each consumer, create the resource under the login `As` with the password read from the
  delivered `Secret` file, for its `Values`; withdraw the login when a consumer leaves the file.
- The adapter stops generating a password and stops returning a credential — it is handed the
  name and the password and only makes the resource exist. Roughly `create({as, password,
  values})` / `remove({as})`, no return.
- `sealKey`, `seal()`, `writeSealedCredential`, `MESH_SEAL_KEY`, and the `.credential` file
  leave entirely; the consumer already receives its copy through the mesh's own channel.

This is proposed as ADR 0048, which defines the corrected provider contract, for ratification.

## Resolution

ADR 0048 was accepted and implemented on the branches this issue is fixed by:

- `mesh-sdk` `src/provisioner/index.ts` now reconciles the mesh's `receives` contributions and,
  per consumer, reads the mesh-minted password from the file the host unsealed, calling the
  adapter to create the resource under the mesh's login. `$MESH_SEAL_KEY`, the symmetric seal,
  `writeSealedCredential`, and the `*.grant.json` / `*.credential` files are gone. The symmetric
  `seal()`/`unseal()` primitive had no other caller and was removed.
- The four adapters (redis, postgres, minio, umami) were re-pointed at the new contract —
  `create({ as, password, values })` / `remove({ as })`, returning nothing. minio's client gained
  a secret-key argument so it sets the mesh's secret rather than generating one.
- Proven in the mesh-lab: `provider-uses-mesh-credential` is green — redis creates the consumer's
  login with the password the mesh minted, a client authenticates as that consumer and gets PONG,
  with no seal key set anywhere.

Two things were carved out deliberately, neither blocking:

- **Data provisions are a separate shape.** umami's `analytics` returns a `siteId` umami
  *generates*, not a secret the mesh mints, and a contract that returns nothing cannot hand that
  back. ADR 0048 is scoped to credential provisions and says so; the provider→consumer return
  path for generated data is left to a separate decision. umami compiles and reconciles under the
  new harness; only that return is unaddressed, and it never had the seal-key fault.
- **Teardown beyond "remove the login"** — an object store's leftover data — is each adapter's to
  name (minio leaves a non-empty bucket for an operator rather than deleting a consumer's data),
  not the harness's.