94 lines
5.9 KiB
Markdown
94 lines
5.9 KiB
Markdown
---
|
|
status: resolved
|
|
opened: 2026-09-10
|
|
located-in: [mesh-controller internal/catalogue, mesh-catalog modules/mesh-controller]
|
|
fixed-by: ADR 0086; mesh-controller feat/secret-not-in-environment (envfile twins for the broker settings, catalogue refusal of secrets in env / undeclared env-file); mesh-catalog (the controller reads its six credentials from files; 35 containers declare their exception); mesh-host (the installer delivers any MESH_…_FILE own secret)
|
|
amended-design: 03-DESIGN/01-to-be/13-credentials-and-their-rotation.md
|
|
---
|
|
|
|
# 041 — A credential the mesh took care to seal ends up in the process environment
|
|
|
|
## Symptom
|
|
|
|
The mesh generates a module's own secret, **seals it to the machine and discards the plaintext** —
|
|
it cannot read the value back even if asked. The host unseals it into a file the module names, at
|
|
`0600`.
|
|
|
|
Then the module hands it to its container as an environment variable, and the runtime puts it
|
|
where anything on that machine that can talk to the runtime can read it: `docker inspect` prints
|
|
it, and `/proc/<pid>/environ` holds it for the life of the process.
|
|
|
|
Observed while making the control plane an ordinary module. Its **store** connections were moved to
|
|
files, read by a `…_FILE` variable naming the path. Its **broker** credentials have no such variable,
|
|
so they are still delivered through an env-file — which the runtime turns into exactly the
|
|
environment above. Same credential handling, same machine, two different exposures, decided by
|
|
whether the program that reads it happens to accept a path.
|
|
|
|
## Why this matters
|
|
|
|
**The care taken elsewhere is what makes this stand out.** Sealing to a machine and discarding the
|
|
plaintext is expensive and deliberate: it exists so that a credential is readable only where it is
|
|
used. Handing that same value to the runtime as an environment variable gives it back to anything
|
|
that can run `inspect` — and `inspect` is a routine operation. It lands in support output, in
|
|
captured logs, in a screenshot of a terminal, and in any tooling that dumps container state.
|
|
|
|
**It is not a module's mistake.** Nothing in the manifest format is being misused: placing a secret
|
|
into an env-file with `${secret:…}` is a supported shape and other modules use it. So each module is
|
|
correct on its own, and the property — *a sealed credential is not readable by everything on the
|
|
machine* — holds or fails per variable, by accident of what each program accepts.
|
|
|
|
**And the two halves now disagree inside one module.** The control plane reads its store connection
|
|
from a file and its broker credential from the environment. A reader cannot tell from the manifest
|
|
which secrets are protected from `inspect` and which are not, because the manifest looks the same
|
|
either way.
|
|
|
|
## Open questions
|
|
|
|
- Should every program the mesh runs accept a path for anything secret — a `…_FILE` twin as a
|
|
convention rather than a thing each program decides? That is a small change in several programs
|
|
and a large one in what the manifest can promise.
|
|
- Should the manifest layer **refuse** `${secret:…}` inside a container's `env`, or inside an
|
|
`env-file`, once a path-shaped alternative exists? A rule nothing enforces is the shape this
|
|
repository keeps finding.
|
|
- Is there a case where the environment is genuinely the only channel — a program that cannot be
|
|
changed and reads no file? If so, what should the mesh say about that module, out loud, rather
|
|
than treating it as equivalent?
|
|
- What is the actual reach of the exposure on a node — which identities can talk to the container
|
|
runtime, and is that set smaller than "anything running as the operator"? The answer decides
|
|
whether this is a hardening item or something sharper.
|
|
|
|
## Reconciled 2026-09-21
|
|
|
|
Still open, and worse than reported: the controller's own manifest now delivers its store
|
|
connections **and** its broker credentials through an env-file, so both halves reach the process
|
|
environment; 28 catalogue manifests use env-file for a secret, and nothing in the catalogue
|
|
engine refuses a `${secret:…}` placeholder in one. The vault work of
|
|
[ADR 0085](../../02-DECISIONS/0085-a-secret-is-a-provision.md) rests on the seal this weakens.
|
|
|
|
## Resolved 2026-09-21
|
|
|
|
By [ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md): a secret reaches
|
|
a process as a file, an environment exception is declared with a reason, and the catalogue engine
|
|
refuses the undeclared shape. The controller, the instance this was opened on, reads all six of
|
|
its credentials from files. The 35 other containers carry a declared reason; converting each where
|
|
its software accepts a path remains per-module work and is not this issue's.
|
|
|
|
## The declared exceptions, surveyed 2026-09-21
|
|
|
|
Of the 35 containers marked when the rule landed, a survey of each image's own configuration
|
|
loader (read at the pinned digest where it was cached) found: 24 variables convertible now, 8 not
|
|
convertible (baserow, mssql, only-office, umami, mailu's initial admin), 9 read by the code of
|
|
Novox's own applications outside the mesh repositories (amqp-email-forwarder, de-spiegel,
|
|
invoicing, photos), 4 unverifiable (keycloak's admin, letta, mailu's other containers), and 2
|
|
dead deliveries that nothing read. Converted and proven in a bed: amqp-ping, minio, mongodb,
|
|
mesh-catalog and model-usage; the two dead deliveries removed. One image, mongodb's, drops to its
|
|
own user before it reads the file, so its manifest names that user as the owner of its secrets — the
|
|
first use of the owner field outside the controller, and the check any converted image needs: run it
|
|
with a file its user cannot read. The 25 that remain carry the
|
|
surveyed reason on the container. Gitea, nextcloud, mailu-admin, influxdb, step-ca and n8n are
|
|
convertible and wait for a bed that exercises their credential
|
|
([issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md)); icecast, keycloak's
|
|
database, searxng and nextcloud's object store convert through a generated configuration file
|
|
rather than a variable.
|
|
|