jschoubben
61f61b5d06
An audit of the six code repositories found eleven open issues fixed on main with commits and beds to show (025, 027, 033, 036, 037, 040, 045, 047, 050, 052, 053), three partly (007, 026, 035), nine not (020, 031, 041, 046, 049, 054, 064, 065, 066) and one whose fix would live outside those repos (006). Resolved ones name their evidence; partly ones say what remains; 041 records that the exposure has widened since it was reported.
3.3 KiB
status, opened, located-in, fixed-by, amended-design
| status | opened | located-in | fixed-by | amended-design | ||
|---|---|---|---|---|---|---|
| resolved | 2026-09-01 |
|
mesh-host aa441ba, 652984b (container restart-on, PR |
027 — A container cannot follow a file, and a rotated credential is the case
Symptom
A service can say restart-on: these files changed, so I must be restarted. A container cannot.
It is not in the shape, and the host refuses a declaration that tries.
So a container reading its password from a file keeps the password it started with, for ever. Nothing reports anything: the file is right, the container is up, every check passes.
Why this is the same fault the mechanism exists for
restart-on is written against exactly this, in the host's own words:
a running service does not re-read its configuration. Replace the file, find the service already running, do nothing, and the machine keeps behaving the way it did before — while every check passes, because the file is right and the service is up.
Every word applies to a container, and more so. Nearly everything the mesh runs is a container — a database, a forge, a mail system — and a credential arrives as a file it reads at start.
What it costs, concretely
Rotation does not reach a container. Rotating a credential replaces the file on the machine and tells the provider to accept the new one. The provider is a program that reconciles, so it takes the change. The consumer is usually a container, so it does not. The two ends then hold different passwords, which is the fault the whole design is arranged to prevent (ADR 0001 records it costing two days).
The end-to-end test that proves rotation works uses a consumer that reads the file on each attempt, so it does not meet this.
Why it was not noticed
The gap is invisible from the control plane. A manifest carrying restart-on on a container
composes into a declaration without complaint and is refused on the machine, so the only way to
learn is to run one — which is how it was found, after nine of them had shipped across seven
modules.
What a fix has to keep
- Declared state, not a command.
restart-onis deliberately not restart this: it says the running thing must reflect these files, and the host works out that it does not. Whatever containers get must keep that shape, because the link may not carry an action (ADR 0005). - Recreate, not restart, where that is the honest verb. A container's environment is fixed at
creation. If what changed is an
env-file, restarting the container is not enough — it has to be made again. That is a different act from a service reload and should not be described as one. - It must not fire on every reconcile. A container that is recreated whenever the host looks at it is worse than one that never follows the file.
The near alternative, and why it is not enough
A module can avoid the problem by having its program read the file on each use rather than at start. That works for something written for this mesh, and is not available for a database, a forge or a mail system — which is the whole population this is about.