2026-09-05 10:27:11 +00:00
1 changed files with 65 additions and 0 deletions
@@ -0,0 +1,65 @@
+---
+status: located
+opened: 2026-09-01
+located-in: [mesh-host, mesh-control]
+fixed-by:
+amended-design:
+---
+
+# 027 — A container cannot follow a file, and a rotated credential is the case
+
+## Symptom
+
+A service can say `restart-on`: *these files changed, so I must be restarted*. A container cannot.
+It is not in the shape, and the host refuses a declaration that tries.
+
+So a container reading its password from a file keeps the password it started with, for ever.
+Nothing reports anything: the file is right, the container is up, every check passes.
+
+## Why this is the same fault the mechanism exists for
+
+`restart-on` is written against exactly this, in the host's own words:
+
+> a running service does not re-read its configuration. Replace the file, find the service already
+> running, do nothing, and the machine keeps behaving the way it did before — while every check
+> passes, because the file is right and the service is up.
+
+Every word applies to a container, and more so. **Nearly everything the mesh runs is a container**
+— a database, a forge, a mail system — and a credential arrives as a file it reads at start.
+
+## What it costs, concretely
+
+**Rotation does not reach a container.** Rotating a credential replaces the file on the machine and
+tells the provider to accept the new one. The provider is a program that reconciles, so it takes
+the change. The consumer is usually a container, so it does not. The two ends then hold different
+passwords, which is the fault the whole design is arranged to prevent
+([ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) records it costing
+two days).
+
+The end-to-end test that proves rotation works uses a consumer that reads the file on each
+attempt, so it does not meet this.
+
+## Why it was not noticed
+
+The gap is invisible from the control plane. A manifest carrying `restart-on` on a container
+composes into a declaration without complaint and is refused on the machine, so the only way to
+learn is to run one — which is how it was found, after nine of them had shipped across seven
+modules.
+
+## What a fix has to keep
+
+- **Declared state, not a command.** `restart-on` is deliberately not *restart this*: it says the
+  running thing must reflect these files, and the host works out that it does not. Whatever
+  containers get must keep that shape, because the link may not carry an action
+  ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
+- **Recreate, not restart, where that is the honest verb.** A container's environment is fixed at
+  creation. If what changed is an `env-file`, restarting the container is not enough — it has to be
+  made again. That is a different act from a service reload and should not be described as one.
+- **It must not fire on every reconcile.** A container that is recreated whenever the host looks at
+  it is worse than one that never follows the file.
+
+## The near alternative, and why it is not enough
+
+A module can avoid the problem by having its program read the file on each use rather than at
+start. That works for something written for this mesh, and is not available for a database, a
+forge or a mail system — which is the whole population this is about.