hq/04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md

status, opened, located-in, fixed-by, amended-design

status	opened	located-in	fixed-by	amended-design
resolved	2026-09-23	mesh-host internal/apply	mesh-host PR

103 — A container is not recreated when a file it reads changes

What was observed

The control-node, 2026-09-23. The store was given the predecessor's port. The host rewrote the forge's and the analytics service's environment files with the new port — correctly — and left both containers running with the old one in their environment. Both lost their database. Both stayed "up" and healthy-looking for the twenty minutes it took to notice, then answered 502.

A restart did not help: a container reads its env-file when it is created, not when it starts, so docker restart handed both containers the same stale environment. Only removing them and letting the host recreate them fixed it.

The host decides whether a container needs recreating by comparing a hash of its declared spec. The spec names the env file's path; the file's content is not part of it. So a change that alters everything the process will see alters nothing the host compares.

Why it matters beyond this instance

Every module with an env-file — which is most of them — has a configuration the host writes and a container that reads it once. Any change to that configuration that the host applies without recreating the container is applied to the disk and not to the service. The mesh then reports the node as running what it was told, because the file is right; only the process is wrong.

The ports move is the obvious trigger, and it is the migration's whole method. But a rotated credential, a re-provisioned database, a changed binding — anything the host substitutes into a file a container reads — has the same shape.

Open questions

• Should the spec hash cover the content of every file the container mounts or reads, so a changed file recreates it — accepting that every such change is a restart of the service?
• Or should the host recreate on content change only for env-file and mounted secrets, and leave bind-mounted data alone — a file the service reads at start versus a directory it reads while running?
• How does the host report the difference between "the file is right" and "the process has read it"? Today it cannot, and that is what made this silent.