ADR 0102: the mesh writes into a shared file, never over it; issue 084 located

2026-09-22 17:46:06 +02:00
parent 1901a90d68
commit 84761f0600
3 changed files with 98 additions and 2 deletions
@@ -0,0 +1,84 @@
+---
+topic: the mesh
+status: accepted
+date: 2026-09-22
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
+---
+
+# 102. The mesh writes into a shared file, never over it
+
+## Context
+
+[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
+found that the networking module declares the container runtime's configuration file to state
+one fact in it: the mesh's registry is trusted over the private network. Reading the code
+closer showed it worse than reported. The controller merges an operator's settings into the
+module's own content, but the host writes the result **whole**. Whatever the machine had in that
+file is replaced, including where the runtime keeps its data. On a machine in use, that is every
+image and container gone from the runtime's view at its next start. The runtime's service is then
+restarted, which stops every container on the machine.
+
+Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with
+no restart, and a running container with no restart policy keeps running through it. The
+runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards.
+
+Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found
+on an adopted node and held until networking is taken. Holding it is safe, but it means an
+adopted machine cannot pull the mesh's images until then, and taking networking would restart
+the runtime.
+
+## Considered Options
+
+1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it
+   still replaces the machine's settings, and the cutover stops everything on the machine.
+2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such
+   directory for its daemon settings.
+3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted.
+
+## Decision
+
+**A file the mesh shares with software it did not install is written into, never over.** A
+file resource may say it is written *into* a structured file. The host then reads what is there,
+sets only the keys the mesh declares, keeps every other key as it found it, and records what each
+of its keys held before. Undeclared later, each key goes back to what it held, and a file the
+mesh created is removed only if nothing but its own keys is left. A file written into replaces
+nothing, so on an adopted node it is never held: it is written whether or not its module has
+been taken.
+
+**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service
+resource may name what it must be *reloaded* on, beside what it must be restarted on. The
+container runtime is reloaded for the registry's trust.
+
+**The networking module writes the runtime's trust into its file and reloads it.** An adopted
+node therefore trusts the mesh's registry as soon as it is on the private network, and taking
+networking no longer touches the runtime. The hosts file networking writes is still written whole
+and stays held until networking is taken; a converge preview names it among the files it replaces.
+
+## Consequences
+
+- The runtime's file on a machine in use keeps its data directory, its logging settings and
+  everything else the predecessor set.
+- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts
+  are upgraded before the controller that emits them — the same order ADR 0100 needs.
+- One shape more for every host: a file written into a structured document. Only JSON is spoken;
+  another format is refused until written.
+- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text
+  file and is not decided here.
+
+## How it is checked
+
+Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each
+key and removing only a file it created when the resource is undeclared, refusing a file that is
+not a JSON object rather than overwriting it, never holding a file written into on an adopted
+node, and reloading rather than restarting a service whose reload-on resource changed. A test in
+the controller holds the networking module to declaring the runtime's file written into and the
+runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own
+and asserts it survives adoption with the registry trusted added, and that a container without a
+restart policy is still running afterwards.
+
+## References
+
+- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
+- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
@@ -90,6 +90,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
 - **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
 - **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md)
+- **0102** — [The mesh writes into a shared file, never over it](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)

 ### Its tiers, from the bottom up

@@ -1,7 +1,7 @@
 ---
-status: open
+status: located
 opened: 2026-09-22
-located-in: []
+located-in: [mesh-control internal/overlay, mesh-host internal/apply]
 fixed-by:
 amended-design:
 ---
@@ -55,3 +55,14 @@ and nothing checks for it today.
 - Which other modules declare a whole file that other software on the machine also writes?
 - Should an adopted node that cannot trust the registry be refused a module that needs to pull?
  Or should the refusal come earlier, when the node joins?
+
+## Diagnosis
+
+*2026-09-22.* Worse than reported. The controller's "merge" of the runtime's file merges an
+operator's settings into the module's content. The host then writes the result **whole**, so a
+machine's own runtime settings, including its data directory, are replaced, not added to.
+Measured on a lab machine: the runtime takes a new trusted-registry list on a reload, and a
+running container without a restart policy survives it. Decided in
+[ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md): the
+runtime's file is written into and the runtime reloaded. The hosts file stays a whole file, held
+until networking is taken.