From 84761f0600e1fe9950090fd25c18e20ee60b0547 Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 22 Sep 2026 17:46:06 +0200 Subject: [PATCH] ADR 0102: the mesh writes into a shared file, never over it; issue 084 located --- ...writes-into-a-shared-file-never-over-it.md | 84 +++++++++++++++++++ 02-DECISIONS/README.md | 1 + .../00-report.md | 15 +++- 3 files changed, 98 insertions(+), 2 deletions(-) create mode 100644 02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md diff --git a/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md new file mode 100644 index 0000000..12ab4e2 --- /dev/null +++ b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md @@ -0,0 +1,84 @@ +--- +topic: the mesh +status: accepted +date: 2026-09-22 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md +--- + +# 102. The mesh writes into a shared file, never over it + +## Context + +[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md) +found that the networking module declares the container runtime's configuration file to state +one fact in it: the mesh's registry is trusted over the private network. Reading the code +closer showed it worse than reported. The controller merges an operator's settings into the +module's own content, but the host writes the result **whole**. Whatever the machine had in that +file is replaced, including where the runtime keeps its data. On a machine in use, that is every +image and container gone from the runtime's view at its next start. The runtime's service is then +restarted, which stops every container on the machine. + +Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with +no restart, and a running container with no restart policy keeps running through it. The +runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards. + +Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found +on an adopted node and held until networking is taken. Holding it is safe, but it means an +adopted machine cannot pull the mesh's images until then, and taking networking would restart +the runtime. + +## Considered Options + +1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it + still replaces the machine's settings, and the cutover stops everything on the machine. +2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such + directory for its daemon settings. +3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted. + +## Decision + +**A file the mesh shares with software it did not install is written into, never over.** A +file resource may say it is written *into* a structured file. The host then reads what is there, +sets only the keys the mesh declares, keeps every other key as it found it, and records what each +of its keys held before. Undeclared later, each key goes back to what it held, and a file the +mesh created is removed only if nothing but its own keys is left. A file written into replaces +nothing, so on an adopted node it is never held: it is written whether or not its module has +been taken. + +**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service +resource may name what it must be *reloaded* on, beside what it must be restarted on. The +container runtime is reloaded for the registry's trust. + +**The networking module writes the runtime's trust into its file and reloads it.** An adopted +node therefore trusts the mesh's registry as soon as it is on the private network, and taking +networking no longer touches the runtime. The hosts file networking writes is still written whole +and stays held until networking is taken; a converge preview names it among the files it replaces. + +## Consequences + +- The runtime's file on a machine in use keeps its data directory, its logging settings and + everything else the predecessor set. +- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts + are upgraded before the controller that emits them — the same order ADR 0100 needs. +- One shape more for every host: a file written into a structured document. Only JSON is spoken; + another format is refused until written. +- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text + file and is not decided here. + +## How it is checked + +Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each +key and removing only a file it created when the resource is undeclared, refusing a file that is +not a JSON object rather than overwriting it, never holding a file written into on an adopted +node, and reloading rather than restarting a service whose reload-on resource changed. A test in +the controller holds the networking module to declaring the runtime's file written into and the +runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own +and asserts it survives adoption with the registry trusted added, and that a container without a +restart policy is still running afterwards. + +## References + +- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md) +- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index eb60615..a56570c 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -90,6 +90,7 @@ python3 00-META/checks/index.py fail if stale - **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md) - **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md) - **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md) +- **0102** — [The mesh writes into a shared file, never over it](0102-the-mesh-writes-into-a-shared-file-never-over-it.md) ### Its tiers, from the bottom up diff --git a/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md index 95d3da0..cb639de 100644 --- a/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md +++ b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md @@ -1,7 +1,7 @@ --- -status: open +status: located opened: 2026-09-22 -located-in: [] +located-in: [mesh-control internal/overlay, mesh-host internal/apply] fixed-by: amended-design: --- @@ -55,3 +55,14 @@ and nothing checks for it today. - Which other modules declare a whole file that other software on the machine also writes? - Should an adopted node that cannot trust the registry be refused a module that needs to pull? Or should the refusal come earlier, when the node joins? + +## Diagnosis + +*2026-09-22.* Worse than reported. The controller's "merge" of the runtime's file merges an +operator's settings into the module's content. The host then writes the result **whole**, so a +machine's own runtime settings, including its data directory, are replaced, not added to. +Measured on a lab machine: the runtime takes a new trusted-registry list on a reload, and a +running container without a restart policy survives it. Decided in +[ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md): the +runtime's file is written into and the runtime reloaded. The hosts file stays a whole file, held +until networking is taken.