ADR 0102: the mesh writes into a shared file, never over it; issue 084 located
This commit is contained in:
@@ -0,0 +1,84 @@
|
|||||||
|
---
|
||||||
|
topic: the mesh
|
||||||
|
status: accepted
|
||||||
|
date: 2026-09-22
|
||||||
|
deciders: jochen
|
||||||
|
reconstructed: false
|
||||||
|
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# 102. The mesh writes into a shared file, never over it
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
||||||
|
found that the networking module declares the container runtime's configuration file to state
|
||||||
|
one fact in it: the mesh's registry is trusted over the private network. Reading the code
|
||||||
|
closer showed it worse than reported. The controller merges an operator's settings into the
|
||||||
|
module's own content, but the host writes the result **whole**. Whatever the machine had in that
|
||||||
|
file is replaced, including where the runtime keeps its data. On a machine in use, that is every
|
||||||
|
image and container gone from the runtime's view at its next start. The runtime's service is then
|
||||||
|
restarted, which stops every container on the machine.
|
||||||
|
|
||||||
|
Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with
|
||||||
|
no restart, and a running container with no restart policy keeps running through it. The
|
||||||
|
runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards.
|
||||||
|
|
||||||
|
Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found
|
||||||
|
on an adopted node and held until networking is taken. Holding it is safe, but it means an
|
||||||
|
adopted machine cannot pull the mesh's images until then, and taking networking would restart
|
||||||
|
the runtime.
|
||||||
|
|
||||||
|
## Considered Options
|
||||||
|
|
||||||
|
1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it
|
||||||
|
still replaces the machine's settings, and the cutover stops everything on the machine.
|
||||||
|
2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such
|
||||||
|
directory for its daemon settings.
|
||||||
|
3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**A file the mesh shares with software it did not install is written into, never over.** A
|
||||||
|
file resource may say it is written *into* a structured file. The host then reads what is there,
|
||||||
|
sets only the keys the mesh declares, keeps every other key as it found it, and records what each
|
||||||
|
of its keys held before. Undeclared later, each key goes back to what it held, and a file the
|
||||||
|
mesh created is removed only if nothing but its own keys is left. A file written into replaces
|
||||||
|
nothing, so on an adopted node it is never held: it is written whether or not its module has
|
||||||
|
been taken.
|
||||||
|
|
||||||
|
**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service
|
||||||
|
resource may name what it must be *reloaded* on, beside what it must be restarted on. The
|
||||||
|
container runtime is reloaded for the registry's trust.
|
||||||
|
|
||||||
|
**The networking module writes the runtime's trust into its file and reloads it.** An adopted
|
||||||
|
node therefore trusts the mesh's registry as soon as it is on the private network, and taking
|
||||||
|
networking no longer touches the runtime. The hosts file networking writes is still written whole
|
||||||
|
and stays held until networking is taken; a converge preview names it among the files it replaces.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The runtime's file on a machine in use keeps its data directory, its logging settings and
|
||||||
|
everything else the predecessor set.
|
||||||
|
- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts
|
||||||
|
are upgraded before the controller that emits them — the same order ADR 0100 needs.
|
||||||
|
- One shape more for every host: a file written into a structured document. Only JSON is spoken;
|
||||||
|
another format is refused until written.
|
||||||
|
- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text
|
||||||
|
file and is not decided here.
|
||||||
|
|
||||||
|
## How it is checked
|
||||||
|
|
||||||
|
Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each
|
||||||
|
key and removing only a file it created when the resource is undeclared, refusing a file that is
|
||||||
|
not a JSON object rather than overwriting it, never holding a file written into on an adopted
|
||||||
|
node, and reloading rather than restarting a service whose reload-on resource changed. A test in
|
||||||
|
the controller holds the networking module to declaring the runtime's file written into and the
|
||||||
|
runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own
|
||||||
|
and asserts it survives adoption with the registry trusted added, and that a container without a
|
||||||
|
restart policy is still running afterwards.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
||||||
|
- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||||
@@ -90,6 +90,7 @@ python3 00-META/checks/index.py fail if stale
|
|||||||
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
|
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
|
||||||
- **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
- **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||||
- **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md)
|
- **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md)
|
||||||
|
- **0102** — [The mesh writes into a shared file, never over it](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)
|
||||||
|
|
||||||
### Its tiers, from the bottom up
|
### Its tiers, from the bottom up
|
||||||
|
|
||||||
|
|||||||
+13
-2
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
status: open
|
status: located
|
||||||
opened: 2026-09-22
|
opened: 2026-09-22
|
||||||
located-in: []
|
located-in: [mesh-control internal/overlay, mesh-host internal/apply]
|
||||||
fixed-by:
|
fixed-by:
|
||||||
amended-design:
|
amended-design:
|
||||||
---
|
---
|
||||||
@@ -55,3 +55,14 @@ and nothing checks for it today.
|
|||||||
- Which other modules declare a whole file that other software on the machine also writes?
|
- Which other modules declare a whole file that other software on the machine also writes?
|
||||||
- Should an adopted node that cannot trust the registry be refused a module that needs to pull?
|
- Should an adopted node that cannot trust the registry be refused a module that needs to pull?
|
||||||
Or should the refusal come earlier, when the node joins?
|
Or should the refusal come earlier, when the node joins?
|
||||||
|
|
||||||
|
## Diagnosis
|
||||||
|
|
||||||
|
*2026-09-22.* Worse than reported. The controller's "merge" of the runtime's file merges an
|
||||||
|
operator's settings into the module's content. The host then writes the result **whole**, so a
|
||||||
|
machine's own runtime settings, including its data directory, are replaced, not added to.
|
||||||
|
Measured on a lab machine: the runtime takes a new trusted-registry list on a reload, and a
|
||||||
|
running container without a restart policy survives it. Decided in
|
||||||
|
[ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md): the
|
||||||
|
runtime's file is written into and the runtime reloaded. The hosts file stays a whole file, held
|
||||||
|
until networking is taken.
|
||||||
|
|||||||
Reference in New Issue
Block a user