94 lines
5.5 KiB
Markdown
94 lines
5.5 KiB
Markdown
---
|
|
topic: the mesh
|
|
status: accepted
|
|
date: 2026-09-22
|
|
deciders: jochen
|
|
reconstructed: false
|
|
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
|
---
|
|
|
|
# 102. The mesh writes into a shared file, never over it
|
|
|
|
## Context
|
|
|
|
[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
|
found that the networking module declares the container runtime's configuration file to state
|
|
one fact in it: the mesh's registry is trusted over the private network. Reading the code
|
|
closer showed it worse than reported. The controller merges an operator's settings into the
|
|
module's own content, but the host writes the result **whole**. Whatever the machine had in that
|
|
file is replaced, including where the runtime keeps its data. On a machine in use, that is every
|
|
image and container gone from the runtime's view at its next start. The runtime's service is then
|
|
restarted, which stops every container on the machine.
|
|
|
|
Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with
|
|
no restart, and a running container with no restart policy keeps running through it. The
|
|
runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards.
|
|
|
|
Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found
|
|
on an adopted node and held until networking is taken. Holding it is safe, but it means an
|
|
adopted machine cannot pull the mesh's images until then, and taking networking would restart
|
|
the runtime.
|
|
|
|
## Considered Options
|
|
|
|
1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it
|
|
still replaces the machine's settings, and the cutover stops everything on the machine.
|
|
2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such
|
|
directory for its daemon settings.
|
|
3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted.
|
|
|
|
## Decision
|
|
|
|
**A file the mesh shares with software it did not install is written into, never over.** A
|
|
file resource may say it is written *into* a structured file. The host then reads what is there,
|
|
sets only the keys the mesh declares, keeps every other key as it found it, and records what each
|
|
of its keys held before. **A list is added to, never replaced**: where the machine already has a
|
|
list under a key the mesh declares, the mesh's members are added to it and the host records
|
|
exactly which members it added — setting the key would replace the operator's own list, the harm
|
|
this record exists to prevent. Undeclared later, each key goes back to what it held and each
|
|
added member is taken out again, and a file the mesh created is removed only if nothing but its
|
|
own keys is left. A file written into replaces nothing, so on an adopted node it is never held:
|
|
it is written whether or not its module has been taken.
|
|
|
|
**Whatever the host writes over without a record of it, it keeps first.** On any node, adopted
|
|
or converged, before the host writes a file over one it has no record of making, it keeps the
|
|
original once and says where; if it cannot keep it, it does not write. A file the mesh takes over
|
|
is then never lost, whatever put it there.
|
|
|
|
**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service
|
|
resource may name what it must be *reloaded* on, beside what it must be restarted on. The
|
|
container runtime is reloaded for the registry's trust.
|
|
|
|
**The networking module writes the runtime's trust into its file and reloads it.** An adopted
|
|
node therefore trusts the mesh's registry as soon as it is on the private network, and taking
|
|
networking no longer touches the runtime. The hosts file networking writes is still written whole
|
|
and stays held until networking is taken; a converge preview names it among the files it replaces.
|
|
|
|
## Consequences
|
|
|
|
- The runtime's file on a machine in use keeps its data directory, its logging settings and
|
|
everything else the predecessor set.
|
|
- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts
|
|
are upgraded before the controller that emits them — the same order ADR 0100 needs.
|
|
- One shape more for every host: a file written into a structured document. Only JSON is spoken;
|
|
another format is refused until written.
|
|
- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text
|
|
file and is not decided here.
|
|
|
|
## How it is checked
|
|
|
|
Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each
|
|
key and removing only a file it created when the resource is undeclared, adding to a list and
|
|
removing only the members it added, keeping the original of a file it writes over without a
|
|
record, refusing a file that is not a JSON object rather than overwriting it, never holding a file written into on an adopted
|
|
node, and reloading rather than restarting a service whose reload-on resource changed. A test in
|
|
the controller holds the networking module to declaring the runtime's file written into and the
|
|
runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own
|
|
and asserts it survives adoption with the registry trusted added, and that a container without a
|
|
restart policy is still running afterwards.
|
|
|
|
## References
|
|
|
|
- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
|
- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|