diff --git a/02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md b/02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md new file mode 100644 index 0000000..79b50c6 --- /dev/null +++ b/02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md @@ -0,0 +1,70 @@ +--- +topic: the mesh +status: accepted +date: 2026-09-22 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md +--- + +# 101. A machine's own resolver does not make it in use + +## Context + +[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) has a converged genesis +refuse a machine in use. It defines *in use* as a container running, or a port listening on an +address other than loopback that is not ssh's. That definition was written before anything was +measured. + +Measured on a freshly installed lab machine, running nothing but its operating system: + +| Listening | Held by | +|---|---| +| TCP and UDP on every address, the link-local name resolution port | the system's name resolver | +| UDP on every address, the multicast name resolution port | the system's name resolver | +| UDP on a link-local address, the address-configuration client port | the system's network manager | +| TCP and UDP on loopback | the resolver's stub and the container runtime | + +By ADR 0100's words, the resolver's TCP listener on every address makes **every** freshly +installed machine a machine in use. A converged genesis would refuse them all, and `--adopted` +would become the only way to raise anything. The refusal exists to catch a forgotten flag on a +working machine. Refusing an empty one defeats it, and teaches operators to pass the flag by +habit. + +## Considered Options + +1. **Keep the words.** Rejected: every fresh machine is refused. +2. **Count TCP only, ignore UDP.** Rejected: the resolver listens on TCP too, and a machine that + serves over UDP alone, a resolver or a tunnel, is in use. +3. **Ignore listeners held by the operating system's own network daemons**, a short named list, + on both protocols. Adopted. + +## Decision + +**A listener held by one of the operating system's own network daemons does not make a machine +in use.** The daemons are the ones the measurement found: the name resolver and the network +manager, named in the installer's code beside that measurement. Everything else in ADR 0100's definition stands: a running container, or any other +listener on an address other than loopback that is not ssh's, makes the machine in use, and +genesis still names every one it counted. + +A daemon is added to the list only with a measurement of a fresh machine that holds it. + +## Consequences + +- A converged genesis on a fresh machine goes ahead, as it did before ADR 0100. +- A machine whose resolver is also serving other machines is not counted as in use by its + resolver alone. It is one of the daemons that serves nobody on a fresh machine, and the one + kind of service this lets through. +- The list is code, not configuration, so it changes by review. + +## How it is checked + +A unit test in the installer feeds the listeners captured from the fresh machine, as the +listening-socket tool printed them, and asserts the machine is not in use. The existing tests +still assert that a serving machine is in use and that every container and listener is named. +The adoption lab bed raises a converged genesis on a fresh machine and asserts it is not refused. + +## References + +- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) +- [research 012, *migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md) diff --git a/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md new file mode 100644 index 0000000..8731e5a --- /dev/null +++ b/02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md @@ -0,0 +1,93 @@ +--- +topic: the mesh +status: accepted +date: 2026-09-22 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md +--- + +# 102. The mesh writes into a shared file, never over it + +## Context + +[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md) +found that the networking module declares the container runtime's configuration file to state +one fact in it: the mesh's registry is trusted over the private network. Reading the code +closer showed it worse than reported. The controller merges an operator's settings into the +module's own content, but the host writes the result **whole**. Whatever the machine had in that +file is replaced, including where the runtime keeps its data. On a machine in use, that is every +image and container gone from the runtime's view at its next start. The runtime's service is then +restarted, which stops every container on the machine. + +Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with +no restart, and a running container with no restart policy keeps running through it. The +runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards. + +Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found +on an adopted node and held until networking is taken. Holding it is safe, but it means an +adopted machine cannot pull the mesh's images until then, and taking networking would restart +the runtime. + +## Considered Options + +1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it + still replaces the machine's settings, and the cutover stops everything on the machine. +2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such + directory for its daemon settings. +3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted. + +## Decision + +**A file the mesh shares with software it did not install is written into, never over.** A +file resource may say it is written *into* a structured file. The host then reads what is there, +sets only the keys the mesh declares, keeps every other key as it found it, and records what each +of its keys held before. **A list is added to, never replaced**: where the machine already has a +list under a key the mesh declares, the mesh's members are added to it and the host records +exactly which members it added — setting the key would replace the operator's own list, the harm +this record exists to prevent. Undeclared later, each key goes back to what it held and each +added member is taken out again, and a file the mesh created is removed only if nothing but its +own keys is left. A file written into replaces nothing, so on an adopted node it is never held: +it is written whether or not its module has been taken. + +**Whatever the host writes over without a record of it, it keeps first.** On any node, adopted +or converged, before the host writes a file over one it has no record of making, it keeps the +original once and says where; if it cannot keep it, it does not write. A file the mesh takes over +is then never lost, whatever put it there. + +**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service +resource may name what it must be *reloaded* on, beside what it must be restarted on. The +container runtime is reloaded for the registry's trust. + +**The networking module writes the runtime's trust into its file and reloads it.** An adopted +node therefore trusts the mesh's registry as soon as it is on the private network, and taking +networking no longer touches the runtime. The hosts file networking writes is still written whole +and stays held until networking is taken; a converge preview names it among the files it replaces. + +## Consequences + +- The runtime's file on a machine in use keeps its data directory, its logging settings and + everything else the predecessor set. +- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts + are upgraded before the controller that emits them — the same order ADR 0100 needs. +- One shape more for every host: a file written into a structured document. Only JSON is spoken; + another format is refused until written. +- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text + file and is not decided here. + +## How it is checked + +Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each +key and removing only a file it created when the resource is undeclared, adding to a list and +removing only the members it added, keeping the original of a file it writes over without a +record, refusing a file that is not a JSON object rather than overwriting it, never holding a file written into on an adopted +node, and reloading rather than restarting a service whose reload-on resource changed. A test in +the controller holds the networking module to declaring the runtime's file written into and the +runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own +and asserts it survives adoption with the registry trusted added, and that a container without a +restart policy is still running afterwards. + +## References + +- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md) +- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) diff --git a/02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md b/02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md new file mode 100644 index 0000000..7955737 --- /dev/null +++ b/02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md @@ -0,0 +1,112 @@ +--- +topic: the mesh +status: accepted +date: 2026-09-22 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md +--- + +# 103. What an adopted node holds, and what its guard refuses + +## Context + +[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) holds a found file or +container until its module is taken. It also has the mesh guard the store's port and the +broker's management port with a table that only refuses. Two independent reviews of the build +found both rules drawn too narrowly and, for the guard, in the wrong place: + +1. **Other kinds of resource reach what was found.** An untaken module's directory is re-owned + and re-moded if a predecessor's directory is already at its path, and a database refuses to + start on a data directory whose mode changed. A service of the same name as a predecessor's + unit is started, stopped or re-enabled. An action run *in* a held container runs inside the + predecessor's service. A container that is not found under its own name is created, and can + mount the predecessor's data beside the predecessor's own container. +2. **The guard's two ports are not the only ones a found firewall misses.** A published + container port is forwarded, not received, and a firewall that filters only incoming traffic + never sees it (ADR 0100's own Context). The broker's plaintext port is published on every + interface and admitted by the filter from the private network only. So on an adopted node it + is reachable from anywhere. The same holds for any published port a module restricts to the + private network. +3. **The guard refuses by port alone.** On a machine that routes for others, such as a + predecessor's private-network hub, a packet for another machine's database port is refused as + well. And a port the guard refuses for a module that is assigned but not taken may still be + the predecessor's own, serving the predecessor's other machines. + +## Decision + +**Found covers every kind that can reach what the machine already has.** On an adopted node, +for a module not yet taken: + +- a **directory** present with no record is held: its mode and owner are left, and nothing in it + is touched; +- a **service** with no record is held when an administrator installed its unit — the service + manager reads the unit from outside the packages' own directory — or when the machine uses it, + running or started at boot. Its state and whether it starts at boot are then left. A reload + named by the module still happens, since a reload stops nothing + ([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). A unit a package merely + ships that nothing runs and nothing enables is not found: the private network's own tunnel unit + is one, and holding it kept the private network from ever coming up; +- an **archive** whose target is present with no record is not unpacked or re-owned; a + **process** whose unit file is present with no record is not written over or restarted; a + **user** that exists with no record keeps its shell and groups; +- a **container** not found by its name is still held if it would mount a path or a volume that + is present with no record, because it would share the predecessor's data; +- an **action** or one-off step run *in* a held container is held until the module is taken. + +What is held is reported as held and is never removed, as ADR 0100 decides for files and +containers. + +**The guard refuses what the found firewall may miss, for taken modules only.** Its ports are +derived from each *taken* module: every machine port it publishes that the filter would admit +from the private network only, and the ports its manifest names under `guards` — which is how +the store's port and the broker's management port are included. Only TCP is guarded. A port of a module that is assigned but not taken is not guarded: it may still +be the predecessor's. The guard matches only packets addressed to this machine. Traffic the +machine routes for others is never its business. + +**An opening already answered by a found rule is not added.** If the found firewall already +admits what an opening says, the opening is reported as satisfied by the found rule, and the +mesh adds nothing and later removes nothing. The found firewall treats two rules differing only +in their action, log setting or comment as one rule, so adding the mesh's would take over the +operator's. Where the found rule is such a twin but is not a plain allow — a deny, a limit, a +logged allow — the opening is refused, naming that rule, and nothing is added. + +**A machine raised adopted stays adopted if genesis is run again.** The installer reads the +node's mode from what the machine records, not only from the operator's flag. A run without +the flag on an adopted machine is refused. + +**What of ADR 0100 this replaces.** ADR 0100's guard refused two ports, the foundation's own, +checked free at genesis, and could therefore close nothing the machine served. That guard is +replaced by the one above: a guarded port of a taken module may be one the predecessor served +more widely, and taking the module narrows it to the private network. Everything else in ADR 0100 +stands, and its rules for files and containers now cover the other kinds listed here. + +## Consequences + +- Taking a module can narrow a port the predecessor served to anyone: the guard then refuses it + from outside the private network, as the module declares. Nothing says so yet + ([issue 086](../04-ISSUES/086-taking-a-module-narrows-a-port-without-saying-so/00-report.md)). +- An untaken module on an adopted node can come up only beside what was found, never on top of + it: whatever would share the predecessor's data waits for the cutover. +- The guard grows with the modules taken, and a module's published private-network port is + protected on an adopted node the way the derived filter protects it on a converged one. +- Guarding only taken modules means a foundation port on a node joining adopted is guarded once + its module is taken, not when it is assigned. Genesis takes the foundation's modules, so the + control-node's store is guarded from the first push. + +## How it is checked + +Unit tests hold the host to holding a found directory, a found service, a found archive, process +and user, and a container that would mount found data; to not holding a unit nothing runs; to +deferring an action run in a held container; to adding no opening a found rule answers, and to +refusing one a conflicting found rule would absorb. They hold the controller to deriving the guard from taken modules' +private-network published ports, and the guard's text to matching only this machine's +addresses. The installer's tests hold a re-run without the flag on an adopted machine to a +refusal. The adoption lab bed asserts that the broker's plaintext port is unreachable from +outside the private network even with the found firewall admitting it, and that an operator's +own rule for a port the mesh opens survives the opening being removed. + +## References + +- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), + [ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index e928743..c94167e 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -89,6 +89,9 @@ python3 00-META/checks/index.py fail if stale - **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md) - **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md) - **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md) +- **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md) +- **0102** — [The mesh writes into a shared file, never over it](0102-the-mesh-writes-into-a-shared-file-never-over-it.md) +- **0103** — [What an adopted node holds, and what its guard refuses](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md) ### Its tiers, from the bottom up diff --git a/03-DESIGN/01-to-be/05-the-node-host.md b/03-DESIGN/01-to-be/05-the-node-host.md index b78d4f3..35300b6 100644 --- a/03-DESIGN/01-to-be/05-the-node-host.md +++ b/03-DESIGN/01-to-be/05-the-node-host.md @@ -5,6 +5,8 @@ code: [mesh-host] updated: 2026-09-22 decisions: - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md + - 02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md + - 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md - 02-DECISIONS/0019-how-this-repository-works.md - 02-DECISIONS/0004-a-node-and-how-it-joins.md - 02-DECISIONS/0005-the-node-host.md @@ -150,6 +152,21 @@ checked:* unit tests hold the host to keeping a found file and container, conver taken, never removing a held file and reporting one that changed; the adoption bed asserts a found file byte for byte unchanged until its module is taken. +**Found reaches every kind that can touch what the machine has** +([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record +keeps its mode and owner, a unit present with no record keeps its state and boot setting, a +container that would mount found data is not created, and an action run in a held container +waits for the cutover. **A file the machine shares is written into, never over** +([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)): the host sets the mesh's keys in the object already there, +keeps every other key, adds its members to a list already there rather than replacing it, +records what each of its keys held and which members it added, and gives them back when the +file is undeclared. Such a file replaces nothing, so it is never held. And on any node, before +the host writes over a file it has no record of making, it keeps the original once and names +where; if it cannot keep it, it does not write. A service that re-reads +its configuration is **reloaded** for what it names in `reload-on`, never restarted. *How it is +checked:* unit tests hold the host to each of these, and the adoption bed asserts the runtime's +own settings survive adoption and a container without a restart policy keeps running. + ## Where a declaration comes from One behaviour, two sources diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index 0ecc114..2d3c85d 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -9,6 +9,7 @@ code: - mesh-host internal/apply (the service that reflects a rule set) updated: 2026-09-22 decisions: + - 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md - 02-DECISIONS/0099-a-step-that-runs-once-names-what-it-reads.md - 02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md @@ -550,9 +551,13 @@ the found firewall in that firewall's own terms, marks it as the mesh's, removes marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for longer than one reconcile. An opening is state, not a command, so it travels over the link like any other resource. **The mesh guards its own ports in a table of its own that only refuses** — -passing everything by default and holding nothing but refusals of the foundation's own two ports, -checked free at genesis, so it cannot close what the machine serves, and the found firewall's -reload does not touch it. It refuses the store's port and the broker's management port except from +passing everything by default and holding nothing but refusals, and the found firewall's reload +does not touch it. Its ports are derived ([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)): every machine port a +*taken* module publishes that the filter admits from the private network only, with the store's +port and the broker's management port. A port of a module not yet taken may still be the +predecessor's, so it is not guarded; and only packets addressed to this machine are matched, so +traffic it routes for others passes. An opening a found rule already answers is not added, so +removing it never removes the operator's rule. It refuses those ports except from the private network and the machine itself — loopback and the container runtime's networks, known by the interface a packet arrives on, never by its source address alone — at the prerouting hook, before the container runtime redirects the packet, so it matches the port the diff --git a/03-DESIGN/01-to-be/09-the-node-lifecycle.md b/03-DESIGN/01-to-be/09-the-node-lifecycle.md index fc6d16e..61db9c4 100644 --- a/03-DESIGN/01-to-be/09-the-node-lifecycle.md +++ b/03-DESIGN/01-to-be/09-the-node-lifecycle.md @@ -300,8 +300,12 @@ cutover. The firewall found there stays in force and the mesh opens what it need ([08-connectivity](08-connectivity.md)). **Converging is one act per node, previewed**: it refuses while an assigned module still holds a found container; otherwise it lists what is reachable on the machine now — listening sockets and published ports — whether an assigned module declares each or -it will close, and which modules it will take, then takes them, loads the mesh's own filter in place -of its refusal-only table and disables the found firewall without flushing it. Returning a +it will close, which modules it will take and every held thing each will replace, and ends with a +short digest of all of that. **The flip is made only with that digest** — the operator confirms +the preview they read, and a preview that has changed since, or whose account of the machine is +more than fifteen minutes old, is refused. It then takes the modules, loads the mesh's own filter +in place of its refusal-only table and disables the found firewall without flushing it, putting +back the forwarding policy the found firewall had set. Returning a converged node to adopted unloads the derived filter, restores the refusal-only table, enables the found firewall again and converges the openings through it; what was taken stays taken. *How it is checked:* a lab bed prepares a machine the way a predecessor leaves one and asserts nothing that serves changes until a module is taken or the diff --git a/03-DESIGN/01-to-be/17-raising-a-mesh.md b/03-DESIGN/01-to-be/17-raising-a-mesh.md index a9d062b..c7c7e59 100644 --- a/03-DESIGN/01-to-be/17-raising-a-mesh.md +++ b/03-DESIGN/01-to-be/17-raising-a-mesh.md @@ -8,6 +8,8 @@ code: updated: 2026-09-22 decisions: - 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md + - 02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md + - 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md - 02-DECISIONS/0067-genesis-is-a-pivot.md - 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md - 02-DECISIONS/0006-the-substrate-and-the-control-plane.md @@ -113,8 +115,13 @@ checks the private network's range does not overlap a tunnel the predecessor run **does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its foundation's ports through it and keeps the store from outside with a table of its own that only refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — -a container running, or a port listening on an address other than loopback that is not ssh's — -and names every one it counted, so a forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and +a container running, or a port listening on an address other than loopback that is neither ssh's +nor held by one of the operating system's own network daemons +([ADR 0101](../../02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md)) — and +names every one it counted, so a forgotten flag cannot close a working machine. **A machine +raised adopted stays adopted when genesis is run again** +([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)): the installer reads the mode from what the machine records, +refuses a run without the flag on an adopted machine, and refuses the flag on a converged one. *How it is checked:* the adoption bed raises genesis converged on a machine in use and asserts the refusal, then adopted with the registry's port held and asserts the refusal names its holder, then with another port given asserts the foundation comes up, stays on that port once adopted as modules, and the machine's service is still reachable. diff --git a/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md new file mode 100644 index 0000000..00977bb --- /dev/null +++ b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md @@ -0,0 +1,58 @@ +--- +status: located +opened: 2026-09-22 +located-in: [mesh-controller internal/overlay, mesh-host internal/apply] +fixed-by: +amended-design: 03-DESIGN/01-to-be/05-the-node-host.md +--- + +# 084 — Taking the networking module on an adopted node restarts every container on the machine + +## What was observed + +Found while planning the build of adoption mode +([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), by +reading the controller's networking module rather than by running it. + +The networking module declares two machine-wide files, and each one **whole**: + +- the container runtime's configuration file. It carries the trust for the mesh's plain-HTTP + registry. The runtime's service is declared to restart whenever this file changes. +- the machine's hosts file, as the fact that gives nodes their names. + +On a node the predecessor still serves, both files already exist, written by the predecessor. +Under ADR 0100 they are *found*, and they are held until the networking module is taken. Holding +them is safe. The trouble starts on either side of the hold: + +1. **Taking networking is not "one service replaced".** The first write of the runtime's + configuration restarts the container runtime. That restarts every container on the machine, + and any predecessor container without a restart policy stays down. The record promises that + taking a module replaces one service and that each step says what it changes first. This step + would replace one file and stop everything the machine serves. +2. **While the file is held, the node cannot pull the mesh's images.** The registry's trust lives + in the held file. An adopted machine that joins can therefore not pull from the mesh's + registry over the private network until networking is taken, and nothing in the modules it + is assigned says why. + +The lab does not show either problem. Its base image writes the runtime's configuration before +any bed starts, so the file the module declares is never found. + +## Why it matters beyond this instance + +A module that owns a whole machine-wide file shared with software it did not install makes +taking that module a cutover for everything on the machine. Adoption mode assumes the opposite: +that modules can be taken one at a time, each when its own data has moved. Every such file, +including the hosts file and the resolver's configuration, breaks that assumption in the same way, +and nothing checks for it today. + +## Open questions + +- Should the mesh write its part of a shared machine-wide file, beside what is already there, + instead of the whole file? And does the container runtime offer a way to add registry trust + without its main configuration file? +- If networking must replace the file whole, should taking it be a cutover of its own? Its + preview would name the restart and every container it will stop. +- Which other modules declare a whole file that other software on the machine also writes? +- Should an adopted node that cannot trust the registry be refused a module that needs to pull? + Or should the refusal come earlier, when the node joins? + diff --git a/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/01-diagnosis.md b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/01-diagnosis.md new file mode 100644 index 0000000..1a36968 --- /dev/null +++ b/04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/01-diagnosis.md @@ -0,0 +1,10 @@ +# 084 — Diagnosis + +*2026-09-22.* Worse than reported. The controller's "merge" of the runtime's file merges an +operator's settings into the module's content. The host then writes the result **whole**, so a +machine's own runtime settings, including its data directory, are replaced, not added to. +Measured on a lab machine: the runtime takes a new trusted-registry list on a reload, and a +running container without a restart policy survives it. Decided in +[ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md): the +runtime's file is written into and the runtime reloaded. The hosts file stays a whole file, held +until networking is taken. diff --git a/04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md b/04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md new file mode 100644 index 0000000..58af427 --- /dev/null +++ b/04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md @@ -0,0 +1,49 @@ +--- +status: open +opened: 2026-09-22 +located-in: [] +fixed-by: +amended-design: +--- + +# 085 — The packages port given at genesis is not a setting, and a later module can undo it + +## What was observed + +Found while fixing review findings in the build of adoption mode +([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), by +reading code rather than by running it. + +Genesis can now be given the foundation's ports, so that a mesh raised beside a predecessor does +not bind a port the predecessor holds. For the store, the bus, the broker's management port and +the registry, the port given becomes a per-node setting of the module that serves it. Every +reader then follows it: the module's container, the filter, the guard and the addresses +consumers are told. + +The package registry's port is the exception. The package registry runs as a bootstrap forge +during genesis, and the builder reaches it through a binding whose port is written into the +builder's manifest. The installer fixes the given port into that one manifest by rewriting its +text when it registers the builder. So: + +- a later registration of the builder from the catalogue puts the manifest's own port back; +- when the forge's own module takes the bootstrap forge over, the forge is configured with the + catalogue's port, not the one given, and so is the address it tells others. + +On a control-node where the predecessor's forge holds the default port, either one points the +builder, and its registry credential, at the predecessor's forge. + +## Why it matters beyond this instance + +"The foundation's ports are the node's" (ADR 0100) holds only where the port is a setting of the +module that serves it. A port fixed by rewriting text at genesis holds only until something +registers or takes over that module again, and nothing checks that it stays given. + +## Open questions + +- Should the forge's module carry the packages port as its own node setting, the way the store + does, so that the builder's binding is resolved from what the forge serves rather than written + in the builder's manifest? +- Should a binding in a manifest ever name a port, or only what it needs, with the port resolved + from the provider? +- What should check that every foundation port given at genesis is still the port in use after + the foundation is adopted as modules? diff --git a/04-ISSUES/086-taking-a-module-narrows-a-port-without-saying-so/00-report.md b/04-ISSUES/086-taking-a-module-narrows-a-port-without-saying-so/00-report.md new file mode 100644 index 0000000..d9c55a4 --- /dev/null +++ b/04-ISSUES/086-taking-a-module-narrows-a-port-without-saying-so/00-report.md @@ -0,0 +1,37 @@ +--- +status: open +opened: 2026-09-22 +located-in: [] +fixed-by: +amended-design: +--- + +# 086 — Taking a module narrows a port the predecessor served, without saying so + +## What was observed + +In the adoption lab bed, 2026-09-22. A service found on the machine served its port to anyone, +and the predecessor's firewall allowed it from anywhere. The catalogue module that replaces it +declares the port reachable from the private network only. After the module was **taken**, the +mesh's guard refused that port from everywhere outside the private network. That is +[ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md) +working as written: a taken module's published private-network ports are guarded. The service +answered over the private network, and the bed passed. + +Nothing said the port had narrowed. Taking a module names the files and containers it replaces, +but not the ports whose reach changes. The converge preview names every narrowing, but the flip +comes after taking, so by then the change has already happened. + +## Why it matters beyond this instance + +On a machine in use, taking a module is the cutover for that service. If the predecessor served +the port publicly and the module says private, taking it closes the port to every client outside +the private network. Nothing warns the operator first. ADR 0100 promises that each step says what +it changes before it changes it, and for taking a module this one does not. + +## Open questions + +- Should taking a module preview the reach of each port it publishes, found firewall and guard + included, and ask for the same kind of confirmation as the flip? +- Or should taking refuse while a port of the module is reachable more widely than the module + declares, until the operator either changes the module's exposure or confirms the narrowing? diff --git a/04-ISSUES/087-the-controller-cannot-tell-a-host-is-too-old/00-report.md b/04-ISSUES/087-the-controller-cannot-tell-a-host-is-too-old/00-report.md new file mode 100644 index 0000000..aaaf12c --- /dev/null +++ b/04-ISSUES/087-the-controller-cannot-tell-a-host-is-too-old/00-report.md @@ -0,0 +1,42 @@ +--- +status: open +opened: 2026-09-22 +located-in: [] +fixed-by: +amended-design: +--- + +# 087 — The controller cannot tell that a node's host is too old for what it sends + +## What was observed + +Found in review of the adoption-mode build, 2026-09-22, by reading the code. + +A host parses a declaration strictly: a field it does not know makes it refuse the whole +declaration, and nothing is applied. That is deliberate, and it is what keeps a half-understood +declaration off a machine. It also means every new field in a declaration is a flag day: hosts +must be upgraded before the controller sends it. + +The controller has no record of which version of the host a node runs. It is not in what a node +reports, and it is not in the node's profile. So when a node refuses a declaration for that +reason, the mesh shows the node failing and relays the host's refusal. Nothing says *the host on +this machine is older than what the mesh now sends*, and nothing could have said so beforehand. + +Adoption mode adds four such fields, one of them on the declaration of **every** node, and its +merge order exists for this reason alone: the host merges first, every machine is upgraded, then +the controller. + +## Why it matters beyond this instance + +Every future field in a declaration has the same flag day, and the mesh upgrades itself. A mesh +that cannot say which machines are ready for what it is about to send has to be upgraded by +someone remembering the order. The rule that a host refuses what it does not understand is right; +the missing half is that the controller should know before it sends. + +## Open questions + +- Should a node report the version of its host, so the controller can refuse to send a + declaration a node cannot parse, and say which machines are behind? +- Should the mesh refuse to send a field no node understands yet, or refuse per node and say so? +- Is there a general shape here — a declaration saying which version of the host it needs — or is + a reported version enough?