Merge pull request 'Adoption mode as built: ADRs 0101–0103, issues 084–086' (#75) from feat/adoption-mode into main
This commit was merged in pull request #75.
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-09-22
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 101. A machine's own resolver does not make it in use
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) has a converged genesis
|
||||
refuse a machine in use. It defines *in use* as a container running, or a port listening on an
|
||||
address other than loopback that is not ssh's. That definition was written before anything was
|
||||
measured.
|
||||
|
||||
Measured on a freshly installed lab machine, running nothing but its operating system:
|
||||
|
||||
| Listening | Held by |
|
||||
|---|---|
|
||||
| TCP and UDP on every address, the link-local name resolution port | the system's name resolver |
|
||||
| UDP on every address, the multicast name resolution port | the system's name resolver |
|
||||
| UDP on a link-local address, the address-configuration client port | the system's network manager |
|
||||
| TCP and UDP on loopback | the resolver's stub and the container runtime |
|
||||
|
||||
By ADR 0100's words, the resolver's TCP listener on every address makes **every** freshly
|
||||
installed machine a machine in use. A converged genesis would refuse them all, and `--adopted`
|
||||
would become the only way to raise anything. The refusal exists to catch a forgotten flag on a
|
||||
working machine. Refusing an empty one defeats it, and teaches operators to pass the flag by
|
||||
habit.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the words.** Rejected: every fresh machine is refused.
|
||||
2. **Count TCP only, ignore UDP.** Rejected: the resolver listens on TCP too, and a machine that
|
||||
serves over UDP alone, a resolver or a tunnel, is in use.
|
||||
3. **Ignore listeners held by the operating system's own network daemons**, a short named list,
|
||||
on both protocols. Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A listener held by one of the operating system's own network daemons does not make a machine
|
||||
in use.** The daemons are the ones the measurement found: the name resolver and the network
|
||||
manager, named in the installer's code beside that measurement. Everything else in ADR 0100's definition stands: a running container, or any other
|
||||
listener on an address other than loopback that is not ssh's, makes the machine in use, and
|
||||
genesis still names every one it counted.
|
||||
|
||||
A daemon is added to the list only with a measurement of a fresh machine that holds it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A converged genesis on a fresh machine goes ahead, as it did before ADR 0100.
|
||||
- A machine whose resolver is also serving other machines is not counted as in use by its
|
||||
resolver alone. It is one of the daemons that serves nobody on a fresh machine, and the one
|
||||
kind of service this lets through.
|
||||
- The list is code, not configuration, so it changes by review.
|
||||
|
||||
## How it is checked
|
||||
|
||||
A unit test in the installer feeds the listeners captured from the fresh machine, as the
|
||||
listening-socket tool printed them, and asserts the machine is not in use. The existing tests
|
||||
still assert that a serving machine is in use and that every container and listener is named.
|
||||
The adoption lab bed raises a converged genesis on a fresh machine and asserts it is not refused.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||
- [research 012, *migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-09-22
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 102. The mesh writes into a shared file, never over it
|
||||
|
||||
## Context
|
||||
|
||||
[Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
||||
found that the networking module declares the container runtime's configuration file to state
|
||||
one fact in it: the mesh's registry is trusted over the private network. Reading the code
|
||||
closer showed it worse than reported. The controller merges an operator's settings into the
|
||||
module's own content, but the host writes the result **whole**. Whatever the machine had in that
|
||||
file is replaced, including where the runtime keeps its data. On a machine in use, that is every
|
||||
image and container gone from the runtime's view at its next start. The runtime's service is then
|
||||
restarted, which stops every container on the machine.
|
||||
|
||||
Measured on a lab machine: the runtime takes a new list of trusted registries on a reload, with
|
||||
no restart, and a running container with no restart policy keeps running through it. The
|
||||
runtime's log says it reloaded its configuration, and the registry reads as trusted afterwards.
|
||||
|
||||
Under [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md), the file is found
|
||||
on an adopted node and held until networking is taken. Holding it is safe, but it means an
|
||||
adopted machine cannot pull the mesh's images until then, and taking networking would restart
|
||||
the runtime.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep writing the whole file; take networking as a cutover of its own.** Rejected: it
|
||||
still replaces the machine's settings, and the cutover stops everything on the machine.
|
||||
2. **A drop-in the runtime reads beside its main file.** Rejected: the runtime has no such
|
||||
directory for its daemon settings.
|
||||
3. **Write into the file: set the mesh's keys, keep the rest; reload, don't restart.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A file the mesh shares with software it did not install is written into, never over.** A
|
||||
file resource may say it is written *into* a structured file. The host then reads what is there,
|
||||
sets only the keys the mesh declares, keeps every other key as it found it, and records what each
|
||||
of its keys held before. **A list is added to, never replaced**: where the machine already has a
|
||||
list under a key the mesh declares, the mesh's members are added to it and the host records
|
||||
exactly which members it added — setting the key would replace the operator's own list, the harm
|
||||
this record exists to prevent. Undeclared later, each key goes back to what it held and each
|
||||
added member is taken out again, and a file the mesh created is removed only if nothing but its
|
||||
own keys is left. A file written into replaces nothing, so on an adopted node it is never held:
|
||||
it is written whether or not its module has been taken.
|
||||
|
||||
**Whatever the host writes over without a record of it, it keeps first.** On any node, adopted
|
||||
or converged, before the host writes a file over one it has no record of making, it keeps the
|
||||
original once and says where; if it cannot keep it, it does not write. A file the mesh takes over
|
||||
is then never lost, whatever put it there.
|
||||
|
||||
**A service that re-reads its configuration on a reload is reloaded, not restarted.** A service
|
||||
resource may name what it must be *reloaded* on, beside what it must be restarted on. The
|
||||
container runtime is reloaded for the registry's trust.
|
||||
|
||||
**The networking module writes the runtime's trust into its file and reloads it.** An adopted
|
||||
node therefore trusts the mesh's registry as soon as it is on the private network, and taking
|
||||
networking no longer touches the runtime. The hosts file networking writes is still written whole
|
||||
and stays held until networking is taken; a converge preview names it among the files it replaces.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The runtime's file on a machine in use keeps its data directory, its logging settings and
|
||||
everything else the predecessor set.
|
||||
- A host that does not know *into* or *reload-on* refuses a declaration carrying them, so hosts
|
||||
are upgraded before the controller that emits them — the same order ADR 0100 needs.
|
||||
- One shape more for every host: a file written into a structured document. Only JSON is spoken;
|
||||
another format is refused until written.
|
||||
- The hosts file remains a whole file. Writing a marked block into it is the same idea for a text
|
||||
file and is not decided here.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Unit tests hold the host to setting only the declared keys and keeping the rest, restoring each
|
||||
key and removing only a file it created when the resource is undeclared, adding to a list and
|
||||
removing only the members it added, keeping the original of a file it writes over without a
|
||||
record, refusing a file that is not a JSON object rather than overwriting it, never holding a file written into on an adopted
|
||||
node, and reloading rather than restarting a service whose reload-on resource changed. A test in
|
||||
the controller holds the networking module to declaring the runtime's file written into and the
|
||||
runtime reloaded. The adoption lab bed gives the machine a runtime file with a setting of its own
|
||||
and asserts it survives adoption with the registry trusted added, and that a container without a
|
||||
restart policy is still running afterwards.
|
||||
|
||||
## References
|
||||
|
||||
- [Issue 084](../04-ISSUES/084-taking-networking-on-an-adopted-node-restarts-every-container/00-report.md)
|
||||
- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md), [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||
@@ -0,0 +1,112 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-09-22
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 103. What an adopted node holds, and what its guard refuses
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md) holds a found file or
|
||||
container until its module is taken. It also has the mesh guard the store's port and the
|
||||
broker's management port with a table that only refuses. Two independent reviews of the build
|
||||
found both rules drawn too narrowly and, for the guard, in the wrong place:
|
||||
|
||||
1. **Other kinds of resource reach what was found.** An untaken module's directory is re-owned
|
||||
and re-moded if a predecessor's directory is already at its path, and a database refuses to
|
||||
start on a data directory whose mode changed. A service of the same name as a predecessor's
|
||||
unit is started, stopped or re-enabled. An action run *in* a held container runs inside the
|
||||
predecessor's service. A container that is not found under its own name is created, and can
|
||||
mount the predecessor's data beside the predecessor's own container.
|
||||
2. **The guard's two ports are not the only ones a found firewall misses.** A published
|
||||
container port is forwarded, not received, and a firewall that filters only incoming traffic
|
||||
never sees it (ADR 0100's own Context). The broker's plaintext port is published on every
|
||||
interface and admitted by the filter from the private network only. So on an adopted node it
|
||||
is reachable from anywhere. The same holds for any published port a module restricts to the
|
||||
private network.
|
||||
3. **The guard refuses by port alone.** On a machine that routes for others, such as a
|
||||
predecessor's private-network hub, a packet for another machine's database port is refused as
|
||||
well. And a port the guard refuses for a module that is assigned but not taken may still be
|
||||
the predecessor's own, serving the predecessor's other machines.
|
||||
|
||||
## Decision
|
||||
|
||||
**Found covers every kind that can reach what the machine already has.** On an adopted node,
|
||||
for a module not yet taken:
|
||||
|
||||
- a **directory** present with no record is held: its mode and owner are left, and nothing in it
|
||||
is touched;
|
||||
- a **service** with no record is held when an administrator installed its unit — the service
|
||||
manager reads the unit from outside the packages' own directory — or when the machine uses it,
|
||||
running or started at boot. Its state and whether it starts at boot are then left. A reload
|
||||
named by the module still happens, since a reload stops nothing
|
||||
([ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). A unit a package merely
|
||||
ships that nothing runs and nothing enables is not found: the private network's own tunnel unit
|
||||
is one, and holding it kept the private network from ever coming up;
|
||||
- an **archive** whose target is present with no record is not unpacked or re-owned; a
|
||||
**process** whose unit file is present with no record is not written over or restarted; a
|
||||
**user** that exists with no record keeps its shell and groups;
|
||||
- a **container** not found by its name is still held if it would mount a path or a volume that
|
||||
is present with no record, because it would share the predecessor's data;
|
||||
- an **action** or one-off step run *in* a held container is held until the module is taken.
|
||||
|
||||
What is held is reported as held and is never removed, as ADR 0100 decides for files and
|
||||
containers.
|
||||
|
||||
**The guard refuses what the found firewall may miss, for taken modules only.** Its ports are
|
||||
derived from each *taken* module: every machine port it publishes that the filter would admit
|
||||
from the private network only, and the ports its manifest names under `guards` — which is how
|
||||
the store's port and the broker's management port are included. Only TCP is guarded. A port of a module that is assigned but not taken is not guarded: it may still
|
||||
be the predecessor's. The guard matches only packets addressed to this machine. Traffic the
|
||||
machine routes for others is never its business.
|
||||
|
||||
**An opening already answered by a found rule is not added.** If the found firewall already
|
||||
admits what an opening says, the opening is reported as satisfied by the found rule, and the
|
||||
mesh adds nothing and later removes nothing. The found firewall treats two rules differing only
|
||||
in their action, log setting or comment as one rule, so adding the mesh's would take over the
|
||||
operator's. Where the found rule is such a twin but is not a plain allow — a deny, a limit, a
|
||||
logged allow — the opening is refused, naming that rule, and nothing is added.
|
||||
|
||||
**A machine raised adopted stays adopted if genesis is run again.** The installer reads the
|
||||
node's mode from what the machine records, not only from the operator's flag. A run without
|
||||
the flag on an adopted machine is refused.
|
||||
|
||||
**What of ADR 0100 this replaces.** ADR 0100's guard refused two ports, the foundation's own,
|
||||
checked free at genesis, and could therefore close nothing the machine served. That guard is
|
||||
replaced by the one above: a guarded port of a taken module may be one the predecessor served
|
||||
more widely, and taking the module narrows it to the private network. Everything else in ADR 0100
|
||||
stands, and its rules for files and containers now cover the other kinds listed here.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Taking a module can narrow a port the predecessor served to anyone: the guard then refuses it
|
||||
from outside the private network, as the module declares. Nothing says so yet
|
||||
([issue 086](../04-ISSUES/086-taking-a-module-narrows-a-port-without-saying-so/00-report.md)).
|
||||
- An untaken module on an adopted node can come up only beside what was found, never on top of
|
||||
it: whatever would share the predecessor's data waits for the cutover.
|
||||
- The guard grows with the modules taken, and a module's published private-network port is
|
||||
protected on an adopted node the way the derived filter protects it on a converged one.
|
||||
- Guarding only taken modules means a foundation port on a node joining adopted is guarded once
|
||||
its module is taken, not when it is assigned. Genesis takes the foundation's modules, so the
|
||||
control-node's store is guarded from the first push.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Unit tests hold the host to holding a found directory, a found service, a found archive, process
|
||||
and user, and a container that would mount found data; to not holding a unit nothing runs; to
|
||||
deferring an action run in a held container; to adding no opening a found rule answers, and to
|
||||
refusing one a conflicting found rule would absorb. They hold the controller to deriving the guard from taken modules'
|
||||
private-network published ports, and the guard's text to matching only this machine's
|
||||
addresses. The installer's tests hold a re-run without the flag on an adopted machine to a
|
||||
refusal. The adoption lab bed asserts that the broker's plaintext port is unreachable from
|
||||
outside the private network even with the found firewall admitting it, and that an operator's
|
||||
own rule for a port the mesh opens survives the opening being removed.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0100](0100-a-node-in-use-is-adopted-before-it-is-converged.md),
|
||||
[ADR 0102](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)
|
||||
@@ -89,6 +89,9 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md)
|
||||
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
|
||||
- **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||
- **0101** — [A machine's own resolver does not make it in use](0101-a-machines-own-resolver-does-not-make-it-in-use.md)
|
||||
- **0102** — [The mesh writes into a shared file, never over it](0102-the-mesh-writes-into-a-shared-file-never-over-it.md)
|
||||
- **0103** — [What an adopted node holds, and what its guard refuses](0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
@@ -5,6 +5,8 @@ code: [mesh-host]
|
||||
updated: 2026-09-22
|
||||
decisions:
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
- 02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md
|
||||
- 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md
|
||||
- 02-DECISIONS/0019-how-this-repository-works.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
@@ -150,6 +152,21 @@ checked:* unit tests hold the host to keeping a found file and container, conver
|
||||
taken, never removing a held file and reporting one that changed; the adoption bed asserts a found
|
||||
file byte for byte unchanged until its module is taken.
|
||||
|
||||
**Found reaches every kind that can touch what the machine has**
|
||||
([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record
|
||||
keeps its mode and owner, a unit present with no record keeps its state and boot setting, a
|
||||
container that would mount found data is not created, and an action run in a held container
|
||||
waits for the cutover. **A file the machine shares is written into, never over**
|
||||
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)): the host sets the mesh's keys in the object already there,
|
||||
keeps every other key, adds its members to a list already there rather than replacing it,
|
||||
records what each of its keys held and which members it added, and gives them back when the
|
||||
file is undeclared. Such a file replaces nothing, so it is never held. And on any node, before
|
||||
the host writes over a file it has no record of making, it keeps the original once and names
|
||||
where; if it cannot keep it, it does not write. A service that re-reads
|
||||
its configuration is **reloaded** for what it names in `reload-on`, never restarted. *How it is
|
||||
checked:* unit tests hold the host to each of these, and the adoption bed asserts the runtime's
|
||||
own settings survive adoption and a container without a restart policy keeps running.
|
||||
|
||||
## Where a declaration comes from
|
||||
|
||||
One behaviour, two sources
|
||||
|
||||
@@ -9,6 +9,7 @@ code:
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-09-22
|
||||
decisions:
|
||||
- 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
- 02-DECISIONS/0099-a-step-that-runs-once-names-what-it-reads.md
|
||||
- 02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md
|
||||
@@ -550,9 +551,13 @@ the found firewall in that firewall's own terms, marks it as the mesh's, removes
|
||||
marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for
|
||||
longer than one reconcile. An opening is state, not a command, so it travels over the link like any
|
||||
other resource. **The mesh guards its own ports in a table of its own that only refuses** —
|
||||
passing everything by default and holding nothing but refusals of the foundation's own two ports,
|
||||
checked free at genesis, so it cannot close what the machine serves, and the found firewall's
|
||||
reload does not touch it. It refuses the store's port and the broker's management port except from
|
||||
passing everything by default and holding nothing but refusals, and the found firewall's reload
|
||||
does not touch it. Its ports are derived ([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)): every machine port a
|
||||
*taken* module publishes that the filter admits from the private network only, with the store's
|
||||
port and the broker's management port. A port of a module not yet taken may still be the
|
||||
predecessor's, so it is not guarded; and only packets addressed to this machine are matched, so
|
||||
traffic it routes for others passes. An opening a found rule already answers is not added, so
|
||||
removing it never removes the operator's rule. It refuses those ports except from
|
||||
the private network and the machine itself — loopback and the container runtime's networks, known by the interface a packet arrives on, never by
|
||||
its source address alone — at
|
||||
the prerouting hook, before the container runtime redirects the packet, so it matches the port the
|
||||
|
||||
@@ -300,8 +300,12 @@ cutover. The firewall found there stays in force and the mesh opens what it need
|
||||
([08-connectivity](08-connectivity.md)). **Converging is one act per node, previewed**: it refuses
|
||||
while an assigned module still holds a found container; otherwise it lists what is reachable on the
|
||||
machine now — listening sockets and published ports — whether an assigned module declares each or
|
||||
it will close, and which modules it will take, then takes them, loads the mesh's own filter in place
|
||||
of its refusal-only table and disables the found firewall without flushing it. Returning a
|
||||
it will close, which modules it will take and every held thing each will replace, and ends with a
|
||||
short digest of all of that. **The flip is made only with that digest** — the operator confirms
|
||||
the preview they read, and a preview that has changed since, or whose account of the machine is
|
||||
more than fifteen minutes old, is refused. It then takes the modules, loads the mesh's own filter
|
||||
in place of its refusal-only table and disables the found firewall without flushing it, putting
|
||||
back the forwarding policy the found firewall had set. Returning a
|
||||
converged node to adopted unloads the derived filter, restores the refusal-only table, enables the
|
||||
found firewall again and converges the openings through it; what was taken stays taken. *How it is checked:* a lab bed prepares a machine the way
|
||||
a predecessor leaves one and asserts nothing that serves changes until a module is taken or the
|
||||
|
||||
@@ -8,6 +8,8 @@ code:
|
||||
updated: 2026-09-22
|
||||
decisions:
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
- 02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md
|
||||
- 02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md
|
||||
- 02-DECISIONS/0067-genesis-is-a-pivot.md
|
||||
- 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md
|
||||
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
|
||||
@@ -113,8 +115,13 @@ checks the private network's range does not overlap a tunnel the predecessor run
|
||||
**does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its
|
||||
foundation's ports through it and keeps the store from outside with a table of its own that only
|
||||
refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** —
|
||||
a container running, or a port listening on an address other than loopback that is not ssh's —
|
||||
and names every one it counted, so a forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
|
||||
a container running, or a port listening on an address other than loopback that is neither ssh's
|
||||
nor held by one of the operating system's own network daemons
|
||||
([ADR 0101](../../02-DECISIONS/0101-a-machines-own-resolver-does-not-make-it-in-use.md)) — and
|
||||
names every one it counted, so a forgotten flag cannot close a working machine. **A machine
|
||||
raised adopted stays adopted when genesis is run again**
|
||||
([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)): the installer reads the mode from what the machine records,
|
||||
refuses a run without the flag on an adopted machine, and refuses the flag on a converged one. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
|
||||
asserts the refusal, then adopted with the registry's port held and asserts the refusal names its
|
||||
holder, then with another port given asserts the foundation comes up, stays on that port once
|
||||
adopted as modules, and the machine's service is still reachable.
|
||||
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-09-22
|
||||
located-in: [mesh-controller internal/overlay, mesh-host internal/apply]
|
||||
fixed-by:
|
||||
amended-design: 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
---
|
||||
|
||||
# 084 — Taking the networking module on an adopted node restarts every container on the machine
|
||||
|
||||
## What was observed
|
||||
|
||||
Found while planning the build of adoption mode
|
||||
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), by
|
||||
reading the controller's networking module rather than by running it.
|
||||
|
||||
The networking module declares two machine-wide files, and each one **whole**:
|
||||
|
||||
- the container runtime's configuration file. It carries the trust for the mesh's plain-HTTP
|
||||
registry. The runtime's service is declared to restart whenever this file changes.
|
||||
- the machine's hosts file, as the fact that gives nodes their names.
|
||||
|
||||
On a node the predecessor still serves, both files already exist, written by the predecessor.
|
||||
Under ADR 0100 they are *found*, and they are held until the networking module is taken. Holding
|
||||
them is safe. The trouble starts on either side of the hold:
|
||||
|
||||
1. **Taking networking is not "one service replaced".** The first write of the runtime's
|
||||
configuration restarts the container runtime. That restarts every container on the machine,
|
||||
and any predecessor container without a restart policy stays down. The record promises that
|
||||
taking a module replaces one service and that each step says what it changes first. This step
|
||||
would replace one file and stop everything the machine serves.
|
||||
2. **While the file is held, the node cannot pull the mesh's images.** The registry's trust lives
|
||||
in the held file. An adopted machine that joins can therefore not pull from the mesh's
|
||||
registry over the private network until networking is taken, and nothing in the modules it
|
||||
is assigned says why.
|
||||
|
||||
The lab does not show either problem. Its base image writes the runtime's configuration before
|
||||
any bed starts, so the file the module declares is never found.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
A module that owns a whole machine-wide file shared with software it did not install makes
|
||||
taking that module a cutover for everything on the machine. Adoption mode assumes the opposite:
|
||||
that modules can be taken one at a time, each when its own data has moved. Every such file,
|
||||
including the hosts file and the resolver's configuration, breaks that assumption in the same way,
|
||||
and nothing checks for it today.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the mesh write its part of a shared machine-wide file, beside what is already there,
|
||||
instead of the whole file? And does the container runtime offer a way to add registry trust
|
||||
without its main configuration file?
|
||||
- If networking must replace the file whole, should taking it be a cutover of its own? Its
|
||||
preview would name the restart and every container it will stop.
|
||||
- Which other modules declare a whole file that other software on the machine also writes?
|
||||
- Should an adopted node that cannot trust the registry be refused a module that needs to pull?
|
||||
Or should the refusal come earlier, when the node joins?
|
||||
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
# 084 — Diagnosis
|
||||
|
||||
*2026-09-22.* Worse than reported. The controller's "merge" of the runtime's file merges an
|
||||
operator's settings into the module's content. The host then writes the result **whole**, so a
|
||||
machine's own runtime settings, including its data directory, are replaced, not added to.
|
||||
Measured on a lab machine: the runtime takes a new trusted-registry list on a reload, and a
|
||||
running container without a restart policy survives it. Decided in
|
||||
[ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md): the
|
||||
runtime's file is written into and the runtime reloaded. The hosts file stays a whole file, held
|
||||
until networking is taken.
|
||||
@@ -0,0 +1,49 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-22
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 085 — The packages port given at genesis is not a setting, and a later module can undo it
|
||||
|
||||
## What was observed
|
||||
|
||||
Found while fixing review findings in the build of adoption mode
|
||||
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), by
|
||||
reading code rather than by running it.
|
||||
|
||||
Genesis can now be given the foundation's ports, so that a mesh raised beside a predecessor does
|
||||
not bind a port the predecessor holds. For the store, the bus, the broker's management port and
|
||||
the registry, the port given becomes a per-node setting of the module that serves it. Every
|
||||
reader then follows it: the module's container, the filter, the guard and the addresses
|
||||
consumers are told.
|
||||
|
||||
The package registry's port is the exception. The package registry runs as a bootstrap forge
|
||||
during genesis, and the builder reaches it through a binding whose port is written into the
|
||||
builder's manifest. The installer fixes the given port into that one manifest by rewriting its
|
||||
text when it registers the builder. So:
|
||||
|
||||
- a later registration of the builder from the catalogue puts the manifest's own port back;
|
||||
- when the forge's own module takes the bootstrap forge over, the forge is configured with the
|
||||
catalogue's port, not the one given, and so is the address it tells others.
|
||||
|
||||
On a control-node where the predecessor's forge holds the default port, either one points the
|
||||
builder, and its registry credential, at the predecessor's forge.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
"The foundation's ports are the node's" (ADR 0100) holds only where the port is a setting of the
|
||||
module that serves it. A port fixed by rewriting text at genesis holds only until something
|
||||
registers or takes over that module again, and nothing checks that it stays given.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the forge's module carry the packages port as its own node setting, the way the store
|
||||
does, so that the builder's binding is resolved from what the forge serves rather than written
|
||||
in the builder's manifest?
|
||||
- Should a binding in a manifest ever name a port, or only what it needs, with the port resolved
|
||||
from the provider?
|
||||
- What should check that every foundation port given at genesis is still the port in use after
|
||||
the foundation is adopted as modules?
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-22
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 086 — Taking a module narrows a port the predecessor served, without saying so
|
||||
|
||||
## What was observed
|
||||
|
||||
In the adoption lab bed, 2026-09-22. A service found on the machine served its port to anyone,
|
||||
and the predecessor's firewall allowed it from anywhere. The catalogue module that replaces it
|
||||
declares the port reachable from the private network only. After the module was **taken**, the
|
||||
mesh's guard refused that port from everywhere outside the private network. That is
|
||||
[ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)
|
||||
working as written: a taken module's published private-network ports are guarded. The service
|
||||
answered over the private network, and the bed passed.
|
||||
|
||||
Nothing said the port had narrowed. Taking a module names the files and containers it replaces,
|
||||
but not the ports whose reach changes. The converge preview names every narrowing, but the flip
|
||||
comes after taking, so by then the change has already happened.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
On a machine in use, taking a module is the cutover for that service. If the predecessor served
|
||||
the port publicly and the module says private, taking it closes the port to every client outside
|
||||
the private network. Nothing warns the operator first. ADR 0100 promises that each step says what
|
||||
it changes before it changes it, and for taking a module this one does not.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should taking a module preview the reach of each port it publishes, found firewall and guard
|
||||
included, and ask for the same kind of confirmation as the flip?
|
||||
- Or should taking refuse while a port of the module is reachable more widely than the module
|
||||
declares, until the operator either changes the module's exposure or confirms the narrowing?
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-09-22
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 087 — The controller cannot tell that a node's host is too old for what it sends
|
||||
|
||||
## What was observed
|
||||
|
||||
Found in review of the adoption-mode build, 2026-09-22, by reading the code.
|
||||
|
||||
A host parses a declaration strictly: a field it does not know makes it refuse the whole
|
||||
declaration, and nothing is applied. That is deliberate, and it is what keeps a half-understood
|
||||
declaration off a machine. It also means every new field in a declaration is a flag day: hosts
|
||||
must be upgraded before the controller sends it.
|
||||
|
||||
The controller has no record of which version of the host a node runs. It is not in what a node
|
||||
reports, and it is not in the node's profile. So when a node refuses a declaration for that
|
||||
reason, the mesh shows the node failing and relays the host's refusal. Nothing says *the host on
|
||||
this machine is older than what the mesh now sends*, and nothing could have said so beforehand.
|
||||
|
||||
Adoption mode adds four such fields, one of them on the declaration of **every** node, and its
|
||||
merge order exists for this reason alone: the host merges first, every machine is upgraded, then
|
||||
the controller.
|
||||
|
||||
## Why it matters beyond this instance
|
||||
|
||||
Every future field in a declaration has the same flag day, and the mesh upgrades itself. A mesh
|
||||
that cannot say which machines are ready for what it is about to send has to be upgraded by
|
||||
someone remembering the order. The rule that a host refuses what it does not understand is right;
|
||||
the missing half is that the controller should know before it sends.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a node report the version of its host, so the controller can refuse to send a
|
||||
declaration a node cannot parse, and say which machines are behind?
|
||||
- Should the mesh refuse to send a field no node understands yet, or refuse per node and say so?
|
||||
- Is there a general shape here — a declaration saying which version of the host it needs — or is
|
||||
a reported version enough?
|
||||
Reference in New Issue
Block a user