novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Research 012: migrating a node that is in use, measured on the control-node; ADR 0100 proposed — a node in use is adopted before it is converged

Browse Source
This commit is contained in:
jschoubben
2026-09-22 16:16:28 +02:00
parent 21baf397a8
commit 5fc3cbde4c
4 changed files with 295 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
+145
View File
@@ -0,0 +1,145 @@
---
topic: the mesh
status: proposed
date: 2026-09-22
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0078-the-store-and-broker-are-modules.md
---
# 100. A node in use is adopted before it is converged
## Context
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
people use. The control-node is the machine that already carries the predecessor's broker and
build pipeline. The operator proposed the migration's shape: stop the predecessor's control on a
machine, bring the mesh up there in an adoption mode that keeps the machine's configuration in
force, migrate its modules one at a time, move to the next machine, and flip adoption off when every
machine is done.
Measured on the control-node ([research 012,
*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)):
60 predecessor containers; the predecessor's control in user-level daemons separate from its
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
explicitly; 12 files its configuration sync writes, 3 of them system files.
Three things in the mesh as it stands break that shape:
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every table at a hook is
run in turn and a drop in any is final, so the predecessor's allowed ports — web, mail, stores —
would close at genesis.
2. **The host replaces what it finds.** A declared file is written whatever is at its path. A
file the predecessor left is replaced as soon as any module declaring its path is assigned.
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
predecessor publishes its own elsewhere — but the registry's port and the broker's management
port are held by the predecessor's, and the private network's port by its tunnel. The
foundation's ports are fixed in the installer's bundle and the catalogue's manifests, so the
collision surfaces as a container that fails to bind.
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
modules, keyed on the container that is already running
([ADR 0078](0078-the-store-and-broker-are-modules.md)). The research this record draws on already
settled the conflict rule for adoption: *on conflict, what is on the machine stays*.
## Considered Options
1. **Cut each machine over in one go** — stop the predecessor's store, broker, registry and proxy,
raise the foundation in their place. Rejected: every predecessor service on the machine is down
until it has migrated, the predecessor's other machines lose their broker, and the rollback is
restarting the predecessor — a recovery, not a step.
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
found files are replaced before their services move.
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the
bind collisions and neither the firewall nor the files.
5. **Adoption as a mode per node, ended by an explicit flip.** Adopted.
## Decision
**A node is either adopted or converged, and the mesh records which.** A node joins adopted when
its machine is in use; the operator says so at genesis for the control-node and at enrolment for
the others. It stays adopted until the operator converges it. Adopted is not a one-time import
before generating starts: it lasts for as long as the machine is being migrated.
**Before a node is adopted, the predecessor's control on it is stopped** — the daemons that write
its configuration. Its services keep running on the configuration they have.
**On an adopted node:**
- **The firewall found on the machine stays in force.** The mesh does not load its own table
there — neither genesis's base ruleset nor the filter module's derived one. What the mesh needs
reachable — its foundation's ports and, as each module migrates, that module's declared
`listens` — it opens *through the found firewall*, as rules marked as the mesh's. It removes only
rules it marked, and edits nothing else.
- **A file found at a path the mesh declares is kept.** The host records what it found there and
does not replace it while the node is adopted. A module's files converge when that module is
assigned, because assigning a module is migrating it. Files no module owns — the hosts file, the
resolver, the ssh drop-in — converge at the flip.
- **The foundation takes the ports it is given.** Every port the foundation binds is an input to
genesis rather than a constant, and genesis checks each is free before raising anything,
refusing with the name of what holds it.
**Converging a node is one act, previewed.** It first lists every port the found firewall allows
and says, for each, whether an assigned module declares it or it will close; then it loads the
mesh's derived filter, retires the found firewall, and converges the files kept at adoption. A
node converges when its migration is done. The mesh is migrated when every node has converged.
**The order is the operator's:** the control-node first, adopted, its modules migrated one at a
time; then each other machine, adopted, migrated, converged in turn. The predecessor's pipeline
runs on the control-node, so its updates stop for every machine while the migration runs; that is
accepted.
## Consequences
The migration becomes a sequence of reversible steps. A module moved is one service changed; a node
adopted is a node on which nothing changed; the flip is the one act that changes what is reachable,
and it says beforehand what it will change.
What got harder:
- **The mesh must speak a firewall it did not install.** Opening a port through the found
firewall means writing to it in its own terms. One kind is found on the machines measured; a
machine with another is not covered until someone writes for it.
- **The host gains a guard it did not have** — *do not replace what you found* — and its report
has to say which files it is holding rather than converging, or an adopted node looks converged.
That is the companion to the host's rule of never touching what it did not create
([ADR 0005](0005-the-node-host.md)) that the research asked for.
- **Genesis grows inputs.** The foundation's ports stop being constants in the bundle and the
catalogue; a mesh that never needed to move them now carries the choice.
- **A node can sit adopted indefinitely.** Nothing forces the flip, and an adopted node filters
with a firewall the mesh does not derive; the mesh's own status has to say which nodes are
adopted, so that one left behind is visible.
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
nodes**: the base ruleset is not loaded on a node that is adopted at genesis, because the
machine's own firewall already filters and a second, stricter table would close it.
## How it is checked
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
port and denying the rest, a service listening on that port, a file at a path a mesh module
declares, and a container holding the registry's port. Then:
- **Genesis adopted, with the registry's port held**, refuses and names what holds it; with
another port given, the foundation comes up.
- **Nothing on the machine changed**: the service is still reachable from a second machine, the
found file is byte for byte what it was, and the found firewall's rules differ only by rules
marked as the mesh's.
- **The mesh works through the found firewall**: the second machine enrols over the bus the mesh
opened there.
- **A module assigned migrates only itself**: its port is opened, its files converge, the found
file it does not own is untouched.
- **Converging previews, then changes**: the preview names the service's port as closing unless
declared; after the flip the mesh's derived filter is loaded, the found firewall is retired, the
declared port is open and the undeclared one is closed.
Unit tests hold the host to keeping a found file on an adopted node and replacing it on a converged
one, and genesis to refusing a held port.
## References
- [research 012 — the minimum viable node, and adopting what is already there](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md),
and its document [*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)
- [ADR 0005](0005-the-node-host.md), [ADR 0011](0011-managed-files-are-generated-never-edited.md),
[ADR 0078](0078-the-store-and-broker-are-modules.md), [ADR 0088](0088-the-foundation-filters-before-anything-listens.md)
02-DECISIONS/README.md
+1
View File
@@ -88,6 +88,7 @@ python3 00-META/checks/index.py fail if stale
- **0083** — [One push leaves the mesh consistent](0083-one-push-leaves-the-mesh-consistent.md)
- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md)
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
- **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md) *(proposed)*
### Its tiers, from the bottom up