ADR 0100 after review: found means unrecorded; assigning prepares, taking cuts over; openings through the found firewall on both paths; the mesh guards its own ports; ports kept as node settings; a converged genesis refuses a machine in use; designs 05, 07, 08, 09 and 17 in step

This commit is contained in:
2026-09-22 16:28:31 +02:00
parent 111456abb5
commit 02c40bcab4
7 changed files with 250 additions and 136 deletions
@@ -1,7 +1,7 @@
# Migrating a node that is in use — adoption as a mode, not a moment
*2026-09-22. Measured on the machine that will be the control-node, which is running the
predecessor mesh today. Nothing below names it; the counts and port numbers are its own.*
predecessor mesh today. Nothing below names it; the counts are its own.*
## The question
@@ -60,21 +60,27 @@ What does collide:
|---|---|---|
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
| the broker's management port, on loopback | the predecessor's broker | at genesis |
| the private network's port | the predecessor's own tunnel | when the node is placed on the overlay |
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
Three of these fall at genesis and cannot be deferred; two only when a particular module is
assigned, which is the moment that module migrates and its predecessor stops anyway. Today the
foundation's ports are **fixed**: written in the installer's bundle and in the catalogue's
manifests, so a collision is found when a container fails to bind, not before.
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
port changed at genesis would be changed back when the foundation is adopted as modules, since the
applier recreates a container whose declared spec differs. The private network's address range
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
distinct.
## What would break if the mesh came up as it is today
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
predecessor's firewall is a different table. The kernel runs every table hooked at the same point
in turn: an accept in one lets the packet go on to the next, and a drop in any is final. So the
predecessor's firewall is a different table. The kernel runs every base chain registered at the
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
So the
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
declared — every web, mail and database port in the table above — the moment genesis ran.
@@ -85,8 +91,12 @@ has moved.
**The container names.** The applier keys a container on its name. A catalogue module whose
container carries the same name as the predecessor's service it replaces takes that container over
the moment it is assigned — which is exactly the cutover moment, so this is the migration's step
and not a fault, but it means *assigning a module is migrating it*, never a rehearsal.
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
**The published ports.** The foundation publishes its ports on every interface, and a published
container port reaches the container through the forwarded path, not the incoming one. A firewall
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
## The pipeline freezes while the control-node migrates
@@ -101,19 +111,23 @@ is being replaced, and it does not need to keep changing.
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
intention, and each thing the mesh would otherwise take must say what it does in that state:
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
path or name with no record in the host's store. A found file or container is held, its original
recorded, until the operator **takes** the module on that node — the cutover, done when the
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
never fires: the host only ever sees what assigned modules declare.
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
adopted node. What the mesh needs open — its foundation, and each module as it migrates — it opens
*through the found firewall*, as rules it marks as its own, and it removes only what it added.
An accept in a table of its own would not help: the found firewall's drop would still be final.
- **A file found at a path the mesh declares is kept**, recorded with what it contained, and
converged only when the module that declares it migrates. Node-level files no module owns — the
hosts file, the resolver, the ssh drop-in — converge at the flip.
- **The foundation's ports are the node's to give.** Every port the foundation binds is set at
genesis rather than written into a manifest, and genesis checks each one is free before it
raises anything, naming what holds it.
- **The flip is per node and previewed.** Converging a node replaces the found firewall with the
mesh's derived one in one step, after saying which currently open ports would close — every one
of them either declared by an assigned module or deliberately dropped.
adopted node. What it needs open it declares as openings the host converges *through the found
firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every
reconcile so a reload or reboot does not lose them. An accept in a table of its own would not
help: the found firewall's drop would still be final. And the mesh protects its own ports itself,
on the forwarded path, since the found firewall may not.
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
that node's settings, read everywhere they are used, so adopting the foundation as modules does
not move them back.
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
published ports, not the found firewall's allow list, which does not see what a container runtime
forwards.
## Options weighed
@@ -123,6 +137,7 @@ intention, and each thing the mesh would otherwise take must say what it does in
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
## What this leaves open