ADR 0100 after review: found means unrecorded; assigning prepares, taking cuts over; openings through the found firewall on both paths; the mesh guards its own ports; ports kept as node settings; a converged genesis refuses a machine in use; designs 05, 07, 08, 09 and 17 in step
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# Migrating a node that is in use — adoption as a mode, not a moment
|
||||
|
||||
*2026-09-22. Measured on the machine that will be the control-node, which is running the
|
||||
predecessor mesh today. Nothing below names it; the counts and port numbers are its own.*
|
||||
predecessor mesh today. Nothing below names it; the counts are its own.*
|
||||
|
||||
## The question
|
||||
|
||||
@@ -60,21 +60,27 @@ What does collide:
|
||||
|---|---|---|
|
||||
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
|
||||
| the broker's management port, on loopback | the predecessor's broker | at genesis |
|
||||
| the private network's port | the predecessor's own tunnel | when the node is placed on the overlay |
|
||||
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
|
||||
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
|
||||
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
|
||||
|
||||
Three of these fall at genesis and cannot be deferred; two only when a particular module is
|
||||
assigned, which is the moment that module migrates and its predecessor stops anyway. Today the
|
||||
foundation's ports are **fixed**: written in the installer's bundle and in the catalogue's
|
||||
manifests, so a collision is found when a container fails to bind, not before.
|
||||
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
|
||||
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
|
||||
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
|
||||
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
|
||||
port changed at genesis would be changed back when the foundation is adopted as modules, since the
|
||||
applier recreates a container whose declared spec differs. The private network's address range
|
||||
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
|
||||
distinct.
|
||||
|
||||
## What would break if the mesh came up as it is today
|
||||
|
||||
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
|
||||
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
|
||||
predecessor's firewall is a different table. The kernel runs every table hooked at the same point
|
||||
in turn: an accept in one lets the packet go on to the next, and a drop in any is final. So the
|
||||
predecessor's firewall is a different table. The kernel runs every base chain registered at the
|
||||
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
|
||||
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
|
||||
So the
|
||||
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
|
||||
declared — every web, mail and database port in the table above — the moment genesis ran.
|
||||
|
||||
@@ -85,8 +91,12 @@ has moved.
|
||||
|
||||
**The container names.** The applier keys a container on its name. A catalogue module whose
|
||||
container carries the same name as the predecessor's service it replaces takes that container over
|
||||
the moment it is assigned — which is exactly the cutover moment, so this is the migration's step
|
||||
and not a fault, but it means *assigning a module is migrating it*, never a rehearsal.
|
||||
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
|
||||
|
||||
**The published ports.** The foundation publishes its ports on every interface, and a published
|
||||
container port reaches the container through the forwarded path, not the incoming one. A firewall
|
||||
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
|
||||
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
|
||||
|
||||
## The pipeline freezes while the control-node migrates
|
||||
|
||||
@@ -101,19 +111,23 @@ is being replaced, and it does not need to keep changing.
|
||||
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
|
||||
intention, and each thing the mesh would otherwise take must say what it does in that state:
|
||||
|
||||
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
|
||||
path or name with no record in the host's store. A found file or container is held, its original
|
||||
recorded, until the operator **takes** the module on that node — the cutover, done when the
|
||||
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
|
||||
never fires: the host only ever sees what assigned modules declare.
|
||||
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
|
||||
adopted node. What the mesh needs open — its foundation, and each module as it migrates — it opens
|
||||
*through the found firewall*, as rules it marks as its own, and it removes only what it added.
|
||||
An accept in a table of its own would not help: the found firewall's drop would still be final.
|
||||
- **A file found at a path the mesh declares is kept**, recorded with what it contained, and
|
||||
converged only when the module that declares it migrates. Node-level files no module owns — the
|
||||
hosts file, the resolver, the ssh drop-in — converge at the flip.
|
||||
- **The foundation's ports are the node's to give.** Every port the foundation binds is set at
|
||||
genesis rather than written into a manifest, and genesis checks each one is free before it
|
||||
raises anything, naming what holds it.
|
||||
- **The flip is per node and previewed.** Converging a node replaces the found firewall with the
|
||||
mesh's derived one in one step, after saying which currently open ports would close — every one
|
||||
of them either declared by an assigned module or deliberately dropped.
|
||||
adopted node. What it needs open it declares as openings the host converges *through the found
|
||||
firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every
|
||||
reconcile so a reload or reboot does not lose them. An accept in a table of its own would not
|
||||
help: the found firewall's drop would still be final. And the mesh protects its own ports itself,
|
||||
on the forwarded path, since the found firewall may not.
|
||||
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
|
||||
that node's settings, read everywhere they are used, so adopting the foundation as modules does
|
||||
not move them back.
|
||||
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
|
||||
published ports, not the found firewall's allow list, which does not see what a container runtime
|
||||
forwards.
|
||||
|
||||
## Options weighed
|
||||
|
||||
@@ -123,6 +137,7 @@ intention, and each thing the mesh would otherwise take must say what it does in
|
||||
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
|
||||
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
|
||||
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
|
||||
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
|
||||
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
|
||||
|
||||
## What this leaves open
|
||||
|
||||
Reference in New Issue
Block a user