Migrating a node that is in use — adoption as a mode, not a moment

2026-09-22. Measured on the machine that will be the control-node, which is running the predecessor mesh today. Nothing below names it; the counts and port numbers are its own.

The question

The mesh replaces a predecessor mesh that is running, on the same machines, with the services people use. The control-node is decided: it is the machine that already carries the predecessor's broker and build pipeline. So the question is not where the mesh starts but how a machine running the predecessor becomes a node of the mesh without its services noticing — and then how each of the other machines follows.

The operator's proposal

Proposed by the operator, 2026-09-22, and the shape this document tests:

Stop the predecessor's control on a machine — its daemons that write configuration: the network and firewall configuration above all, which decide what is reachable and what is blocked. Its services keep running; only the control over their configuration stops.
Bring the mesh up on that machine in adoption mode. It takes custody of those files and keeps what it finds in force.
Migrate the modules one at a time, data preserved, per the cutover procedure.
Move to the next machine and repeat — adopted first, keeping its local configuration, then migrated.
When every machine is migrated, flip adoption mode, and the mesh takes full control of the configuration it has been holding.

This is the research above made concrete. It keeps the conflict rule already decided here — on conflict, what is on the machine stays — and gives the middle state, adopted, a length: not a one-time import before generating starts, but a mode that lasts for as long as the machine is being migrated, ended by an explicit act.

What the machine actually looks like

Measured, read-only:


Containers running	60, all the predecessor's services and their stores
The predecessor's control	user-level daemons, separate from the services; none of the 20 running system services is the predecessor's control
Firewall	the predecessor's, active: 54 incoming rules and 52 forwarding rules, each served port allowed explicitly — how a default-deny firewall reads
Files the predecessor's configuration sync writes	12, of which 3 are system files (an ssh server drop-in, the package manager's configuration, one service's configuration); the rest are the operator's shell and agent files
Per-service configuration	environment and composition files per service, written by the predecessor's service tooling

Stopping the predecessor's control stops nothing that serves. The services are containers and system units that run without it; what stops is the rewriting of their configuration. Nothing changes on the machine until something else writes.

What collides, measured rather than assumed

An earlier note assumed the mesh's foundation could not stand beside the predecessor because both want the store's and the broker's standard ports. The measurement says otherwise. The predecessor publishes its own store on a non-standard port and its broker on another; the standard ports the foundation binds for its store and its bus are free.

What does collide:

The mesh wants	Held by	When
the registry's port	the predecessor's registry	at genesis — the foundation raises a registry
the broker's management port, on loopback	the predecessor's broker	at genesis
the private network's port	the predecessor's own tunnel	when the node is placed on the overlay
the web ports	the predecessor's reverse proxy	when the route proxy is assigned
the resolver's port	a resolver the predecessor runs	when a resolver module is assigned

Three of these fall at genesis and cannot be deferred; two only when a particular module is assigned, which is the moment that module migrates and its predecessor stops anyway. Today the foundation's ports are fixed: written in the installer's bundle and in the catalogue's manifests, so a collision is found when a container fails to bind, not before.

What would break if the mesh came up as it is today

The firewall. Genesis loads a base ruleset — drop anything undeclared — in its own table (ADR 0088). The predecessor's firewall is a different table. The kernel runs every table hooked at the same point in turn: an accept in one lets the packet go on to the next, and a drop in any is final. So the mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not declared — every web, mail and database port in the table above — the moment genesis ran.

The files. The host writes a declared file whatever it finds at the path, reporting it as updated. A file the predecessor left — the ssh server drop-in, the resolver's configuration — is replaced the first time a module declaring that path is assigned, before that module's service has moved.

The container names. The applier keys a container on its name. A catalogue module whose container carries the same name as the predecessor's service it replaces takes that container over the moment it is assigned — which is exactly the cutover moment, so this is the migration's step and not a fault, but it means assigning a module is migrating it, never a rehearsal.

The pipeline freezes while the control-node migrates

The predecessor's build pipeline and its coordinator run on the control-node. Stopping its control there stops the predecessor's updates for every machine it manages, until the migration is done. Their services keep running; they receive nothing new. That is the price of the proposal and it is worth stating, not a reason against it: the migration is the period in which the predecessor is being replaced, and it does not need to keep changing.

What adoption mode has to mean

For the proposal to hold, adopted must be a state the mesh records per node, not an intention, and each thing the mesh would otherwise take must say what it does in that state:

The firewall found on the machine stays in force. The mesh does not load its own table on an adopted node. What the mesh needs open — its foundation, and each module as it migrates — it opens through the found firewall, as rules it marks as its own, and it removes only what it added. An accept in a table of its own would not help: the found firewall's drop would still be final.
A file found at a path the mesh declares is kept, recorded with what it contained, and converged only when the module that declares it migrates. Node-level files no module owns — the hosts file, the resolver, the ssh drop-in — converge at the flip.
The foundation's ports are the node's to give. Every port the foundation binds is set at genesis rather than written into a manifest, and genesis checks each one is free before it raises anything, naming what holds it.
The flip is per node and previewed. Converging a node replaces the found firewall with the mesh's derived one in one step, after saying which currently open ports would close — every one of them either declared by an assigned module or deliberately dropped.

Options weighed

Option	Verdict
Cut the machine over in one go: stop the predecessor's store, broker, registry and proxy, raise the foundation in their place	Rejected. Every predecessor service goes down until it has migrated, and the predecessor's other machines lose their broker. The rollback is restarting the predecessor, which is a recovery, not a step.
A separate machine as the control-node	Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline.
Converge on joining, as the mesh does today	Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move.
Make only the foundation's ports configurable and otherwise converge	Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones.
Adoption as a mode, per node, ended by an explicit flip	The proposal. It is the conflict rule already decided here, given a duration.

What this leaves open

Answered by the decision record this feeds — ADR 0100 — only for the migration's needs. The general questions above stay open: whether a module can say which settings are load-bearing, what shape a briefing takes, how a flagged conflict is reconciled. One is newly sharp: the mesh opening ports through a firewall it did not install needs to speak that firewall. There is one kind on the machines measured; a machine with another is not covered until someone writes for it.

8.9 KiB Raw Blame History