novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Research 012: migrating a node that is in use, measured on the control-node; ADR 0100 proposed — a node in use is adopted before it is converged

Browse Source
This commit is contained in:
jschoubben
2026-09-22 16:16:28 +02:00
parent 21baf397a8
commit 5fc3cbde4c
4 changed files with 295 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/012-the-minimum-viable-node/00-overview.md
+14
View File
@@ -7,6 +7,9 @@ touches:
- 03-DESIGN/01-to-be/05-the-node-host.md
- 03-DESIGN/00-as-is/05-runtime-and-installation.md
- 01-RESEARCH/011-the-module-graph/00-overview.md
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
---
# 012 — The minimum viable node, and adopting what is already there
@@ -180,6 +183,16 @@ same makes a node where something the mesh needed never happened indistinguishab
where a log level differed. Whether a failed line still lets adoption complete is therefore
reopened by adding severity, and is not decided here.
## Adoption as a mode, for the migration
*2026-09-22.* The migration from the predecessor mesh gave the middle state a length. A machine
running the predecessor is **adopted** when the mesh comes up on it — the predecessor's control
stopped, the machine's firewall and files kept in force, the mesh opening what it needs through
them — and stays adopted while its modules migrate one at a time, until the operator **converges**
it. Measured on the control-node, and weighed against the alternatives, in
[*migrating a node that is in use*](migrating-a-node-in-use.md); proposed as
[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).
## Open questions
| Question | Why it is open |
@@ -191,6 +204,7 @@ reopened by adding severity, and is not decided here.
| Can a module say which of its settings are load-bearing? | The question that dissolves the conflict rule rather than choosing a side. A setting the module *requires* cannot be kept from the machine without producing something installed and broken; a setting it merely *prefers* should always yield. Until a module can say which is which, adoption is defaulting in the dark. Belongs with the graph. |
| Does a `failed` line still let adoption complete? | *Flags inform, they do not block* was decided about conflicts, where the mesh chose and the machine works. A failure is *we could not*, which is different in kind — and treating them alike hides the worse one behind the commoner one. |
| How is a flagged conflict reconciled, and by whom? | The briefing hands it to a session. What that session is empowered to change, and whether the resolution is recorded so the next adoption does not re-raise it, is undecided. |
| ~~How long is a machine adopted?~~ | **Proposed** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — for as long as it is being migrated: a mode per node, recorded, ended by an explicit and previewed flip. |
| Where does the kept original live, and for how long? | Whether it is recorded in the node's state so adoption is visibly reversible, and whether it is returned when the mesh stops managing the thing. |
| What shape is a briefing? | Structured enough to be acted on, prose enough to be read. It is the first thing a session on a new node sees, which makes it an interface rather than a log. |
| Does owning a package mean owning its version? | Owning configuration and owning the package are different scopes. The second means the mesh decides which version is installed, and that decision then has to survive the machine's own package manager updating it. |
01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md
+135
View File
@@ -0,0 +1,135 @@
# Migrating a node that is in use — adoption as a mode, not a moment
*2026-09-22. Measured on the machine that will be the control-node, which is running the
predecessor mesh today. Nothing below names it; the counts and port numbers are its own.*
## The question
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
people use. The control-node is decided: it is the machine that already carries the predecessor's
broker and build pipeline. So the question is not *where* the mesh starts but **how a machine
running the predecessor becomes a node of the mesh without its services noticing** — and then how
each of the other machines follows.
## The operator's proposal
Proposed by the operator, 2026-09-22, and the shape this document tests:
1. **Stop the predecessor's control on a machine** — its daemons that write configuration: the
network and firewall configuration above all, which decide what is reachable and what is
blocked. Its services keep running; only the control over their configuration stops.
2. **Bring the mesh up on that machine in adoption mode.** It takes custody of those files and
keeps what it finds in force.
3. **Migrate the modules one at a time**, data preserved, per the cutover procedure.
4. **Move to the next machine and repeat** — adopted first, keeping its local configuration, then
migrated.
5. **When every machine is migrated, flip adoption mode**, and the mesh takes full control of the
configuration it has been holding.
This is the research above made concrete. It keeps the conflict rule already decided here — *on
conflict, what is on the machine stays* — and gives the middle state, *adopted*, a length: not a
one-time import before generating starts, but a mode that lasts for as long as the machine is
being migrated, ended by an explicit act.
## What the machine actually looks like
Measured, read-only:
| | |
|---|---|
| Containers running | 60, all the predecessor's services and their stores |
| The predecessor's control | user-level daemons, separate from the services; none of the 20 running system services is the predecessor's control |
| Firewall | the predecessor's, active: 54 incoming rules and 52 forwarding rules, each served port allowed explicitly — how a default-deny firewall reads |
| Files the predecessor's configuration sync writes | 12, of which 3 are system files (an ssh server drop-in, the package manager's configuration, one service's configuration); the rest are the operator's shell and agent files |
| Per-service configuration | environment and composition files per service, written by the predecessor's service tooling |
**Stopping the predecessor's control stops nothing that serves.** The services are containers and
system units that run without it; what stops is the rewriting of their configuration. Nothing
changes on the machine until something else writes.
## What collides, measured rather than assumed
An earlier note assumed the mesh's foundation could not stand beside the predecessor because both
want the store's and the broker's standard ports. **The measurement says otherwise.** The
predecessor publishes its own store on a non-standard port and its broker on another; the standard
ports the foundation binds for its store and its bus are free.
What does collide:
| The mesh wants | Held by | When |
|---|---|---|
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
| the broker's management port, on loopback | the predecessor's broker | at genesis |
| the private network's port | the predecessor's own tunnel | when the node is placed on the overlay |
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
Three of these fall at genesis and cannot be deferred; two only when a particular module is
assigned, which is the moment that module migrates and its predecessor stops anyway. Today the
foundation's ports are **fixed**: written in the installer's bundle and in the catalogue's
manifests, so a collision is found when a container fails to bind, not before.
## What would break if the mesh came up as it is today
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
predecessor's firewall is a different table. The kernel runs every table hooked at the same point
in turn: an accept in one lets the packet go on to the next, and a drop in any is final. So the
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
declared — every web, mail and database port in the table above — the moment genesis ran.
**The files.** The host writes a declared file whatever it finds at the path, reporting it as
updated. A file the predecessor left — the ssh server drop-in, the resolver's configuration — is
replaced the first time a module declaring that path is assigned, before that module's service
has moved.
**The container names.** The applier keys a container on its name. A catalogue module whose
container carries the same name as the predecessor's service it replaces takes that container over
the moment it is assigned — which is exactly the cutover moment, so this is the migration's step
and not a fault, but it means *assigning a module is migrating it*, never a rehearsal.
## The pipeline freezes while the control-node migrates
The predecessor's build pipeline and its coordinator run on the control-node. Stopping its control
there stops the predecessor's updates for **every** machine it manages, until the migration is
done. Their services keep running; they receive nothing new. That is the price of the proposal and
it is worth stating, not a reason against it: the migration is the period in which the predecessor
is being replaced, and it does not need to keep changing.
## What adoption mode has to mean
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
intention, and each thing the mesh would otherwise take must say what it does in that state:
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
adopted node. What the mesh needs open — its foundation, and each module as it migrates — it opens
*through the found firewall*, as rules it marks as its own, and it removes only what it added.
An accept in a table of its own would not help: the found firewall's drop would still be final.
- **A file found at a path the mesh declares is kept**, recorded with what it contained, and
converged only when the module that declares it migrates. Node-level files no module owns — the
hosts file, the resolver, the ssh drop-in — converge at the flip.
- **The foundation's ports are the node's to give.** Every port the foundation binds is set at
genesis rather than written into a manifest, and genesis checks each one is free before it
raises anything, naming what holds it.
- **The flip is per node and previewed.** Converging a node replaces the found firewall with the
mesh's derived one in one step, after saying which currently open ports would close — every one
of them either declared by an assigned module or deliberately dropped.
## Options weighed
| Option | Verdict |
|---|---|
| Cut the machine over in one go: stop the predecessor's store, broker, registry and proxy, raise the foundation in their place | Rejected. Every predecessor service goes down until it has migrated, and the predecessor's other machines lose their broker. The rollback is restarting the predecessor, which is a recovery, not a step. |
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
## What this leaves open
Answered by the decision record this feeds — [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)
— only for the migration's needs. The general questions above stay open: whether a module can say
which settings are load-bearing, what shape a briefing takes, how a flagged conflict is reconciled.
One is newly sharp: the mesh opening ports through a firewall it did not install needs to speak that
firewall. There is one kind on the machines measured; a machine with another is not covered until
someone writes for it.