146 lines
8.7 KiB
Markdown
146 lines
8.7 KiB
Markdown
---
|
|
topic: the mesh
|
|
status: proposed
|
|
date: 2026-09-22
|
|
deciders: jochen
|
|
reconstructed: false
|
|
extends: 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
|
---
|
|
|
|
# 100. A node in use is adopted before it is converged
|
|
|
|
## Context
|
|
|
|
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
|
|
people use. The control-node is the machine that already carries the predecessor's broker and
|
|
build pipeline. The operator proposed the migration's shape: stop the predecessor's control on a
|
|
machine, bring the mesh up there in an adoption mode that keeps the machine's configuration in
|
|
force, migrate its modules one at a time, move to the next machine, and flip adoption off when every
|
|
machine is done.
|
|
|
|
Measured on the control-node ([research 012,
|
|
*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)):
|
|
60 predecessor containers; the predecessor's control in user-level daemons separate from its
|
|
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
|
|
explicitly; 12 files its configuration sync writes, 3 of them system files.
|
|
|
|
Three things in the mesh as it stands break that shape:
|
|
|
|
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
|
|
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every table at a hook is
|
|
run in turn and a drop in any is final, so the predecessor's allowed ports — web, mail, stores —
|
|
would close at genesis.
|
|
2. **The host replaces what it finds.** A declared file is written whatever is at its path. A
|
|
file the predecessor left is replaced as soon as any module declaring its path is assigned.
|
|
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
|
|
predecessor publishes its own elsewhere — but the registry's port and the broker's management
|
|
port are held by the predecessor's, and the private network's port by its tunnel. The
|
|
foundation's ports are fixed in the installer's bundle and the catalogue's manifests, so the
|
|
collision surfaces as a container that fails to bind.
|
|
|
|
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
|
|
modules, keyed on the container that is already running
|
|
([ADR 0078](0078-the-store-and-broker-are-modules.md)). The research this record draws on already
|
|
settled the conflict rule for adoption: *on conflict, what is on the machine stays*.
|
|
|
|
## Considered Options
|
|
|
|
1. **Cut each machine over in one go** — stop the predecessor's store, broker, registry and proxy,
|
|
raise the foundation in their place. Rejected: every predecessor service on the machine is down
|
|
until it has migrated, the predecessor's other machines lose their broker, and the rollback is
|
|
restarting the predecessor — a recovery, not a step.
|
|
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
|
|
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
|
|
found files are replaced before their services move.
|
|
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the
|
|
bind collisions and neither the firewall nor the files.
|
|
5. **Adoption as a mode per node, ended by an explicit flip.** Adopted.
|
|
|
|
## Decision
|
|
|
|
**A node is either adopted or converged, and the mesh records which.** A node joins adopted when
|
|
its machine is in use; the operator says so at genesis for the control-node and at enrolment for
|
|
the others. It stays adopted until the operator converges it. Adopted is not a one-time import
|
|
before generating starts: it lasts for as long as the machine is being migrated.
|
|
|
|
**Before a node is adopted, the predecessor's control on it is stopped** — the daemons that write
|
|
its configuration. Its services keep running on the configuration they have.
|
|
|
|
**On an adopted node:**
|
|
|
|
- **The firewall found on the machine stays in force.** The mesh does not load its own table
|
|
there — neither genesis's base ruleset nor the filter module's derived one. What the mesh needs
|
|
reachable — its foundation's ports and, as each module migrates, that module's declared
|
|
`listens` — it opens *through the found firewall*, as rules marked as the mesh's. It removes only
|
|
rules it marked, and edits nothing else.
|
|
- **A file found at a path the mesh declares is kept.** The host records what it found there and
|
|
does not replace it while the node is adopted. A module's files converge when that module is
|
|
assigned, because assigning a module is migrating it. Files no module owns — the hosts file, the
|
|
resolver, the ssh drop-in — converge at the flip.
|
|
- **The foundation takes the ports it is given.** Every port the foundation binds is an input to
|
|
genesis rather than a constant, and genesis checks each is free before raising anything,
|
|
refusing with the name of what holds it.
|
|
|
|
**Converging a node is one act, previewed.** It first lists every port the found firewall allows
|
|
and says, for each, whether an assigned module declares it or it will close; then it loads the
|
|
mesh's derived filter, retires the found firewall, and converges the files kept at adoption. A
|
|
node converges when its migration is done. The mesh is migrated when every node has converged.
|
|
|
|
**The order is the operator's:** the control-node first, adopted, its modules migrated one at a
|
|
time; then each other machine, adopted, migrated, converged in turn. The predecessor's pipeline
|
|
runs on the control-node, so its updates stop for every machine while the migration runs; that is
|
|
accepted.
|
|
|
|
## Consequences
|
|
|
|
The migration becomes a sequence of reversible steps. A module moved is one service changed; a node
|
|
adopted is a node on which nothing changed; the flip is the one act that changes what is reachable,
|
|
and it says beforehand what it will change.
|
|
|
|
What got harder:
|
|
|
|
- **The mesh must speak a firewall it did not install.** Opening a port through the found
|
|
firewall means writing to it in its own terms. One kind is found on the machines measured; a
|
|
machine with another is not covered until someone writes for it.
|
|
- **The host gains a guard it did not have** — *do not replace what you found* — and its report
|
|
has to say which files it is holding rather than converging, or an adopted node looks converged.
|
|
That is the companion to the host's rule of never touching what it did not create
|
|
([ADR 0005](0005-the-node-host.md)) that the research asked for.
|
|
- **Genesis grows inputs.** The foundation's ports stop being constants in the bundle and the
|
|
catalogue; a mesh that never needed to move them now carries the choice.
|
|
- **A node can sit adopted indefinitely.** Nothing forces the flip, and an adopted node filters
|
|
with a firewall the mesh does not derive; the mesh's own status has to say which nodes are
|
|
adopted, so that one left behind is visible.
|
|
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
|
|
nodes**: the base ruleset is not loaded on a node that is adopted at genesis, because the
|
|
machine's own firewall already filters and a second, stricter table would close it.
|
|
|
|
## How it is checked
|
|
|
|
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
|
|
port and denying the rest, a service listening on that port, a file at a path a mesh module
|
|
declares, and a container holding the registry's port. Then:
|
|
|
|
- **Genesis adopted, with the registry's port held**, refuses and names what holds it; with
|
|
another port given, the foundation comes up.
|
|
- **Nothing on the machine changed**: the service is still reachable from a second machine, the
|
|
found file is byte for byte what it was, and the found firewall's rules differ only by rules
|
|
marked as the mesh's.
|
|
- **The mesh works through the found firewall**: the second machine enrols over the bus the mesh
|
|
opened there.
|
|
- **A module assigned migrates only itself**: its port is opened, its files converge, the found
|
|
file it does not own is untouched.
|
|
- **Converging previews, then changes**: the preview names the service's port as closing unless
|
|
declared; after the flip the mesh's derived filter is loaded, the found firewall is retired, the
|
|
declared port is open and the undeclared one is closed.
|
|
|
|
Unit tests hold the host to keeping a found file on an adopted node and replacing it on a converged
|
|
one, and genesis to refusing a held port.
|
|
|
|
## References
|
|
|
|
- [research 012 — the minimum viable node, and adopting what is already there](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md),
|
|
and its document [*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)
|
|
- [ADR 0005](0005-the-node-host.md), [ADR 0011](0011-managed-files-are-generated-never-edited.md),
|
|
[ADR 0078](0078-the-store-and-broker-are-modules.md), [ADR 0088](0088-the-foundation-filters-before-anything-listens.md)
|