ADR 0100 after review: found means unrecorded; assigning prepares, taking cuts over; openings through the found firewall on both paths; the mesh guards its own ports; ports kept as node settings; a converged genesis refuses a machine in use; designs 05, 07, 08, 09 and 17 in step

This commit is contained in:
2026-09-22 16:28:31 +02:00
parent 111456abb5
commit 02c40bcab4
7 changed files with 250 additions and 136 deletions
@@ -24,19 +24,27 @@ Measured on the control-node ([research 012,
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
explicitly; 12 files its configuration sync writes, 3 of them system files.
Three things in the mesh as it stands break that shape:
Four things in the mesh as it stands break that shape:
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every table at a hook is
run in turn and a drop in any is final, so the predecessor's allowed ports — web, mail, stores —
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every base chain at a hook
runs in priority order; an accept ends only its own chain and a drop in any is final — whether
the other firewall's chains are nftables or legacy iptables. So the predecessor's allowed ports
would close at genesis.
2. **The host replaces what it finds.** A declared file is written whatever is at its path. A
file the predecessor left is replaced as soon as any module declaring its path is assigned.
2. **The host replaces what it finds.** A declared file is written whatever is at its path, except
a file declared create-once. A declared container replaces a running one of the same name. So
assigning a module the predecessor also runs replaces the predecessor's service and its files at
once.
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
predecessor publishes its own elsewhere — but the registry's port and the broker's management
port are held by the predecessor's, and the private network's port by its tunnel. The
foundation's ports are fixed in the installer's bundle and the catalogue's manifests, so the
collision surfaces as a container that fails to bind.
port are held, and on a control-node that is the private network's hub, so is the private
network's port. The foundation's ports are fixed in the installer's bundle and the catalogue's
manifests, so a collision surfaces as a container that fails to bind, and a port changed at
genesis would be changed back when the foundation is adopted as modules.
4. **Published container ports are forwarded, not received.** The foundation publishes its ports
on every interface; a predecessor firewall that filters only incoming traffic never sees them.
The base ruleset is what keeps the store unreachable from outside, and it is the thing that
cannot be loaded.
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
modules, keyed on the container that is already running
@@ -51,91 +59,133 @@ settled the conflict rule for adoption: *on conflict, what is on the machine sta
restarting the predecessor — a recovery, not a step.
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
found files are replaced before their services move.
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the
bind collisions and neither the firewall nor the files.
5. **Adoption as a mode per node, ended by an explicit flip.** Adopted.
the predecessor's services and files are replaced as soon as any module naming them is assigned.
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the bind
collisions and neither the firewall nor the files.
5. **Treat assigning a module as migrating it.** Considered and rejected on review: it makes the
rule that keeps found files never fire — the host only ever sees files of assigned modules — and
it makes an assignment on an adopted node an outage rather than a preparation.
6. **Adoption as a mode per node; each module taken explicitly; the node converged by an explicit
flip.** Adopted.
## Decision
**A node is either adopted or converged, and the mesh records which.** A node joins adopted when
its machine is in use; the operator says so at genesis for the control-node and at enrolment for
the others. It stays adopted until the operator converges it. Adopted is not a one-time import
before generating starts: it lasts for as long as the machine is being migrated.
**A node is adopted or converged, and the controller records which.** The operator says so: at
genesis for the control-node, and in the enrolment token for the others. The controller is
authoritative, and every declaration it sends says whether the node is adopted and which modules
have been taken on it. A node stays adopted until the operator converges it. An adopted node is
said to be adopted wherever the mesh reports a node's state.
**Before a node is adopted, the predecessor's control on it is stopped** — the daemons that write
its configuration. Its services keep running on the configuration they have.
**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with
an active firewall or services listening that are not the mesh's, genesis refuses and names what
it found — a forgotten flag must not close a working machine.
**On an adopted node:**
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
that write its configuration. Its services keep running on what they have.
- **The firewall found on the machine stays in force.** The mesh does not load its own table
there — neither genesis's base ruleset nor the filter module's derived one. What the mesh needs
reachable — its foundation's ports and, as each module migrates, that module's declared
`listens` — it opens *through the found firewall*, as rules marked as the mesh's. It removes only
rules it marked, and edits nothing else.
- **A file found at a path the mesh declares is kept.** The host records what it found there and
does not replace it while the node is adopted. A module's files converge when that module is
assigned, because assigning a module is migrating it. Files no module owns — the hosts file, the
resolver, the ssh drop-in — converge at the flip.
- **The foundation takes the ports it is given.** Every port the foundation binds is an input to
genesis rather than a constant, and genesis checks each is free before raising anything,
refusing with the name of what holds it.
**Found means present with no record.** A file at a declared path, or a container at a declared
name, that the host's store has no record of writing is *found*. A file the host wrote in an
earlier life of the node is not found; its record says so.
**Converging a node is one act, previewed.** It first lists every port the found firewall allows
and says, for each, whether an assigned module declares it or it will close; then it loads the
mesh's derived filter, retires the found firewall, and converges the files kept at adoption. A
node converges when its migration is done. The mesh is migrated when every node has converged.
**On an adopted node, what is found is kept until its module is taken.** The host keeps a found
file and a found container as they are, records the file's original content before anything else
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
the module declares that is not found is created; what is found is held. **Taking a module** on a
node is its cutover — the operator's act, done when that module's data has moved — and from then
on the module's resources converge on that node like any other. A held file that changes while
held is reported as changed by something else, not reverted: that is how a predecessor still
writing is caught. A held file whose module is unassigned is left where it is, never removed.
**The order is the operator's:** the control-node first, adopted, its modules migrated one at a
time; then each other machine, adopted, migrated, converged in turn. The predecessor's pipeline
runs on the control-node, so its updates stop for every machine while the migration runs; that is
accepted.
**The firewall found on the machine stays in force.** The mesh loads no table of its own on an
adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh
needs is declared as **openings**: a resource that says a port is reachable, from where, on the
incoming path or the forwarded path — a published container port is forwarded. The host converges
an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and
removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot
of the found firewall does not lose it. An opening is a state, not a command, which is what lets it
travel over the link. The host reports which firewall it found, and a machine with a kind no host
speaks is refused adoption rather than adopted with nothing protecting the mesh's ports.
**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings
from the private network and marked refusals from anywhere else, on the forwarded path — so the
store is unreachable from outside whether or not the found firewall filters forwarded traffic.
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
checked free before anything is raised, refused with the name of what holds it. The ports given
become that node's settings for the foundation's modules — the catalogue's numbers are only their
defaults — and every place that uses them reads them from there: the modules' containers, the
filter, the base ruleset, the private network's endpoint, and the addresses consumers are given.
The private network's address range must not overlap a tunnel the predecessor still runs; genesis
checks that too.
**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now
— every listening socket and every published container port — and for each whether an assigned
module declares it or it will close. The flip then takes every module not yet taken, loads the
mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the
container runtime's rules and the found firewall's own configuration stay on disk. Returning a
converged node to adopted re-enables the firewall it retired. A node converges when its migration
is done; the mesh is migrated when every node has converged.
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
pipeline runs on the control-node, so its updates stop for every machine while the migration runs;
that is accepted.
## Consequences
The migration becomes a sequence of reversible steps. A module moved is one service changed; a node
adopted is a node on which nothing changed; the flip is the one act that changes what is reachable,
and it says beforehand what it will change.
Each step says what it changes before it changes it. Adopting a node changes nothing that serves;
assigning a module adds what is not there; taking a module replaces one service; the flip replaces
the firewall, after naming every port it will close. Two steps are not undone by the mesh: taking a
module replaces the predecessor's container, and the kept original of a file is recorded but not
yet restored by any act of the mesh
([research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) leaves where it lives
open).
What got harder:
- **The mesh must speak a firewall it did not install.** Opening a port through the found
firewall means writing to it in its own terms. One kind is found on the machines measured; a
machine with another is not covered until someone writes for it.
- **The host gains a guard it did not have** — *do not replace what you found* — and its report
has to say which files it is holding rather than converging, or an adopted node looks converged.
That is the companion to the host's rule of never touching what it did not create
([ADR 0005](0005-the-node-host.md)) that the research asked for.
- **Genesis grows inputs.** The foundation's ports stop being constants in the bundle and the
catalogue; a mesh that never needed to move them now carries the choice.
- **A node can sit adopted indefinitely.** Nothing forces the flip, and an adopted node filters
with a firewall the mesh does not derive; the mesh's own status has to say which nodes are
adopted, so that one left behind is visible.
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
path. One kind is found on the machines measured; another is refused until a host speaks it.
- **The host gains a guard it did not have** — keep what you found — and its report must say
which files and containers it holds, or an adopted node reads as converged.
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
- **Genesis grows inputs, and they outlive genesis.** The foundation's ports stop being constants;
every reader of them reads the node's settings.
- **A node can sit adopted indefinitely.** Nothing forces the flip; the mesh's status says which
nodes are adopted, so one left behind is visible.
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
nodes**: the base ruleset is not loaded on a node that is adopted at genesis, because the
machine's own firewall already filters and a second, stricter table would close it.
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
reachable from outside — passes to the mesh's own openings and refusals on the forwarded path.
## How it is checked
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
port and denying the rest, a service listening on that port, a file at a path a mesh module
declares, and a container holding the registry's port. Then:
port and denying the rest, a service container listening on that port under a name a catalogue
module also uses, a file at a path that module declares, a stand-in for the predecessor's control
that would rewrite that file, and a container holding the registry's port. Then:
- **Genesis adopted, with the registry's port held**, refuses and names what holds it; with
another port given, the foundation comes up.
- **Nothing on the machine changed**: the service is still reachable from a second machine, the
found file is byte for byte what it was, and the found firewall's rules differ only by rules
marked as the mesh's.
- **The mesh works through the found firewall**: the second machine enrols over the bus the mesh
opened there.
- **A module assigned migrates only itself**: its port is opened, its files converge, the found
file it does not own is untouched.
- **Converging previews, then changes**: the preview names the service's port as closing unless
declared; after the flip the mesh's derived filter is loaded, the found firewall is retired, the
declared port is open and the undeclared one is closed.
- **Genesis converged** on it refuses and names the firewall and the listener.
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
port given, the foundation comes up — and adopting the foundation as modules leaves it on that
port.
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
- **The store is unreachable from outside** — probed from a machine off the private network — and
reachable over it.
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the
machine reboots**: the second machine enrols, and the openings are there again.
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change
is reported and not reverted.
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
taken, it replaces them and its port is opened.
- **Converging previews, then changes**: the preview names the service's port and a published port
no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall
is disabled with its configuration still on disk, the declared port is open and the undeclared one
closed. Returned to adopted, the found firewall is enabled again.
Unit tests hold the host to keeping a found file on an adopted node and replacing it on a converged
one, and genesis to refusing a held port.
Unit tests hold the host to keeping a found file and container on an adopted node, converging them
once taken, never removing a held file, and reporting a held file that changed; genesis to refusing
a held port and a converged raise on a machine in use; the controller to carrying the mode and the
taken modules in every declaration.
## References