ADR 0100 after review: found means unrecorded; assigning prepares, taking cuts over; openings through the found firewall on both paths; the mesh guards its own ports; ports kept as node settings; a converged genesis refuses a machine in use; designs 05, 07, 08, 09 and 17 in step
This commit is contained in:
@@ -24,19 +24,27 @@ Measured on the control-node ([research 012,
|
||||
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
|
||||
explicitly; 12 files its configuration sync writes, 3 of them system files.
|
||||
|
||||
Three things in the mesh as it stands break that shape:
|
||||
Four things in the mesh as it stands break that shape:
|
||||
|
||||
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
|
||||
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every table at a hook is
|
||||
run in turn and a drop in any is final, so the predecessor's allowed ports — web, mail, stores —
|
||||
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every base chain at a hook
|
||||
runs in priority order; an accept ends only its own chain and a drop in any is final — whether
|
||||
the other firewall's chains are nftables or legacy iptables. So the predecessor's allowed ports
|
||||
would close at genesis.
|
||||
2. **The host replaces what it finds.** A declared file is written whatever is at its path. A
|
||||
file the predecessor left is replaced as soon as any module declaring its path is assigned.
|
||||
2. **The host replaces what it finds.** A declared file is written whatever is at its path, except
|
||||
a file declared create-once. A declared container replaces a running one of the same name. So
|
||||
assigning a module the predecessor also runs replaces the predecessor's service and its files at
|
||||
once.
|
||||
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
|
||||
predecessor publishes its own elsewhere — but the registry's port and the broker's management
|
||||
port are held by the predecessor's, and the private network's port by its tunnel. The
|
||||
foundation's ports are fixed in the installer's bundle and the catalogue's manifests, so the
|
||||
collision surfaces as a container that fails to bind.
|
||||
port are held, and on a control-node that is the private network's hub, so is the private
|
||||
network's port. The foundation's ports are fixed in the installer's bundle and the catalogue's
|
||||
manifests, so a collision surfaces as a container that fails to bind, and a port changed at
|
||||
genesis would be changed back when the foundation is adopted as modules.
|
||||
4. **Published container ports are forwarded, not received.** The foundation publishes its ports
|
||||
on every interface; a predecessor firewall that filters only incoming traffic never sees them.
|
||||
The base ruleset is what keeps the store unreachable from outside, and it is the thing that
|
||||
cannot be loaded.
|
||||
|
||||
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
|
||||
modules, keyed on the container that is already running
|
||||
@@ -51,91 +59,133 @@ settled the conflict rule for adoption: *on conflict, what is on the machine sta
|
||||
restarting the predecessor — a recovery, not a step.
|
||||
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
|
||||
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
|
||||
found files are replaced before their services move.
|
||||
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the
|
||||
bind collisions and neither the firewall nor the files.
|
||||
5. **Adoption as a mode per node, ended by an explicit flip.** Adopted.
|
||||
the predecessor's services and files are replaced as soon as any module naming them is assigned.
|
||||
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the bind
|
||||
collisions and neither the firewall nor the files.
|
||||
5. **Treat assigning a module as migrating it.** Considered and rejected on review: it makes the
|
||||
rule that keeps found files never fire — the host only ever sees files of assigned modules — and
|
||||
it makes an assignment on an adopted node an outage rather than a preparation.
|
||||
6. **Adoption as a mode per node; each module taken explicitly; the node converged by an explicit
|
||||
flip.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A node is either adopted or converged, and the mesh records which.** A node joins adopted when
|
||||
its machine is in use; the operator says so at genesis for the control-node and at enrolment for
|
||||
the others. It stays adopted until the operator converges it. Adopted is not a one-time import
|
||||
before generating starts: it lasts for as long as the machine is being migrated.
|
||||
**A node is adopted or converged, and the controller records which.** The operator says so: at
|
||||
genesis for the control-node, and in the enrolment token for the others. The controller is
|
||||
authoritative, and every declaration it sends says whether the node is adopted and which modules
|
||||
have been taken on it. A node stays adopted until the operator converges it. An adopted node is
|
||||
said to be adopted wherever the mesh reports a node's state.
|
||||
|
||||
**Before a node is adopted, the predecessor's control on it is stopped** — the daemons that write
|
||||
its configuration. Its services keep running on the configuration they have.
|
||||
**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with
|
||||
an active firewall or services listening that are not the mesh's, genesis refuses and names what
|
||||
it found — a forgotten flag must not close a working machine.
|
||||
|
||||
**On an adopted node:**
|
||||
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
|
||||
that write its configuration. Its services keep running on what they have.
|
||||
|
||||
- **The firewall found on the machine stays in force.** The mesh does not load its own table
|
||||
there — neither genesis's base ruleset nor the filter module's derived one. What the mesh needs
|
||||
reachable — its foundation's ports and, as each module migrates, that module's declared
|
||||
`listens` — it opens *through the found firewall*, as rules marked as the mesh's. It removes only
|
||||
rules it marked, and edits nothing else.
|
||||
- **A file found at a path the mesh declares is kept.** The host records what it found there and
|
||||
does not replace it while the node is adopted. A module's files converge when that module is
|
||||
assigned, because assigning a module is migrating it. Files no module owns — the hosts file, the
|
||||
resolver, the ssh drop-in — converge at the flip.
|
||||
- **The foundation takes the ports it is given.** Every port the foundation binds is an input to
|
||||
genesis rather than a constant, and genesis checks each is free before raising anything,
|
||||
refusing with the name of what holds it.
|
||||
**Found means present with no record.** A file at a declared path, or a container at a declared
|
||||
name, that the host's store has no record of writing is *found*. A file the host wrote in an
|
||||
earlier life of the node is not found; its record says so.
|
||||
|
||||
**Converging a node is one act, previewed.** It first lists every port the found firewall allows
|
||||
and says, for each, whether an assigned module declares it or it will close; then it loads the
|
||||
mesh's derived filter, retires the found firewall, and converges the files kept at adoption. A
|
||||
node converges when its migration is done. The mesh is migrated when every node has converged.
|
||||
**On an adopted node, what is found is kept until its module is taken.** The host keeps a found
|
||||
file and a found container as they are, records the file's original content before anything else
|
||||
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
|
||||
the module declares that is not found is created; what is found is held. **Taking a module** on a
|
||||
node is its cutover — the operator's act, done when that module's data has moved — and from then
|
||||
on the module's resources converge on that node like any other. A held file that changes while
|
||||
held is reported as changed by something else, not reverted: that is how a predecessor still
|
||||
writing is caught. A held file whose module is unassigned is left where it is, never removed.
|
||||
|
||||
**The order is the operator's:** the control-node first, adopted, its modules migrated one at a
|
||||
time; then each other machine, adopted, migrated, converged in turn. The predecessor's pipeline
|
||||
runs on the control-node, so its updates stop for every machine while the migration runs; that is
|
||||
accepted.
|
||||
**The firewall found on the machine stays in force.** The mesh loads no table of its own on an
|
||||
adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh
|
||||
needs is declared as **openings**: a resource that says a port is reachable, from where, on the
|
||||
incoming path or the forwarded path — a published container port is forwarded. The host converges
|
||||
an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and
|
||||
removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot
|
||||
of the found firewall does not lose it. An opening is a state, not a command, which is what lets it
|
||||
travel over the link. The host reports which firewall it found, and a machine with a kind no host
|
||||
speaks is refused adoption rather than adopted with nothing protecting the mesh's ports.
|
||||
|
||||
**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings
|
||||
from the private network and marked refusals from anywhere else, on the forwarded path — so the
|
||||
store is unreachable from outside whether or not the found firewall filters forwarded traffic.
|
||||
|
||||
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
|
||||
checked free before anything is raised, refused with the name of what holds it. The ports given
|
||||
become that node's settings for the foundation's modules — the catalogue's numbers are only their
|
||||
defaults — and every place that uses them reads them from there: the modules' containers, the
|
||||
filter, the base ruleset, the private network's endpoint, and the addresses consumers are given.
|
||||
The private network's address range must not overlap a tunnel the predecessor still runs; genesis
|
||||
checks that too.
|
||||
|
||||
**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now
|
||||
— every listening socket and every published container port — and for each whether an assigned
|
||||
module declares it or it will close. The flip then takes every module not yet taken, loads the
|
||||
mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the
|
||||
container runtime's rules and the found firewall's own configuration stay on disk. Returning a
|
||||
converged node to adopted re-enables the firewall it retired. A node converges when its migration
|
||||
is done; the mesh is migrated when every node has converged.
|
||||
|
||||
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken
|
||||
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
|
||||
pipeline runs on the control-node, so its updates stop for every machine while the migration runs;
|
||||
that is accepted.
|
||||
|
||||
## Consequences
|
||||
|
||||
The migration becomes a sequence of reversible steps. A module moved is one service changed; a node
|
||||
adopted is a node on which nothing changed; the flip is the one act that changes what is reachable,
|
||||
and it says beforehand what it will change.
|
||||
Each step says what it changes before it changes it. Adopting a node changes nothing that serves;
|
||||
assigning a module adds what is not there; taking a module replaces one service; the flip replaces
|
||||
the firewall, after naming every port it will close. Two steps are not undone by the mesh: taking a
|
||||
module replaces the predecessor's container, and the kept original of a file is recorded but not
|
||||
yet restored by any act of the mesh
|
||||
([research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) leaves where it lives
|
||||
open).
|
||||
|
||||
What got harder:
|
||||
|
||||
- **The mesh must speak a firewall it did not install.** Opening a port through the found
|
||||
firewall means writing to it in its own terms. One kind is found on the machines measured; a
|
||||
machine with another is not covered until someone writes for it.
|
||||
- **The host gains a guard it did not have** — *do not replace what you found* — and its report
|
||||
has to say which files it is holding rather than converging, or an adopted node looks converged.
|
||||
That is the companion to the host's rule of never touching what it did not create
|
||||
([ADR 0005](0005-the-node-host.md)) that the research asked for.
|
||||
- **Genesis grows inputs.** The foundation's ports stop being constants in the bundle and the
|
||||
catalogue; a mesh that never needed to move them now carries the choice.
|
||||
- **A node can sit adopted indefinitely.** Nothing forces the flip, and an adopted node filters
|
||||
with a firewall the mesh does not derive; the mesh's own status has to say which nodes are
|
||||
adopted, so that one left behind is visible.
|
||||
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
|
||||
path. One kind is found on the machines measured; another is refused until a host speaks it.
|
||||
- **The host gains a guard it did not have** — keep what you found — and its report must say
|
||||
which files and containers it holds, or an adopted node reads as converged.
|
||||
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
|
||||
- **Genesis grows inputs, and they outlive genesis.** The foundation's ports stop being constants;
|
||||
every reader of them reads the node's settings.
|
||||
- **A node can sit adopted indefinitely.** Nothing forces the flip; the mesh's status says which
|
||||
nodes are adopted, so one left behind is visible.
|
||||
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
|
||||
nodes**: the base ruleset is not loaded on a node that is adopted at genesis, because the
|
||||
machine's own firewall already filters and a second, stricter table would close it.
|
||||
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
|
||||
reachable from outside — passes to the mesh's own openings and refusals on the forwarded path.
|
||||
|
||||
## How it is checked
|
||||
|
||||
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
|
||||
port and denying the rest, a service listening on that port, a file at a path a mesh module
|
||||
declares, and a container holding the registry's port. Then:
|
||||
port and denying the rest, a service container listening on that port under a name a catalogue
|
||||
module also uses, a file at a path that module declares, a stand-in for the predecessor's control
|
||||
that would rewrite that file, and a container holding the registry's port. Then:
|
||||
|
||||
- **Genesis adopted, with the registry's port held**, refuses and names what holds it; with
|
||||
another port given, the foundation comes up.
|
||||
- **Nothing on the machine changed**: the service is still reachable from a second machine, the
|
||||
found file is byte for byte what it was, and the found firewall's rules differ only by rules
|
||||
marked as the mesh's.
|
||||
- **The mesh works through the found firewall**: the second machine enrols over the bus the mesh
|
||||
opened there.
|
||||
- **A module assigned migrates only itself**: its port is opened, its files converge, the found
|
||||
file it does not own is untouched.
|
||||
- **Converging previews, then changes**: the preview names the service's port as closing unless
|
||||
declared; after the flip the mesh's derived filter is loaded, the found firewall is retired, the
|
||||
declared port is open and the undeclared one is closed.
|
||||
- **Genesis converged** on it refuses and names the firewall and the listener.
|
||||
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
|
||||
port given, the foundation comes up — and adopting the foundation as modules leaves it on that
|
||||
port.
|
||||
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
|
||||
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
|
||||
- **The store is unreachable from outside** — probed from a machine off the private network — and
|
||||
reachable over it.
|
||||
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the
|
||||
machine reboots**: the second machine enrols, and the openings are there again.
|
||||
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change
|
||||
is reported and not reverted.
|
||||
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
|
||||
taken, it replaces them and its port is opened.
|
||||
- **Converging previews, then changes**: the preview names the service's port and a published port
|
||||
no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall
|
||||
is disabled with its configuration still on disk, the declared port is open and the undeclared one
|
||||
closed. Returned to adopted, the found firewall is enabled again.
|
||||
|
||||
Unit tests hold the host to keeping a found file on an adopted node and replacing it on a converged
|
||||
one, and genesis to refusing a held port.
|
||||
Unit tests hold the host to keeping a found file and container on an adopted node, converging them
|
||||
once taken, never removing a held file, and reporting a held file that changed; genesis to refusing
|
||||
a held port and a converged raise on a machine in use; the controller to carrying the mode and the
|
||||
taken modules in every declaration.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
Reference in New Issue
Block a user