ADR 0100 after review: found means unrecorded; assigning prepares, taking cuts over; openings through the found firewall on both paths; the mesh guards its own ports; ports kept as node settings; a converged genesis refuses a machine in use; designs 05, 07, 08, 09 and 17 in step
This commit is contained in:
@@ -1,7 +1,7 @@
|
|||||||
# Migrating a node that is in use — adoption as a mode, not a moment
|
# Migrating a node that is in use — adoption as a mode, not a moment
|
||||||
|
|
||||||
*2026-09-22. Measured on the machine that will be the control-node, which is running the
|
*2026-09-22. Measured on the machine that will be the control-node, which is running the
|
||||||
predecessor mesh today. Nothing below names it; the counts and port numbers are its own.*
|
predecessor mesh today. Nothing below names it; the counts are its own.*
|
||||||
|
|
||||||
## The question
|
## The question
|
||||||
|
|
||||||
@@ -60,21 +60,27 @@ What does collide:
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
|
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
|
||||||
| the broker's management port, on loopback | the predecessor's broker | at genesis |
|
| the broker's management port, on loopback | the predecessor's broker | at genesis |
|
||||||
| the private network's port | the predecessor's own tunnel | when the node is placed on the overlay |
|
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
|
||||||
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
|
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
|
||||||
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
|
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
|
||||||
|
|
||||||
Three of these fall at genesis and cannot be deferred; two only when a particular module is
|
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
|
||||||
assigned, which is the moment that module migrates and its predecessor stops anyway. Today the
|
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
|
||||||
foundation's ports are **fixed**: written in the installer's bundle and in the catalogue's
|
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
|
||||||
manifests, so a collision is found when a container fails to bind, not before.
|
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
|
||||||
|
port changed at genesis would be changed back when the foundation is adopted as modules, since the
|
||||||
|
applier recreates a container whose declared spec differs. The private network's address range
|
||||||
|
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
|
||||||
|
distinct.
|
||||||
|
|
||||||
## What would break if the mesh came up as it is today
|
## What would break if the mesh came up as it is today
|
||||||
|
|
||||||
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
|
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
|
||||||
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
|
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
|
||||||
predecessor's firewall is a different table. The kernel runs every table hooked at the same point
|
predecessor's firewall is a different table. The kernel runs every base chain registered at the
|
||||||
in turn: an accept in one lets the packet go on to the next, and a drop in any is final. So the
|
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
|
||||||
|
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
|
||||||
|
So the
|
||||||
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
|
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
|
||||||
declared — every web, mail and database port in the table above — the moment genesis ran.
|
declared — every web, mail and database port in the table above — the moment genesis ran.
|
||||||
|
|
||||||
@@ -85,8 +91,12 @@ has moved.
|
|||||||
|
|
||||||
**The container names.** The applier keys a container on its name. A catalogue module whose
|
**The container names.** The applier keys a container on its name. A catalogue module whose
|
||||||
container carries the same name as the predecessor's service it replaces takes that container over
|
container carries the same name as the predecessor's service it replaces takes that container over
|
||||||
the moment it is assigned — which is exactly the cutover moment, so this is the migration's step
|
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
|
||||||
and not a fault, but it means *assigning a module is migrating it*, never a rehearsal.
|
|
||||||
|
**The published ports.** The foundation publishes its ports on every interface, and a published
|
||||||
|
container port reaches the container through the forwarded path, not the incoming one. A firewall
|
||||||
|
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
|
||||||
|
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
|
||||||
|
|
||||||
## The pipeline freezes while the control-node migrates
|
## The pipeline freezes while the control-node migrates
|
||||||
|
|
||||||
@@ -101,19 +111,23 @@ is being replaced, and it does not need to keep changing.
|
|||||||
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
|
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
|
||||||
intention, and each thing the mesh would otherwise take must say what it does in that state:
|
intention, and each thing the mesh would otherwise take must say what it does in that state:
|
||||||
|
|
||||||
|
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
|
||||||
|
path or name with no record in the host's store. A found file or container is held, its original
|
||||||
|
recorded, until the operator **takes** the module on that node — the cutover, done when the
|
||||||
|
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
|
||||||
|
never fires: the host only ever sees what assigned modules declare.
|
||||||
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
|
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
|
||||||
adopted node. What the mesh needs open — its foundation, and each module as it migrates — it opens
|
adopted node. What it needs open it declares as openings the host converges *through the found
|
||||||
*through the found firewall*, as rules it marks as its own, and it removes only what it added.
|
firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every
|
||||||
An accept in a table of its own would not help: the found firewall's drop would still be final.
|
reconcile so a reload or reboot does not lose them. An accept in a table of its own would not
|
||||||
- **A file found at a path the mesh declares is kept**, recorded with what it contained, and
|
help: the found firewall's drop would still be final. And the mesh protects its own ports itself,
|
||||||
converged only when the module that declares it migrates. Node-level files no module owns — the
|
on the forwarded path, since the found firewall may not.
|
||||||
hosts file, the resolver, the ssh drop-in — converge at the flip.
|
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
|
||||||
- **The foundation's ports are the node's to give.** Every port the foundation binds is set at
|
that node's settings, read everywhere they are used, so adopting the foundation as modules does
|
||||||
genesis rather than written into a manifest, and genesis checks each one is free before it
|
not move them back.
|
||||||
raises anything, naming what holds it.
|
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
|
||||||
- **The flip is per node and previewed.** Converging a node replaces the found firewall with the
|
published ports, not the found firewall's allow list, which does not see what a container runtime
|
||||||
mesh's derived one in one step, after saying which currently open ports would close — every one
|
forwards.
|
||||||
of them either declared by an assigned module or deliberately dropped.
|
|
||||||
|
|
||||||
## Options weighed
|
## Options weighed
|
||||||
|
|
||||||
@@ -123,6 +137,7 @@ intention, and each thing the mesh would otherwise take must say what it does in
|
|||||||
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
|
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
|
||||||
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
|
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
|
||||||
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
|
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
|
||||||
|
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
|
||||||
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
|
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
|
||||||
|
|
||||||
## What this leaves open
|
## What this leaves open
|
||||||
|
|||||||
@@ -24,19 +24,27 @@ Measured on the control-node ([research 012,
|
|||||||
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
|
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
|
||||||
explicitly; 12 files its configuration sync writes, 3 of them system files.
|
explicitly; 12 files its configuration sync writes, 3 of them system files.
|
||||||
|
|
||||||
Three things in the mesh as it stands break that shape:
|
Four things in the mesh as it stands break that shape:
|
||||||
|
|
||||||
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
|
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
|
||||||
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every table at a hook is
|
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every base chain at a hook
|
||||||
run in turn and a drop in any is final, so the predecessor's allowed ports — web, mail, stores —
|
runs in priority order; an accept ends only its own chain and a drop in any is final — whether
|
||||||
|
the other firewall's chains are nftables or legacy iptables. So the predecessor's allowed ports
|
||||||
would close at genesis.
|
would close at genesis.
|
||||||
2. **The host replaces what it finds.** A declared file is written whatever is at its path. A
|
2. **The host replaces what it finds.** A declared file is written whatever is at its path, except
|
||||||
file the predecessor left is replaced as soon as any module declaring its path is assigned.
|
a file declared create-once. A declared container replaces a running one of the same name. So
|
||||||
|
assigning a module the predecessor also runs replaces the predecessor's service and its files at
|
||||||
|
once.
|
||||||
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
|
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
|
||||||
predecessor publishes its own elsewhere — but the registry's port and the broker's management
|
predecessor publishes its own elsewhere — but the registry's port and the broker's management
|
||||||
port are held by the predecessor's, and the private network's port by its tunnel. The
|
port are held, and on a control-node that is the private network's hub, so is the private
|
||||||
foundation's ports are fixed in the installer's bundle and the catalogue's manifests, so the
|
network's port. The foundation's ports are fixed in the installer's bundle and the catalogue's
|
||||||
collision surfaces as a container that fails to bind.
|
manifests, so a collision surfaces as a container that fails to bind, and a port changed at
|
||||||
|
genesis would be changed back when the foundation is adopted as modules.
|
||||||
|
4. **Published container ports are forwarded, not received.** The foundation publishes its ports
|
||||||
|
on every interface; a predecessor firewall that filters only incoming traffic never sees them.
|
||||||
|
The base ruleset is what keeps the store unreachable from outside, and it is the thing that
|
||||||
|
cannot be loaded.
|
||||||
|
|
||||||
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
|
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
|
||||||
modules, keyed on the container that is already running
|
modules, keyed on the container that is already running
|
||||||
@@ -51,91 +59,133 @@ settled the conflict rule for adoption: *on conflict, what is on the machine sta
|
|||||||
restarting the predecessor — a recovery, not a step.
|
restarting the predecessor — a recovery, not a step.
|
||||||
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
|
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
|
||||||
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
|
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
|
||||||
found files are replaced before their services move.
|
the predecessor's services and files are replaced as soon as any module naming them is assigned.
|
||||||
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the
|
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the bind
|
||||||
bind collisions and neither the firewall nor the files.
|
collisions and neither the firewall nor the files.
|
||||||
5. **Adoption as a mode per node, ended by an explicit flip.** Adopted.
|
5. **Treat assigning a module as migrating it.** Considered and rejected on review: it makes the
|
||||||
|
rule that keeps found files never fire — the host only ever sees files of assigned modules — and
|
||||||
|
it makes an assignment on an adopted node an outage rather than a preparation.
|
||||||
|
6. **Adoption as a mode per node; each module taken explicitly; the node converged by an explicit
|
||||||
|
flip.** Adopted.
|
||||||
|
|
||||||
## Decision
|
## Decision
|
||||||
|
|
||||||
**A node is either adopted or converged, and the mesh records which.** A node joins adopted when
|
**A node is adopted or converged, and the controller records which.** The operator says so: at
|
||||||
its machine is in use; the operator says so at genesis for the control-node and at enrolment for
|
genesis for the control-node, and in the enrolment token for the others. The controller is
|
||||||
the others. It stays adopted until the operator converges it. Adopted is not a one-time import
|
authoritative, and every declaration it sends says whether the node is adopted and which modules
|
||||||
before generating starts: it lasts for as long as the machine is being migrated.
|
have been taken on it. A node stays adopted until the operator converges it. An adopted node is
|
||||||
|
said to be adopted wherever the mesh reports a node's state.
|
||||||
|
|
||||||
**Before a node is adopted, the predecessor's control on it is stopped** — the daemons that write
|
**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with
|
||||||
its configuration. Its services keep running on the configuration they have.
|
an active firewall or services listening that are not the mesh's, genesis refuses and names what
|
||||||
|
it found — a forgotten flag must not close a working machine.
|
||||||
|
|
||||||
**On an adopted node:**
|
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
|
||||||
|
that write its configuration. Its services keep running on what they have.
|
||||||
|
|
||||||
- **The firewall found on the machine stays in force.** The mesh does not load its own table
|
**Found means present with no record.** A file at a declared path, or a container at a declared
|
||||||
there — neither genesis's base ruleset nor the filter module's derived one. What the mesh needs
|
name, that the host's store has no record of writing is *found*. A file the host wrote in an
|
||||||
reachable — its foundation's ports and, as each module migrates, that module's declared
|
earlier life of the node is not found; its record says so.
|
||||||
`listens` — it opens *through the found firewall*, as rules marked as the mesh's. It removes only
|
|
||||||
rules it marked, and edits nothing else.
|
|
||||||
- **A file found at a path the mesh declares is kept.** The host records what it found there and
|
|
||||||
does not replace it while the node is adopted. A module's files converge when that module is
|
|
||||||
assigned, because assigning a module is migrating it. Files no module owns — the hosts file, the
|
|
||||||
resolver, the ssh drop-in — converge at the flip.
|
|
||||||
- **The foundation takes the ports it is given.** Every port the foundation binds is an input to
|
|
||||||
genesis rather than a constant, and genesis checks each is free before raising anything,
|
|
||||||
refusing with the name of what holds it.
|
|
||||||
|
|
||||||
**Converging a node is one act, previewed.** It first lists every port the found firewall allows
|
**On an adopted node, what is found is kept until its module is taken.** The host keeps a found
|
||||||
and says, for each, whether an assigned module declares it or it will close; then it loads the
|
file and a found container as they are, records the file's original content before anything else
|
||||||
mesh's derived filter, retires the found firewall, and converges the files kept at adoption. A
|
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
|
||||||
node converges when its migration is done. The mesh is migrated when every node has converged.
|
the module declares that is not found is created; what is found is held. **Taking a module** on a
|
||||||
|
node is its cutover — the operator's act, done when that module's data has moved — and from then
|
||||||
|
on the module's resources converge on that node like any other. A held file that changes while
|
||||||
|
held is reported as changed by something else, not reverted: that is how a predecessor still
|
||||||
|
writing is caught. A held file whose module is unassigned is left where it is, never removed.
|
||||||
|
|
||||||
**The order is the operator's:** the control-node first, adopted, its modules migrated one at a
|
**The firewall found on the machine stays in force.** The mesh loads no table of its own on an
|
||||||
time; then each other machine, adopted, migrated, converged in turn. The predecessor's pipeline
|
adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh
|
||||||
runs on the control-node, so its updates stop for every machine while the migration runs; that is
|
needs is declared as **openings**: a resource that says a port is reachable, from where, on the
|
||||||
accepted.
|
incoming path or the forwarded path — a published container port is forwarded. The host converges
|
||||||
|
an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and
|
||||||
|
removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot
|
||||||
|
of the found firewall does not lose it. An opening is a state, not a command, which is what lets it
|
||||||
|
travel over the link. The host reports which firewall it found, and a machine with a kind no host
|
||||||
|
speaks is refused adoption rather than adopted with nothing protecting the mesh's ports.
|
||||||
|
|
||||||
|
**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings
|
||||||
|
from the private network and marked refusals from anywhere else, on the forwarded path — so the
|
||||||
|
store is unreachable from outside whether or not the found firewall filters forwarded traffic.
|
||||||
|
|
||||||
|
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
|
||||||
|
checked free before anything is raised, refused with the name of what holds it. The ports given
|
||||||
|
become that node's settings for the foundation's modules — the catalogue's numbers are only their
|
||||||
|
defaults — and every place that uses them reads them from there: the modules' containers, the
|
||||||
|
filter, the base ruleset, the private network's endpoint, and the addresses consumers are given.
|
||||||
|
The private network's address range must not overlap a tunnel the predecessor still runs; genesis
|
||||||
|
checks that too.
|
||||||
|
|
||||||
|
**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now
|
||||||
|
— every listening socket and every published container port — and for each whether an assigned
|
||||||
|
module declares it or it will close. The flip then takes every module not yet taken, loads the
|
||||||
|
mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the
|
||||||
|
container runtime's rules and the found firewall's own configuration stay on disk. Returning a
|
||||||
|
converged node to adopted re-enables the firewall it retired. A node converges when its migration
|
||||||
|
is done; the mesh is migrated when every node has converged.
|
||||||
|
|
||||||
|
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken
|
||||||
|
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
|
||||||
|
pipeline runs on the control-node, so its updates stop for every machine while the migration runs;
|
||||||
|
that is accepted.
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
The migration becomes a sequence of reversible steps. A module moved is one service changed; a node
|
Each step says what it changes before it changes it. Adopting a node changes nothing that serves;
|
||||||
adopted is a node on which nothing changed; the flip is the one act that changes what is reachable,
|
assigning a module adds what is not there; taking a module replaces one service; the flip replaces
|
||||||
and it says beforehand what it will change.
|
the firewall, after naming every port it will close. Two steps are not undone by the mesh: taking a
|
||||||
|
module replaces the predecessor's container, and the kept original of a file is recorded but not
|
||||||
|
yet restored by any act of the mesh
|
||||||
|
([research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) leaves where it lives
|
||||||
|
open).
|
||||||
|
|
||||||
What got harder:
|
What got harder:
|
||||||
|
|
||||||
- **The mesh must speak a firewall it did not install.** Opening a port through the found
|
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
|
||||||
firewall means writing to it in its own terms. One kind is found on the machines measured; a
|
path. One kind is found on the machines measured; another is refused until a host speaks it.
|
||||||
machine with another is not covered until someone writes for it.
|
- **The host gains a guard it did not have** — keep what you found — and its report must say
|
||||||
- **The host gains a guard it did not have** — *do not replace what you found* — and its report
|
which files and containers it holds, or an adopted node reads as converged.
|
||||||
has to say which files it is holding rather than converging, or an adopted node looks converged.
|
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
|
||||||
That is the companion to the host's rule of never touching what it did not create
|
- **Genesis grows inputs, and they outlive genesis.** The foundation's ports stop being constants;
|
||||||
([ADR 0005](0005-the-node-host.md)) that the research asked for.
|
every reader of them reads the node's settings.
|
||||||
- **Genesis grows inputs.** The foundation's ports stop being constants in the bundle and the
|
- **A node can sit adopted indefinitely.** Nothing forces the flip; the mesh's status says which
|
||||||
catalogue; a mesh that never needed to move them now carries the choice.
|
nodes are adopted, so one left behind is visible.
|
||||||
- **A node can sit adopted indefinitely.** Nothing forces the flip, and an adopted node filters
|
|
||||||
with a firewall the mesh does not derive; the mesh's own status has to say which nodes are
|
|
||||||
adopted, so that one left behind is visible.
|
|
||||||
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
|
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
|
||||||
nodes**: the base ruleset is not loaded on a node that is adopted at genesis, because the
|
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
|
||||||
machine's own firewall already filters and a second, stricter table would close it.
|
reachable from outside — passes to the mesh's own openings and refusals on the forwarded path.
|
||||||
|
|
||||||
## How it is checked
|
## How it is checked
|
||||||
|
|
||||||
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
|
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
|
||||||
port and denying the rest, a service listening on that port, a file at a path a mesh module
|
port and denying the rest, a service container listening on that port under a name a catalogue
|
||||||
declares, and a container holding the registry's port. Then:
|
module also uses, a file at a path that module declares, a stand-in for the predecessor's control
|
||||||
|
that would rewrite that file, and a container holding the registry's port. Then:
|
||||||
|
|
||||||
- **Genesis adopted, with the registry's port held**, refuses and names what holds it; with
|
- **Genesis converged** on it refuses and names the firewall and the listener.
|
||||||
another port given, the foundation comes up.
|
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
|
||||||
- **Nothing on the machine changed**: the service is still reachable from a second machine, the
|
port given, the foundation comes up — and adopting the foundation as modules leaves it on that
|
||||||
found file is byte for byte what it was, and the found firewall's rules differ only by rules
|
port.
|
||||||
marked as the mesh's.
|
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
|
||||||
- **The mesh works through the found firewall**: the second machine enrols over the bus the mesh
|
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
|
||||||
opened there.
|
- **The store is unreachable from outside** — probed from a machine off the private network — and
|
||||||
- **A module assigned migrates only itself**: its port is opened, its files converge, the found
|
reachable over it.
|
||||||
file it does not own is untouched.
|
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the
|
||||||
- **Converging previews, then changes**: the preview names the service's port as closing unless
|
machine reboots**: the second machine enrols, and the openings are there again.
|
||||||
declared; after the flip the mesh's derived filter is loaded, the found firewall is retired, the
|
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change
|
||||||
declared port is open and the undeclared one is closed.
|
is reported and not reverted.
|
||||||
|
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
|
||||||
|
taken, it replaces them and its port is opened.
|
||||||
|
- **Converging previews, then changes**: the preview names the service's port and a published port
|
||||||
|
no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall
|
||||||
|
is disabled with its configuration still on disk, the declared port is open and the undeclared one
|
||||||
|
closed. Returned to adopted, the found firewall is enabled again.
|
||||||
|
|
||||||
Unit tests hold the host to keeping a found file on an adopted node and replacing it on a converged
|
Unit tests hold the host to keeping a found file and container on an adopted node, converging them
|
||||||
one, and genesis to refusing a held port.
|
once taken, never removing a held file, and reporting a held file that changed; genesis to refusing
|
||||||
|
a held port and a converged raise on a machine in use; the controller to carrying the mode and the
|
||||||
|
taken modules in every declaration.
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
|
|||||||
@@ -133,16 +133,21 @@ is the component; that one is what happens to it.
|
|||||||
|
|
||||||
## What it finds, on an adopted node
|
## What it finds, on an adopted node
|
||||||
|
|
||||||
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A declaration says whether the node is adopted. On an adopted node
|
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A declaration says whether the node is adopted, and which of its
|
||||||
the host does not replace a file it finds at a declared path: it records what was there — its
|
modules have been **taken**. *Found* is a file at a declared path, or a container at a declared
|
||||||
content, so the original is kept before anything is written — and holds the file as found. A
|
name, that the host's store has no record of writing. On an adopted node the host keeps what it
|
||||||
module's files are converged when that module is assigned, because assigning a module is
|
found for any module not yet taken: it records a found file's original content before anything
|
||||||
migrating it; the rest when the node is converged. The host's report says which files it is
|
else, and it reports the file or container as held — a report that says what it holds, so an
|
||||||
holding rather than converging, so an adopted node never reads as converged. This is the
|
adopted node never reads as converged. Once the module is taken, its resources converge like any
|
||||||
|
other. A held file that changes while held is reported as changed by something else, not
|
||||||
|
reverted; a held file is never removed, even when its module is unassigned. The host also
|
||||||
|
converges a new resource, the **opening** — a port made reachable through the firewall it found
|
||||||
|
([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the
|
||||||
companion the host's ownership rule needed: *never touch what you did not create, unless adoption
|
companion the host's ownership rule needed: *never touch what you did not create, unless adoption
|
||||||
made it yours — and while the node is adopted, keep it as you found it.* *How it is checked:*
|
made it yours — and while the node is adopted, not until its module is taken.* *How it is
|
||||||
unit tests hold the host to keeping a found file on an adopted node and replacing it on a
|
checked:* unit tests hold the host to keeping a found file and container, converging them once
|
||||||
converged one, and the adoption bed asserts a found file byte for byte unchanged.
|
taken, never removing a held file and reporting one that changed; the adoption bed asserts a found
|
||||||
|
file byte for byte unchanged until its module is taken.
|
||||||
|
|
||||||
## Where a declaration comes from
|
## Where a declaration comes from
|
||||||
|
|
||||||
|
|||||||
@@ -11,8 +11,9 @@ code:
|
|||||||
- mesh-catalog modules/postgres
|
- mesh-catalog modules/postgres
|
||||||
- mesh-catalog modules/lavinmq
|
- mesh-catalog modules/lavinmq
|
||||||
- mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
|
- mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
|
||||||
updated: 2026-09-21
|
updated: 2026-09-22
|
||||||
decisions:
|
decisions:
|
||||||
|
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||||
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
|
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
|
||||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||||
@@ -172,6 +173,16 @@ ruleset the machine loaded. **Checked** by the installer's bundle test (order an
|
|||||||
the genesis bed, which probes the machine from outside for the length of the install: the store's
|
the genesis bed, which probes the machine from outside for the length of the install: the store's
|
||||||
port never answers, the bus's does.
|
port never answers, the bus's does.
|
||||||
|
|
||||||
|
**Except on a machine raised adopted**
|
||||||
|
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). A
|
||||||
|
machine already serving under a predecessor has a firewall of its own, and a second, stricter
|
||||||
|
table would close everything it serves. There the base ruleset is not loaded and the filter module
|
||||||
|
is not assigned until the node converges; the foundation's ports are opened through the found
|
||||||
|
firewall from the private network and refused from anywhere else on the forwarded path, which
|
||||||
|
keeps the same promise — the store's port never answers from outside — by other means. The
|
||||||
|
foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked**
|
||||||
|
by the adoption bed, which makes the same outside probe.
|
||||||
|
|
||||||
## Raising it
|
## Raising it
|
||||||
|
|
||||||
The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md):
|
The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md):
|
||||||
|
|||||||
@@ -442,7 +442,8 @@ stopgap until the manifest layer can carry a label and a domain separately
|
|||||||
|
|
||||||
**Derived from what is assigned here, and from the overlay's shape** — a node's open ports are a
|
**Derived from what is assigned here, and from the overlay's shape** — a node's open ports are a
|
||||||
consequence of what runs on it and who must reach it, not an independent declaration to keep in
|
consequence of what runs on it and who must reach it, not an independent declaration to keep in
|
||||||
step by hand.
|
step by hand. That is true of a converged node; an adopted one keeps the firewall it was found
|
||||||
|
with until it converges (below).
|
||||||
|
|
||||||
**A rule names its source** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)).
|
**A rule names its source** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)).
|
||||||
A rule with no source is open, and must say so rather than appear to restrict something. `scope:`
|
A rule with no source is open, and must say so rather than appear to restrict something. `scope:`
|
||||||
@@ -536,16 +537,24 @@ is why the check reads packets.*
|
|||||||
### On an adopted node
|
### On an adopted node
|
||||||
|
|
||||||
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh
|
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh
|
||||||
loads no table of its own there — neither genesis's base ruleset nor the derived one. The kernel
|
loads no table of its own there — neither genesis's base ruleset nor the derived one. Every base
|
||||||
runs every table at a hook and a drop in any is final, so a second, stricter table would close
|
chain at a hook runs and a drop in any is final, whatever the other firewall is written in, so a
|
||||||
every port the machine serves. What the mesh needs reachable — its foundation's ports and each
|
second, stricter table would close every port the machine serves. What the mesh needs reachable
|
||||||
migrated module's `listens` — it opens *through the found firewall*, as rules marked as the mesh's,
|
it declares as **openings**: a port, from where, on the incoming path or the forwarded path — a
|
||||||
and it removes only rules it marked. Converging the node replaces the found firewall with the
|
published container port is forwarded, and a firewall that filters only incoming traffic never
|
||||||
derived filter in one step, after previewing which open ports will close. The mesh must speak the
|
sees it. The host converges each opening through the found firewall in that firewall's own terms,
|
||||||
found firewall in its own terms; one kind is found on the machines measured, and a machine with
|
marks it as the mesh's, removes only what it marked, and re-checks every opening on each reconcile
|
||||||
another is not covered until someone writes for it. *How it is checked:* the adoption bed asserts
|
so a reload or a reboot does not lose it. An opening is state, not a command, so it travels over
|
||||||
the found firewall's rules differ only by the mesh's marked rules, that a second machine enrols
|
the link like any other resource. **The mesh protects its own ports itself**: its foundation's ports
|
||||||
through them, and that after the flip the declared port is open and the undeclared one closed.
|
are opened from the private network and refused from anywhere else on the forwarded path, so the
|
||||||
|
store stays unreachable from outside whether or not the found firewall filters forwarded traffic.
|
||||||
|
One kind of found firewall is spoken; a machine with another is refused adoption rather than
|
||||||
|
adopted unprotected. Converging the node previews what is reachable now — listening sockets and
|
||||||
|
published ports — and what will close, then loads the derived filter and disables the found
|
||||||
|
firewall without flushing it. *How it is checked:* the adoption bed asserts the found firewall's
|
||||||
|
rules differ only by the mesh's marked rules, that the store is unreachable from off the private
|
||||||
|
network, that a second machine enrols through the openings before and after a reload and a reboot,
|
||||||
|
and that after the flip the declared port is open and the undeclared one closed.
|
||||||
|
|
||||||
## 5 — Certificates
|
## 5 — Certificates
|
||||||
|
|
||||||
|
|||||||
@@ -292,14 +292,18 @@ until its migration is done and the operator converges it. A machine that was em
|
|||||||
converged, as before
|
converged, as before
|
||||||
([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
|
([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
|
||||||
|
|
||||||
On an adopted node the firewall found there stays in force and the mesh opens what it needs
|
The operator says a node is adopted — at genesis for the control-node, in the enrolment token for
|
||||||
through it ([08-connectivity](08-connectivity.md)); a file found at a path the mesh declares is
|
the others — and the controller records it and says so in every declaration, with the modules
|
||||||
kept until the module declaring it migrates ([05-the-node-host](05-the-node-host.md)).
|
**taken** on that node. On an adopted node what is found is held until its module is taken
|
||||||
**Converging is one act per node, previewed**: it lists every port the found firewall allows and
|
([05-the-node-host](05-the-node-host.md)): assigning a module prepares it, taking it is its
|
||||||
whether an assigned module declares it or it will close, then loads the mesh's own filter, retires
|
cutover. The firewall found there stays in force and the mesh opens what it needs through it
|
||||||
the found one and converges what was kept. *How it is checked:* a lab bed prepares a machine the
|
([08-connectivity](08-connectivity.md)). **Converging is one act per node, previewed**: it lists
|
||||||
way a predecessor leaves one and asserts nothing on it changes until the flip, and that the flip
|
what is reachable on the machine now — listening sockets and published ports — and whether an
|
||||||
closes exactly what the preview said.
|
assigned module declares each or it will close, then takes every module not yet taken, loads the
|
||||||
|
mesh's own filter and disables the found one without flushing it. Returning a converged node to
|
||||||
|
adopted enables the found firewall again. *How it is checked:* a lab bed prepares a machine the way
|
||||||
|
a predecessor leaves one and asserts nothing that serves changes until a module is taken or the
|
||||||
|
node is converged, and that the flip closes exactly what the preview said.
|
||||||
|
|
||||||
A candidate machine is not empty. It has a package manager, probably a container runtime,
|
A candidate machine is not empty. It has a package manager, probably a container runtime,
|
||||||
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||||
@@ -316,11 +320,16 @@ an installation. This is a *never* rule, and it earns that from the worst loss i
|
|||||||
a tool acting on a path it did not own.
|
a tool acting on a path it did not own.
|
||||||
|
|
||||||
**On conflict, the machine's configuration wins.** Adoption always completes; the conflict is
|
**On conflict, the machine's configuration wins.** Adoption always completes; the conflict is
|
||||||
flagged and reconciled afterwards. A machine in use keeps working exactly as it did.
|
flagged and reconciled afterwards. A machine in use keeps working exactly as it did. Two things are
|
||||||
|
not conflicts in this sense ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)):
|
||||||
|
a module the operator has *taken* replaces what it found, because that is its cutover; and genesis
|
||||||
|
refuses a port the foundation needs that something else holds, because a foundation that cannot
|
||||||
|
bind is not adopted but broken.
|
||||||
|
|
||||||
**Adoption produces a briefing**, not just a result: what it found, what it took over, and what
|
**Adoption produces a briefing**, not just a result: what it found, what it took over, and what
|
||||||
it could not resolve — with each line marked `ok`, `kept`, `unknown` or `failed`, and the overall
|
it could not resolve — with each line marked `ok`, `kept`, `unknown` or `failed`, and the overall
|
||||||
outcome **derived** from the worst line rather than stated alongside it.
|
outcome **derived** from the worst line rather than stated alongside it. A file or container the
|
||||||
|
host is holding on an adopted node is a `kept` line for as long as it is held.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -384,6 +393,11 @@ runtime because a declaration changed would stop every container on the node.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
|
*On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)),
|
||||||
|
what the host is holding was found, not made, so nothing held is ever removed: a held file whose
|
||||||
|
module is unassigned stays where it is.
|
||||||
|
|
||||||
## enrolled ⇄ disconnected
|
## enrolled ⇄ disconnected
|
||||||
|
|
||||||
Not a failure. Not degraded. A situation
|
Not a failure. Not degraded. A situation
|
||||||
|
|||||||
@@ -101,16 +101,23 @@ that runs it.
|
|||||||
### Genesis on a machine in use
|
### Genesis on a machine in use
|
||||||
|
|
||||||
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A control-node that is already running a predecessor mesh is raised
|
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A control-node that is already running a predecessor mesh is raised
|
||||||
**adopted**. The predecessor's control on it is stopped first — the daemons that write its
|
**adopted**, said so by the operator. The operator stops the predecessor's control on it first —
|
||||||
configuration — and its services keep running. **Every port the foundation binds is an input to
|
the daemons that write its configuration — and its services keep running. **Every port the
|
||||||
genesis**, not a constant in the bundle or the catalogue, and genesis checks each is free before it
|
foundation binds is an input to genesis**, checked free before anything is raised, refused with the
|
||||||
raises anything, refusing with the name of what holds it. On such a machine the store's and the
|
name of what holds it; the ports given become the node's settings for the foundation's modules, so
|
||||||
bus's usual ports were free and the registry's, the broker's management port and the private
|
adopting the foundation as modules keeps them, and every reader of them — the filter, the base
|
||||||
network's port were held. Genesis adopted **does not load the base ruleset**: the machine's own
|
ruleset, the private network's endpoint, the addresses consumers are given — reads them there. On
|
||||||
firewall already filters, and the mesh opens its foundation's ports through it
|
the control-node measured, the store's and the bus's usual ports were free and the registry's, the
|
||||||
([08-connectivity](08-connectivity.md)). *How it is checked:* the adoption bed raises genesis
|
broker's management port and, as the private network's hub, its port were held. Genesis also
|
||||||
with the registry's port held and asserts the refusal names its holder, then with another port
|
checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted
|
||||||
given asserts the foundation comes up and the machine's service is still reachable.
|
**does not load the base ruleset**: the machine's own firewall already filters, and the mesh opens
|
||||||
|
its foundation's ports through it and refuses them from outside itself
|
||||||
|
([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — an
|
||||||
|
active firewall or listeners that are not the mesh's — so a forgotten flag cannot close a working
|
||||||
|
machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
|
||||||
|
asserts the refusal, then adopted with the registry's port held and asserts the refusal names its
|
||||||
|
holder, then with another port given asserts the foundation comes up, stays on that port once
|
||||||
|
adopted as modules, and the machine's service is still reachable.
|
||||||
|
|
||||||
## After the pivot, and still part of installing
|
## After the pivot, and still part of installing
|
||||||
|
|
||||||
@@ -157,7 +164,10 @@ What remains after *that* belongs to somebody else: adding machines, and decidin
|
|||||||
|
|
||||||
A machine joins with the host binary and a token. It does not raise a foundation, does not install a
|
A machine joins with the host binary and a token. It does not raise a foundation, does not install a
|
||||||
registry, and is never enrolled twice. The mesh already knows how to tell a machine what to be;
|
registry, and is never enrolled twice. The mesh already knows how to tell a machine what to be;
|
||||||
joining is the point at which a machine starts listening.
|
joining is the point at which a machine starts listening. A machine in use joins **adopted**: the
|
||||||
|
token says so, the operator has stopped the predecessor's control on it first, and from then on it
|
||||||
|
keeps what it has until each module is taken
|
||||||
|
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)).
|
||||||
|
|
||||||
## Where the line falls
|
## Where the line falls
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user