ADR 0100 (proposed): a node in use is adopted before it is converged #74
@@ -116,12 +116,15 @@ intention, and each thing the mesh would otherwise take must say what it does in
|
||||
recorded, until the operator **takes** the module on that node — the cutover, done when the
|
||||
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
|
||||
never fires: the host only ever sees what assigned modules declare.
|
||||
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an
|
||||
adopted node. What it needs open it declares as openings the host converges *through the found
|
||||
firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every
|
||||
reconcile so a reload or reboot does not lose them. An accept in a table of its own would not
|
||||
help: the found firewall's drop would still be final. And the mesh protects its own ports itself,
|
||||
on the forwarded path, since the found firewall may not.
|
||||
- **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
|
||||
that drops by default or accepts. What it needs open it declares as openings the host converges
|
||||
*through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and
|
||||
re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of
|
||||
its own would not help: the found firewall's drop would still be final. What a table of its own
|
||||
*can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's
|
||||
management port from outside the private network in a table that only refuses, which the found
|
||||
firewall may not do and cannot undo. The bus, the registry and the hub's port stay open to
|
||||
anywhere: a node enrols before it has a private-network address.
|
||||
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
|
||||
that node's settings, read everywhere they are used, so adopting the foundation as modules does
|
||||
not move them back.
|
||||
|
||||
@@ -76,9 +76,10 @@ authoritative, and every declaration it sends says whether the node is adopted a
|
||||
have been taken on it. A node stays adopted until the operator converges it. An adopted node is
|
||||
said to be adopted wherever the mesh reports a node's state.
|
||||
|
||||
**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with
|
||||
an active firewall or services listening that are not the mesh's, genesis refuses and names what
|
||||
it found — a forgotten flag must not close a working machine.
|
||||
**A converged genesis refuses a machine in use.** A machine is in use when a container is running
|
||||
on it or a port is listening that is neither ssh nor one of the operating system's own services.
|
||||
Raised without saying adopted on such a machine, genesis refuses and names what it found — a
|
||||
forgotten flag must not close a working machine.
|
||||
|
||||
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
|
||||
that write its configuration. Its services keep running on what they have.
|
||||
@@ -92,23 +93,34 @@ file and a found container as they are, records the file's original content befo
|
||||
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
|
||||
the module declares that is not found is created; what is found is held. **Taking a module** on a
|
||||
node is its cutover — the operator's act, done when that module's data has moved — and from then
|
||||
on the module's resources converge on that node like any other. A held file that changes while
|
||||
held is reported as changed by something else, not reverted: that is how a predecessor still
|
||||
writing is caught. A held file whose module is unassigned is left where it is, never removed.
|
||||
on the module's resources converge on that node like any other. What is held is never removed,
|
||||
even when its module is unassigned, and a held file or container that changes while held — a file
|
||||
rewritten, a container stopped or replaced — is reported as changed by something else, not reverted
|
||||
or restarted: that is how a predecessor still writing is caught.
|
||||
|
||||
**The firewall found on the machine stays in force.** The mesh loads no table of its own on an
|
||||
adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh
|
||||
needs is declared as **openings**: a resource that says a port is reachable, from where, on the
|
||||
incoming path or the forwarded path — a published container port is forwarded. The host converges
|
||||
an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and
|
||||
removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot
|
||||
of the found firewall does not lose it. An opening is a state, not a command, which is what lets it
|
||||
travel over the link. The host reports which firewall it found, and a machine with a kind no host
|
||||
speaks is refused adoption rather than adopted with nothing protecting the mesh's ports.
|
||||
**The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
|
||||
that drops by default or that accepts — neither genesis's base ruleset nor the filter module's
|
||||
derived one. What the mesh needs reachable is declared as **openings**: a resource that says a port
|
||||
is reachable, from where, on the incoming path or the forwarded path — a published container port is
|
||||
forwarded. The controller derives them from the same inputs as the filter: the `listens` of the
|
||||
modules assigned there, the private network's hub port, and the foundation's ports. The host
|
||||
converges an opening through the found firewall in that firewall's own terms, marks it as the
|
||||
mesh's, and removes only what it marked; it re-checks each opening on every reconcile, so a reload
|
||||
or a reboot of the found firewall does not lose it for longer than one reconcile. An opening is a
|
||||
state, not a command, which is what lets it travel over the link. The host reports which firewall
|
||||
it found. A machine with no firewall needs no openings; a machine with a kind no host speaks is
|
||||
refused adoption.
|
||||
|
||||
**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings
|
||||
from the private network and marked refusals from anywhere else, on the forwarded path — so the
|
||||
store is unreachable from outside whether or not the found firewall filters forwarded traffic.
|
||||
**The mesh guards its own ports itself, in a table of its own that only refuses.** It accepts by
|
||||
default and holds nothing but refusals, so it cannot close anything the machine serves — a drop in
|
||||
any chain is final and an accept in it would change nothing — and it is the mesh's, so the found
|
||||
firewall reloading does not touch it. It refuses the store's port and the broker's management port
|
||||
from anywhere but the private network, matched on the port the packet was sent to, before the
|
||||
container runtime redirects it. The bus and the registry stay reachable from anywhere, as a node
|
||||
enrols over the bus and pulls from the registry before it has a private-network address
|
||||
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)); so does the private network's
|
||||
hub port. The store is unreachable from outside whatever the found firewall does, and on a machine
|
||||
with none.
|
||||
|
||||
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
|
||||
checked free before anything is raised, refused with the name of what holds it. The ports given
|
||||
@@ -118,13 +130,16 @@ filter, the base ruleset, the private network's endpoint, and the addresses cons
|
||||
The private network's address range must not overlap a tunnel the predecessor still runs; genesis
|
||||
checks that too.
|
||||
|
||||
**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now
|
||||
— every listening socket and every published container port — and for each whether an assigned
|
||||
module declares it or it will close. The flip then takes every module not yet taken, loads the
|
||||
mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the
|
||||
container runtime's rules and the found firewall's own configuration stay on disk. Returning a
|
||||
converged node to adopted re-enables the firewall it retired. A node converges when its migration
|
||||
is done; the mesh is migrated when every node has converged.
|
||||
**Converging a node is one act, previewed.** It refuses while an assigned module still holds a
|
||||
found container: each service is taken on its own, when its data has moved, never by the flip. The
|
||||
preview lists what is reachable on the machine now — every listening socket and every published
|
||||
container port — and for each whether an assigned module declares it or it will close, and every
|
||||
module the flip will take, with what each will replace. The flip then takes those modules, loads the
|
||||
mesh's derived filter in place of its refusal-only table, and retires the found firewall by
|
||||
disabling it, never by flushing: the container runtime's rules and the found firewall's own
|
||||
configuration stay on disk. Returning a converged node to adopted unloads the derived filter,
|
||||
restores the refusal-only table and enables the found firewall again; what was taken stays taken. A
|
||||
node converges when its migration is done; the mesh is migrated when every node has converged.
|
||||
|
||||
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken
|
||||
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
|
||||
@@ -144,7 +159,8 @@ open).
|
||||
What got harder:
|
||||
|
||||
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
|
||||
path. One kind is found on the machines measured; another is refused until a host speaks it.
|
||||
path. One kind is found on the machines measured; another is refused until a host speaks it. The
|
||||
mesh's own guard does not depend on it: that table is the mesh's.
|
||||
- **The host gains a guard it did not have** — keep what you found — and its report must say
|
||||
which files and containers it holds, or an adopted node reads as converged.
|
||||
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
|
||||
@@ -154,7 +170,7 @@ What got harder:
|
||||
nodes are adopted, so one left behind is visible.
|
||||
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
|
||||
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
|
||||
reachable from outside — passes to the mesh's own openings and refusals on the forwarded path.
|
||||
reachable from outside — passes to a table of the mesh's that only refuses.
|
||||
|
||||
## How it is checked
|
||||
|
||||
@@ -163,29 +179,33 @@ port and denying the rest, a service container listening on that port under a na
|
||||
module also uses, a file at a path that module declares, a stand-in for the predecessor's control
|
||||
that would rewrite that file, and a container holding the registry's port. Then:
|
||||
|
||||
- **Genesis converged** on it refuses and names the firewall and the listener.
|
||||
- **Genesis converged** on it refuses and names the running container and the listener.
|
||||
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
|
||||
port given, the foundation comes up — and adopting the foundation as modules leaves it on that
|
||||
port.
|
||||
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
|
||||
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
|
||||
- **The store is unreachable from outside** — probed from a machine off the private network — and
|
||||
reachable over it.
|
||||
- **The store is unreachable from outside** — probed from a machine off the private network, and
|
||||
again after the found firewall is reloaded — and reachable over it; the bus is reachable from a
|
||||
machine that has not yet enrolled.
|
||||
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the
|
||||
machine reboots**: the second machine enrols, and the openings are there again.
|
||||
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change
|
||||
is reported and not reverted.
|
||||
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
|
||||
taken, it replaces them and its port is opened.
|
||||
- **Converging previews, then changes**: the preview names the service's port and a published port
|
||||
no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall
|
||||
is disabled with its configuration still on disk, the declared port is open and the undeclared one
|
||||
closed. Returned to adopted, the found firewall is enabled again.
|
||||
- **Converging previews, then changes**: it refuses while the service's module holds its found
|
||||
container; once that module is taken, the preview names the service's port and a published port no
|
||||
firewall rule mentions, and the modules it will take; after the flip the mesh's derived filter is
|
||||
loaded, the found firewall is disabled with its configuration still on disk, the declared port is
|
||||
open and the undeclared one closed. Returned to adopted, the found firewall is enabled again and
|
||||
the derived filter is gone.
|
||||
|
||||
Unit tests hold the host to keeping a found file and container on an adopted node, converging them
|
||||
once taken, never removing a held file, and reporting a held file that changed; genesis to refusing
|
||||
a held port and a converged raise on a machine in use; the controller to carrying the mode and the
|
||||
taken modules in every declaration.
|
||||
once taken, never removing what it holds, and reporting a held file or container that changed;
|
||||
genesis to refusing a held port and a converged raise on a machine in use; the controller to
|
||||
carrying the mode and the taken modules in every declaration, deriving the openings, and refusing
|
||||
a flip while a found container is held.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -139,8 +139,9 @@ name, that the host's store has no record of writing. On an adopted node the hos
|
||||
found for any module not yet taken: it records a found file's original content before anything
|
||||
else, and it reports the file or container as held — a report that says what it holds, so an
|
||||
adopted node never reads as converged. Once the module is taken, its resources converge like any
|
||||
other. A held file that changes while held is reported as changed by something else, not
|
||||
reverted; a held file is never removed, even when its module is unassigned. The host also
|
||||
other. What is held is never removed, even when its module is unassigned, and a held file or
|
||||
container that changes while held is reported as changed by something else, not reverted or
|
||||
restarted. The host also
|
||||
converges a new resource, the **opening** — a port made reachable through the firewall it found
|
||||
([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the
|
||||
companion the host's ownership rule needed: *never touch what you did not create, unless adoption
|
||||
@@ -212,6 +213,11 @@ ready; the current bundle simply does not. **All of them are built:**
|
||||
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
|
||||
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
|
||||
|
||||
One more shape is decided and not yet built: **`opening`**, on adopted nodes only
|
||||
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — a port
|
||||
made reachable, from where, on the incoming or forwarded path, through the firewall found on the
|
||||
machine, marked as the mesh's and re-checked on every reconcile.
|
||||
|
||||
**A service says what it must reflect, and that is declared state rather than a command.**
|
||||
`restart-on` names files whose change means the unit must be restarted — because a running service
|
||||
does not re-read its configuration, and replacing a file, finding the service already running and
|
||||
|
||||
@@ -178,8 +178,10 @@ port never answers, the bus's does.
|
||||
machine already serving under a predecessor has a firewall of its own, and a second, stricter
|
||||
table would close everything it serves. There the base ruleset is not loaded and the filter module
|
||||
is not assigned until the node converges; the foundation's ports are opened through the found
|
||||
firewall from the private network and refused from anywhere else on the forwarded path, which
|
||||
keeps the same promise — the store's port never answers from outside — by other means. The
|
||||
firewall, and a table of the mesh's that only refuses keeps the store's port and the broker's
|
||||
management port from anyone off the private network — the same promise, the store's port never
|
||||
answering from outside, kept by other means. The bus and the registry stay reachable from anywhere,
|
||||
as they are here, because a node enrols and pulls before it has a private-network address. The
|
||||
foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked**
|
||||
by the adoption bed, which makes the same outside probe.
|
||||
|
||||
|
||||
@@ -537,24 +537,32 @@ is why the check reads packets.*
|
||||
### On an adopted node
|
||||
|
||||
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh
|
||||
loads no table of its own there — neither genesis's base ruleset nor the derived one. Every base
|
||||
chain at a hook runs and a drop in any is final, whatever the other firewall is written in, so a
|
||||
second, stricter table would close every port the machine serves. What the mesh needs reachable
|
||||
it declares as **openings**: a port, from where, on the incoming path or the forwarded path — a
|
||||
loads no table there that drops by default or that accepts — neither genesis's base ruleset nor
|
||||
the derived one. Every base chain at a hook runs and a drop in any is final, whatever the other
|
||||
firewall is written in, so a second, stricter table would close every port the machine serves, and
|
||||
an accept in one would open nothing the found firewall drops. What the mesh needs reachable it
|
||||
declares as **openings**: a port, from where, on the incoming path or the forwarded path — a
|
||||
published container port is forwarded, and a firewall that filters only incoming traffic never
|
||||
sees it. The host converges each opening through the found firewall in that firewall's own terms,
|
||||
marks it as the mesh's, removes only what it marked, and re-checks every opening on each reconcile
|
||||
so a reload or a reboot does not lose it. An opening is state, not a command, so it travels over
|
||||
the link like any other resource. **The mesh protects its own ports itself**: its foundation's ports
|
||||
are opened from the private network and refused from anywhere else on the forwarded path, so the
|
||||
store stays unreachable from outside whether or not the found firewall filters forwarded traffic.
|
||||
One kind of found firewall is spoken; a machine with another is refused adoption rather than
|
||||
adopted unprotected. Converging the node previews what is reachable now — listening sockets and
|
||||
published ports — and what will close, then loads the derived filter and disables the found
|
||||
firewall without flushing it. *How it is checked:* the adoption bed asserts the found firewall's
|
||||
rules differ only by the mesh's marked rules, that the store is unreachable from off the private
|
||||
network, that a second machine enrols through the openings before and after a reload and a reboot,
|
||||
and that after the flip the declared port is open and the undeclared one closed.
|
||||
sees it. The controller derives them from what the filter would be derived from: the assigned
|
||||
modules' `listens`, the hub's port, the foundation's ports. The host converges each opening through
|
||||
the found firewall in that firewall's own terms, marks it as the mesh's, removes only what it
|
||||
marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for
|
||||
longer than one reconcile. An opening is state, not a command, so it travels over the link like any
|
||||
other resource. **The mesh guards its own ports in a table of its own that only refuses** —
|
||||
accepting by default and holding nothing but refusals, so it cannot close what the machine serves,
|
||||
and the found firewall's reload does not touch it. It refuses the store's port and the broker's
|
||||
management port from outside the private network, matched on the port the packet was sent to; the
|
||||
bus, the registry and the hub's port stay reachable from anywhere, as a node enrols and pulls
|
||||
before it has a private-network address. One kind of found firewall is spoken; a machine with none
|
||||
needs no openings, and a machine with another kind is refused adoption. Converging the node refuses
|
||||
while a found container is still held; otherwise it previews what is reachable now — listening
|
||||
sockets and published ports — what will close and which modules it will take, then loads the
|
||||
derived filter in place of the refusal-only table and disables the found firewall without flushing
|
||||
it. *How it is checked:* the adoption bed asserts the found firewall's rules differ only by the
|
||||
mesh's marked rules, that the store is unreachable from off the private network before and after
|
||||
the found firewall reloads, that a machine not yet enrolled reaches the bus, that a second machine
|
||||
enrols through the openings before and after a reload and a reboot, and that after the flip the
|
||||
declared port is open and the undeclared one closed.
|
||||
|
||||
## 5 — Certificates
|
||||
|
||||
|
||||
@@ -391,12 +391,11 @@ should be and sends it; the host applies the difference and removes what is no l
|
||||
The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container
|
||||
runtime because a declaration changed would stop every container on the node.
|
||||
|
||||
---
|
||||
|
||||
|
||||
*On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)),
|
||||
what the host is holding was found, not made, so nothing held is ever removed: a held file whose
|
||||
module is unassigned stays where it is.
|
||||
what the host is holding was found, not made, so nothing held is ever removed: a held file or
|
||||
container whose module is unassigned stays where it is.
|
||||
|
||||
---
|
||||
|
||||
## enrolled ⇄ disconnected
|
||||
|
||||
|
||||
@@ -110,11 +110,11 @@ ruleset, the private network's endpoint, the addresses consumers are given — r
|
||||
the control-node measured, the store's and the bus's usual ports were free and the registry's, the
|
||||
broker's management port and, as the private network's hub, its port were held. Genesis also
|
||||
checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted
|
||||
**does not load the base ruleset**: the machine's own firewall already filters, and the mesh opens
|
||||
its foundation's ports through it and refuses them from outside itself
|
||||
([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — an
|
||||
active firewall or listeners that are not the mesh's — so a forgotten flag cannot close a working
|
||||
machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
|
||||
**does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its
|
||||
foundation's ports through it and keeps the store from outside with a table of its own that only
|
||||
refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** —
|
||||
a container running, or a port listening that is neither ssh nor the operating system's own — so a
|
||||
forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
|
||||
asserts the refusal, then adopted with the registry's port held and asserts the refusal names its
|
||||
holder, then with another port given asserts the foundation comes up, stays on that port once
|
||||
adopted as modules, and the machine's service is still reachable.
|
||||
|
||||
Reference in New Issue
Block a user