diff --git a/01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md b/01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md index b37efd3..c0a8013 100644 --- a/01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md +++ b/01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md @@ -116,12 +116,15 @@ intention, and each thing the mesh would otherwise take must say what it does in recorded, until the operator **takes** the module on that node — the cutover, done when the module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule never fires: the host only ever sees what assigned modules declare. -- **The firewall found on the machine stays in force.** The mesh does not load its own table on an - adopted node. What it needs open it declares as openings the host converges *through the found - firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every - reconcile so a reload or reboot does not lose them. An accept in a table of its own would not - help: the found firewall's drop would still be final. And the mesh protects its own ports itself, - on the forwarded path, since the found firewall may not. +- **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node + that drops by default or accepts. What it needs open it declares as openings the host converges + *through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and + re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of + its own would not help: the found firewall's drop would still be final. What a table of its own + *can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's + management port from outside the private network in a table that only refuses, which the found + firewall may not do and cannot undo. The bus, the registry and the hub's port stay open to + anywhere: a node enrols before it has a private-network address. - **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as that node's settings, read everywhere they are used, so adopting the foundation as modules does not move them back. diff --git a/02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md b/02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md index 34272a1..fd1df4a 100644 --- a/02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md +++ b/02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md @@ -76,9 +76,10 @@ authoritative, and every declaration it sends says whether the node is adopted a have been taken on it. A node stays adopted until the operator converges it. An adopted node is said to be adopted wherever the mesh reports a node's state. -**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with -an active firewall or services listening that are not the mesh's, genesis refuses and names what -it found — a forgotten flag must not close a working machine. +**A converged genesis refuses a machine in use.** A machine is in use when a container is running +on it or a port is listening that is neither ssh nor one of the operating system's own services. +Raised without saying adopted on such a machine, genesis refuses and names what it found — a +forgotten flag must not close a working machine. **Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons that write its configuration. Its services keep running on what they have. @@ -92,23 +93,34 @@ file and a found container as they are, records the file's original content befo happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what the module declares that is not found is created; what is found is held. **Taking a module** on a node is its cutover — the operator's act, done when that module's data has moved — and from then -on the module's resources converge on that node like any other. A held file that changes while -held is reported as changed by something else, not reverted: that is how a predecessor still -writing is caught. A held file whose module is unassigned is left where it is, never removed. +on the module's resources converge on that node like any other. What is held is never removed, +even when its module is unassigned, and a held file or container that changes while held — a file +rewritten, a container stopped or replaced — is reported as changed by something else, not reverted +or restarted: that is how a predecessor still writing is caught. -**The firewall found on the machine stays in force.** The mesh loads no table of its own on an -adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh -needs is declared as **openings**: a resource that says a port is reachable, from where, on the -incoming path or the forwarded path — a published container port is forwarded. The host converges -an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and -removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot -of the found firewall does not lose it. An opening is a state, not a command, which is what lets it -travel over the link. The host reports which firewall it found, and a machine with a kind no host -speaks is refused adoption rather than adopted with nothing protecting the mesh's ports. +**The firewall found on the machine stays in force.** The mesh loads no table on an adopted node +that drops by default or that accepts — neither genesis's base ruleset nor the filter module's +derived one. What the mesh needs reachable is declared as **openings**: a resource that says a port +is reachable, from where, on the incoming path or the forwarded path — a published container port is +forwarded. The controller derives them from the same inputs as the filter: the `listens` of the +modules assigned there, the private network's hub port, and the foundation's ports. The host +converges an opening through the found firewall in that firewall's own terms, marks it as the +mesh's, and removes only what it marked; it re-checks each opening on every reconcile, so a reload +or a reboot of the found firewall does not lose it for longer than one reconcile. An opening is a +state, not a command, which is what lets it travel over the link. The host reports which firewall +it found. A machine with no firewall needs no openings; a machine with a kind no host speaks is +refused adoption. -**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings -from the private network and marked refusals from anywhere else, on the forwarded path — so the -store is unreachable from outside whether or not the found firewall filters forwarded traffic. +**The mesh guards its own ports itself, in a table of its own that only refuses.** It accepts by +default and holds nothing but refusals, so it cannot close anything the machine serves — a drop in +any chain is final and an accept in it would change nothing — and it is the mesh's, so the found +firewall reloading does not touch it. It refuses the store's port and the broker's management port +from anywhere but the private network, matched on the port the packet was sent to, before the +container runtime redirects it. The bus and the registry stay reachable from anywhere, as a node +enrols over the bus and pulls from the registry before it has a private-network address +([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)); so does the private network's +hub port. The store is unreachable from outside whatever the found firewall does, and on a machine +with none. **The foundation's ports are the node's.** Every port the foundation binds is an input to genesis, checked free before anything is raised, refused with the name of what holds it. The ports given @@ -118,13 +130,16 @@ filter, the base ruleset, the private network's endpoint, and the addresses cons The private network's address range must not overlap a tunnel the predecessor still runs; genesis checks that too. -**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now -— every listening socket and every published container port — and for each whether an assigned -module declares it or it will close. The flip then takes every module not yet taken, loads the -mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the -container runtime's rules and the found firewall's own configuration stay on disk. Returning a -converged node to adopted re-enables the firewall it retired. A node converges when its migration -is done; the mesh is migrated when every node has converged. +**Converging a node is one act, previewed.** It refuses while an assigned module still holds a +found container: each service is taken on its own, when its data has moved, never by the flip. The +preview lists what is reachable on the machine now — every listening socket and every published +container port — and for each whether an assigned module declares it or it will close, and every +module the flip will take, with what each will replace. The flip then takes those modules, loads the +mesh's derived filter in place of its refusal-only table, and retires the found firewall by +disabling it, never by flushing: the container runtime's rules and the found firewall's own +configuration stay on disk. Returning a converged node to adopted unloads the derived filter, +restores the refusal-only table and enables the found firewall again; what was taken stays taken. A +node converges when its migration is done; the mesh is migrated when every node has converged. **The order is the operator's:** the control-node first, adopted, its modules assigned and taken one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's @@ -144,7 +159,8 @@ open). What got harder: - **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded - path. One kind is found on the machines measured; another is refused until a host speaks it. + path. One kind is found on the machines measured; another is refused until a host speaks it. The + mesh's own guard does not depend on it: that table is the mesh's. - **The host gains a guard it did not have** — keep what you found — and its report must say which files and containers it holds, or an adopted node reads as converged. - **The declaration gains a node's mode and its taken modules**, and a resource, the opening. @@ -154,7 +170,7 @@ What got harder: nodes are adopted, so one left behind is visible. - **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never - reachable from outside — passes to the mesh's own openings and refusals on the forwarded path. + reachable from outside — passes to a table of the mesh's that only refuses. ## How it is checked @@ -163,29 +179,33 @@ port and denying the rest, a service container listening on that port under a na module also uses, a file at a path that module declares, a stand-in for the predecessor's control that would rewrite that file, and a container holding the registry's port. Then: -- **Genesis converged** on it refuses and names the firewall and the listener. +- **Genesis converged** on it refuses and names the running container and the listener. - **Genesis adopted, with the registry's port held**, refuses and names the holder; with another port given, the foundation comes up — and adopting the foundation as modules leaves it on that port. - **Nothing that serves changed**: the service is reachable from a second machine, the file is byte for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's. -- **The store is unreachable from outside** — probed from a machine off the private network — and - reachable over it. +- **The store is unreachable from outside** — probed from a machine off the private network, and + again after the found firewall is reloaded — and reachable over it; the bus is reachable from a + machine that has not yet enrolled. - **The mesh works through the found firewall, and keeps working after it is reloaded and after the machine reboots**: the second machine enrols, and the openings are there again. - **A predecessor still writing is caught**: with the stand-in left running, the held file's change is reported and not reverted. - **Assigning prepares, taking cuts over**: the module assigned holds the found container and file; taken, it replaces them and its port is opened. -- **Converging previews, then changes**: the preview names the service's port and a published port - no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall - is disabled with its configuration still on disk, the declared port is open and the undeclared one - closed. Returned to adopted, the found firewall is enabled again. +- **Converging previews, then changes**: it refuses while the service's module holds its found + container; once that module is taken, the preview names the service's port and a published port no + firewall rule mentions, and the modules it will take; after the flip the mesh's derived filter is + loaded, the found firewall is disabled with its configuration still on disk, the declared port is + open and the undeclared one closed. Returned to adopted, the found firewall is enabled again and + the derived filter is gone. Unit tests hold the host to keeping a found file and container on an adopted node, converging them -once taken, never removing a held file, and reporting a held file that changed; genesis to refusing -a held port and a converged raise on a machine in use; the controller to carrying the mode and the -taken modules in every declaration. +once taken, never removing what it holds, and reporting a held file or container that changed; +genesis to refusing a held port and a converged raise on a machine in use; the controller to +carrying the mode and the taken modules in every declaration, deriving the openings, and refusing +a flip while a found container is held. ## References diff --git a/03-DESIGN/01-to-be/05-the-node-host.md b/03-DESIGN/01-to-be/05-the-node-host.md index a9f3068..b78d4f3 100644 --- a/03-DESIGN/01-to-be/05-the-node-host.md +++ b/03-DESIGN/01-to-be/05-the-node-host.md @@ -139,8 +139,9 @@ name, that the host's store has no record of writing. On an adopted node the hos found for any module not yet taken: it records a found file's original content before anything else, and it reports the file or container as held — a report that says what it holds, so an adopted node never reads as converged. Once the module is taken, its resources converge like any -other. A held file that changes while held is reported as changed by something else, not -reverted; a held file is never removed, even when its module is unassigned. The host also +other. What is held is never removed, even when its module is unassigned, and a held file or +container that changes while held is reported as changed by something else, not reverted or +restarted. The host also converges a new resource, the **opening** — a port made reachable through the firewall it found ([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the companion the host's ownership rule needed: *never touch what you did not create, unless adoption @@ -212,6 +213,11 @@ ready; the current bundle simply does not. **All of them are built:** | `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift | | `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back | +One more shape is decided and not yet built: **`opening`**, on adopted nodes only +([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — a port +made reachable, from where, on the incoming or forwarded path, through the firewall found on the +machine, marked as the mesh's and re-checked on every reconcile. + **A service says what it must reflect, and that is declared state rather than a command.** `restart-on` names files whose change means the unit must be restarted — because a running service does not re-read its configuration, and replacing a file, finding the service already running and diff --git a/03-DESIGN/01-to-be/07-the-foundation.md b/03-DESIGN/01-to-be/07-the-foundation.md index 41479dd..3033842 100644 --- a/03-DESIGN/01-to-be/07-the-foundation.md +++ b/03-DESIGN/01-to-be/07-the-foundation.md @@ -178,8 +178,10 @@ port never answers, the bus's does. machine already serving under a predecessor has a firewall of its own, and a second, stricter table would close everything it serves. There the base ruleset is not loaded and the filter module is not assigned until the node converges; the foundation's ports are opened through the found -firewall from the private network and refused from anywhere else on the forwarded path, which -keeps the same promise — the store's port never answers from outside — by other means. The +firewall, and a table of the mesh's that only refuses keeps the store's port and the broker's +management port from anyone off the private network — the same promise, the store's port never +answering from outside, kept by other means. The bus and the registry stay reachable from anywhere, +as they are here, because a node enrols and pulls before it has a private-network address. The foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked** by the adoption bed, which makes the same outside probe. diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index cf0a89f..31db740 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -537,24 +537,32 @@ is why the check reads packets.* ### On an adopted node *2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh -loads no table of its own there — neither genesis's base ruleset nor the derived one. Every base -chain at a hook runs and a drop in any is final, whatever the other firewall is written in, so a -second, stricter table would close every port the machine serves. What the mesh needs reachable -it declares as **openings**: a port, from where, on the incoming path or the forwarded path — a +loads no table there that drops by default or that accepts — neither genesis's base ruleset nor +the derived one. Every base chain at a hook runs and a drop in any is final, whatever the other +firewall is written in, so a second, stricter table would close every port the machine serves, and +an accept in one would open nothing the found firewall drops. What the mesh needs reachable it +declares as **openings**: a port, from where, on the incoming path or the forwarded path — a published container port is forwarded, and a firewall that filters only incoming traffic never -sees it. The host converges each opening through the found firewall in that firewall's own terms, -marks it as the mesh's, removes only what it marked, and re-checks every opening on each reconcile -so a reload or a reboot does not lose it. An opening is state, not a command, so it travels over -the link like any other resource. **The mesh protects its own ports itself**: its foundation's ports -are opened from the private network and refused from anywhere else on the forwarded path, so the -store stays unreachable from outside whether or not the found firewall filters forwarded traffic. -One kind of found firewall is spoken; a machine with another is refused adoption rather than -adopted unprotected. Converging the node previews what is reachable now — listening sockets and -published ports — and what will close, then loads the derived filter and disables the found -firewall without flushing it. *How it is checked:* the adoption bed asserts the found firewall's -rules differ only by the mesh's marked rules, that the store is unreachable from off the private -network, that a second machine enrols through the openings before and after a reload and a reboot, -and that after the flip the declared port is open and the undeclared one closed. +sees it. The controller derives them from what the filter would be derived from: the assigned +modules' `listens`, the hub's port, the foundation's ports. The host converges each opening through +the found firewall in that firewall's own terms, marks it as the mesh's, removes only what it +marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for +longer than one reconcile. An opening is state, not a command, so it travels over the link like any +other resource. **The mesh guards its own ports in a table of its own that only refuses** — +accepting by default and holding nothing but refusals, so it cannot close what the machine serves, +and the found firewall's reload does not touch it. It refuses the store's port and the broker's +management port from outside the private network, matched on the port the packet was sent to; the +bus, the registry and the hub's port stay reachable from anywhere, as a node enrols and pulls +before it has a private-network address. One kind of found firewall is spoken; a machine with none +needs no openings, and a machine with another kind is refused adoption. Converging the node refuses +while a found container is still held; otherwise it previews what is reachable now — listening +sockets and published ports — what will close and which modules it will take, then loads the +derived filter in place of the refusal-only table and disables the found firewall without flushing +it. *How it is checked:* the adoption bed asserts the found firewall's rules differ only by the +mesh's marked rules, that the store is unreachable from off the private network before and after +the found firewall reloads, that a machine not yet enrolled reaches the bus, that a second machine +enrols through the openings before and after a reload and a reboot, and that after the flip the +declared port is open and the undeclared one closed. ## 5 — Certificates diff --git a/03-DESIGN/01-to-be/09-the-node-lifecycle.md b/03-DESIGN/01-to-be/09-the-node-lifecycle.md index 8c42f41..6b50933 100644 --- a/03-DESIGN/01-to-be/09-the-node-lifecycle.md +++ b/03-DESIGN/01-to-be/09-the-node-lifecycle.md @@ -391,12 +391,11 @@ should be and sends it; the host applies the difference and removes what is no l The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container runtime because a declaration changed would stop every container on the node. ---- - - *On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), -what the host is holding was found, not made, so nothing held is ever removed: a held file whose -module is unassigned stays where it is. +what the host is holding was found, not made, so nothing held is ever removed: a held file or +container whose module is unassigned stays where it is. + +--- ## enrolled ⇄ disconnected diff --git a/03-DESIGN/01-to-be/17-raising-a-mesh.md b/03-DESIGN/01-to-be/17-raising-a-mesh.md index 9683ab0..429d874 100644 --- a/03-DESIGN/01-to-be/17-raising-a-mesh.md +++ b/03-DESIGN/01-to-be/17-raising-a-mesh.md @@ -110,11 +110,11 @@ ruleset, the private network's endpoint, the addresses consumers are given — r the control-node measured, the store's and the bus's usual ports were free and the registry's, the broker's management port and, as the private network's hub, its port were held. Genesis also checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted -**does not load the base ruleset**: the machine's own firewall already filters, and the mesh opens -its foundation's ports through it and refuses them from outside itself -([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — an -active firewall or listeners that are not the mesh's — so a forgotten flag cannot close a working -machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and +**does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its +foundation's ports through it and keeps the store from outside with a table of its own that only +refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — +a container running, or a port listening that is neither ssh nor the operating system's own — so a +forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and asserts the refusal, then adopted with the registry's port held and asserts the refusal names its holder, then with another port given asserts the foundation comes up, stays on that port once adopted as modules, and the machine's service is still reachable.